system

Abstract

We provide the system. [Solution] An input method for entering student learning-related information, A generation means that generates individually optimized educational content based on student learning-related information obtained from the input means, A storage means for storing the educational content generated by the generation means in a database, A means for teachers to access the educational content stored in the aforementioned storage means, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In special needs schools, despite the requirement for educational programs tailored to the diverse disabilities of individual students, due to a shortage of teaching staff and a heavy burden in preparing teaching materials, there is a problem that it is difficult to quickly provide individually optimized educational content. As a result, appropriate learning opportunities according to the characteristics of each student are not sufficiently provided, and the improvement of educational effects is hindered.

Means for Solving the Problems

[0005] This invention comprises an input means for inputting student learning-related information, a generation means for generating individually optimized educational content using an artificial intelligence model based on the input information, a storage means for saving the generated educational content to a database, and a provision means for enabling teachers to access the saved content. This enables teachers in special needs schools to efficiently and quickly provide individually optimized teaching materials to students, thereby solving current educational challenges.

[0006] The "input method" is an interface for collecting and incorporating students' learning-related information into the system.

[0007] "Generation means" refers to the process or apparatus for creating individually optimized educational content based on acquired student information.

[0008] A "storage method" refers to a system for safely and efficiently storing generated educational content in a database.

[0009] "Means of delivery" refers to functions and systems that enable teachers to access stored educational content and use it as appropriate teaching material for students.

[0010] A "generative artificial intelligence model" is a program that utilizes algorithms and machine learning to generate individualized learning programs based on student information. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention relates to an automated system for generating educational materials in special needs schools, and provides technology for individually optimizing student learning. This system is realized through the interaction of several entities.

[0033] First, the user (teacher) accesses the system using a dedicated terminal. The user enters detailed learning-related information about each student, including the student's name, type and severity of disability, and areas of expertise and interests, using an interface that allows for accurate understanding of each student's individual characteristics.

[0034] Next, the terminal securely transmits this input information to the server. The data is encrypted and delivered accurately to the server while preventing unauthorized access through an authentication process. This transmitted information is securely stored in a database on the server and organized so that teachers can refer to it later.

[0035] Information arriving on the server is processed using a generative AI model. This AI model, similar to a human teacher, considers various educational elements based on students' learning needs and quickly generates individually optimized educational content. Specifically, the generated content includes practice problems to improve basic academic skills and activities that accommodate disabilities. Furthermore, the design and layout of the teaching materials are automatically adjusted to enhance student learning.

[0036] The generated content is stored in the server's database, awaiting later access. When a user logs back into the system via their terminal, they gain access to this stored educational content, allowing them to easily download or print materials tailored to the individual needs of their students.

[0037] In this way, the system of the present invention aims to improve the quality of education by enabling teachers to efficiently and effectively provide individually optimized learning materials to diverse students in special needs schools.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, hobbies, and interests. Users can easily and quickly enter information using an intuitive interface.

[0041] Step 2:

[0042] The terminal encrypts the entered information and sends it to the server using a secure data communication protocol. During transmission, an authentication process is performed, and measures are taken to protect the data from unauthorized access.

[0043] Step 3:

[0044] The server analyzes the received data and stores it appropriately in the database. During storage, a verification process is performed to ensure data integrity, and the data is organized in preparation for subsequent processing.

[0045] Step 4:

[0046] The server runs a generation AI model based on the stored information. The AI ​​model generates individually optimized educational content based on the students' attribute information. At this stage, the AI ​​combines various educational elements to determine the appropriate teaching materials and activities for each student.

[0047] Step 5:

[0048] The server stores the generated educational content in a database and categorizes it as needed. This process allows users to access the content in a format that is easy to use later.

[0049] Step 6:

[0050] Users access the system again from their terminals, search for and select educational content for the desired students, and download or print it. This allows teachers to efficiently utilize prepared materials in the classroom and provide instruction tailored to the individual characteristics of each student.

[0051] (Example 1)

[0052] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0053] Providing educational materials tailored to the individual characteristics of students in special needs schools is a significant burden for teachers using traditional methods. Furthermore, creating these materials is time-consuming and labor-intensive, making it difficult to quickly generate materials suitable for each student's needs. Security and accessibility of information also pose challenges.

[0054] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0055] In this invention, the server includes a registration means for inputting student attribute information, a creation means for generating individually optimized educational materials based on the student attribute information obtained from the registration means, and a storage means for storing the educational materials generated by the creation means in a storage medium. This enables the rapid generation of personalized educational materials and safe and easy access to them.

[0056] "Registration method" refers to an interface for inputting student attribute information and recording it in the system.

[0057] "Creative means" refers to a device or process for automatically generating educational materials optimized for individual learners based on inputted student attribute information.

[0058] "Storage means" refers to a device or function that securely stores generated educational materials on a storage medium and manages them so that they can be accessed later.

[0059] "Communication methods" refer to protocols and devices that use encryption technology to send and receive data and maintain the confidentiality and integrity of information.

[0060] "Means of acquisition" refers to an interface that allows educators to easily access and acquire educational materials stored on a storage medium through an electronic device.

[0061] This invention relates to a system for automatically generating individualized educational materials in special needs education. The system is configured to efficiently create customized educational content based on the characteristics of specific students.

[0062] The user (teacher) first accesses the system using a dedicated terminal. This terminal is equipped with an input interface that allows the user to input attribute information for each student (e.g., grade level, favorite subjects, activities of interest, special support needs, etc.). For example, if student A expresses interest in "mathematics teaching materials that make extensive use of visual elements," the user would input that information.

[0063] The terminal securely transmits the entered information to the server using encryption technology. High-security encryption protocols such as AES and TLS are used.

[0064] The server stores the received information in a database and uses a generative AI model to generate educational materials based on that information. The generative AI model can quickly design optimal educational content using the information about the students that has been input. For example, for student A, who needs visual learning materials, the generative AI model can be used to provide math practice problems using colorful shapes.

[0065] The generated teaching materials are saved in PDF format in the server's database and can be accessed by the user later. The user can then log back into the system and download or print the educational materials via their terminal.

[0066] A concrete example of a prompt might be, "Provide an integer addition lesson with enhanced visual support." This allows teachers to quickly and easily provide lessons tailored to students with special needs. The aim is to improve the quality of education and reduce the burden on teachers.

[0067] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0068] Step 1:

[0069] The user enters student attribute information using a device. Through the input interface, they fill in information such as the student's name, grade level, special needs, and interests in the input form. The entered information is temporarily stored on the device.

[0070] Step 2:

[0071] The terminal encrypts the entered student information using AES encryption. The encrypted data is sent to the server using the secure HTTPS protocol. The input is student attribute information, and the output is encrypted data.

[0072] Step 3:

[0073] The server receives encrypted data sent from the terminal. The server decrypts the data and constructs a data structure based on the received student attribute information. The input is encrypted data, and the output is decrypted student information data.

[0074] Step 4:

[0075] The server inputs the decrypted student information into a generating AI model. The AI ​​model analyzes this information and generates educational content optimized for the specific student. For example, if visual learning materials are needed, colorful math practice problems will be created. The input is student information, and the output is the generated educational content.

[0076] Step 5:

[0077] The generated educational content is saved in PDF format in the server's database. The server organizes and manages this content so that users can access it later. The input is the educational content, and the output is the file stored in the database.

[0078] Step 6:

[0079] The user re-accesses the system using a terminal. The user selects the necessary student materials from the database and downloads or prints them. This provides personalized materials to the students. The input is the user's access request, and the output is the downloaded or printed educational content.

[0080] (Application Example 1)

[0081] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0082] Special education requires educational activities tailored to each student, but resources for providing individually optimized teaching materials are limited. Furthermore, it is currently difficult to provide products or experiences suitable for specific students in physical stores. To address these challenges, a system is needed that automatically generates and provides teaching materials and product information according to the characteristics of each student.

[0083] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0084] In this invention, the server includes an input means for inputting student learning-related information, a generation means for generating individually optimized educational content based on the learning-related information obtained from the input means, a storage means for storing the educational content generated by the generation means in a data storage unit, a recommendation means for generating product information suitable for students based on the stored educational content, and a display means for communicating the product information generated by the recommendation means via a display device. This enables the provision of educational materials in educational settings and the provision of optimal products in physical stores.

[0085] "Input means" refers to a device or interface for importing student learning-related information into the system.

[0086] "Generation means" refers to a device or program for creating individually optimized educational content or product information based on input student learning-related information.

[0087] "Storage means" refers to a device or function for recording and storing generated educational content in a data storage unit.

[0088] "Recommendation method" refers to a device or function for creating product information suitable for students based on saved educational content.

[0089] "Display means" refers to a device or interface for providing generated product information to a user through a display device.

[0090] This invention realizes a system that provides teaching materials and product information optimized for the educational environment of special needs schools. The main components are a server, terminals, and users.

[0091] First, the user uses a terminal to input student learning-related information. This terminal functions as an input method for collecting detailed information such as name, interests, and type and degree of disability through a dedicated interface.

[0092] The input information is transferred to the server via a secure communication protocol. The server automatically generates educational content tailored to each student's learning characteristics based on the input information. Here, a generation AI model is used to create individually optimized content. This is the generation method. The generated content is securely stored in a data storage unit. This storage unit organizes the content so that users can easily access it later.

[0093] Furthermore, the server incorporates a recommended system that generates personalized product information for students based on their stored educational content. This information is then provided to students through display devices used by employees. This allows for, for example, supporting the shopping experience in physical stores.

[0094] As a concrete example, if a student interested in science is identified based on information entered by the user, the server generates information on science-related products and activity recommendations. This information is then displayed on a display device such as smart glasses to support customer service in physical stores.

[0095] An example of a prompt for a generative AI model would be, "This student is interested in science. Please recommend appropriate products." In response to this prompt, the system dynamically generates and provides optimized information.

[0096] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0097] Step 1:

[0098] The user enters student learning-related information on a terminal. This information includes the student's name, interests, and the type and severity of their disability. This information is formatted into a database through the terminal's interface and prepared for transmission to the server.

[0099] Step 2:

[0100] The terminal encrypts the entered student information and sends it to the server. To ensure the security and privacy of the information, this process uses the SSL / TLS protocol to encrypt the data and securely transfer it to the server over the network.

[0101] Step 3:

[0102] The server decrypts the received encrypted data and obtains student learning-related information. Based on the obtained information, it sends prompt messages such as "This student is interested in science. Please recommend appropriate products." to the generating AI model, and generates educational content tailored to the student's learning characteristics.

[0103] Step 4:

[0104] The server stores the generated educational content in its data storage unit. In this process, the content is saved in a database format along with metadata to efficiently manage the information and facilitate later access.

[0105] Step 5:

[0106] The server generates product information suitable for students using recommendation methods based on stored educational content. Here, an AI model extracts and generates highly relevant product information from input information and educational content.

[0107] Step 6:

[0108] The server outputs the generated product information and sends it to the display device. Specifically, it formats the information in an appropriate format for devices such as smart glasses and tablets, and delivers it to the display device using a communication protocol.

[0109] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0110] This invention combines an emotion engine with an automated system for generating educational materials in special needs schools. Its aim is to provide a more individually optimized educational experience by considering not only students' learning progress but also the user's emotional state. This system recognizes the user's emotions in real time and generates and provides educational content based on that information. The implementation method will be described in detail below.

[0111] First, the user (teacher) logs into the system via a dedicated terminal. This terminal is equipped with a camera and microphone for facial recognition, and the emotion engine analyzes the user's voice tone and facial expressions in real time through these data input devices. As a result of the analysis, the user's current emotional state (e.g., reassurance, confusion, anxiety, etc.) is quantified, and this emotional data is sent to the system.

[0112] The device aggregates emotional data and student learning-related information and sends it to the server using a security protocol. The server stores the received data in a database, and the emotional data is recorded and linked to each student's information.

[0113] Next, the server generates educational content using a generative AI model based on the accumulated data. Unlike traditional generation methods that rely solely on student information, this generative model reflects the user's emotional state. This process allows for adjustments such as suggesting easier-to-handle materials if the user is stressed, or incorporating more challenging content if they are relaxed.

[0114] The generated educational content is organized and stored in a database, after which users can access it using their devices. Users can search for the stored content as needed and utilize it in classes and learning activities through downloading and printing. This allows for the provision of appropriate educational programs for each student, reducing the burden on teachers while improving student learning effectiveness.

[0115] The following describes the processing flow.

[0116] Step 1:

[0117] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, and activities of interest. During input, a camera and microphone capture the user's facial expressions and voice.

[0118] Step 2:

[0119] The device transmits the acquired user voice and facial expression data to the emotion engine. The emotion engine analyzes this data and generates the user's emotional state as numerical or categorical data. For example, it determines whether the user is relaxed, focused, or stressed.

[0120] Step 3:

[0121] The device transmits the entered student information and generated user sentiment data to the server via a security protocol. This data is encrypted, ensuring it reaches the server safely.

[0122] Step 4:

[0123] The server stores the received data in a database. Sentimental data is managed in conjunction with student information and referenced later in the generation of educational content.

[0124] Step 5:

[0125] The server runs a generative AI model based on stored data. The AI ​​model combines student information and the user's emotional state to generate individually optimized educational content. For example, if a user is feeling stressed, the system will suggest experiential tasks to help them relax.

[0126] Step 6:

[0127] The generated educational content is stored in a database, where it is organized and indexed. This ensures that the content can be effectively searched and used later.

[0128] Step 7:

[0129] Users access content via their devices and review the generated educational materials. They can download or print materials as needed, enabling them to create lesson plans optimized for their students.

[0130] (Example 2)

[0131] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0132] To achieve individually optimized education, it is crucial to consider not only students' learning progress but also teachers' emotional states in real time. However, conventional systems have struggled to grasp emotional states and generate appropriate educational content based on them. This has led to increased burdens on teachers and the inability to provide appropriate educational support to students.

[0133] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0134] In this invention, the server includes an analysis means that analyzes the user's facial expressions and voice and quantifies their emotional state; a generation means that generates individually optimized educational content based on the data acquired from the input means and the analysis means; and a storage means that stores the generated educational content in a data storage device. This makes it possible to provide teaching materials that take into account the emotional state of teachers, thereby improving the quality of education and reducing the burden on teachers.

[0135] "Input means" refers to devices or methods for acquiring student learning-related information and user sentiment data.

[0136] The "analysis method" is a technology that analyzes the user's facial expressions and voice in real time and quantifies their emotional state.

[0137] "Generation means" refers to a process or system for creating individually optimized educational content based on acquired learning-related information and emotional data.

[0138] A "storage method" refers to a system that saves generated educational content to a storage device or similar, allowing it to be retrieved and used later.

[0139] "Means of delivery" refers to the methods and technologies used to deliver stored educational content so that teachers can access it.

[0140] This invention is an automated system for generating educational materials in special needs education, which realizes an individually optimized educational experience by taking into account the user's emotional state. Specifically, the system is configured as follows.

[0141] Users access and log into the system using a dedicated information terminal. The terminal is equipped with a camera and microphone, which capture the user's facial expressions and voice in real time. This data is analyzed using an emotion engine, and the user's emotional state is quantified. For example, if a user has a smiling face and a calm voice tone, the system will determine that the user is "at ease."

[0142] The device centralizes captured emotional data and student learning-related information and sends it to the server using a secure communication protocol. This ensures the safety and reliability of the data.

[0143] The server stores received emotional data and learning information in a database. Here, the emotional data is linked to each student's learning history and recorded as comprehensive information. Subsequently, the server uses a generative AI model to generate educational content based on the received data. This model enables the customization of educational content according to the user's emotional state, achieving individual optimization.

[0144] As a concrete example, by instructing the generative AI model with the prompt "Provide content appropriate for when students are feeling at ease," appropriate teaching materials will be provided. This prompt serves as an indicator of what kind of content should be emphasized in the content presented by the model.

[0145] Users can access, download, and print the generated educational content through their devices for use in their learning activities. This enables the provision of educational programs optimized for individual students, thereby improving the quality of education.

[0146] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0147] Step 1:

[0148] The user logs into the system using an information terminal. Upon logging in, the camera and microphone built into the terminal activate, capturing the user's face and voice in real time. The input at this time consists of the user's facial image data and voice data, which are acquired to provide basic data for sentiment analysis.

[0149] Step 2:

[0150] The device transmits the acquired facial image data and audio data to the emotion engine. This engine uses machine learning algorithms to analyze the data and quantify the user's emotional state. This analysis process analyzes changes in facial expressions and tone of voice to generate emotion tags such as reassurance, confusion, and anxiety. The output is numerical data indicating the user's emotional state.

[0151] Step 3:

[0152] The device packages emotional data and student learning-related information together, encrypts it, and sends it to the server. The input consists of emotional data and learning-related information, and by securely sending these to the server using a security protocol, the output prevents information leakage.

[0153] Step 4:

[0154] The server stores the received sentiment data and learning information in a database. Simultaneously, it links the sentiment data to the corresponding student profile, combining sentiment with learning history. In this process, the input is the received data, and the output is the updated database entry.

[0155] Step 5:

[0156] The server runs a generative AI model using stored data to generate educational content. Using the prompt "Provide content appropriate for when students are feeling secure," it generates materials for students who are in a secure state. The input is stored emotion data and learning information, and the output is individually optimized educational content.

[0157] Step 6:

[0158] Users can access the generated educational content through their devices and search, download, or print the necessary learning materials. In this case, the input is the user's actions, and the output is physical or digital educational materials. This enables effective learning support in educational settings.

[0159] (Application Example 2)

[0160] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0161] In face-to-face sales, it is difficult for salespeople to accurately grasp a customer's emotional state, which contributes to inappropriate service and low customer satisfaction. Furthermore, it is not easy for salespeople with little customer service experience to immediately decide on the appropriate response. Therefore, there is a need for a system that can acquire customer emotional information in real time and provide optimal customer service advice based on that information.

[0162] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0163] In this invention, the server includes an input means for inputting customer facial expressions and voice data, a generation means for generating individually optimized customer service advice based on the customer's emotional state, and a provision means for sales staff to access and provide the generated customer service advice through a display device. This allows sales staff to grasp the customer's emotional state in real time and respond based on appropriate advice.

[0164] An "input device" is a device used to electronically acquire facial expressions and voice data collected from customers.

[0165] The "generation method" refers to a mechanism that creates individually optimized customer service advice based on acquired data.

[0166] "Storage means" refers to a device that stores the generated customer service advice in a landscape information recording device.

[0167] "Means of provision" refers to a system that makes stored customer service advice accessible to sales staff and presents it via a display device.

[0168] A "generative intelligence model" is an artificial intelligence algorithm used to derive appropriate advice from acquired data.

[0169] A "display device" is a device that allows sales staff to visually confirm the customer service advice that has been generated.

[0170] This embodiment of the invention relates to a sales staff support system for physical stores. This system enables sales staff to understand the customer's emotional state in real time and provide appropriate customer service. The specific details of the system are described below.

[0171] The server first collects customer facial expressions and voice data through input means. Wearable devices such as smart glasses are used as hardware. These devices are equipped with cameras and microphones to acquire data for analyzing the customer's facial expressions and voice tone.

[0172] Next, a lightweight machine learning model is used as a generation method to quantify the customer's emotional state (e.g., reassurance, confusion, etc.). The analyzed data is sent to a generative intelligence model (e.g., OpenAI's generative AI model), where personalized customer service advice based on the customer's emotional state is generated.

[0173] The generated customer service advice is stored as digital data in a landscape information recording device via a storage means. Salespeople can access this advice via a display device (e.g., smart glasses display) using a delivery means. This allows salespeople to implement the optimal customer service strategy at the appropriate time.

[0174] As a concrete example, while a salesperson is assisting a customer, the smart glasses display might show advice such as, "The customer is confused about which product to choose, so please provide clear explanations." Another example of a prompt for a generative intelligence model might be, "Please create a product suggestion that will interest the customer." In this way, the system aims to streamline the sales process and improve customer satisfaction.

[0175] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0176] Step 1:

[0177] The device acquires customer facial and audio data via the smart glasses' camera and microphone. The inputs are camera video and audio, and these data are temporarily stored on the device as output. Specifically, the camera captures the customer's face, and the microphone records their voice.

[0178] Step 2:

[0179] The device uses the acquired facial expression and voice data to perform an initial analysis with a lightweight machine learning model. The input is the data saved in step 1, and the output is data that quantifies the customer's emotional state. This process involves facial feature point extraction and voice tone analysis to identify the customer's emotional state (e.g., reassured, confused).

[0180] Step 3:

[0181] The terminal sends digitized emotion data to the server. The input is digitized emotion data, and the output is the completion of data transmission to the server. Here, data transfer takes place over the network, and the emotion data is registered in the server's database.

[0182] Step 4:

[0183] The server generates customer service advice using a generative AI model based on the received sentiment data. The input is sentiment data stored on the server, and the output is customer service advice based on a specific prompt. For example, the prompt might be "Suggest products that the customer is interested in."

[0184] Step 5:

[0185] The server saves the generated customer service advice to the scenery information recording device. The input is the generated customer service advice, and the output is the advice registered in the scenery information recording device. This operation ensures that the advice is saved in an easily searchable format.

[0186] Step 6:

[0187] The user views and confirms customer service advice stored in a landscape information recording device through smart glasses. The input is the stored advice information, and the output is the text of the advice displayed on the smart glasses' screen. This allows the user to take the most appropriate action in real time when interacting with customers.

[0188] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0189] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0190] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0191] [Second Embodiment]

[0192] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0193] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0194] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0195] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0196] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0197] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0198] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0199] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0200] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0201] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0202] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0203] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0204] This invention relates to an automated system for generating educational materials in special needs schools, and provides technology for individually optimizing student learning. This system is realized through the interaction of several entities.

[0205] First, the user (teacher) accesses the system using a dedicated terminal. The user enters detailed learning-related information about each student, including the student's name, type and severity of disability, and areas of expertise and interests, using an interface that allows for accurate understanding of each student's individual characteristics.

[0206] Next, the terminal securely transmits this input information to the server. The data is encrypted and delivered accurately to the server while preventing unauthorized access through an authentication process. This transmitted information is securely stored in a database on the server and organized so that teachers can refer to it later.

[0207] Information arriving on the server is processed using a generative AI model. This AI model, similar to a human teacher, considers various educational elements based on students' learning needs and quickly generates individually optimized educational content. Specifically, the generated content includes practice problems to improve basic academic skills and activities that accommodate disabilities. Furthermore, the design and layout of the teaching materials are automatically adjusted to enhance student learning.

[0208] The generated content is stored in the server's database, awaiting later access. When a user logs back into the system via their terminal, they gain access to this stored educational content, allowing them to easily download or print materials tailored to the individual needs of their students.

[0209] In this way, the system of the present invention aims to improve the quality of education by enabling teachers to efficiently and effectively provide individually optimized learning materials to diverse students in special needs schools.

[0210] The following describes the processing flow.

[0211] Step 1:

[0212] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, hobbies, and interests. Users can easily and quickly enter information using an intuitive interface.

[0213] Step 2:

[0214] The terminal encrypts the entered information and sends it to the server using a secure data communication protocol. During transmission, an authentication process is performed, and measures are taken to protect the data from unauthorized access.

[0215] Step 3:

[0216] The server analyzes the received data and stores it appropriately in the database. During storage, a verification process is performed to ensure data integrity, and the data is organized in preparation for subsequent processing.

[0217] Step 4:

[0218] The server runs a generation AI model based on the stored information. The AI ​​model generates individually optimized educational content based on the students' attribute information. At this stage, the AI ​​combines various educational elements to determine the appropriate teaching materials and activities for each student.

[0219] Step 5:

[0220] The server stores the generated educational content in a database and categorizes it as needed. This process allows users to access the content in a format that is easy to use later.

[0221] Step 6:

[0222] Users access the system again from their terminals, search for and select educational content for the desired students, and download or print it. This allows teachers to efficiently utilize prepared materials in the classroom and provide instruction tailored to the individual characteristics of each student.

[0223] (Example 1)

[0224] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0225] Providing educational materials tailored to the individual characteristics of students in special needs schools is a significant burden for teachers using traditional methods. Furthermore, creating these materials is time-consuming and labor-intensive, making it difficult to quickly generate materials suitable for each student's needs. Security and accessibility of information also pose challenges.

[0226] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0227] In this invention, the server includes a registration means for inputting student attribute information, a creation means for generating individually optimized educational materials based on the student attribute information obtained from the registration means, and a storage means for storing the educational materials generated by the creation means in a storage medium. This enables the rapid generation of personalized educational materials and safe and easy access to them.

[0228] "Registration method" refers to an interface for inputting student attribute information and recording it in the system.

[0229] "Creative means" refers to a device or process for automatically generating educational materials optimized for individual learners based on inputted student attribute information.

[0230] "Storage means" refers to a device or function that securely stores generated educational materials on a storage medium and manages them so that they can be accessed later.

[0231] "Communication methods" refer to protocols and devices that use encryption technology to send and receive data and maintain the confidentiality and integrity of information.

[0232] "Means of acquisition" refers to an interface that allows educators to easily access and acquire educational materials stored on a storage medium through an electronic device.

[0233] This invention relates to a system for automatically generating individualized educational materials in special needs education. The system is configured to efficiently create customized educational content based on the characteristics of specific students.

[0234] The user (teacher) first accesses the system using a dedicated terminal. This terminal is equipped with an input interface that allows the user to input attribute information for each student (e.g., grade level, favorite subjects, activities of interest, special support needs, etc.). For example, if student A expresses interest in "mathematics teaching materials that make extensive use of visual elements," the user would input that information.

[0235] The terminal securely transmits the entered information to the server using encryption technology. High-security encryption protocols such as AES and TLS are used.

[0236] The server stores the received information in a database and uses a generative AI model to generate educational materials based on that information. The generative AI model can quickly design optimal educational content using the information about the students that has been input. For example, for student A, who needs visual learning materials, the generative AI model can be used to provide math practice problems using colorful shapes.

[0237] The generated teaching materials are saved in PDF format in the server's database and can be accessed by the user later. The user can then log back into the system and download or print the educational materials via their terminal.

[0238] A concrete example of a prompt might be, "Provide an integer addition lesson with enhanced visual support." This allows teachers to quickly and easily provide lessons tailored to students with special needs. The aim is to improve the quality of education and reduce the burden on teachers.

[0239] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0240] Step 1:

[0241] The user enters student attribute information using a device. Through the input interface, they fill in information such as the student's name, grade level, special needs, and interests in the input form. The entered information is temporarily stored on the device.

[0242] Step 2:

[0243] The terminal encrypts the entered student information using AES encryption. The encrypted data is sent to the server using the secure HTTPS protocol. The input is student attribute information, and the output is encrypted data.

[0244] Step 3:

[0245] The server receives encrypted data sent from the terminal. The server decrypts the data and constructs a data structure based on the received student attribute information. The input is encrypted data, and the output is decrypted student information data.

[0246] Step 4:

[0247] The server inputs the decrypted student information into a generating AI model. The AI ​​model analyzes this information and generates educational content optimized for the specific student. For example, if visual learning materials are needed, colorful math practice problems will be created. The input is student information, and the output is the generated educational content.

[0248] Step 5:

[0249] The generated educational content is saved in PDF format in the server's database. The server organizes and manages this content so that users can access it later. The input is the educational content, and the output is the file stored in the database.

[0250] Step 6:

[0251] The user re-accesses the system using a terminal. The user selects the necessary student materials from the database and downloads or prints them. This provides personalized materials to the students. The input is the user's access request, and the output is the downloaded or printed educational content.

[0252] (Application Example 1)

[0253] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0254] Special education requires educational activities tailored to each student, but resources for providing individually optimized teaching materials are limited. Furthermore, it is currently difficult to provide products or experiences suitable for specific students in physical stores. To address these challenges, a system is needed that automatically generates and provides teaching materials and product information according to the characteristics of each student.

[0255] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0256] In this invention, the server includes an input means for inputting student learning-related information, a generation means for generating individually optimized educational content based on the learning-related information obtained from the input means, a storage means for storing the educational content generated by the generation means in a data storage unit, a recommendation means for generating product information suitable for students based on the stored educational content, and a display means for communicating the product information generated by the recommendation means via a display device. This enables the provision of educational materials in educational settings and the provision of optimal products in physical stores.

[0257] "Input means" refers to a device or interface for importing student learning-related information into the system.

[0258] "Generation means" refers to a device or program for creating individually optimized educational content or product information based on input student learning-related information.

[0259] "Storage means" refers to a device or function for recording and storing generated educational content in a data storage unit.

[0260] "Recommendation method" refers to a device or function for creating product information suitable for students based on saved educational content.

[0261] "Display means" refers to a device or interface for providing generated product information to a user through a display device.

[0262] This invention realizes a system that provides teaching materials and product information optimized for the educational environment of special needs schools. The main components are a server, terminals, and users.

[0263] First, the user uses a terminal to input student learning-related information. This terminal functions as an input method for collecting detailed information such as name, interests, and type and degree of disability through a dedicated interface.

[0264] The input information is transferred to the server via a secure communication protocol. The server automatically generates educational content tailored to each student's learning characteristics based on the input information. Here, a generation AI model is used to create individually optimized content. This is the generation method. The generated content is securely stored in a data storage unit. This storage unit organizes the content so that users can easily access it later.

[0265] Furthermore, the server incorporates a recommended system that generates personalized product information for students based on their stored educational content. This information is then provided to students through display devices used by employees. This allows for, for example, supporting the shopping experience in physical stores.

[0266] As a concrete example, if a student interested in science is identified based on information entered by the user, the server generates information on science-related products and activity recommendations. This information is then displayed on a display device such as smart glasses to support customer service in physical stores.

[0267] An example of a prompt for a generative AI model would be, "This student is interested in science. Please recommend appropriate products." In response to this prompt, the system dynamically generates and provides optimized information.

[0268] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0269] Step 1:

[0270] The user enters student learning-related information on a terminal. This information includes the student's name, interests, and the type and severity of their disability. This information is formatted into a database through the terminal's interface and prepared for transmission to the server.

[0271] Step 2:

[0272] The terminal encrypts the entered student information and sends it to the server. To ensure the security and privacy of the information, this process uses the SSL / TLS protocol to encrypt the data and securely transfer it to the server over the network.

[0273] Step 3:

[0274] The server decrypts the received encrypted data and obtains student learning-related information. Based on the obtained information, it sends prompt messages such as "This student is interested in science. Please recommend appropriate products." to the generating AI model, and generates educational content tailored to the student's learning characteristics.

[0275] Step 4:

[0276] The server stores the generated educational content in its data storage unit. In this process, the content is saved in a database format along with metadata to efficiently manage the information and facilitate later access.

[0277] Step 5:

[0278] The server generates product information suitable for students using recommendation methods based on stored educational content. Here, an AI model extracts and generates highly relevant product information from input information and educational content.

[0279] Step 6:

[0280] The server outputs the generated product information and sends it to the display device. Specifically, it formats the information in an appropriate format for devices such as smart glasses and tablets, and delivers it to the display device using a communication protocol.

[0281] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0282] The present invention combines an emotion engine with an automatic generation system for educational materials in special needs schools, aiming to provide a more individually optimized educational experience by considering not only the learning situation of students but also the emotional state of users. This system recognizes the emotions of users in real-time and generates and provides educational content based on that information. The implementation method will be specifically described below.

[0283] First, the user (teacher) logs in to the system via a dedicated terminal. Here, a camera and microphone for face recognition of the user are provided, and the emotion engine analyzes the user's voice tone and expression in real-time through these data input devices. As a result of the analysis, the current emotional state of the user (e.g., calm, confused, anxious, etc.) is quantified, and this emotion data is sent to the system.

[0284] The terminal combines the emotion data and the learning-related information of the students and sends it to the server using a security protocol. The server saves the received data in a database, and the emotion data is recorded linked to the information of each student.

[0285] Next, the server generates educational content using a generation AI model based on the accumulated data. The generation model here reflects the emotional state of the user, different from the generation based only on conventional student information in the past. Through this process, for example, when the user is feeling stressed, teaching materials that are easier to handle are proposed, or conversely, challenging content is incorporated when the user is relaxed.

[0286] The generated educational content is organized and saved in the database, and then the user accesses it using the terminal. The user can search for the saved content as needed and utilize it for classes and learning activities through downloading or printing. As a result, an appropriate educational program for each student can be provided, reducing the burden on teachers and enhancing the learning effect of students.

[0287] The following describes the processing flow.

[0288] Step 1:

[0289] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, and activities of interest. During input, a camera and microphone capture the user's facial expressions and voice.

[0290] Step 2:

[0291] The device transmits the acquired user voice and facial expression data to the emotion engine. The emotion engine analyzes this data and generates the user's emotional state as numerical or categorical data. For example, it determines whether the user is relaxed, focused, or stressed.

[0292] Step 3:

[0293] The device transmits the entered student information and generated user sentiment data to the server via a security protocol. This data is encrypted, ensuring it reaches the server safely.

[0294] Step 4:

[0295] The server stores the received data in a database. Sentimental data is managed in conjunction with student information and referenced later in the generation of educational content.

[0296] Step 5:

[0297] The server runs a generative AI model based on stored data. The AI ​​model combines student information and the user's emotional state to generate individually optimized educational content. For example, if a user is feeling stressed, the system will suggest experiential tasks to help them relax.

[0298] Step 6:

[0299] The generated educational content is stored in a database, where it is organized and indexed. This ensures that the content can be effectively searched and used later.

[0300] Step 7:

[0301] Users access content via their devices and review the generated educational materials. They can download or print materials as needed, enabling them to create lesson plans optimized for their students.

[0302] (Example 2)

[0303] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0304] To achieve individually optimized education, it is crucial to consider not only students' learning progress but also teachers' emotional states in real time. However, conventional systems have struggled to grasp emotional states and generate appropriate educational content based on them. This has led to increased burdens on teachers and the inability to provide appropriate educational support to students.

[0305] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0306] In this invention, the server includes an analysis means that analyzes the user's facial expressions and voice and quantifies their emotional state; a generation means that generates individually optimized educational content based on the data acquired from the input means and the analysis means; and a storage means that stores the generated educational content in a data storage device. This makes it possible to provide teaching materials that take into account the emotional state of teachers, thereby improving the quality of education and reducing the burden on teachers.

[0307] The "input means" is a device or method for acquiring students' learning-related information and users' emotional data.

[0308] The "analysis means" is a technology for analyzing users' expressions and voices in real time and quantifying their emotional states.

[0309] The "generation means" is a process or system for creating individually optimized educational content based on the acquired learning-related information and emotional data.

[0310] The "storage means" is a mechanism for storing the generated educational content in a storage device or the like so that it can be retrieved and used later.

[0311] The "provision means" is a method or technology for delivering the stored educational content so that it can be accessed by teachers.

[0312] This invention is an automatic generation system for educational materials in special needs education, which realizes an individually optimized educational experience by considering the emotional state of the user. Specifically, the system is configured as follows.

[0313] The user accesses and logs in to the system using a dedicated information terminal. The terminal is equipped with a camera and a microphone, and the user's facial expressions and voice are captured in real time. These data are analyzed using an emotion engine, and the user's emotional state is quantified. For example, when the expression is a smile and the tone of voice is gentle, the system determines that the user is "at ease".

[0314] The terminal unifies the captured emotional data and the learning-related information of the student and transmits it to the server using a secure communication protocol. This ensures the security and reliability of the data.

[0315] The server stores received emotional data and learning information in a database. Here, the emotional data is linked to each student's learning history and recorded as comprehensive information. Subsequently, the server uses a generative AI model to generate educational content based on the received data. This model enables the customization of educational content according to the user's emotional state, achieving individual optimization.

[0316] As a concrete example, by instructing the generative AI model with the prompt "Provide content appropriate for when students are feeling at ease," appropriate teaching materials will be provided. This prompt serves as an indicator of what kind of content should be emphasized in the content presented by the model.

[0317] Users can access, download, and print the generated educational content through their devices for use in their learning activities. This enables the provision of educational programs optimized for individual students, thereby improving the quality of education.

[0318] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0319] Step 1:

[0320] The user logs into the system using an information terminal. Upon logging in, the camera and microphone built into the terminal activate, capturing the user's face and voice in real time. The input at this time consists of the user's facial image data and voice data, which are acquired to provide basic data for sentiment analysis.

[0321] Step 2:

[0322] The device transmits the acquired facial image data and audio data to the emotion engine. This engine uses machine learning algorithms to analyze the data and quantify the user's emotional state. This analysis process analyzes changes in facial expressions and tone of voice to generate emotion tags such as reassurance, confusion, and anxiety. The output is numerical data indicating the user's emotional state.

[0323] Step 3:

[0324] The device packages emotional data and student learning-related information together, encrypts it, and sends it to the server. The input consists of emotional data and learning-related information, and by securely sending these to the server using a security protocol, the output prevents information leakage.

[0325] Step 4:

[0326] The server stores the received sentiment data and learning information in a database. Simultaneously, it links the sentiment data to the corresponding student profile, combining sentiment with learning history. In this process, the input is the received data, and the output is the updated database entry.

[0327] Step 5:

[0328] The server runs a generative AI model using stored data to generate educational content. Using the prompt "Provide content appropriate for when students are feeling secure," it generates materials for students who are in a secure state. The input is stored emotion data and learning information, and the output is individually optimized educational content.

[0329] Step 6:

[0330] Users can access the generated educational content through their devices and search, download, or print the necessary learning materials. In this case, the input is the user's actions, and the output is physical or digital educational materials. This enables effective learning support in educational settings.

[0331] (Application Example 2)

[0332] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0333] In face-to-face sales, it is difficult for salespeople to accurately grasp a customer's emotional state, which contributes to inappropriate service and low customer satisfaction. Furthermore, it is not easy for salespeople with little customer service experience to immediately decide on the appropriate response. Therefore, there is a need for a system that can acquire customer emotional information in real time and provide optimal customer service advice based on that information.

[0334] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0335] In this invention, the server includes an input means for inputting customer facial expressions and voice data, a generation means for generating individually optimized customer service advice based on the customer's emotional state, and a provision means for sales staff to access and provide the generated customer service advice through a display device. This allows sales staff to grasp the customer's emotional state in real time and respond based on appropriate advice.

[0336] An "input device" is a device used to electronically acquire facial expressions and voice data collected from customers.

[0337] The "generation method" refers to a mechanism that creates individually optimized customer service advice based on acquired data.

[0338] "Storage means" refers to a device that stores the generated customer service advice in a landscape information recording device.

[0339] "Means of provision" refers to a system that makes stored customer service advice accessible to sales staff and presents it via a display device.

[0340] A "generative intelligence model" is an artificial intelligence algorithm used to derive appropriate advice from acquired data.

[0341] A "display device" is a device that allows sales staff to visually confirm the customer service advice that has been generated.

[0342] This embodiment of the invention relates to a sales staff support system for physical stores. This system enables sales staff to understand the customer's emotional state in real time and provide appropriate customer service. The specific details of the system are described below.

[0343] The server first collects customer facial expressions and voice data through input means. Wearable devices such as smart glasses are used as hardware. These devices are equipped with cameras and microphones to acquire data for analyzing the customer's facial expressions and voice tone.

[0344] Next, a lightweight machine learning model is used as a generation tool to quantify the customer's emotional state (e.g., reassurance, confusion, etc.). The analyzed data is sent to a generative intelligence model (e.g., OpenAI's generative AI model), where personalized customer service advice based on the customer's emotional state is generated.

[0345] The generated customer service advice is stored as digital data in a landscape information recording device via a storage means. Salespeople can access this advice via a display device (e.g., smart glasses display) using a delivery means. This allows salespeople to implement the optimal customer service strategy at the appropriate time.

[0346] As a concrete example, while a salesperson is assisting a customer, the smart glasses display might show advice such as, "The customer is confused about which product to choose, so please provide clear explanations." Another example of a prompt for a generative intelligence model might be, "Please create a product suggestion that will interest the customer." In this way, the system aims to streamline the sales process and improve customer satisfaction.

[0347] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0348] Step 1:

[0349] The device acquires customer facial and audio data via the smart glasses' camera and microphone. The inputs are camera video and audio, and these data are temporarily stored on the device as output. Specifically, the camera captures the customer's face, and the microphone records their voice.

[0350] Step 2:

[0351] The device uses the acquired facial expression and voice data to perform an initial analysis with a lightweight machine learning model. The input is the data saved in step 1, and the output is data that quantifies the customer's emotional state. This process involves facial feature point extraction and voice tone analysis to identify the customer's emotional state (e.g., reassured, confused).

[0352] Step 3:

[0353] The terminal sends digitized emotion data to the server. The input is digitized emotion data, and the output is the completion of data transmission to the server. Here, data transfer takes place over the network, and the emotion data is registered in the server's database.

[0354] Step 4:

[0355] The server generates customer service advice using a generative AI model based on the received sentiment data. The input is sentiment data stored on the server, and the output is customer service advice based on a specific prompt. For example, the prompt might be "Suggest products that the customer is interested in."

[0356] Step 5:

[0357] The server saves the generated customer service advice to the scenery information recording device. The input is the generated customer service advice, and the output is the advice registered in the scenery information recording device. This operation ensures that the advice is saved in an easily searchable format.

[0358] Step 6:

[0359] The user views and confirms customer service advice stored in a landscape information recording device through smart glasses. The input is the stored advice information, and the output is the text of the advice displayed on the smart glasses' screen. This allows the user to take the most appropriate action in real time when interacting with customers.

[0360] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0361] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0362] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0363] [Third Embodiment]

[0364] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0365] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0366] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0367] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0368] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0369] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0370] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0371] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0372] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0373] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0374] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0375] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0376] This invention relates to an automated system for generating educational materials in special needs schools, and provides technology for individually optimizing student learning. This system is realized through the interaction of several entities.

[0377] First, the user (teacher) accesses the system using a dedicated terminal. The user enters detailed learning-related information about each student, including the student's name, type and severity of disability, and areas of expertise and interests, using an interface that allows for accurate understanding of each student's individual characteristics.

[0378] Next, the terminal securely transmits this input information to the server. The data is encrypted and delivered accurately to the server while preventing unauthorized access through an authentication process. This transmitted information is securely stored in a database on the server and organized so that teachers can refer to it later.

[0379] Information arriving on the server is processed using a generative AI model. This AI model, similar to a human teacher, considers various educational elements based on students' learning needs and quickly generates individually optimized educational content. Specifically, the generated content includes practice problems to improve basic academic skills and activities that accommodate disabilities. Furthermore, the design and layout of the teaching materials are automatically adjusted to enhance student learning.

[0380] The generated content is stored in the server's database, awaiting later access. When a user logs back into the system via their terminal, they gain access to this stored educational content, allowing them to easily download or print materials tailored to the individual needs of their students.

[0381] In this way, the system of the present invention aims to improve the quality of education by enabling teachers to efficiently and effectively provide individually optimized learning materials to diverse students in special needs schools.

[0382] The following describes the processing flow.

[0383] Step 1:

[0384] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, hobbies, and interests. Users can easily and quickly enter information using an intuitive interface.

[0385] Step 2:

[0386] The terminal encrypts the entered information and sends it to the server using a secure data communication protocol. During transmission, an authentication process is performed, and measures are taken to protect the data from unauthorized access.

[0387] Step 3:

[0388] The server analyzes the received data and stores it appropriately in the database. During storage, a verification process is performed to ensure data integrity, and the data is organized in preparation for subsequent processing.

[0389] Step 4:

[0390] The server runs a generation AI model based on the stored information. The AI ​​model generates individually optimized educational content based on the students' attribute information. At this stage, the AI ​​combines various educational elements to determine the appropriate teaching materials and activities for each student.

[0391] Step 5:

[0392] The server stores the generated educational content in a database and categorizes it as needed. This process allows users to access the content in a format that is easy to use later.

[0393] Step 6:

[0394] Users access the system again from their terminals, search for and select educational content for the desired students, and download or print it. This allows teachers to efficiently utilize prepared materials in the classroom and provide instruction tailored to the individual characteristics of each student.

[0395] (Example 1)

[0396] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0397] Providing educational materials tailored to the individual characteristics of students in special needs schools is a significant burden for teachers using traditional methods. Furthermore, creating these materials is time-consuming and labor-intensive, making it difficult to quickly generate materials suitable for each student's needs. Security and accessibility of information also pose challenges.

[0398] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0399] In this invention, the server includes a registration means for inputting student attribute information, a creation means for generating individually optimized educational materials based on the student attribute information obtained from the registration means, and a storage means for storing the educational materials generated by the creation means in a storage medium. This enables the rapid generation of personalized educational materials and safe and easy access to them.

[0400] "Registration method" refers to an interface for inputting student attribute information and recording it in the system.

[0401] "Creative means" refers to a device or process for automatically generating educational materials optimized for individual learners based on inputted student attribute information.

[0402] "Storage means" refers to a device or function that securely stores generated educational materials on a storage medium and manages them so that they can be accessed later.

[0403] "Communication methods" refer to protocols and devices that use encryption technology to send and receive data and maintain the confidentiality and integrity of information.

[0404] "Means of acquisition" refers to an interface that allows educators to easily access and acquire educational materials stored on a storage medium through an electronic device.

[0405] This invention relates to a system for automatically generating individualized educational materials in special needs education. The system is configured to efficiently create customized educational content based on the characteristics of specific students.

[0406] The user (teacher) first accesses the system using a dedicated terminal. This terminal is equipped with an input interface that allows the user to input attribute information for each student (e.g., grade level, favorite subjects, activities of interest, special support needs, etc.). For example, if student A expresses interest in "mathematics teaching materials that make extensive use of visual elements," the user would input that information.

[0407] The terminal securely transmits the entered information to the server using encryption technology. High-security encryption protocols such as AES and TLS are used.

[0408] The server stores the received information in a database and uses a generative AI model to generate educational materials based on that information. The generative AI model can quickly design optimal educational content using the information about the students that has been input. For example, for student A, who needs visual learning materials, the generative AI model can be used to provide math practice problems using colorful shapes.

[0409] The generated teaching materials are saved in PDF format in the server's database and can be accessed by the user later. The user can then log back into the system and download or print the educational materials via their terminal.

[0410] A concrete example of a prompt might be, "Provide an integer addition lesson with enhanced visual support." This allows teachers to quickly and easily provide lessons tailored to students with special needs. The aim is to improve the quality of education and reduce the burden on teachers.

[0411] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0412] Step 1:

[0413] The user enters student attribute information using a device. Through the input interface, they fill in information such as the student's name, grade level, special needs, and interests in the input form. The entered information is temporarily stored on the device.

[0414] Step 2:

[0415] The terminal encrypts the entered student information using AES encryption. The encrypted data is sent to the server using the secure HTTPS protocol. The input is student attribute information, and the output is encrypted data.

[0416] Step 3:

[0417] The server receives encrypted data sent from the terminal. The server decrypts the data and constructs a data structure based on the received student attribute information. The input is encrypted data, and the output is decrypted student information data.

[0418] Step 4:

[0419] The server inputs the decrypted student information into a generating AI model. The AI ​​model analyzes this information and generates educational content optimized for the specific student. For example, if visual learning materials are needed, colorful math practice problems will be created. The input is student information, and the output is the generated educational content.

[0420] Step 5:

[0421] The generated educational content is saved in PDF format in the server's database. The server organizes and manages this content so that users can access it later. The input is the educational content, and the output is the file stored in the database.

[0422] Step 6:

[0423] The user re-accesses the system using a terminal. The user selects the necessary student materials from the database and downloads or prints them. This provides personalized materials to the students. The input is the user's access request, and the output is the downloaded or printed educational content.

[0424] (Application Example 1)

[0425] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0426] Special education requires educational activities tailored to each student, but resources for providing individually optimized teaching materials are limited. Furthermore, it is currently difficult to provide products or experiences suitable for specific students in physical stores. To address these challenges, a system is needed that automatically generates and provides teaching materials and product information according to the characteristics of each student.

[0427] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0428] In this invention, the server includes an input means for inputting student learning-related information, a generation means for generating individually optimized educational content based on the learning-related information obtained from the input means, a storage means for storing the educational content generated by the generation means in a data storage unit, a recommendation means for generating product information suitable for students based on the stored educational content, and a display means for communicating the product information generated by the recommendation means via a display device. This enables the provision of educational materials in educational settings and the provision of optimal products in physical stores.

[0429] "Input means" refers to a device or interface for importing student learning-related information into the system.

[0430] "Generation means" refers to a device or program for creating individually optimized educational content or product information based on input student learning-related information.

[0431] "Storage means" refers to a device or function for recording and storing generated educational content in a data storage unit.

[0432] "Recommendation method" refers to a device or function for creating product information suitable for students based on saved educational content.

[0433] "Display means" refers to a device or interface for providing generated product information to a user through a display device.

[0434] This invention realizes a system that provides teaching materials and product information optimized for the educational environment of special needs schools. The main components are a server, terminals, and users.

[0435] First, the user uses a terminal to input student learning-related information. This terminal functions as an input method for collecting detailed information such as name, interests, and type and degree of disability through a dedicated interface.

[0436] The input information is transferred to the server via a secure communication protocol. The server automatically generates educational content tailored to each student's learning characteristics based on the input information. Here, a generation AI model is used to create individually optimized content. This is the generation method. The generated content is securely stored in a data storage unit. This storage unit organizes the content so that users can easily access it later.

[0437] Furthermore, the server incorporates a recommended system that generates personalized product information for students based on their stored educational content. This information is then provided to students through display devices used by employees. This allows for, for example, supporting the shopping experience in physical stores.

[0438] As a concrete example, if a student interested in science is identified based on information entered by the user, the server generates information on science-related products and activity recommendations. This information is then displayed on a display device such as smart glasses to support customer service in physical stores.

[0439] An example of a prompt for a generative AI model would be, "This student is interested in science. Please recommend appropriate products." In response to this prompt, the system dynamically generates and provides optimized information.

[0440] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0441] Step 1:

[0442] The user enters student learning-related information on a terminal. This information includes the student's name, interests, and the type and severity of their disability. This information is formatted into a database through the terminal's interface and prepared for transmission to the server.

[0443] Step 2:

[0444] The terminal encrypts the entered student information and sends it to the server. To ensure the security and privacy of the information, this process uses the SSL / TLS protocol to encrypt the data and securely transfer it to the server over the network.

[0445] Step 3:

[0446] The server decrypts the received encrypted data and obtains student learning-related information. Based on the obtained information, it sends prompt messages such as "This student is interested in science. Please recommend appropriate products." to the generating AI model, and generates educational content tailored to the student's learning characteristics.

[0447] Step 4:

[0448] The server stores the generated educational content in its data storage unit. In this process, the content is saved in a database format along with metadata to efficiently manage the information and facilitate later access.

[0449] Step 5:

[0450] The server generates product information suitable for students using recommendation methods based on stored educational content. Here, an AI model extracts and generates highly relevant product information from input information and educational content.

[0451] Step 6:

[0452] The server outputs the generated product information and sends it to the display device. Specifically, it formats the information in an appropriate format for devices such as smart glasses and tablets, and delivers it to the display device using a communication protocol.

[0453] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0454] This invention combines an emotion engine with an automated system for generating educational materials in special needs schools. Its aim is to provide a more individually optimized educational experience by considering not only students' learning progress but also the user's emotional state. This system recognizes the user's emotions in real time and generates and provides educational content based on that information. The implementation method will be described in detail below.

[0455] First, the user (teacher) logs into the system via a dedicated terminal. This terminal is equipped with a camera and microphone for facial recognition, and the emotion engine analyzes the user's voice tone and facial expressions in real time through these data input devices. As a result of the analysis, the user's current emotional state (e.g., reassurance, confusion, anxiety, etc.) is quantified, and this emotional data is sent to the system.

[0456] The device aggregates emotional data and student learning-related information and sends it to the server using a security protocol. The server stores the received data in a database, and the emotional data is recorded and linked to each student's information.

[0457] Next, the server generates educational content using a generative AI model based on the accumulated data. Unlike traditional generation methods that rely solely on student information, this generative model reflects the user's emotional state. This process allows for adjustments such as suggesting easier-to-handle materials if the user is stressed, or incorporating more challenging content if they are relaxed.

[0458] The generated educational content is organized and stored in a database, after which users can access it using their devices. Users can search for the stored content as needed and utilize it in classes and learning activities through downloading and printing. This allows for the provision of appropriate educational programs for each student, reducing the burden on teachers while improving student learning effectiveness.

[0459] The following describes the processing flow.

[0460] Step 1:

[0461] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, and activities of interest. During input, a camera and microphone capture the user's facial expressions and voice.

[0462] Step 2:

[0463] The device transmits the acquired user voice and facial expression data to the emotion engine. The emotion engine analyzes this data and generates the user's emotional state as numerical or categorical data. For example, it determines whether the user is relaxed, focused, or stressed.

[0464] Step 3:

[0465] The device transmits the entered student information and generated user sentiment data to the server via a security protocol. This data is encrypted, ensuring it reaches the server safely.

[0466] Step 4:

[0467] The server stores the received data in a database. Sentimental data is managed in conjunction with student information and referenced later in the generation of educational content.

[0468] Step 5:

[0469] The server runs a generative AI model based on stored data. The AI ​​model combines student information and the user's emotional state to generate individually optimized educational content. For example, if a user is feeling stressed, the system will suggest experiential tasks to help them relax.

[0470] Step 6:

[0471] The generated educational content is stored in a database, where it is organized and indexed. This ensures that the content can be effectively searched and used later.

[0472] Step 7:

[0473] Users access content via their devices and review the generated educational materials. They can download or print materials as needed, enabling them to create lesson plans optimized for their students.

[0474] (Example 2)

[0475] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0476] To achieve individually optimized education, it is crucial to consider not only students' learning progress but also teachers' emotional states in real time. However, conventional systems have struggled to grasp emotional states and generate appropriate educational content based on them. This has led to increased burdens on teachers and the inability to provide appropriate educational support to students.

[0477] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0478] In this invention, the server includes an analysis means that analyzes the user's facial expressions and voice and quantifies their emotional state; a generation means that generates individually optimized educational content based on the data acquired from the input means and the analysis means; and a storage means that stores the generated educational content in a data storage device. This makes it possible to provide teaching materials that take into account the emotional state of teachers, thereby improving the quality of education and reducing the burden on teachers.

[0479] "Input means" refers to devices or methods for acquiring student learning-related information and user sentiment data.

[0480] The "analysis method" is a technology that analyzes the user's facial expressions and voice in real time and quantifies their emotional state.

[0481] "Generation means" refers to a process or system for creating individually optimized educational content based on acquired learning-related information and emotional data.

[0482] A "storage method" refers to a system that saves generated educational content to a storage device or similar, allowing it to be retrieved and used later.

[0483] "Means of delivery" refers to the methods and technologies used to deliver stored educational content so that teachers can access it.

[0484] This invention is an automated system for generating educational materials in special needs education, which realizes an individually optimized educational experience by taking into account the user's emotional state. Specifically, the system is configured as follows.

[0485] Users access and log into the system using a dedicated information terminal. The terminal is equipped with a camera and microphone, which capture the user's facial expressions and voice in real time. This data is analyzed using an emotion engine, and the user's emotional state is quantified. For example, if a user has a smiling face and a calm voice tone, the system will determine that the user is "at ease."

[0486] The device centralizes captured emotional data and student learning-related information and sends it to the server using a secure communication protocol. This ensures the safety and reliability of the data.

[0487] The server stores received emotional data and learning information in a database. Here, the emotional data is linked to each student's learning history and recorded as comprehensive information. Subsequently, the server uses a generative AI model to generate educational content based on the received data. This model enables the customization of educational content according to the user's emotional state, achieving individual optimization.

[0488] As a concrete example, by instructing the generative AI model with the prompt "Provide content appropriate for when students are feeling at ease," appropriate teaching materials will be provided. This prompt serves as an indicator of what kind of content should be emphasized in the content presented by the model.

[0489] Users can access, download, and print the generated educational content through their devices for use in their learning activities. This enables the provision of educational programs optimized for individual students, thereby improving the quality of education.

[0490] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0491] Step 1:

[0492] The user logs into the system using an information terminal. Upon logging in, the camera and microphone built into the terminal activate, capturing the user's face and voice in real time. The input at this time consists of the user's facial image data and voice data, which are acquired to provide basic data for sentiment analysis.

[0493] Step 2:

[0494] The device transmits the acquired facial image data and audio data to the emotion engine. This engine uses machine learning algorithms to analyze the data and quantify the user's emotional state. This analysis process analyzes changes in facial expressions and tone of voice to generate emotion tags such as reassurance, confusion, and anxiety. The output is numerical data indicating the user's emotional state.

[0495] Step 3:

[0496] The device packages emotional data and student learning-related information together, encrypts it, and sends it to the server. The input consists of emotional data and learning-related information, and by securely sending these to the server using a security protocol, the output prevents information leakage.

[0497] Step 4:

[0498] The server stores the received sentiment data and learning information in a database. Simultaneously, it links the sentiment data to the corresponding student profile, combining sentiment with learning history. In this process, the input is the received data, and the output is the updated database entry.

[0499] Step 5:

[0500] The server runs a generative AI model using stored data to generate educational content. Using the prompt "Provide content appropriate for when students are feeling secure," it generates materials for students who are in a secure state. The input is stored emotion data and learning information, and the output is individually optimized educational content.

[0501] Step 6:

[0502] Users can access the generated educational content through their devices and search, download, or print the necessary learning materials. In this case, the input is the user's actions, and the output is physical or digital educational materials. This enables effective learning support in educational settings.

[0503] (Application Example 2)

[0504] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0505] In face-to-face sales, it is difficult for salespeople to accurately grasp a customer's emotional state, which contributes to inappropriate service and low customer satisfaction. Furthermore, it is not easy for salespeople with little customer service experience to immediately decide on the appropriate response. Therefore, there is a need for a system that can acquire customer emotional information in real time and provide optimal customer service advice based on that information.

[0506] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0507] In this invention, the server includes an input means for inputting customer facial expressions and voice data, a generation means for generating individually optimized customer service advice based on the customer's emotional state, and a provision means for sales staff to access and provide the generated customer service advice through a display device. This allows sales staff to grasp the customer's emotional state in real time and respond based on appropriate advice.

[0508] An "input device" is a device used to electronically acquire facial expressions and voice data collected from customers.

[0509] The "generation method" refers to a mechanism that creates individually optimized customer service advice based on acquired data.

[0510] "Storage means" refers to a device that stores the generated customer service advice in a landscape information recording device.

[0511] "Means of provision" refers to a system that makes stored customer service advice accessible to sales staff and presents it via a display device.

[0512] A "generative intelligence model" is an artificial intelligence algorithm used to derive appropriate advice from acquired data.

[0513] A "display device" is a device that allows sales staff to visually confirm the customer service advice that has been generated.

[0514] This embodiment of the invention relates to a sales staff support system for physical stores. This system enables sales staff to understand the customer's emotional state in real time and provide appropriate customer service. The specific details of the system are described below.

[0515] The server first collects customer facial expressions and voice data through input means. Wearable devices such as smart glasses are used as hardware. These devices are equipped with cameras and microphones to acquire data for analyzing the customer's facial expressions and voice tone.

[0516] Next, a lightweight machine learning model is used as a generation tool to quantify the customer's emotional state (e.g., reassurance, confusion, etc.). The analyzed data is sent to a generative intelligence model (e.g., OpenAI's generative AI model), where personalized customer service advice based on the customer's emotional state is generated.

[0517] The generated customer service advice is stored as digital data in a landscape information recording device via a storage means. Salespeople can access this advice via a display device (e.g., smart glasses display) using a delivery means. This allows salespeople to implement the optimal customer service strategy at the appropriate time.

[0518] As a concrete example, while a salesperson is assisting a customer, the smart glasses display might show advice such as, "The customer is confused about which product to choose, so please provide clear explanations." Another example of a prompt for a generative intelligence model might be, "Please create a product suggestion that will interest the customer." In this way, the system aims to streamline the sales process and improve customer satisfaction.

[0519] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0520] Step 1:

[0521] The device acquires customer facial and audio data via the smart glasses' camera and microphone. The inputs are camera video and audio, and these data are temporarily stored on the device as output. Specifically, the camera captures the customer's face, and the microphone records their voice.

[0522] Step 2:

[0523] The device uses the acquired facial expression and voice data to perform an initial analysis with a lightweight machine learning model. The input is the data saved in step 1, and the output is data that quantifies the customer's emotional state. This process involves facial feature point extraction and voice tone analysis to identify the customer's emotional state (e.g., reassured, confused).

[0524] Step 3:

[0525] The terminal sends digitized emotion data to the server. The input is digitized emotion data, and the output is the completion of data transmission to the server. Here, data transfer takes place over the network, and the emotion data is registered in the server's database.

[0526] Step 4:

[0527] The server generates customer service advice using a generative AI model based on the received sentiment data. The input is sentiment data stored on the server, and the output is customer service advice based on a specific prompt. For example, the prompt might be "Suggest products that the customer is interested in."

[0528] Step 5:

[0529] The server saves the generated customer service advice to the scenery information recording device. The input is the generated customer service advice, and the output is the advice registered in the scenery information recording device. This operation ensures that the advice is saved in an easily searchable format.

[0530] Step 6:

[0531] The user views and confirms customer service advice stored in a landscape information recording device through smart glasses. The input is the stored advice information, and the output is the text of the advice displayed on the smart glasses' screen. This allows the user to take the most appropriate action in real time when interacting with customers.

[0532] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0533] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0534] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0535] [Fourth Embodiment]

[0536] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0537] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0538] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0539] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0540] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0541] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0542] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0543] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0544] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0545] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0546] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0547] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0548] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0549] This invention relates to an automated system for generating educational materials in special needs schools, and provides technology for individually optimizing student learning. This system is realized through the interaction of several entities.

[0550] First, the user (teacher) accesses the system using a dedicated terminal. The user enters detailed learning-related information about each student, including the student's name, type and severity of disability, and areas of expertise and interests, using an interface that allows for accurate understanding of each student's individual characteristics.

[0551] Next, the terminal securely transmits this input information to the server. The data is encrypted and delivered accurately to the server while preventing unauthorized access through an authentication process. This transmitted information is securely stored in a database on the server and organized so that teachers can refer to it later.

[0552] Information arriving on the server is processed using a generative AI model. This AI model, similar to a human teacher, considers various educational elements based on students' learning needs and quickly generates individually optimized educational content. Specifically, the generated content includes practice problems to improve basic academic skills and activities that accommodate disabilities. Furthermore, the design and layout of the teaching materials are automatically adjusted to enhance student learning.

[0553] The generated content is stored in the server's database, awaiting later access. When a user logs back into the system via their terminal, they gain access to this stored educational content, allowing them to easily download or print materials tailored to the individual needs of their students.

[0554] In this way, the system of the present invention aims to improve the quality of education by enabling teachers to efficiently and effectively provide individually optimized learning materials to diverse students in special needs schools.

[0555] The following describes the processing flow.

[0556] Step 1:

[0557] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, hobbies, and interests. Users can easily and quickly enter information using an intuitive interface.

[0558] Step 2:

[0559] The terminal encrypts the entered information and sends it to the server using a secure data communication protocol. During transmission, an authentication process is performed, and measures are taken to protect the data from unauthorized access.

[0560] Step 3:

[0561] The server analyzes the received data and stores it appropriately in the database. During storage, a verification process is performed to ensure data integrity, and the data is organized in preparation for subsequent processing.

[0562] Step 4:

[0563] The server runs a generation AI model based on the stored information. The AI ​​model generates individually optimized educational content based on the students' attribute information. At this stage, the AI ​​combines various educational elements to determine the appropriate teaching materials and activities for each student.

[0564] Step 5:

[0565] The server stores the generated educational content in a database and categorizes it as needed. This process allows users to access the content in a format that is easy to use later.

[0566] Step 6:

[0567] Users access the system again from their terminals, search for and select educational content for the desired students, and download or print it. This allows teachers to efficiently utilize prepared materials in the classroom and provide instruction tailored to the individual characteristics of each student.

[0568] (Example 1)

[0569] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0570] Providing educational materials tailored to the individual characteristics of students in special needs schools is a significant burden for teachers using traditional methods. Furthermore, creating these materials is time-consuming and labor-intensive, making it difficult to quickly generate materials suitable for each student's needs. Security and accessibility of information also pose challenges.

[0571] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0572] In this invention, the server includes a registration means for inputting student attribute information, a creation means for generating individually optimized educational materials based on the student attribute information obtained from the registration means, and a storage means for storing the educational materials generated by the creation means in a storage medium. This enables the rapid generation of personalized educational materials and safe and easy access to them.

[0573] "Registration method" refers to an interface for inputting student attribute information and recording it in the system.

[0574] "Creative means" refers to a device or process for automatically generating educational materials optimized for individual learners based on inputted student attribute information.

[0575] "Storage means" refers to a device or function that securely stores generated educational materials on a storage medium and manages them so that they can be accessed later.

[0576] "Communication methods" refer to protocols and devices that use encryption technology to send and receive data and maintain the confidentiality and integrity of information.

[0577] "Means of acquisition" refers to an interface that allows educators to easily access and acquire educational materials stored on a storage medium through an electronic device.

[0578] This invention relates to a system for automatically generating individualized educational materials in special needs education. The system is configured to efficiently create customized educational content based on the characteristics of specific students.

[0579] The user (teacher) first accesses the system using a dedicated terminal. This terminal is equipped with an input interface that allows the user to input attribute information for each student (e.g., grade level, favorite subjects, activities of interest, special support needs, etc.). For example, if student A expresses interest in "mathematics teaching materials that make extensive use of visual elements," the user would input that information.

[0580] The terminal securely transmits the entered information to the server using encryption technology. High-security encryption protocols such as AES and TLS are used.

[0581] The server stores the received information in a database and uses a generative AI model to generate educational materials based on that information. The generative AI model can quickly design optimal educational content using the information about the students that has been input. For example, for student A, who needs visual learning materials, the generative AI model can be used to provide math practice problems using colorful shapes.

[0582] The generated teaching materials are saved in PDF format in the server's database and can be accessed by the user later. The user can then log back into the system and download or print the educational materials via their terminal.

[0583] A concrete example of a prompt might be, "Provide an integer addition lesson with enhanced visual support." This allows teachers to quickly and easily provide lessons tailored to students with special needs. The aim is to improve the quality of education and reduce the burden on teachers.

[0584] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0585] Step 1:

[0586] The user enters student attribute information using a device. Through the input interface, they fill in information such as the student's name, grade level, special needs, and interests in the input form. The entered information is temporarily stored on the device.

[0587] Step 2:

[0588] The terminal encrypts the entered student information using AES encryption. The encrypted data is sent to the server using the secure HTTPS protocol. The input is student attribute information, and the output is encrypted data.

[0589] Step 3:

[0590] The server receives encrypted data sent from the terminal. The server decrypts the data and constructs a data structure based on the received student attribute information. The input is encrypted data, and the output is decrypted student information data.

[0591] Step 4:

[0592] The server inputs the decrypted student information into a generating AI model. The AI ​​model analyzes this information and generates educational content optimized for the specific student. For example, if visual learning materials are needed, colorful math practice problems will be created. The input is student information, and the output is the generated educational content.

[0593] Step 5:

[0594] The generated educational content is saved in PDF format in the server's database. The server organizes and manages this content so that users can access it later. The input is the educational content, and the output is the file stored in the database.

[0595] Step 6:

[0596] The user re-accesses the system using a terminal. The user selects the necessary student materials from the database and downloads or prints them. This provides personalized materials to the students. The input is the user's access request, and the output is the downloaded or printed educational content.

[0597] (Application Example 1)

[0598] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0599] Special education requires educational activities tailored to each student, but resources for providing individually optimized teaching materials are limited. Furthermore, it is currently difficult to provide products or experiences suitable for specific students in physical stores. To address these challenges, a system is needed that automatically generates and provides teaching materials and product information according to the characteristics of each student.

[0600] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0601] In this invention, the server includes an input means for inputting student learning-related information, a generation means for generating individually optimized educational content based on the learning-related information obtained from the input means, a storage means for storing the educational content generated by the generation means in a data storage unit, a recommendation means for generating product information suitable for students based on the stored educational content, and a display means for communicating the product information generated by the recommendation means via a display device. This enables the provision of educational materials in educational settings and the provision of optimal products in physical stores.

[0602] "Input means" refers to a device or interface for importing student learning-related information into the system.

[0603] "Generation means" refers to a device or program for creating individually optimized educational content or product information based on input student learning-related information.

[0604] "Storage means" refers to a device or function for recording and storing generated educational content in a data storage unit.

[0605] "Recommendation method" refers to a device or function for creating product information suitable for students based on saved educational content.

[0606] "Display means" refers to a device or interface for providing generated product information to a user through a display device.

[0607] This invention realizes a system that provides teaching materials and product information optimized for the educational environment of special needs schools. The main components are a server, terminals, and users.

[0608] First, the user uses a terminal to input student learning-related information. This terminal functions as an input method for collecting detailed information such as name, interests, and type and degree of disability through a dedicated interface.

[0609] The input information is transferred to the server via a secure communication protocol. The server automatically generates educational content tailored to each student's learning characteristics based on the input information. Here, a generation AI model is used to create individually optimized content. This is the generation method. The generated content is securely stored in a data storage unit. This storage unit organizes the content so that users can easily access it later.

[0610] Furthermore, the server incorporates a recommended system that generates personalized product information for students based on their stored educational content. This information is then provided to students through display devices used by employees. This allows for, for example, supporting the shopping experience in physical stores.

[0611] As a concrete example, if a student interested in science is identified based on information entered by the user, the server generates information on science-related products and activity recommendations. This information is then displayed on a display device such as smart glasses to support customer service in physical stores.

[0612] An example of a prompt for a generative AI model would be, "This student is interested in science. Please recommend appropriate products." In response to this prompt, the system dynamically generates and provides optimized information.

[0613] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0614] Step 1:

[0615] The user enters student learning-related information on a terminal. This information includes the student's name, interests, and the type and severity of their disability. This information is formatted into a database through the terminal's interface and prepared for transmission to the server.

[0616] Step 2:

[0617] The terminal encrypts the entered student information and sends it to the server. To ensure the security and privacy of the information, this process uses the SSL / TLS protocol to encrypt the data and securely transfer it to the server over the network.

[0618] Step 3:

[0619] The server decrypts the received encrypted data and obtains student learning-related information. Based on the obtained information, it sends prompt messages such as "This student is interested in science. Please recommend appropriate products." to the generating AI model, and generates educational content tailored to the student's learning characteristics.

[0620] Step 4:

[0621] The server stores the generated educational content in its data storage unit. In this process, the content is saved in a database format along with metadata to efficiently manage the information and facilitate later access.

[0622] Step 5:

[0623] The server generates product information suitable for students using recommendation methods based on stored educational content. Here, an AI model extracts and generates highly relevant product information from input information and educational content.

[0624] Step 6:

[0625] The server outputs the generated product information and sends it to the display device. Specifically, it formats the information in an appropriate format for devices such as smart glasses and tablets, and delivers it to the display device using a communication protocol.

[0626] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0627] This invention combines an emotion engine with an automated system for generating educational materials in special needs schools. Its aim is to provide a more individually optimized educational experience by considering not only students' learning progress but also the user's emotional state. This system recognizes the user's emotions in real time and generates and provides educational content based on that information. The implementation method will be described in detail below.

[0628] First, the user (teacher) logs into the system via a dedicated terminal. This terminal is equipped with a camera and microphone for facial recognition, and the emotion engine analyzes the user's voice tone and facial expressions in real time through these data input devices. As a result of the analysis, the user's current emotional state (e.g., reassurance, confusion, anxiety, etc.) is quantified, and this emotional data is sent to the system.

[0629] The device aggregates emotional data and student learning-related information and sends it to the server using a security protocol. The server stores the received data in a database, and the emotional data is recorded and linked to each student's information.

[0630] Next, the server generates educational content using a generative AI model based on the accumulated data. Unlike traditional generation methods that rely solely on student information, this generative model reflects the user's emotional state. This process allows for adjustments such as suggesting easier-to-handle materials if the user is stressed, or incorporating more challenging content if they are relaxed.

[0631] The generated educational content is organized and stored in a database, after which users can access it using their devices. Users can search for the stored content as needed and utilize it in classes and learning activities through downloading and printing. This allows for the provision of appropriate educational programs for each student, reducing the burden on teachers while improving student learning effectiveness.

[0632] The following describes the processing flow.

[0633] Step 1:

[0634] Users log in to a dedicated terminal and enter student learning-related information. This information includes the student's name, type of disability, severity of disability, areas of expertise, and activities of interest. During input, a camera and microphone capture the user's facial expressions and voice.

[0635] Step 2:

[0636] The device transmits the acquired user voice and facial expression data to the emotion engine. The emotion engine analyzes this data and generates the user's emotional state as numerical or categorical data. For example, it determines whether the user is relaxed, focused, or stressed.

[0637] Step 3:

[0638] The device transmits the entered student information and generated user sentiment data to the server via a security protocol. This data is encrypted, ensuring it reaches the server safely.

[0639] Step 4:

[0640] The server stores the received data in a database. Sentimental data is managed in conjunction with student information and referenced later in the generation of educational content.

[0641] Step 5:

[0642] The server runs a generative AI model based on stored data. The AI ​​model combines student information and the user's emotional state to generate individually optimized educational content. For example, if a user is feeling stressed, the system will suggest experiential tasks to help them relax.

[0643] Step 6:

[0644] The generated educational content is stored in a database, where it is organized and indexed. This ensures that the content can be effectively searched and used later.

[0645] Step 7:

[0646] Users access content via their devices and review the generated educational materials. They can download or print materials as needed, enabling them to create lesson plans optimized for their students.

[0647] (Example 2)

[0648] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0649] To achieve individually optimized education, it is crucial to consider not only students' learning progress but also teachers' emotional states in real time. However, conventional systems have struggled to grasp emotional states and generate appropriate educational content based on them. This has led to increased burdens on teachers and the inability to provide appropriate educational support to students.

[0650] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0651] In this invention, the server includes an analysis means that analyzes the user's facial expressions and voice and quantifies their emotional state; a generation means that generates individually optimized educational content based on the data acquired from the input means and the analysis means; and a storage means that stores the generated educational content in a data storage device. This makes it possible to provide teaching materials that take into account the emotional state of teachers, thereby improving the quality of education and reducing the burden on teachers.

[0652] "Input means" refers to devices or methods for acquiring student learning-related information and user sentiment data.

[0653] The "analysis method" is a technology that analyzes the user's facial expressions and voice in real time and quantifies their emotional state.

[0654] "Generation means" refers to a process or system for creating individually optimized educational content based on acquired learning-related information and emotional data.

[0655] A "storage method" refers to a system that saves generated educational content to a storage device or similar, allowing it to be retrieved and used later.

[0656] "Means of delivery" refers to the methods and technologies used to deliver stored educational content so that teachers can access it.

[0657] This invention is an automated system for generating educational materials in special needs education, which realizes an individually optimized educational experience by taking into account the user's emotional state. Specifically, the system is configured as follows.

[0658] Users access and log into the system using a dedicated information terminal. The terminal is equipped with a camera and microphone, which capture the user's facial expressions and voice in real time. This data is analyzed using an emotion engine, and the user's emotional state is quantified. For example, if a user has a smiling face and a calm voice tone, the system will determine that the user is "at ease."

[0659] The device centralizes captured emotional data and student learning-related information and sends it to the server using a secure communication protocol. This ensures the safety and reliability of the data.

[0660] The server stores received emotional data and learning information in a database. Here, the emotional data is linked to each student's learning history and recorded as comprehensive information. Subsequently, the server uses a generative AI model to generate educational content based on the received data. This model enables the customization of educational content according to the user's emotional state, achieving individual optimization.

[0661] As a concrete example, by instructing the generative AI model with the prompt "Provide content appropriate for when students are feeling at ease," appropriate teaching materials will be provided. This prompt serves as an indicator of what kind of content should be emphasized in the content presented by the model.

[0662] Users can access, download, and print the generated educational content through their devices for use in their learning activities. This enables the provision of educational programs optimized for individual students, thereby improving the quality of education.

[0663] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0664] Step 1:

[0665] The user logs into the system using an information terminal. Upon logging in, the camera and microphone built into the terminal activate, capturing the user's face and voice in real time. The input at this time consists of the user's facial image data and voice data, which are acquired to provide basic data for sentiment analysis.

[0666] Step 2:

[0667] The device transmits the acquired facial image data and audio data to the emotion engine. This engine uses machine learning algorithms to analyze the data and quantify the user's emotional state. This analysis process analyzes changes in facial expressions and tone of voice to generate emotion tags such as reassurance, confusion, and anxiety. The output is numerical data indicating the user's emotional state.

[0668] Step 3:

[0669] The device packages emotional data and student learning-related information together, encrypts it, and sends it to the server. The input consists of emotional data and learning-related information, and by securely sending these to the server using a security protocol, the output prevents information leakage.

[0670] Step 4:

[0671] The server stores the received sentiment data and learning information in a database. Simultaneously, it links the sentiment data to the corresponding student profile, combining sentiment with learning history. In this process, the input is the received data, and the output is the updated database entry.

[0672] Step 5:

[0673] The server runs a generative AI model using stored data to generate educational content. Using the prompt "Provide content appropriate for when students are feeling secure," it generates materials for students who are in a secure state. The input is stored emotion data and learning information, and the output is individually optimized educational content.

[0674] Step 6:

[0675] Users can access the generated educational content through their devices and search, download, or print the necessary learning materials. In this case, the input is the user's actions, and the output is physical or digital educational materials. This enables effective learning support in educational settings.

[0676] (Application Example 2)

[0677] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] In face-to-face sales, it is difficult for salespeople to accurately grasp a customer's emotional state, which contributes to inappropriate service and low customer satisfaction. Furthermore, it is not easy for salespeople with little customer service experience to immediately decide on the appropriate response. Therefore, there is a need for a system that can acquire customer emotional information in real time and provide optimal customer service advice based on that information.

[0679] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0680] In this invention, the server includes an input means for inputting customer facial expressions and voice data, a generation means for generating individually optimized customer service advice based on the customer's emotional state, and a provision means for sales staff to access and provide the generated customer service advice through a display device. This allows sales staff to grasp the customer's emotional state in real time and respond based on appropriate advice.

[0681] An "input device" is a device used to electronically acquire facial expressions and voice data collected from customers.

[0682] The "generation method" refers to a mechanism that creates individually optimized customer service advice based on acquired data.

[0683] "Storage means" refers to a device that stores the generated customer service advice in a landscape information recording device.

[0684] "Means of provision" refers to a system that makes stored customer service advice accessible to sales staff and presents it via a display device.

[0685] A "generative intelligence model" is an artificial intelligence algorithm used to derive appropriate advice from acquired data.

[0686] A "display device" is a device that allows sales staff to visually confirm the customer service advice that has been generated.

[0687] This embodiment of the invention relates to a sales staff support system for physical stores. This system enables sales staff to understand the customer's emotional state in real time and provide appropriate customer service. The specific details of the system are described below.

[0688] The server first collects customer facial expressions and voice data through input means. Wearable devices such as smart glasses are used as hardware. These devices are equipped with cameras and microphones to acquire data for analyzing the customer's facial expressions and voice tone.

[0689] Next, a lightweight machine learning model is used as a generation tool to quantify the customer's emotional state (e.g., reassurance, confusion, etc.). The analyzed data is sent to a generative intelligence model (e.g., OpenAI's generative AI model), where personalized customer service advice based on the customer's emotional state is generated.

[0690] The generated customer service advice is stored as digital data in a landscape information recording device via a storage means. Salespeople can access this advice via a display device (e.g., smart glasses display) using a delivery means. This allows salespeople to implement the optimal customer service strategy at the appropriate time.

[0691] As a concrete example, while a salesperson is assisting a customer, the smart glasses display might show advice such as, "The customer is confused about which product to choose, so please provide clear explanations." Another example of a prompt for a generative intelligence model might be, "Please create a product suggestion that will interest the customer." In this way, the system aims to streamline the sales process and improve customer satisfaction.

[0692] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0693] Step 1:

[0694] The device acquires customer facial and audio data via the smart glasses' camera and microphone. The inputs are camera video and audio, and these data are temporarily stored on the device as output. Specifically, the camera captures the customer's face, and the microphone records their voice.

[0695] Step 2:

[0696] The device uses the acquired facial expression and voice data to perform an initial analysis with a lightweight machine learning model. The input is the data saved in step 1, and the output is data that quantifies the customer's emotional state. This process involves facial feature point extraction and voice tone analysis to identify the customer's emotional state (e.g., reassured, confused).

[0697] Step 3:

[0698] The terminal sends digitized emotion data to the server. The input is digitized emotion data, and the output is the completion of data transmission to the server. Here, data transfer takes place over the network, and the emotion data is registered in the server's database.

[0699] Step 4:

[0700] The server generates customer service advice using a generative AI model based on the received sentiment data. The input is sentiment data stored on the server, and the output is customer service advice based on a specific prompt. For example, the prompt might be "Suggest products that the customer is interested in."

[0701] Step 5:

[0702] The server saves the generated customer service advice to the scenery information recording device. The input is the generated customer service advice, and the output is the advice registered in the scenery information recording device. This operation ensures that the advice is saved in an easily searchable format.

[0703] Step 6:

[0704] The user views and confirms customer service advice stored in a landscape information recording device through smart glasses. The input is the stored advice information, and the output is the text of the advice displayed on the smart glasses' screen. This allows the user to take the most appropriate action in real time when interacting with customers.

[0705] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0706] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0707] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0708] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0709] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0710] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0711] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0712] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0713] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0714] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0715] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0716] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0717] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0718] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0719] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0720] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0721] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0722] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0723] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0724] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0725] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0726] The following is further disclosed regarding the embodiments described above.

[0727] (Claim 1)

[0728] An input method for entering student learning-related information,

[0729] A generation means that generates individually optimized educational content based on student learning-related information obtained from the input means,

[0730] A storage means for storing the educational content generated by the generation means in a database,

[0731] A means for teachers to access the educational content stored in the aforementioned storage means,

[0732] A system that includes this.

[0733] (Claim 2)

[0734] The system according to claim 1, wherein the generation means generates educational content using a generation artificial intelligence model.

[0735] (Claim 3)

[0736] The system according to claim 1, wherein the providing means transmits educational content to a teacher via a terminal.

[0737] "Example 1"

[0738] (Claim 1)

[0739] A registration method for entering student attribute information,

[0740] A creative means for generating individually optimized educational materials based on student attribute information obtained from the aforementioned registration means,

[0741] A storage means for storing educational materials generated by the aforementioned creative means in a storage medium,

[0742] An acquisition means for educators to acquire educational materials stored in the aforementioned storage means,

[0743] A communication means for transmitting data using encryption technology in the registration means,

[0744] A system that includes this.

[0745] (Claim 2)

[0746] The system according to claim 1, wherein the creative means generates educational materials by applying artificial intelligence technology.

[0747] (Claim 3)

[0748] The system according to claim 1, wherein the acquisition means provides educational materials to educators via a digital device.

[0749] "Application Example 1"

[0750] (Claim 1)

[0751] An input method for entering student learning-related information,

[0752] A generation means that generates individually optimized educational content based on student learning-related information obtained from the input means,

[0753] A storage means for storing the educational content generated by the generation means in a data storage unit,

[0754] A recommendation means for generating product information suitable for students based on the aforementioned saved educational content,

[0755] A display means that communicates product information generated by the aforementioned recommendation means via a display device,

[0756] A system that includes this.

[0757] (Claim 2)

[0758] The system according to claim 1, wherein the generation means generates educational content and product information using a generation artificial intelligence model.

[0759] (Claim 3)

[0760] The system according to claim 1, wherein the display means transmits educational content and product information to a display device.

[0761] "Example 2 of combining an emotion engine"

[0762] (Claim 1)

[0763] An input method for entering student learning-related information,

[0764] An analytical method that analyzes the user's facial expressions and voice to quantify their emotional state,

[0765] A generation means for generating individually optimized educational content based on data obtained from the input means and analysis means,

[0766] A storage means for storing the educational content generated by the generation means in a data storage device,

[0767] A means for providing educational content stored in the aforementioned storage means to be accessed by teachers via an information terminal,

[0768] A system that includes this.

[0769] (Claim 2)

[0770] The system according to claim 1, wherein the generation means generates educational content taking into account the generating artificial intelligence model and the user's emotional state.

[0771] (Claim 3)

[0772] The system according to claim 1, wherein the providing means transmits educational content to teachers via an information terminal.

[0773] "Application example 2 when combining with an emotional engine"

[0774] (Claim 1)

[0775] An input means for inputting customer facial expressions and voice data,

[0776] A generation means that generates individually optimized customer service advice based on the customer's emotional state obtained from the input means,

[0777] A storage means for storing the customer service advice generated by the generation means in a landscape information recording device,

[0778] A means for providing customer service advice stored in the aforementioned storage means, which is accessed by a salesperson and provided through a display device,

[0779] A system that includes this.

[0780] (Claim 2)

[0781] The system according to claim 1, wherein the generation means generates customer service advice using a generative intelligence model.

[0782] (Claim 3)

[0783] The system according to claim 1, wherein the providing means transmits customer service advice to a salesperson via a display device. [Explanation of Symbols]

[0784] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input method for entering student learning-related information, A generation means that generates individually optimized educational content based on student learning-related information obtained from the input means, A storage means for storing the educational content generated by the generation means in a database, A means for teachers to access the educational content stored in the aforementioned storage means, A system that includes this.

2. The system according to claim 1, wherein the generation means generates educational content using a generation artificial intelligence model.

3. The system according to claim 1, wherein the providing means transmits educational content to a teacher via a terminal.

Images