System

Abstract

A system is provided.SOLUTION: A system comprising: means for recording a student's note description or speech during a class; means for using a generative AI model that analyzes the recorded note description or speech content and extracts important keywords; means for qualitatively evaluating the student's thinking ability or judgment ability based on the keywords extracted by the generative AI model; and means for storing the evaluation results in a database and using them for subsequent evaluation or learning.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional educational evaluation relied primarily on test scores, but this had limitations in properly assessing students' thinking and judgment abilities. Furthermore, the influence of human emotions made it difficult to evaluate fairly, and it was also difficult for teachers to evaluate all notes and comments within a limited time frame. As a result, there was the issue of insufficient assessment of each student's thinking ability and independent learning. [Means for solving the problem]

[0005] This invention solves these problems by utilizing a generative AI model to record and analyze students' notes and comments in class in real time. Specifically, it first provides a means for recording students' notes and comments, then analyzes the recorded content using a generative AI model to extract important keywords. Students' thinking and judgment skills are qualitatively evaluated based on the keywords extracted by the generative AI model, and the evaluation results are stored in a database. It also provides a means for teachers to review the evaluation results and make corrections as necessary, as well as a means for accumulating long-term evaluation data and retraining the generative AI model. This streamlines teachers' evaluation work and enables fair and accurate evaluations.

[0006] "Student notes" refer to notes and writings that students make during class.

[0007] "Classroom comments" refer to opinions and questions that students verbally express during class.

[0008] "Recording means" refers to the method or device that digitally captures student notes and statements.

[0009] "Key words" are words or phrases that receive special attention in assessing a student's thinking and judgment skills.

[0010] A "generative AI model" is an algorithm or system that uses artificial intelligence to analyze text and recognize meaning and patterns.

[0011] A "qualitative evaluation tool" is a method or system that evaluates based on characteristics or features rather than numerical values.

[0012] "Means for storing evaluation results in a database" refers to a system or method for systematically collecting and storing generated evaluation data.

[0013] "Long-term evaluation data" refers to evaluation information accumulated over multiple classes or semesters, rather than a single evaluation.

[0014] "Means for relearning" refers to methods or systems for reviewing AI models based on data accumulated in the past and improving their performance.

[0015] "Means to make teachers' evaluation work more efficient" refers to methods and systems that shorten the time and effort required for teachers to perform evaluation work. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] System Overview

[0038] The present invention is a system that records students' note taking and comments made during class, analyzes them using a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities. The following describes in detail an embodiment of this system.

[0039] 1. Initial Setup Phase

[0040] server:

[0041] It provides an interface for teachers to log in and performs login authentication.

[0042] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[0043] Save important keywords in a database.

[0044] Device:

[0045] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[0046] User (Teacher):

[0047] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[0048] 2. Data Collection Phase

[0049] server:

[0050] Students' notes and comments are received in real time and stored in a database.

[0051] Device:

[0052] Provides an interface for students to enter notes.

[0053] Provides the ability to record student comments and convert them into text.

[0054] User (student):

[0055] Type notes into an electronic device during class.

[0056] Make a statement and record what you say.

[0057] 3. Data analysis phase

[0058] server:

[0059] The received notes and statements are analyzed using a generative AI model.

[0060] Important keywords are extracted and students' thinking and judgment skills are evaluated based on these.

[0061] The primary evaluation results are stored in a database.

[0062] 4. Evaluation confirmation phase

[0063] Device:

[0064] Provide a dashboard for teachers to view assessment results.

[0065] User (Teacher):

[0066] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[0067] server:

[0068] The corrected evaluation results are stored in the database.

[0069] 5. Data accumulation and re-learning phase

[0070] server:

[0071] The evaluation results will be accumulated in a database over a long period of time.

[0072] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[0073] Specific examples

[0074] Example of the initial setup phase

[0075] User (Teacher):

[0076] 1. Teachers log in to their dedicated dashboard.

[0077] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[0078] server:

[0079] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[0080] 2. Save the entered keywords in the database.

[0081] A concrete example of the data collection phase

[0082] User (student):

[0083] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[0084] 2. Speaking at specific class times.

[0085] Device:

[0086] 1. Send note input contents to the server in real time.

[0087] 2. Record the speech as audio, convert it into text, and send it to the server.

[0088] server:

[0089] 1. Receive note entries and comments and store them in a database.

[0090] A concrete example of the data analysis phase

[0091] server:

[0092] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[0093] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[0094] 3. Generate a primary evaluation score of "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and save it in the database.

[0095] Example of the evaluation confirmation phase

[0096] User (Teacher):

[0097] 1. Teachers access the dashboard and check the primary assessment results.

[0098] 2. The teacher makes the necessary corrections to the results.

[0099] server:

[0100] 1. Save the corrected evaluation results in the database.

[0101] Specific example of data accumulation and re-learning phase

[0102] server:

[0103] 1. Compile all evaluation data at the end of the semester.

[0104] 2. Retrain the generative AI model using the aggregated data.

[0105] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[0106] In this way, a system for qualitatively evaluating students' thinking and judgment abilities is constructed, realizing efficient and fair evaluation.

[0107] The processing flow will be explained below.

[0108] Step 1:

[0109] User (Teacher):

[0110] Teachers log in to their dedicated dashboard.

[0111] The teacher selects the "Class Settings" tab.

[0112] server:

[0113] Log in and authenticate to start the teacher session.

[0114] Displays the class settings tab and provides a form for entering important keywords.

[0115] Step 2:

[0116] User (Teacher):

[0117] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[0118] server:

[0119] The entered keywords are received and stored in a database.

[0120] Step 3:

[0121] User (student):

[0122] During class, students take notes using a dedicated application.

[0123] Speak up at specific times in class.

[0124] Device:

[0125] It provides a note-taking interface and sends inputs to a server in real time.

[0126] It provides a function to record speech, convert it into text, and send it to the server.

[0127] Step 4:

[0128] server:

[0129] Receive notes and comments sent from the device in real time.

[0130] The received content is temporarily saved and prepared for analysis.

[0131] Step 5:

[0132] server:

[0133] The note descriptions and speech content are passed to a generative AI model for analysis.

[0134] A generative AI model analyzes the content and extracts important keywords.

[0135] A primary evaluation is generated based on the extracted keywords and the results are stored in a database.

[0136] Step 6:

[0137] User (Teacher):

[0138] Teachers access the dashboard to check assessments.

[0139] Device:

[0140] Display the teacher dashboard and provide primary assessment results.

[0141] Step 7:

[0142] User (Teacher):

[0143] Check the results of the initial evaluation and make corrections as necessary.

[0144] Once the corrections are complete, the final evaluation is sent to the server.

[0145] server:

[0146] The corrected evaluation results are received and stored in a database.

[0147] Step 8:

[0148] server:

[0149] All evaluation data will be compiled at the end of the semester.

[0150] The generative AI model is retrained based on the aggregated data.

[0151] Step 9:

[0152] server:

[0153] Prepare to apply the retrained generative AI model to assessment in the next class.

[0154] Implement a model with new evaluation criteria into the system.

[0155] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and an efficient and fair evaluation is realized.

[0156] Example 1

[0157] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] To effectively evaluate students' thinking and judgment skills, a system is needed that can analyze notes and comments taken during class in real time and accurately reflect the evaluation results. However, conventional systems have difficulty consistently recording and analyzing notes and comments, which means that fairness and efficiency of evaluation cannot be ensured. Furthermore, there is a lack of a way for teachers to easily check evaluation results and make corrections as necessary. Technology that can solve these issues is needed.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0160] In this invention, the server includes means for recording students' notes and comments during class, means for receiving the recorded notes and comments in real time and saving them in a database, means for using a generative AI model that analyzes the recorded notes and comments to extract important keywords, means for inputting prompts to the generative AI model and analyzing the received data, means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model, means for saving the evaluation results in a database and using them for subsequent evaluation and learning, and means for teachers to check and correct the evaluation results. This allows students' learning activities to be recorded and analyzed in real time, enabling fair and efficient evaluation.

[0161] "Notes" are text data that students input into their electronic devices during class to record important information and personal thoughts.

[0162] "Comments" refer to the verbal expressions of opinions and questions that students make during class, and the recordings are then converted into text.

[0163] The "database" is an information storage system for uniformly managing and storing recorded notes, comments, and evaluation results.

[0164] A "generative AI model" is a model that uses artificial intelligence to analyze text data, extract important keywords, and evaluate thinking ability and judgment.

[0165] A "prompt sentence" is a text sentence entered into a generative AI model to instruct it to perform a specific process or analysis.

[0166] The "evaluation results" are the results of a qualitative assessment of students' thinking and judgment abilities based on data analyzed by the generative AI model.

[0167] "Means" refers to specific methods or functions for achieving a certain purpose.

[0168] A "server" is a computer system that processes, stores, and manages data, and has functions such as receiving, analyzing, evaluating, and storing note-taking and comment content.

[0169] A "terminal" is a device that a user directly operates to input and confirm data.

[0170] "User" refers to a person such as a teacher or student who uses this system.

[0171] This invention is a system that records students' notes and comments during class and uses a generative AI model to qualitatively evaluate their thinking and judgment abilities. This system consists of three parties: a server, a terminal, and a user, each of which processes and calculates data using specific hardware and software.

[0172] server

[0173] The server performs the following main roles:

[0174] 1. Login authentication and dashboard provision: Provides an interface for teachers to log in to a dedicated dashboard and authenticates the login information. If login is successful, displays the dashboard for lesson settings and evaluation confirmation.

[0175] 2. Data reception and storage: Students' notes and comments are received in real time and stored in a database.

[0176] 3. Data analysis: Prompt sentences are input into the generative AI model, which analyzes the notes and speech content to extract important keywords.

[0177] 4. Generating and saving evaluations: Based on the extracted keywords, students' thinking and judgment skills are evaluated and the results are saved in the database. If the evaluation results are revised, they are also saved.

[0178] 5. Retraining: Retrain the generative AI model using accumulated evaluation data to improve the accuracy of the evaluation.

[0179] Terminal

[0180] The terminal performs the following main roles:

[0181] 1. Login and settings interface provided: An interface is provided for teachers to log in and enter and save important keywords from the lesson settings tab.

[0182] 2. Note taking and speech recording: Provides an interface for students to take notes and has a function to record speech. This data is sent to the server in real time.

[0183] 3. Displaying assessment results: Provide a dashboard for teachers to view assessment results.

[0184] User

[0185] Users (teachers and students) have the following roles:

[0186] 1. Teachers: Log in to the dedicated dashboard and enter important keywords such as originality and critical thinking in the "Class Settings" tab. Also, check the evaluation results and make corrections as necessary.

[0187] 2. Students: Use electronic devices to take notes and speak during class.

[0188] Hardware and Software Examples

[0189] Server: Lincoln Central Server (Generic Cloud Server), database system (MySQL®)

[0190] Devices: Teacher and student laptops or tablets

[0191] Generative AI models: Text analysis software (e.g., OpenAI® GPT-3®)

[0192] Specific examples and prompts

[0193] Teacher operation example

[0194] 1. Teachers log in to their dedicated dashboard.

[0195] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[0196] 3. Teachers access the dashboard to review the primary assessment results and make corrections as necessary.

[0197] Student operation example

[0198] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[0199] 2. Students will speak during class, and their comments will be recorded audio and converted into text.

[0200] Prompt Sentence Examples

[0201] "Evaluate the students' thinking and judgment based on the text below."

[0202] By implementing such a system, students' learning activities can be recorded and analyzed in real time, enabling fair and efficient evaluation.

[0203] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0204] Step 1:

[0205] User (Teacher):

[0206] Teachers log in to their dedicated dashboard.

[0207] Specific behavior:

[0208] The teacher uses an electronic device to enter a username and password and clicks the login button.

[0209] input:

[0210] Username, Password

[0211] output:

[0212] If the login authentication is successful, the dashboard will be displayed.

[0213] Step 2:

[0214] server:

[0215] Authenticate your login details and view the teacher dashboard.

[0216] Specific behavior:

[0217] The server checks the received username and password against the information in its database. If authentication is successful, the class settings tab is displayed.

[0218] input:

[0219] Username, Password

[0220] output:

[0221] Upon successful login, a dashboard containing the Course Settings tab will be displayed.

[0222] Step 3:

[0223] User (Teacher):

[0224] Select the lesson settings tab, enter important keywords, and save.

[0225] Specific behavior:

[0226] Teachers enter key keywords such as originality, critical thinking, trial and error, and click the save button.

[0227] input:

[0228] Important keywords (e.g., "originality," "critical thinking," "trial and error")

[0229] output:

[0230] A save confirmation message will be displayed.

[0231] Step 4:

[0232] server:

[0233] The entered important keywords are saved in the database.

[0234] Specific behavior:

[0235] The server receives the important keywords entered by the teacher and stores them in a database.

[0236] input:

[0237] Important Keywords

[0238] output:

[0239] Notification that saving to the database is complete.

[0240] Step 5:

[0241] User (student):

[0242] Take notes during class.

[0243] Specific behavior:

[0244] Students use their electronic devices to enter text into a note-taking interface, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[0245] input:

[0246] Note contents

[0247] output:

[0248] The note contents will be displayed on the device.

[0249] Step 6:

[0250] Device:

[0251] The note description input contents are sent to the server in real time.

[0252] Specific behavior:

[0253] The contents entered in the note entry interface are periodically sent to the server.

[0254] input:

[0255] Note contents

[0256] output:

[0257] Data sent to the server.

[0258] Step 7:

[0259] User (student):

[0260] Speak during class.

[0261] Specific behavior:

[0262] Students speak aloud during class.

[0263] input:

[0264] Voice remarks

[0265] output:

[0266] The speech is recorded as audio data.

[0267] Step 8:

[0268] Device:

[0269] Speech is recorded as audio, converted into text and sent to the server.

[0270] Specific behavior:

[0271] The recorded voice data is converted into text using a text conversion engine and sent to the server.

[0272] input:

[0273] Audio data

[0274] output:

[0275] The speech converted to text.

[0276] Step 9:

[0277] server:

[0278] The received note contents and speech content are passed to the generation AI model.

[0279] Specific behavior:

[0280] The server reads the received note descriptions and speech text from the database and prepares them to be passed to the generative AI model.

[0281] input:

[0282] Note writing, speech text

[0283] output:

[0284] Text data input into a generative AI model.

[0285] Step 10:

[0286] server:

[0287] The prompt sentence is fed into a generative AI model to analyze the data.

[0288] Specific behavior:

[0289] The server passes a prompt statement, for example, "Please evaluate the student's thinking ability and judgment based on the text below," to the generative AI model and begins analysis.

[0290] input:

[0291] Note description, speech text, prompt text

[0292] output:

[0293] Important keywords as analysis results.

[0294] Step 11:

[0295] server:

[0296] Students' thinking and judgment skills are evaluated based on the extracted keywords, and the evaluation results are stored in a database.

[0297] Specific behavior:

[0298] Based on the keywords extracted by the generative AI model, an evaluation score is generated and stored in a database.

[0299] input:

[0300] Important Keywords

[0301] output:

[0302] Evaluation scores are stored in a database.

[0303] Step 12:

[0304] User (Teacher):

[0305] Check the assessment results on the dashboard and make corrections as necessary.

[0306] Specific behavior:

[0307] Teachers can access the dashboard, check the assessment results, make corrections if necessary, and click the save button.

[0308] input:

[0309] Evaluation results

[0310] output:

[0311] Corrected evaluation results.

[0312] Step 13:

[0313] server:

[0314] The corrected evaluation results are stored in the database.

[0315] Specific behavior:

[0316] The server receives the teacher-corrected evaluation results and stores them in a database.

[0317] input:

[0318] Corrected evaluation results

[0319] output:

[0320] Notification that saving to the database is complete.

[0321] Step 14:

[0322] server:

[0323] The generative AI model is retrained using accumulated evaluation data.

[0324] Specific behavior:

[0325] The server aggregates the evaluation data at the end of the semester and builds a dataset for retraining the generative AI model. The retraining of the generative AI model is then performed, and the new model is applied to the next semester's evaluations.

[0326] input:

[0327] Evaluation Data

[0328] output:

[0329] Retrained generative AI model.

[0330] The above are the specific processing steps of this system.

[0331] (Application example 1)

[0332] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] Factory workplaces require accurate evaluation of worker efficiency and skill levels, and the provision of appropriate feedback and training. However, conventional evaluation methods require cumbersome recording and evaluation criteria, making efficient and objective evaluation difficult. Additionally, there is a lack of means to evaluate skills and efficiency based on worker comments, making comprehensive evaluation difficult.

[0334] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0335] In this invention, the server includes a means for recording work records and speech contents, a means for using a generative AI model that analyzes the recorded records and speech contents to extract important keywords, a means for qualitatively evaluating work efficiency and skill level based on the keywords extracted by the generative AI model, and a means for storing the evaluation results in a database and using them for subsequent evaluation and learning, thereby enabling a comprehensive evaluation of work efficiency and skill level.

[0336] "Work records" are data that records in detail the procedures and results of work performed by workers in a factory.

[0337] "Speech content" refers to data recorded in text format, including verbal communications, questions, and responses made by workers while they were working.

[0338] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze text data, extract important keywords, and understand the text.

[0339] "Key words" are important words or phrases related to specific evaluation metrics extracted by the generative AI model.

[0340] "Work efficiency" is an indicator of how effectively and quickly a worker can perform a given task.

[0341] "Skill level" is an indicator of the degree of technical ability possessed by a worker.

[0342] "Qualitative evaluation" refers to evaluation based on text data and important keywords, rather than on quantified data.

[0343] A "database" is a data storage system that systematically stores collected records and evaluation results so that they can be reused as needed.

[0344] "Future evaluation and learning" refers to future re-evaluation based on the stored data and re-learning to improve the generative AI model.

[0345] MODE FOR CARRYING OUT THE INVENTION

[0346] One embodiment of the present invention provides a system for evaluating the efficiency and skill level of factory workers. This system records work logs and speech, analyzes them using a generative AI model, and stores the evaluation results in a database.

[0347] 1. Initial Setup Phase

[0348] server

[0349] The server provides an interface for administrators to log in, performs login authentication, provides a work setting tab, and displays a form for administrators to input key evaluation indicators, which are then stored in a database.

[0350] Terminal

[0351] The terminal provides an interface for administrators to log in and enter and save evaluation indicators from the work settings tab.

[0352] User (Administrator)

[0353] Administrators log in to a dedicated dashboard and enter and save key evaluation indicators such as "operation accuracy," "efficiency," and "adaptability" in the "Work Settings" tab.

[0354] 2. Data Collection Phase

[0355] server

[0356] The server receives the workers' work records and comments in real time and stores them in a database.

[0357] Terminal

[0358] The terminal provides an interface for workers to input their work, and also provides the function of recording what the workers say and converting it into text.

[0359] User (worker)

[0360] While working, workers input their work details into the electronic device, and also make statements, which are then recorded.

[0361] 3. Data analysis phase

[0362] server

[0363] The server analyzes the received work records and comments using a generative AI model. It extracts important evaluation indicators and evaluates work efficiency and skill level based on these. The evaluation results are stored in a database.

[0364] 4. Evaluation confirmation phase

[0365] Terminal

[0366] The terminal provides a dashboard for administrators to view evaluation results.

[0367] User (Administrator)

[0368] Administrators can check the evaluation results through the dashboard and make corrections as necessary.

[0369] server

[0370] The server stores the modified evaluation results in a database.

[0371] 5. Data accumulation and re-learning phase

[0372] server

[0373] The server accumulates the evaluation results in a database over a long period of time and uses the accumulated data to retrain the generative AI model, thereby improving the accuracy of the evaluation.

[0374] Hardware and software used

[0375] This system uses the following hardware and software:

[0376] Hardware: Factory robots, voice input devices

[0377] Software: databases (e.g., MySQL), speech-to-text tools (e.g., Google® Speech-to-Text API), generative AI models (e.g., OpenAI GPT-4®)

[0378] Specific examples

[0379] For example, consider a situation where a worker in a particular factory needs to learn how to operate a new machine. The robot records the worker's machine operation logs and conversations, which are then analyzed by a generative AI model. For example, the evaluation result may be that "the efficiency of the new machine operation has improved, but more skill is needed." In this case, the manager can plan additional training based on this. The manager can view the evaluation results in real time on a dashboard.

[0380] Prompt Sentence Examples

[0381] Work log data:

[0382] "The workers operated the CNC machines, adjusted the settings, and completed the assembly of the parts."

[0383] Speech data:

[0384] "Is there a way to tweak this setting to improve the quality of the finished result?"

[0385] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0386] Step 1:

[0387] The administrator logs in.

[0388] The administrator accesses the dedicated dashboard and enters their login information. The server receives the information and performs authentication. If authentication is successful, the dashboard is displayed to the administrator. The input is the administrator's login information, and the output is the administrator's dashboard.

[0389] Step 2:

[0390] Managers set key metrics.

[0391] The administrator selects the "Work Settings" tab on the dashboard, enters evaluation indicators such as "operation accuracy," "efficiency," and "adaptability," and clicks the save button. The terminal receives the input and sends it to the server, which stores the information in a database. The input is the evaluation indicator entered by the administrator, and the output is the evaluation indicator stored in the database.

[0392] Step 3:

[0393] The worker enters the work record.

[0394] While working, workers enter details of the machines they operated and the work they performed into a terminal. The terminal sends the entered data in real time to a server, which stores the data in a database. The input is the work record entered by the worker, and the output is the work record stored in the database.

[0395] Step 4:

[0396] The worker records what is said.

[0397] Conversations and questions asked by workers while they are working are recorded using a voice input device. The device converts the voice data into text and sends it to a server. The server stores the data in a database. The input is the worker's voice data, and the output is the text of what was said.

[0398] Step 5:

[0399] The server passes the data to the generative AI model.

[0400] The server extracts work records and speech content from the database and inputs them into the generative AI model. The generative AI model analyzes the data, extracts important keywords, and evaluates work efficiency and skill level. The inputs are work records and speech content, and the output is the extracted keywords and evaluation results.

[0401] Step 6:

[0402] The evaluation results are stored in a database.

[0403] The server stores the evaluation results obtained from the generative AI model in a database. The evaluation results include indicators such as "operation accuracy," "efficiency," and "adaptability." The input is the evaluation results from the generative AI model, and the output is the evaluation results stored in the database.

[0404] Step 7:

[0405] The administrator checks and corrects the evaluation results.

[0406] Administrators can access the dashboard to check the evaluation results and make corrections as necessary. The corrections made by the administrator are sent from the terminal to the server and stored in the database. The input is the corrections made by the administrator, and the output is the corrected evaluation results.

[0407] Step 8:

[0408] Evaluation data is stored for a long period of time and re-learned.

[0409] The server stores the evaluation results in a database for a long period of time. The accumulated data is used to periodically retrain the generative AI model. The input is the accumulated evaluation data, and the output is the retrained generative AI model.

[0410] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0411] System Overview

[0412] The present invention provides a system that records students' notes and comments during class, analyzes them with a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities, and also provides a system that combines an emotion engine. A specific embodiment of this system is described below.

[0413] 1. Initial Setup Phase

[0414] server:

[0415] It provides an interface for teachers to log in and performs login authentication.

[0416] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[0417] Save important keywords in a database.

[0418] Device:

[0419] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[0420] User (Teacher):

[0421] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[0422] 2. Data Collection Phase

[0423] server:

[0424] Students' notes and comments are received in real time and stored in a database.

[0425] Device:

[0426] Provides an interface for students to enter notes.

[0427] Provides the ability to record student comments and convert them into text.

[0428] User (student):

[0429] Type notes into an electronic device during class.

[0430] Make a statement and record what you say.

[0431] 3. Emotional data collection phase

[0432] Device:

[0433] Students' facial expressions and vocal tones are analyzed in real time using an emotion engine to obtain emotional data.

[0434] The emotion data is sent to the server.

[0435] server:

[0436] Emotion data sent from the device is received and saved in a linked manner with the note description and the content of the remarks.

[0437] 4. Data analysis phase

[0438] server:

[0439] The received notes and statements are passed to a generative AI model for analysis.

[0440] A generative AI model analyzes the content and extracts important keywords.

[0441] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[0442] 5. Evaluation confirmation phase

[0443] Device:

[0444] Provide a dashboard for teachers to view assessment results.

[0445] User (Teacher):

[0446] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[0447] server:

[0448] The corrected evaluation results are stored in the database.

[0449] 6. Data accumulation and re-learning phase

[0450] server:

[0451] The evaluation results will be accumulated in a database over a long period of time.

[0452] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[0453] Specific examples

[0454] Example of the initial setup phase

[0455] User (Teacher):

[0456] 1. Teachers log in to their dedicated dashboard.

[0457] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[0458] server:

[0459] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[0460] 2. Save the entered keywords in the database.

[0461] A concrete example of the data collection phase

[0462] User (student):

[0463] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[0464] 2. Speaking at specific class times.

[0465] Device:

[0466] 1. Send note input contents to the server in real time.

[0467] 2. Record the speech as audio, convert it into text, and send it to the server.

[0468] server:

[0469] 1. Receive note entries and comments and store them in a database.

[0470] A concrete example of the emotion data collection phase

[0471] Device:

[0472] 1. The camera captures students' facial expressions as they speak.

[0473] 2. The emotion engine analyzes this and detects emotions such as "tension," "excitement," and "anxiety."

[0474] 3. The detected emotion data is sent to the server.

[0475] server:

[0476] 1. The received emotion data is saved by linking it to the note description and the speech content.

[0477] A concrete example of the data analysis phase

[0478] server:

[0479] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[0480] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[0481] 3. Correct the result based on the emotional data, and generate a primary evaluation such as "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and store it in the database.

[0482] Example of the evaluation confirmation phase

[0483] User (Teacher):

[0484] 1. Teachers access the dashboard and check the primary assessment results.

[0485] 2. The teacher makes the necessary corrections to the results.

[0486] server:

[0487] 1. Save the corrected evaluation results in the database.

[0488] Specific example of data accumulation and re-learning phase

[0489] server:

[0490] 1. After the semester ends, all evaluation data will be compiled.

[0491] 2. Retrain the generative AI model using the aggregated data.

[0492] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[0493] In this way, by qualitatively evaluating students' thinking and judgment abilities and taking emotional data into consideration, more efficient and fair evaluation becomes possible.

[0494] The processing flow will be explained below.

[0495] Step 1:

[0496] User (Teacher):

[0497] Teachers log in to their dedicated dashboard.

[0498] The teacher selects the "Class Settings" tab.

[0499] server:

[0500] Log in and authenticate to start the teacher session.

[0501] Displays the class settings tab and provides a form for entering important keywords.

[0502] Step 2:

[0503] User (Teacher):

[0504] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[0505] server:

[0506] The entered keywords are received and stored in a database.

[0507] Step 3:

[0508] User (student):

[0509] During class, students take notes using a dedicated application.

[0510] Speak up at specific times in class.

[0511] Device:

[0512] It provides a note-taking interface and sends inputs to a server in real time.

[0513] It provides a function to record speech, convert it into text, and send it to the server.

[0514] Step 4:

[0515] server:

[0516] Receive notes and comments sent from the device in real time.

[0517] The received content is temporarily saved and prepared for analysis.

[0518] Step 5:

[0519] Device:

[0520] The camera captures students' facial expressions and activates an emotion engine that analyzes their voice tone in real time.

[0521] The emotion engine analyzes facial expressions and vocal tone to generate emotion data (e.g., "nervous," "excited," "anxious").

[0522] The emotion data is sent to the server.

[0523] server:

[0524] Emotion data transmitted from the terminal is received.

[0525] Save the notes and link them to the comments.

[0526] Step 6:

[0527] server:

[0528] The note description, speech content, and linked emotional data are passed to a generative AI model for analysis.

[0529] A generative AI model extracts important keywords.

[0530] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[0531] Step 7:

[0532] User (Teacher):

[0533] Teachers access the dashboard to check assessments.

[0534] Device:

[0535] Display the teacher dashboard and provide primary assessment results.

[0536] Step 8:

[0537] User (Teacher):

[0538] Check the results of the initial evaluation and make corrections as necessary.

[0539] Once the corrections are complete, the final evaluation is sent to the server.

[0540] server:

[0541] The corrected evaluation results are received and stored in a database.

[0542] Step 9:

[0543] server:

[0544] All evaluation data will be compiled at the end of the semester.

[0545] The generative AI model is retrained using the aggregated data.

[0546] Step 10:

[0547] server:

[0548] Prepare to apply the retrained generative AI model to assessment in the next class.

[0549] Implement a model with new evaluation criteria into the system.

[0550] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and by taking emotional data into consideration, more efficient and fair evaluation is realized.

[0551] Example 2

[0552] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0553] Conventional student evaluation systems evaluate students based solely on their notes and comments, making it difficult to accurately assess their thinking and judgment abilities. Furthermore, they do not take emotional data into account, making it difficult to provide fair evaluations that reflect students' psychological state and emotional changes. For these reasons, there has been a demand for a more accurate and fair evaluation method.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0555] In this invention, the server includes a means for recording students' notes and comments during class, a means for analyzing the recorded notes and comments and using a generative AI model to extract important keywords, a means for analyzing the students' facial expressions and voice tones during evaluation to obtain emotional data, and a means for qualitatively evaluating the students' thinking ability and judgment ability based on the keywords extracted by the generative AI model. This allows for more accurate evaluation of students' thinking ability and judgment ability, and by taking emotional data into consideration, enables fair and comprehensive evaluation.

[0556] "Note writing" is text data that students input using electronic devices during class.

[0557] "Speech" refers to the verbal content of a student's speech during class, and the content is recorded as audio or converted into text data.

[0558] A "generative AI model" is an artificial intelligence model that analyzes text data and extracts important keywords. For example, it includes models that use natural language processing technology.

[0559] "Important keywords" are particularly noteworthy words and phrases extracted by the generative AI model from students' notes and comments.

[0560] "Thinking ability" refers to the logical thinking skills that students use to solve problems, and specifically includes originality and critical thinking.

[0561] "Judgment" is a student's ability to take appropriate action or make appropriate decisions depending on the situation.

[0562] "Emotional data" refers to data that indicates the emotional state of a student, obtained by analyzing their facial expressions and vocal tones. For example, it includes information on tension, excitement, anxiety, etc.

[0563] A "database" is a system for centrally managing and storing recorded data.

[0564] "Retraining" is the process of using accumulated data to improve the accuracy of a generative AI model.

[0565] "Evaluation results" are evaluation data on students' thinking and judgment abilities generated based on the generative AI model and emotional data.

[0566] A "teacher" is a person who is responsible for teaching and assessing students at an educational institution.

[0567] The present invention is a system that records students' notes and comments in class in real time, extracts important keywords using a generative AI model, and further combines them with emotional data to qualitatively evaluate students' thinking and judgment abilities. An embodiment of this system will be described in detail.

[0568] Hardware and software used

[0569] Server: Database management system, generative AI model (e.g., model using natural language processing technology), login authentication system, evaluation result management system

[0570] Device: Camera, microphone, note input interface, facial expression analysis engine (e.g. emotion engine), voice recognition API

[0571] Users: Electronic devices (tablets and PCs) used by teachers and students

[0572] Specific operation of the system

[0573] Initial Setup Phase

[0574] User (Teacher):

[0575] 1. The teacher logs in to the dedicated dashboard and enters their username and password for authentication.

[0576] 2. After logging in, select the "Class Settings" tab, enter the keywords you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the Save button.

[0577] server:

[0578] Receives the teacher's authentication information and verifies the login via the authentication server. If the login is successful, it provides the lesson settings tab interface.

[0579] Important keywords entered by the teacher are saved in a database.

[0580] Data Collection Phase

[0581] User (student):

[0582] Students typing notes into electronic devices during class.

[0583] Students will speak during class and their comments will be recorded.

[0584] Device:

[0585] It provides an interface for note entry and sends the note contents entered by students to the server in real time.

[0586] It uses a speech recognition API to record what you say and convert it into text, which is then sent to the server.

[0587] server:

[0588] The received note contents and speech contents are saved in a database.

[0589] Emotional data collection phase

[0590] Device:

[0591] A camera captures students' facial expressions as they speak.

[0592] The emotion engine is used to analyze facial expressions and voice tones to detect emotion data (e.g., "tension," "excitement," "anxiety"), and transmit the detected emotion data to the server.

[0593] server:

[0594] The received emotion data is saved by linking it to the note description and the content of the speech.

[0595] Data analysis phase

[0596] server:

[0597] The received note content and speech content are passed to a generative AI model, which uses natural language processing technology such as OpenAI's GPT-3.

[0598] Send the following prompt to the generative AI model: "Analyze the student's notebook entry, 'We held a discussion about the impact of global warming on society, taking into account new perspectives,' and extract key keywords. Additionally, take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error."

[0599] The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database.

[0600] Evaluation confirmation phase

[0601] User (Teacher):

[0602] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page.

[0603] Check the results of the primary evaluation and make any necessary corrections. Once the corrections are complete, click the Save button.

[0604] server:

[0605] The corrections made by the teacher are saved in a database.

[0606] Data accumulation and re-learning phase

[0607] server:

[0608] After each semester, all evaluation data is compiled and stored in a database.

[0609] The generated AI model will be retrained using the accumulated data to improve the accuracy of the evaluation algorithm, and the retrained model will be applied to class evaluations from the next semester.

[0610] Specific examples

[0611] In the initial setup phase, the teacher inputs and saves key keywords such as "originality," "critical thinking," and "trial and error." In the data collection phase, students write in their notebooks, "We held a discussion that considered new perspectives on the impact of global warming on society." In the emotion data collection phase, the students' facial expressions while speaking are captured and analyzed with an emotion engine to detect "tension." In the data analysis phase, the generative AI model extracts "new perspectives" and "discussion" and generates an initial evaluation of "originality: high, critical thinking: medium, trial and error: medium." In the evaluation confirmation phase, the teacher checks the results of the initial evaluation and makes any necessary corrections.

[0612] The present invention can be implemented according to the specific steps set forth in the above detailed description.

[0613] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0614] The specific processing flow of this system program

[0615] Step 1:

[0616] User (Teacher):

[0617] The teacher logs in to the dedicated dashboard. Enter the username and password and click the login button.

[0618] server:

[0619] Receives the teacher's authentication information and verifies the login via the authentication server (input: username, password, output: authentication result). If the login is successful, the lesson settings tab interface is provided.

[0620] Step 2:

[0621] User (Teacher):

[0622] Select the "Class Settings" tab, enter keywords that you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the save button.

[0623] Device:

[0624] It provides a keyword input interface and sends the input keywords to the server (input: important keywords, output: transmission signal).

[0625] server:

[0626] Receives keyword input from teachers and stores it in a database (input: important keywords, output: stored results).

[0627] Step 3:

[0628] User (student):

[0629] A student takes notes on an electronic device during class, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[0630] Device:

[0631] It provides a note input interface and transmits the input note content to the server in real time (input: note content, output: transmission signal).

[0632] server:

[0633] Save the received note contents to the database (input: note contents, output: saved result).

[0634] Step 4:

[0635] User (student):

[0636] He speaks up during class, saying things like, "We should think about environmental issues from a new policy perspective."

[0637] Device:

[0638] The speech is recorded as audio and converted into text using a speech recognition API. The converted text is sent to the server (input: audio data, output: transmission signal).

[0639] server:

[0640] Save the received comment content in the database (input: comment content, output: saved result).

[0641] Step 5:

[0642] Device:

[0643] When a student speaks, the camera captures their facial expressions, and the emotion engine analyzes their facial expressions and vocal tone to detect emotional data (e.g., "nervous," "excited," "anxious") and transmits it to the server (input: facial expressions, vocal data, output: emotional data).

[0644] server:

[0645] The received emotion data is linked to the note description and the speech content and saved (input: emotion data, output: saved result).

[0646] Step 6:

[0647] server:

[0648] The received note content and speech content are passed to the generative AI model (input: note content, speech content, output: prompt). For example, a prompt such as "Analyze the student's note, 'We held a discussion on the impact of global warming on society, taking into account new perspectives,' and extract important keywords. In addition, please take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error" is sent to the generative AI model. The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database (input: generative AI model analysis results, emotional data, output: primary evaluation).

[0649] Step 7:

[0650] User (Teacher):

[0651] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page. Check the primary evaluation results and make corrections as necessary (input: primary evaluation results, corrections, output: final evaluation results).

[0652] server:

[0653] The teacher's corrections are saved in the database (input: final evaluation result, output: saved result).

[0654] Step 8:

[0655] server:

[0656] After each semester, all evaluation data is aggregated and stored in a database (input: each evaluation data, output: stored data). The stored data is used to retrain the generative AI model (input: stored data, output: retrained model). The retrained model is applied to class evaluations from the next semester onwards (input: retrained model, output: application results).

[0657] The above is the specific processing flow of this system.

[0658] (Application example 2)

[0659] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0660] In addition to a system for efficiently evaluating students' thinking and judgment skills, there is a need for a method for qualitatively evaluating the performance and processing capabilities of factory robots by collecting and analyzing their work logs and sensor data in real time. This will enable fairer and more efficient evaluations in both educational and production settings.

[0661] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0662] In this invention, the server includes: means for recording students' notes and comments during class; means for using a generative AI model to analyze the recorded notes and comments and extract important keywords; means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model; means for storing the evaluation results in a database and using them for subsequent evaluation and learning; means for collecting robot work logs and sensor data in real time and evaluating them using the generative AI model; and means for qualitatively evaluating the robot's performance and processing capabilities based on the sensor data analyzed in real time. This makes it possible to highly evaluate both student learning effectiveness and robot work efficiency.

[0663] "Note taking" refers to handwritten or electronic notes or records that students make during class.

[0664] "Content of remarks" refers to the words and opinions expressed by students during class.

[0665] A "generative AI model" is an artificial intelligence model that extracts and analyzes useful information from input data.

[0666] "Important keywords" refer to words or phrases that are particularly meaningful in the recorded notes and statements.

[0667] "Thinking ability" refers to the intellectual abilities necessary for problem-solving and generating new ideas.

[0668] Judgment is the ability to choose the best option depending on the situation.

[0669] "Evaluation results" are the evaluation values ​​of the student's or robot's performance after analysis using the generative AI model.

[0670] A "database" is a computer system for systematically storing, retrieving, and managing digital information.

[0671] A "work log" is a detailed record of the work a robot performs.

[0672] "Sensor data" refers to digital data obtained from sensors that measure the operating conditions of a robot.

[0673] "Performance" refers to the efficiency and accuracy with which a robot carries out a task.

[0674] "Processing capacity" refers to how effectively a robot can perform multiple tasks.

[0675] System Overview

[0676] This invention is a system that qualitatively evaluates students' thinking and judgment skills, as well as robot performance and processing capabilities, by collecting students' note-taking and comments during class, factory robot work logs, and sensor data in real time and analyzing them with a generative AI model.

[0677] 1. Initial Setup Phase

[0678] The server provides an interface for teachers and factory managers to log in and performs login authentication. After login authentication is complete, it displays the lesson settings tab and work settings tab, and provides a form for entering important keywords and evaluation criteria (e.g., efficiency, accuracy, flexibility). The entered evaluation criteria are saved in a database.

[0679] The terminal provides an interface for teachers and factory managers to log in and input and save evaluation criteria.

[0680] Users (teachers or factory managers) log in to a dedicated dashboard, enter important keywords and evaluation criteria in the "Class Settings" or "Work Settings" tab, and save them.

[0681] 2. Data Collection Phase

[0682] The server receives students' notes and comments, as well as the robot's work logs and sensor data in real time, stores them in a database, and prepares them as input for the generative AI model.

[0683] The device provides an interface for students to input notes and a function to record the robot's work log. It also has the ability to record student comments and the robot's operation status and convert them into text as needed.

[0684] During class, users (students or robots) take notes and make comments on electronic devices. The robots perform assigned tasks and send their work logs and sensor data to the terminal.

[0685] 3. Emotional data collection phase

[0686] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data, which is then sent to a server.

[0687] The server receives the emotional data sent from the device and stores it, linking it with note descriptions, statements, the robot's work log, and sensor data.

[0688] 4. Data analysis phase

[0689] The server passes the received note entries, comments, robot work logs, and sensor data to the generative AI model for analysis. The generative AI model analyzes the content and extracts important keywords and evaluation criteria. It generates a primary evaluation based on the extracted keywords and criteria and stores the results in a database.

[0690] 5. Evaluation confirmation phase

[0691] The devices provide a dashboard for teachers and factory managers to view assessment results.

[0692] Users (teachers and factory managers) can check the results of the initial assessment through the dashboard and make corrections as necessary.

[0693] The server stores the modified evaluation results in a database.

[0694] 6. Data accumulation and re-learning phase

[0695] The server accumulates the evaluation results in a database over a long period of time, and uses the accumulated data to retrain the generative AI model and improve the accuracy of the evaluation.

[0696] Specific examples

[0697] Example of the initial setup phase

[0698] The user (factory manager) logs in to the dedicated dashboard, selects the "Work Settings" tab, and enters and saves evaluation criteria such as "Efficiency," "Accuracy," and "Flexibility." After authenticating the manager's login, the server displays the work settings tab and saves the entered evaluation criteria in the database.

[0699] A concrete example of the data collection phase

[0700] The user (robot) inputs "Completed assembly of part A and adjusted the position of part B" into the work log interface. The terminal sends the work log interface to the server in real time. The server receives the work log and sensor data and stores them in a database.

[0701] A concrete example of the emotion data collection phase

[0702] The device captures the robot's movements as it assembles parts, and the emotion engine analyzes them to detect emotions such as "stress" or "relaxation." The detected emotion data is then sent to a server, which then links the received emotion data with work logs and sensor data and stores it.

[0703] A concrete example of the data analysis phase

[0704] The server passes the received work log, "Assembly of part A completed, position of part B adjusted," to the generative AI model. The generative AI model analyzes the content and detects "assembly" and "adjustment." It then makes corrections based on the emotional data, generating a primary evaluation, for example, "Efficiency: High, Accuracy: Medium, Flexibility: Medium," and stores this in the database.

[0705] Example of the evaluation confirmation phase

[0706] The user (factory manager) accesses the dashboard and checks the primary evaluation results. The manager makes any necessary corrections to the results. The server saves the corrected evaluation results in the database.

[0707] Specific example of data accumulation and re-learning phase

[0708] After the end of the semester, the server aggregates all evaluation data. The aggregated data is used to retrain the generative AI model. The model with the new evaluation criteria is then applied to the next semester's class evaluations.

[0709] Prompt Sentence Examples

[0710] "Based on the following work log, rate the robot's efficiency, accuracy, and flexibility.

[0711] Work log: [Log data]

[0712] Sensor Data: [Sensor Data]"

[0713] In this way, the present invention can highly evaluate the thinking ability and judgment ability of students and the working efficiency of robots.

[0714] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0715] Step 1: Initial Setup Phase

[0716] The server provides an interface for the administrator to log in and performs authentication. After successful login, it displays a form for the administrator to input evaluation criteria (efficiency, accuracy, flexibility, etc.). It receives the evaluation criteria entered by the administrator and stores them in a database.

[0717] Input: Administrator login information and evaluation criteria

[0718] Output: Authentication results and stored metrics data

[0719] Step 2: Data collection phase

[0720] The terminal provides an interface for students to input notes and an interface for robots to input work logs. Students and robots input information into the interface, and the terminal transmits this information in real time to a server. The server stores the received data in a database.

[0721] Input: Student notes, robot work log

[0722] Output: Input data stored in a database

[0723] Step 3: Emotional data collection phase

[0724] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data. The emotional data is then sent to a server, which links it to notes and work logs and stores it in a database.

[0725] Input: Student's facial expressions, voice tone, robot's movement data

[0726] Output: Emotion data and linked notes and work logs

[0727] Step 4: Data analysis phase

[0728] The server passes the notes, comments, work logs, sensor data, and emotion data stored in the database to the generative AI model for analysis. The generative AI model extracts important keywords and evaluation criteria from this data and generates a primary evaluation. The evaluation results are stored in the database.

[0729] Input: Note writing, speech content, work log, sensor data, emotion data

[0730] Output: Analysis results and primary evaluation by the generative AI model

[0731] Step 5: Evaluation confirmation phase

[0732] The terminal provides a dashboard for teachers and factory managers to check the evaluation results. Users can check the primary evaluation results through the dashboard and make corrections as necessary. The server then stores the corrected evaluation results back in the database.

[0733] Input: Primary assessment results, teacher and administrator feedback

[0734] Output: Corrected evaluation result

[0735] Step 6: Data accumulation and retraining phase

[0736] The server accumulates the evaluation results in a database over a long period of time. The accumulated data is used to retrain the generative AI model, improving the model's evaluation accuracy. The model with the new evaluation criteria is used in the next cycle.

[0737] Input: Accumulated evaluation data

[0738] Output: The retrained generative AI model and its evaluation criteria

[0739] By using the above steps, the present invention can highly evaluate the thinking ability and judgment ability of students and the work efficiency of robots.

[0740] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0741] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0742] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0743] [Second embodiment]

[0744] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0745] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0746] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0747] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0748] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0749] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0750] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0751] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0752] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0753] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0754] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0755] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0756] System Overview

[0757] The present invention is a system that records students' note taking and comments made during class, analyzes them using a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities. The following describes in detail an embodiment of this system.

[0758] 1. Initial Setup Phase

[0759] server:

[0760] It provides an interface for teachers to log in and performs login authentication.

[0761] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[0762] Save important keywords in a database.

[0763] Device:

[0764] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[0765] User (Teacher):

[0766] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[0767] 2. Data Collection Phase

[0768] server:

[0769] Students' notes and comments are received in real time and stored in a database.

[0770] Device:

[0771] Provides an interface for students to enter notes.

[0772] Provides the ability to record student comments and convert them into text.

[0773] User (student):

[0774] Type notes into an electronic device during class.

[0775] Make a statement and record what you say.

[0776] 3. Data analysis phase

[0777] server:

[0778] The received notes and statements are analyzed using a generative AI model.

[0779] Important keywords are extracted and students' thinking and judgment skills are evaluated based on these.

[0780] The primary evaluation results are stored in a database.

[0781] 4. Evaluation confirmation phase

[0782] Device:

[0783] Provide a dashboard for teachers to view assessment results.

[0784] User (Teacher):

[0785] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[0786] server:

[0787] The corrected evaluation results are stored in the database.

[0788] 5. Data accumulation and re-learning phase

[0789] server:

[0790] The evaluation results will be accumulated in a database over a long period of time.

[0791] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[0792] Specific examples

[0793] Example of the initial setup phase

[0794] User (Teacher):

[0795] 1. Teachers log in to their dedicated dashboard.

[0796] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[0797] server:

[0798] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[0799] 2. Save the entered keywords in the database.

[0800] A concrete example of the data collection phase

[0801] User (student):

[0802] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[0803] 2. Speaking at specific class times.

[0804] Device:

[0805] 1. Send note input contents to the server in real time.

[0806] 2. Record the speech as audio, convert it into text, and send it to the server.

[0807] server:

[0808] 1. Receive note entries and comments and store them in a database.

[0809] A concrete example of the data analysis phase

[0810] server:

[0811] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[0812] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[0813] 3. Generate a primary evaluation score of "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and save it in the database.

[0814] Example of the evaluation confirmation phase

[0815] User (Teacher):

[0816] 1. Teachers access the dashboard and check the primary assessment results.

[0817] 2. The teacher makes the necessary corrections to the results.

[0818] server:

[0819] 1. Save the corrected evaluation results in the database.

[0820] Specific example of data accumulation and re-learning phase

[0821] server:

[0822] 1. Compile all evaluation data at the end of the semester.

[0823] 2. Retrain the generative AI model using the aggregated data.

[0824] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[0825] In this way, a system for qualitatively evaluating students' thinking and judgment abilities is constructed, realizing efficient and fair evaluation.

[0826] The processing flow will be explained below.

[0827] Step 1:

[0828] User (Teacher):

[0829] Teachers log in to their dedicated dashboard.

[0830] The teacher selects the "Class Settings" tab.

[0831] server:

[0832] Log in and authenticate to start the teacher session.

[0833] Displays the class settings tab and provides a form for entering important keywords.

[0834] Step 2:

[0835] User (Teacher):

[0836] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[0837] server:

[0838] The entered keywords are received and stored in a database.

[0839] Step 3:

[0840] User (student):

[0841] During class, students take notes using a dedicated application.

[0842] Speak up at specific times in class.

[0843] Device:

[0844] It provides a note-taking interface and sends inputs to a server in real time.

[0845] It provides a function to record speech, convert it into text, and send it to the server.

[0846] Step 4:

[0847] server:

[0848] Receive notes and comments sent from the device in real time.

[0849] The received content is temporarily saved and prepared for analysis.

[0850] Step 5:

[0851] server:

[0852] The note descriptions and speech content are passed to a generative AI model for analysis.

[0853] A generative AI model analyzes the content and extracts important keywords.

[0854] A primary evaluation is generated based on the extracted keywords and the results are stored in a database.

[0855] Step 6:

[0856] User (Teacher):

[0857] Teachers access the dashboard to check assessments.

[0858] Device:

[0859] Display the teacher dashboard and provide primary assessment results.

[0860] Step 7:

[0861] User (Teacher):

[0862] Check the results of the initial evaluation and make corrections as necessary.

[0863] Once the corrections are complete, the final evaluation is sent to the server.

[0864] server:

[0865] The corrected evaluation results are received and stored in a database.

[0866] Step 8:

[0867] server:

[0868] All evaluation data will be compiled at the end of the semester.

[0869] The generative AI model is retrained based on the aggregated data.

[0870] Step 9:

[0871] server:

[0872] Prepare to apply the retrained generative AI model to assessment in the next class.

[0873] Implement a model with new evaluation criteria into the system.

[0874] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and an efficient and fair evaluation is realized.

[0875] Example 1

[0876] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0877] To effectively evaluate students' thinking and judgment skills, a system is needed that can analyze notes and comments taken during class in real time and accurately reflect the evaluation results. However, conventional systems have difficulty consistently recording and analyzing notes and comments, which means that fairness and efficiency of evaluation cannot be ensured. Furthermore, there is a lack of a way for teachers to easily check evaluation results and make corrections as necessary. Technology that can solve these issues is needed.

[0878] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0879] In this invention, the server includes means for recording students' notes and comments during class, means for receiving the recorded notes and comments in real time and saving them in a database, means for using a generative AI model that analyzes the recorded notes and comments to extract important keywords, means for inputting prompts to the generative AI model and analyzing the received data, means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model, means for saving the evaluation results in a database and using them for subsequent evaluation and learning, and means for teachers to check and correct the evaluation results. This allows students' learning activities to be recorded and analyzed in real time, enabling fair and efficient evaluation.

[0880] "Notes" are text data that students input into their electronic devices during class to record important information and personal thoughts.

[0881] "Comments" refer to the verbal expressions of opinions and questions that students make during class, and the recordings are then converted into text.

[0882] The "database" is an information storage system for uniformly managing and storing recorded notes, comments, and evaluation results.

[0883] A "generative AI model" is a model that uses artificial intelligence to analyze text data, extract important keywords, and evaluate thinking ability and judgment.

[0884] A "prompt sentence" is a text sentence entered into a generative AI model to instruct it to perform a specific process or analysis.

[0885] The "evaluation results" are the results of a qualitative assessment of students' thinking and judgment abilities based on data analyzed by the generative AI model.

[0886] "Means" refers to specific methods or functions for achieving a certain purpose.

[0887] A "server" is a computer system that processes, stores, and manages data, and has functions such as receiving, analyzing, evaluating, and storing note-taking and comment content.

[0888] A "terminal" is a device that a user directly operates to input and confirm data.

[0889] "User" refers to a person such as a teacher or student who uses this system.

[0890] This invention is a system that records students' notes and comments during class and uses a generative AI model to qualitatively evaluate their thinking and judgment abilities. This system consists of three parties: a server, a terminal, and a user, each of which processes and calculates data using specific hardware and software.

[0891] server

[0892] The server performs the following main roles:

[0893] 1. Login authentication and dashboard provision: Provides an interface for teachers to log in to a dedicated dashboard and authenticates the login information. If login is successful, displays the dashboard for lesson settings and evaluation confirmation.

[0894] 2. Data reception and storage: Students' notes and comments are received in real time and stored in a database.

[0895] 3. Data analysis: Prompt sentences are input into the generative AI model, which analyzes the notes and speech content to extract important keywords.

[0896] 4. Generating and saving evaluations: Based on the extracted keywords, students' thinking and judgment skills are evaluated and the results are saved in the database. If the evaluation results are revised, they are also saved.

[0897] 5. Retraining: Retrain the generative AI model using accumulated evaluation data to improve the accuracy of the evaluation.

[0898] Terminal

[0899] The terminal performs the following main roles:

[0900] 1. Login and settings interface provided: An interface is provided for teachers to log in and enter and save important keywords from the lesson settings tab.

[0901] 2. Note taking and speech recording: Provides an interface for students to take notes and has a function to record speech. This data is sent to the server in real time.

[0902] 3. Displaying assessment results: Provide a dashboard for teachers to view assessment results.

[0903] User

[0904] Users (teachers and students) have the following roles:

[0905] 1. Teachers: Log in to the dedicated dashboard and enter important keywords such as originality and critical thinking in the "Class Settings" tab. Also, check the evaluation results and make corrections as necessary.

[0906] 2. Students: Use electronic devices to take notes and speak during class.

[0907] Hardware and Software Examples

[0908] Server: Lincoln Central Server (Generic Cloud Server), Database System (MySQL)

[0909] Devices: Teacher and student laptops or tablets

[0910] Generative AI models: text analysis software (e.g., OpenAI GPT-3)

[0911] Specific examples and prompts

[0912] Teacher operation example

[0913] 1. Teachers log in to their dedicated dashboard.

[0914] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[0915] 3. Teachers access the dashboard to review the primary assessment results and make corrections as necessary.

[0916] Student operation example

[0917] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[0918] 2. Students will speak during class, and their comments will be recorded audio and converted into text.

[0919] Prompt Sentence Examples

[0920] "Evaluate the students' thinking and judgment based on the text below."

[0921] By implementing such a system, students' learning activities can be recorded and analyzed in real time, enabling fair and efficient evaluation.

[0922] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0923] Step 1:

[0924] User (Teacher):

[0925] Teachers log in to their dedicated dashboard.

[0926] Specific behavior:

[0927] The teacher uses an electronic device to enter a username and password and clicks the login button.

[0928] input:

[0929] Username, Password

[0930] output:

[0931] If the login authentication is successful, the dashboard will be displayed.

[0932] Step 2:

[0933] server:

[0934] Authenticate your login details and view the teacher dashboard.

[0935] Specific behavior:

[0936] The server checks the received username and password against the information in its database. If authentication is successful, the class settings tab is displayed.

[0937] input:

[0938] Username, Password

[0939] output:

[0940] Upon successful login, a dashboard containing the Course Settings tab will be displayed.

[0941] Step 3:

[0942] User (Teacher):

[0943] Select the lesson settings tab, enter important keywords, and save.

[0944] Specific behavior:

[0945] Teachers enter key keywords such as originality, critical thinking, trial and error, and click the save button.

[0946] input:

[0947] Important keywords (e.g., "originality," "critical thinking," "trial and error")

[0948] output:

[0949] A save confirmation message will be displayed.

[0950] Step 4:

[0951] server:

[0952] The entered important keywords are saved in the database.

[0953] Specific behavior:

[0954] The server receives the important keywords entered by the teacher and stores them in a database.

[0955] input:

[0956] Important Keywords

[0957] output:

[0958] Notification that saving to the database is complete.

[0959] Step 5:

[0960] User (student):

[0961] Take notes during class.

[0962] Specific behavior:

[0963] Students use their electronic devices to enter text into a note-taking interface, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[0964] input:

[0965] Note contents

[0966] output:

[0967] The note contents will be displayed on the device.

[0968] Step 6:

[0969] Device:

[0970] The note description input contents are sent to the server in real time.

[0971] Specific behavior:

[0972] The contents entered in the note entry interface are periodically sent to the server.

[0973] input:

[0974] Note contents

[0975] output:

[0976] Data sent to the server.

[0977] Step 7:

[0978] User (student):

[0979] Speak during class.

[0980] Specific behavior:

[0981] Students speak aloud during class.

[0982] input:

[0983] Voice remarks

[0984] output:

[0985] The speech is recorded as audio data.

[0986] Step 8:

[0987] Device:

[0988] Speech is recorded as audio, converted into text and sent to the server.

[0989] Specific behavior:

[0990] The recorded voice data is converted into text using a text conversion engine and sent to the server.

[0991] input:

[0992] Audio data

[0993] output:

[0994] The speech converted to text.

[0995] Step 9:

[0996] server:

[0997] The received note contents and speech content are passed to the generation AI model.

[0998] Specific behavior:

[0999] The server reads the received note descriptions and speech text from the database and prepares them to be passed to the generative AI model.

[1000] input:

[1001] Note writing, speech text

[1002] output:

[1003] Text data input into a generative AI model.

[1004] Step 10:

[1005] server:

[1006] The prompt sentence is fed into a generative AI model to analyze the data.

[1007] Specific behavior:

[1008] The server passes a prompt statement, for example, "Please evaluate the student's thinking ability and judgment based on the text below," to the generative AI model and begins analysis.

[1009] input:

[1010] Note description, speech text, prompt text

[1011] output:

[1012] Important keywords as analysis results.

[1013] Step 11:

[1014] server:

[1015] Students' thinking and judgment skills are evaluated based on the extracted keywords, and the evaluation results are stored in a database.

[1016] Specific behavior:

[1017] Based on the keywords extracted by the generative AI model, an evaluation score is generated and stored in a database.

[1018] input:

[1019] Important Keywords

[1020] output:

[1021] Evaluation scores are stored in a database.

[1022] Step 12:

[1023] User (Teacher):

[1024] Check the assessment results on the dashboard and make corrections as necessary.

[1025] Specific behavior:

[1026] Teachers can access the dashboard, check the assessment results, make corrections if necessary, and click the save button.

[1027] input:

[1028] Evaluation results

[1029] output:

[1030] Corrected evaluation results.

[1031] Step 13:

[1032] server:

[1033] The corrected evaluation results are stored in the database.

[1034] Specific behavior:

[1035] The server receives the teacher-corrected evaluation results and stores them in a database.

[1036] input:

[1037] Corrected evaluation results

[1038] output:

[1039] Notification that saving to the database is complete.

[1040] Step 14:

[1041] server:

[1042] The generative AI model is retrained using accumulated evaluation data.

[1043] Specific behavior:

[1044] The server aggregates the evaluation data at the end of the semester and builds a dataset for retraining the generative AI model. The retraining of the generative AI model is then performed, and the new model is applied to the next semester's evaluations.

[1045] input:

[1046] Evaluation Data

[1047] output:

[1048] Retrained generative AI model.

[1049] The above are the specific processing steps of this system.

[1050] (Application example 1)

[1051] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1052] Factory workplaces require accurate evaluation of worker efficiency and skill levels, and the provision of appropriate feedback and training. However, conventional evaluation methods require cumbersome recording and evaluation criteria, making efficient and objective evaluation difficult. Additionally, there is a lack of means to evaluate skills and efficiency based on worker comments, making comprehensive evaluation difficult.

[1053] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1054] In this invention, the server includes a means for recording work records and speech contents, a means for using a generative AI model that analyzes the recorded records and speech contents to extract important keywords, a means for qualitatively evaluating work efficiency and skill level based on the keywords extracted by the generative AI model, and a means for storing the evaluation results in a database and using them for subsequent evaluation and learning, thereby enabling a comprehensive evaluation of work efficiency and skill level.

[1055] "Work records" are data that records in detail the procedures and results of work performed by workers in a factory.

[1056] "Speech content" refers to data recorded in text format, including verbal communications, questions, and responses made by workers while they were working.

[1057] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze text data, extract important keywords, and understand the text.

[1058] "Key words" are important words or phrases related to specific evaluation metrics extracted by the generative AI model.

[1059] "Work efficiency" is an indicator of how effectively and quickly a worker can perform a given task.

[1060] "Skill level" is an indicator of the degree of technical ability possessed by a worker.

[1061] "Qualitative evaluation" refers to evaluation based on text data and important keywords, rather than on quantified data.

[1062] A "database" is a data storage system that systematically stores collected records and evaluation results so that they can be reused as needed.

[1063] "Future evaluation and learning" refers to future re-evaluation based on the stored data and re-learning to improve the generative AI model.

[1064] MODE FOR CARRYING OUT THE INVENTION

[1065] One embodiment of the present invention provides a system for evaluating the efficiency and skill level of factory workers. This system records work logs and speech, analyzes them using a generative AI model, and stores the evaluation results in a database.

[1066] 1. Initial Setup Phase

[1067] server

[1068] The server provides an interface for administrators to log in, performs login authentication, provides a work setting tab, and displays a form for administrators to input key evaluation indicators, which are then stored in a database.

[1069] Terminal

[1070] The terminal provides an interface for administrators to log in and enter and save evaluation indicators from the work settings tab.

[1071] User (Administrator)

[1072] Administrators log in to a dedicated dashboard and enter and save key evaluation indicators such as "operation accuracy," "efficiency," and "adaptability" in the "Work Settings" tab.

[1073] 2. Data Collection Phase

[1074] server

[1075] The server receives the workers' work records and comments in real time and stores them in a database.

[1076] Terminal

[1077] The terminal provides an interface for workers to input their work, and also provides the function of recording what the workers say and converting it into text.

[1078] User (worker)

[1079] While working, workers input their work details into the electronic device, and also make statements, which are then recorded.

[1080] 3. Data analysis phase

[1081] server

[1082] The server analyzes the received work records and comments using a generative AI model. It extracts important evaluation indicators and evaluates work efficiency and skill level based on these. The evaluation results are stored in a database.

[1083] 4. Evaluation confirmation phase

[1084] Terminal

[1085] The terminal provides a dashboard for administrators to view evaluation results.

[1086] User (Administrator)

[1087] Administrators can check the evaluation results through the dashboard and make corrections as necessary.

[1088] server

[1089] The server stores the modified evaluation results in a database.

[1090] 5. Data accumulation and re-learning phase

[1091] server

[1092] The server accumulates the evaluation results in a database over a long period of time and uses the accumulated data to retrain the generative AI model, thereby improving the accuracy of the evaluation.

[1093] Hardware and software used

[1094] This system uses the following hardware and software:

[1095] Hardware: Factory robots, voice input devices

[1096] Software: databases (e.g., MySQL), speech-to-text tools (e.g., Google Speech-to-Text API), generative AI models (e.g., OpenAI GPT-4)

[1097] Specific examples

[1098] For example, consider a situation where a worker in a particular factory needs to learn how to operate a new machine. The robot records the worker's machine operation logs and conversations, which are then analyzed by a generative AI model. For example, the evaluation result may be that "the efficiency of the new machine operation has improved, but more skill is needed." In this case, the manager can plan additional training based on this. The manager can view the evaluation results in real time on a dashboard.

[1099] Prompt Sentence Examples

[1100] Work log data:

[1101] "The workers operated the CNC machines, adjusted the settings, and completed the assembly of the parts."

[1102] Speech data:

[1103] "Is there a way to tweak this setting to improve the quality of the finished result?"

[1104] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1105] Step 1:

[1106] The administrator logs in.

[1107] The administrator accesses the dedicated dashboard and enters their login information. The server receives the information and performs authentication. If authentication is successful, the dashboard is displayed to the administrator. The input is the administrator's login information, and the output is the administrator's dashboard.

[1108] Step 2:

[1109] Managers set key metrics.

[1110] The administrator selects the "Work Settings" tab on the dashboard, enters evaluation indicators such as "operation accuracy," "efficiency," and "adaptability," and clicks the save button. The terminal receives the input and sends it to the server, which stores the information in a database. The input is the evaluation indicator entered by the administrator, and the output is the evaluation indicator stored in the database.

[1111] Step 3:

[1112] The worker enters the work record.

[1113] While working, workers enter details of the machines they operated and the work they performed into a terminal. The terminal sends the entered data in real time to a server, which stores the data in a database. The input is the work record entered by the worker, and the output is the work record stored in the database.

[1114] Step 4:

[1115] The worker records what is said.

[1116] Conversations and questions asked by workers while they are working are recorded using a voice input device. The device converts the voice data into text and sends it to a server. The server stores the data in a database. The input is the worker's voice data, and the output is the text of what was said.

[1117] Step 5:

[1118] The server passes the data to the generative AI model.

[1119] The server extracts work records and speech content from the database and inputs them into the generative AI model. The generative AI model analyzes the data, extracts important keywords, and evaluates work efficiency and skill level. The input is the work records and speech content, and the output is the extracted keywords and evaluation results.

[1120] Step 6:

[1121] The evaluation results are stored in a database.

[1122] The server stores the evaluation results obtained from the generative AI model in a database. The evaluation results include indicators such as "operation accuracy," "efficiency," and "adaptability." The input is the evaluation results from the generative AI model, and the output is the evaluation results stored in the database.

[1123] Step 7:

[1124] The administrator checks and corrects the evaluation results.

[1125] Administrators can access the dashboard to check the evaluation results and make corrections as necessary. The corrections made by the administrator are sent from the terminal to the server and stored in the database. The input is the corrections made by the administrator, and the output is the corrected evaluation results.

[1126] Step 8:

[1127] Evaluation data is stored for a long period of time and re-learned.

[1128] The server stores the evaluation results in a database for a long period of time. The accumulated data is used to periodically retrain the generative AI model. The input is the accumulated evaluation data, and the output is the retrained generative AI model.

[1129] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1130] System Overview

[1131] The present invention provides a system that records students' notes and comments during class, analyzes them with a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities, and also provides a system that combines an emotion engine. A specific embodiment of this system will be described below.

[1132] 1. Initial Setup Phase

[1133] server:

[1134] It provides an interface for teachers to log in and performs login authentication.

[1135] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[1136] Save important keywords in a database.

[1137] Device:

[1138] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[1139] User (Teacher):

[1140] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[1141] 2. Data Collection Phase

[1142] server:

[1143] Students' notes and comments are received in real time and stored in a database.

[1144] Device:

[1145] Provides an interface for students to enter notes.

[1146] Provides the ability to record student comments and convert them into text.

[1147] User (student):

[1148] Type notes into an electronic device during class.

[1149] Make a statement and record what you say.

[1150] 3. Emotional data collection phase

[1151] Device:

[1152] Students' facial expressions and vocal tones are analyzed in real time using an emotion engine to obtain emotional data.

[1153] The emotion data is sent to the server.

[1154] server:

[1155] Emotion data sent from the device is received and saved in a linked manner with the note description and the content of the remarks.

[1156] 4. Data analysis phase

[1157] server:

[1158] The received notes and statements are passed to a generative AI model for analysis.

[1159] A generative AI model analyzes the content and extracts important keywords.

[1160] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[1161] 5. Evaluation confirmation phase

[1162] Device:

[1163] Provide a dashboard for teachers to view assessment results.

[1164] User (Teacher):

[1165] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[1166] server:

[1167] The corrected evaluation results are stored in the database.

[1168] 6. Data accumulation and re-learning phase

[1169] server:

[1170] The evaluation results will be accumulated in a database over a long period of time.

[1171] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[1172] Specific examples

[1173] Example of the initial setup phase

[1174] User (Teacher):

[1175] 1. Teachers log in to their dedicated dashboard.

[1176] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[1177] server:

[1178] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[1179] 2. Save the entered keywords in the database.

[1180] A concrete example of the data collection phase

[1181] User (student):

[1182] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[1183] 2. Speaking at specific class times.

[1184] Device:

[1185] 1. Send note input contents to the server in real time.

[1186] 2. Record the speech as audio, convert it into text, and send it to the server.

[1187] server:

[1188] 1. Receive note entries and comments and store them in a database.

[1189] A concrete example of the emotion data collection phase

[1190] Device:

[1191] 1. The camera captures students' facial expressions as they speak.

[1192] 2. The emotion engine analyzes this and detects emotions such as "tension," "excitement," and "anxiety."

[1193] 3. The detected emotion data is sent to the server.

[1194] server:

[1195] 1. The received emotion data is saved by linking it to the note description and the speech content.

[1196] A concrete example of the data analysis phase

[1197] server:

[1198] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[1199] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[1200] 3. Correct the result based on the emotional data, and generate a primary evaluation such as "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and store it in the database.

[1201] Example of the evaluation confirmation phase

[1202] User (Teacher):

[1203] 1. Teachers access the dashboard and check the primary assessment results.

[1204] 2. The teacher makes the necessary corrections to the results.

[1205] server:

[1206] 1. Save the corrected evaluation results in the database.

[1207] Specific example of data accumulation and re-learning phase

[1208] server:

[1209] 1. After the semester ends, all evaluation data will be compiled.

[1210] 2. Retrain the generative AI model using the aggregated data.

[1211] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[1212] In this way, by qualitatively evaluating students' thinking and judgment abilities and taking emotional data into consideration, more efficient and fair evaluation becomes possible.

[1213] The processing flow will be explained below.

[1214] Step 1:

[1215] User (Teacher):

[1216] Teachers log in to their dedicated dashboard.

[1217] The teacher selects the "Class Settings" tab.

[1218] server:

[1219] Log in and authenticate to start the teacher session.

[1220] Displays the class settings tab and provides a form for entering important keywords.

[1221] Step 2:

[1222] User (Teacher):

[1223] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[1224] server:

[1225] The entered keywords are received and stored in a database.

[1226] Step 3:

[1227] User (student):

[1228] During class, students take notes using a dedicated application.

[1229] Speak up at specific times in class.

[1230] Device:

[1231] It provides a note-taking interface and sends inputs to a server in real time.

[1232] It provides a function to record speech, convert it into text, and send it to the server.

[1233] Step 4:

[1234] server:

[1235] Receive notes and comments sent from the device in real time.

[1236] The received content is temporarily saved and prepared for analysis.

[1237] Step 5:

[1238] Device:

[1239] The camera captures students' facial expressions and activates an emotion engine that analyzes their voice tone in real time.

[1240] The emotion engine analyzes facial expressions and vocal tone to generate emotion data (e.g., "nervous," "excited," "anxious").

[1241] The emotion data is sent to the server.

[1242] server:

[1243] Emotion data transmitted from the terminal is received.

[1244] Save the notes and link them to the comments.

[1245] Step 6:

[1246] server:

[1247] The note description, speech content, and linked emotional data are passed to a generative AI model for analysis.

[1248] A generative AI model extracts important keywords.

[1249] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[1250] Step 7:

[1251] User (Teacher):

[1252] Teachers access the dashboard to check assessments.

[1253] Device:

[1254] Display the teacher dashboard and provide primary assessment results.

[1255] Step 8:

[1256] User (Teacher):

[1257] Check the results of the initial evaluation and make corrections as necessary.

[1258] Once the corrections are complete, the final evaluation is sent to the server.

[1259] server:

[1260] The corrected evaluation results are received and stored in a database.

[1261] Step 9:

[1262] server:

[1263] All evaluation data will be compiled at the end of the semester.

[1264] The generative AI model is retrained using the aggregated data.

[1265] Step 10:

[1266] server:

[1267] Prepare to apply the retrained generative AI model to assessment in the next class.

[1268] Implement a model with new evaluation criteria into the system.

[1269] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and by taking emotional data into consideration, more efficient and fair evaluation is realized.

[1270] Example 2

[1271] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1272] Conventional student evaluation systems evaluate students based solely on their notes and comments, making it difficult to accurately assess their thinking and judgment abilities. Furthermore, they do not take emotional data into account, making it difficult to provide fair evaluations that reflect students' psychological state and emotional changes. For these reasons, there has been a demand for a more accurate and fair evaluation method.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1274] In this invention, the server includes a means for recording students' notes and comments during class, a means for analyzing the recorded notes and comments and using a generative AI model to extract important keywords, a means for analyzing the students' facial expressions and voice tones during evaluation to obtain emotional data, and a means for qualitatively evaluating the students' thinking ability and judgment ability based on the keywords extracted by the generative AI model. This allows for more accurate evaluation of students' thinking ability and judgment ability, and by taking emotional data into consideration, enables fair and comprehensive evaluation.

[1275] "Note writing" is text data that students input using electronic devices during class.

[1276] "Speech" refers to the verbal content of a student's speech during class, and the content is recorded as audio or converted into text data.

[1277] A "generative AI model" is an artificial intelligence model that analyzes text data and extracts important keywords. For example, it includes models that use natural language processing technology.

[1278] "Important keywords" are particularly noteworthy words and phrases extracted by the generative AI model from students' notes and comments.

[1279] "Thinking ability" refers to the logical thinking skills that students use to solve problems, and specifically includes originality and critical thinking.

[1280] "Judgment" is a student's ability to take appropriate action or make appropriate decisions depending on the situation.

[1281] "Emotional data" refers to data that indicates the emotional state of a student, obtained by analyzing their facial expressions and vocal tones. For example, it includes information on tension, excitement, anxiety, etc.

[1282] A "database" is a system for centrally managing and storing recorded data.

[1283] "Retraining" is the process of using accumulated data to improve the accuracy of a generative AI model.

[1284] "Evaluation results" are evaluation data on students' thinking and judgment abilities generated based on the generative AI model and emotional data.

[1285] A "teacher" is a person who is responsible for teaching and assessing students at an educational institution.

[1286] The present invention is a system that records students' notes and comments in class in real time, extracts important keywords using a generative AI model, and further combines them with emotional data to qualitatively evaluate students' thinking and judgment abilities. An embodiment of this system will be described in detail.

[1287] Hardware and software used

[1288] Server: Database management system, generative AI model (e.g., model using natural language processing technology), login authentication system, evaluation result management system

[1289] Device: Camera, microphone, note input interface, facial expression analysis engine (e.g. emotion engine), voice recognition API

[1290] Users: Electronic devices (tablets and PCs) used by teachers and students

[1291] Specific operation of the system

[1292] Initial Setup Phase

[1293] User (Teacher):

[1294] 1. The teacher logs in to the dedicated dashboard and enters their username and password for authentication.

[1295] 2. After logging in, select the "Class Settings" tab, enter the keywords you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the Save button.

[1296] server:

[1297] Receives the teacher's authentication information and verifies the login via the authentication server. If the login is successful, it provides the lesson settings tab interface.

[1298] Important keywords entered by the teacher are saved in a database.

[1299] Data Collection Phase

[1300] User (student):

[1301] Students typing notes into electronic devices during class.

[1302] Students will speak during class and their comments will be recorded.

[1303] Device:

[1304] It provides an interface for note entry and sends the note contents entered by students to the server in real time.

[1305] It uses a speech recognition API to record what you say and convert it into text, which is then sent to the server.

[1306] server:

[1307] The received note contents and speech contents are saved in a database.

[1308] Emotional data collection phase

[1309] Device:

[1310] A camera captures students' facial expressions as they speak.

[1311] The emotion engine is used to analyze facial expressions and voice tones to detect emotion data (e.g., "tension," "excitement," "anxiety"), and transmit the detected emotion data to the server.

[1312] server:

[1313] The received emotion data is saved by linking it to the note description and the content of the speech.

[1314] Data analysis phase

[1315] server:

[1316] The received note content and speech content are passed to a generative AI model, which uses natural language processing technology such as OpenAI's GPT-3.

[1317] Send the following prompt to the generative AI model: "Analyze the student's notebook entry, 'We held a discussion about the impact of global warming on society, taking into account new perspectives,' and extract key keywords. Additionally, take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error."

[1318] The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database.

[1319] Evaluation confirmation phase

[1320] User (Teacher):

[1321] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page.

[1322] Check the results of the primary evaluation and make any necessary corrections. Once the corrections are complete, click the Save button.

[1323] server:

[1324] The corrections made by the teacher are saved in a database.

[1325] Data accumulation and re-learning phase

[1326] server:

[1327] After each semester, all evaluation data is compiled and stored in a database.

[1328] The generated AI model will be retrained using the accumulated data to improve the accuracy of the evaluation algorithm, and the retrained model will be applied to class evaluations from the next semester.

[1329] Specific examples

[1330] In the initial setup phase, the teacher inputs and saves key keywords such as "originality," "critical thinking," and "trial and error." In the data collection phase, students write in their notebooks, "We held a discussion that considered new perspectives on the impact of global warming on society." In the emotion data collection phase, the students' facial expressions while speaking are captured and analyzed with an emotion engine to detect "tension." In the data analysis phase, the generative AI model extracts "new perspectives" and "discussion" and generates an initial evaluation of "originality: high, critical thinking: medium, trial and error: medium." In the evaluation confirmation phase, the teacher checks the results of the initial evaluation and makes any necessary corrections.

[1331] The present invention can be implemented according to the specific steps set forth in the above detailed description.

[1332] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1333] The specific processing flow of this system program

[1334] Step 1:

[1335] User (Teacher):

[1336] The teacher logs in to the dedicated dashboard. Enter the username and password and click the login button.

[1337] server:

[1338] Receives the teacher's authentication information and verifies the login via the authentication server (input: username, password, output: authentication result). If the login is successful, the lesson settings tab interface is provided.

[1339] Step 2:

[1340] User (Teacher):

[1341] Select the "Class Settings" tab, enter keywords that you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the save button.

[1342] Device:

[1343] It provides a keyword input interface and sends the input keywords to the server (input: important keywords, output: transmission signal).

[1344] server:

[1345] Receives keyword input from teachers and stores it in a database (input: important keywords, output: stored results).

[1346] Step 3:

[1347] User (student):

[1348] A student takes notes on an electronic device during class, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[1349] Device:

[1350] It provides a note input interface and transmits the input note content to the server in real time (input: note content, output: transmission signal).

[1351] server:

[1352] Save the received note contents to the database (input: note contents, output: saved result).

[1353] Step 4:

[1354] User (student):

[1355] He speaks up during class, saying things like, "We should think about environmental issues from a new policy perspective."

[1356] Device:

[1357] The speech is recorded as audio and converted into text using a speech recognition API. The converted text is sent to the server (input: audio data, output: transmission signal).

[1358] server:

[1359] Save the received comment content in the database (input: comment content, output: saved result).

[1360] Step 5:

[1361] Device:

[1362] When a student speaks, the camera captures their facial expressions, and the emotion engine analyzes their facial expressions and vocal tone to detect emotional data (e.g., "nervous," "excited," "anxious") and transmits it to the server (input: facial expressions, vocal data, output: emotional data).

[1363] server:

[1364] The received emotion data is linked to the note description and the speech content and saved (input: emotion data, output: saved result).

[1365] Step 6:

[1366] server:

[1367] The received note content and speech content are passed to the generative AI model (input: note content, speech content, output: prompt). For example, a prompt such as "Analyze the student's note, 'We held a discussion on the impact of global warming on society, taking into account new perspectives,' and extract important keywords. In addition, please take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error" is sent to the generative AI model. The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database (input: generative AI model analysis results, emotional data, output: primary evaluation).

[1368] Step 7:

[1369] User (Teacher):

[1370] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page. Check the primary evaluation results and make corrections as necessary (input: primary evaluation results, corrections, output: final evaluation results).

[1371] server:

[1372] The teacher's corrections are saved in the database (input: final evaluation result, output: saved result).

[1373] Step 8:

[1374] server:

[1375] After each semester, all evaluation data is aggregated and stored in a database (input: each evaluation data, output: stored data). The stored data is used to retrain the generative AI model (input: stored data, output: retrained model). The retrained model is applied to class evaluations from the next semester onwards (input: retrained model, output: application results).

[1376] The above is the specific processing flow of this system.

[1377] (Application example 2)

[1378] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1379] In addition to a system for efficiently evaluating students' thinking and judgment skills, there is a need for a method for qualitatively evaluating the performance and processing capabilities of factory robots by collecting and analyzing their work logs and sensor data in real time. This will enable fairer and more efficient evaluations in both educational and production settings.

[1380] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1381] In this invention, the server includes: means for recording students' notes and comments during class; means for using a generative AI model to analyze the recorded notes and comments and extract important keywords; means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model; means for storing the evaluation results in a database and using them for subsequent evaluation and learning; means for collecting robot work logs and sensor data in real time and evaluating them using the generative AI model; and means for qualitatively evaluating the robot's performance and processing capabilities based on the sensor data analyzed in real time. This makes it possible to highly evaluate both student learning effectiveness and robot work efficiency.

[1382] "Note taking" refers to handwritten or electronic notes or records that students make during class.

[1383] "Content of remarks" refers to the words and opinions expressed by students during class.

[1384] A "generative AI model" is an artificial intelligence model that extracts and analyzes useful information from input data.

[1385] "Important keywords" refer to words or phrases that are particularly meaningful in the recorded notes or statements.

[1386] "Thinking ability" refers to the intellectual abilities necessary for problem-solving and generating new ideas.

[1387] "Judgment" is the ability to choose the best option depending on the situation.

[1388] "Evaluation results" are the evaluation values ​​of the student's or robot's performance after analysis using the generative AI model.

[1389] A "database" is a computer system for systematically storing, retrieving, and managing digital information.

[1390] A "work log" is a detailed record of the work a robot performs.

[1391] "Sensor data" refers to digital data obtained from sensors that measure the operating conditions of a robot.

[1392] "Performance" refers to the efficiency and accuracy with which a robot carries out a task.

[1393] "Processing capacity" refers to how effectively a robot can perform multiple tasks.

[1394] System Overview

[1395] This invention is a system that qualitatively evaluates students' thinking and judgment skills, as well as robot performance and processing capabilities, by collecting students' note-taking and comments during class, factory robot work logs, and sensor data in real time and analyzing them with a generative AI model.

[1396] 1. Initial Setup Phase

[1397] The server provides an interface for teachers and factory managers to log in and performs login authentication. After login authentication is complete, it displays the lesson settings tab and work settings tab, and provides a form for entering important keywords and evaluation criteria (e.g., efficiency, accuracy, flexibility). The entered evaluation criteria are saved in a database.

[1398] The terminal provides an interface for teachers and factory managers to log in and input and save evaluation criteria.

[1399] Users (teachers or factory managers) log in to a dedicated dashboard, enter important keywords and evaluation criteria in the "Class Settings" or "Work Settings" tab, and save them.

[1400] 2. Data Collection Phase

[1401] The server receives students' notes and comments, as well as the robot's work logs and sensor data in real time, stores them in a database, and prepares them as input for the generative AI model.

[1402] The device provides an interface for students to input notes and a function to record the robot's work log. It also has the ability to record student comments and the robot's operation status and convert them into text as needed.

[1403] During class, users (students or robots) take notes and make comments on electronic devices. The robots perform assigned tasks and send their work logs and sensor data to the terminal.

[1404] 3. Emotional data collection phase

[1405] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data, which is then sent to a server.

[1406] The server receives the emotional data sent from the device and stores it, linking it with note descriptions, statements, the robot's work log, and sensor data.

[1407] 4. Data analysis phase

[1408] The server passes the received note entries, comments, robot work logs, and sensor data to the generative AI model for analysis. The generative AI model analyzes the content and extracts important keywords and evaluation criteria. It generates a primary evaluation based on the extracted keywords and criteria and stores the results in a database.

[1409] 5. Evaluation confirmation phase

[1410] The devices provide a dashboard for teachers and factory managers to view assessment results.

[1411] Users (teachers and factory managers) can check the results of the initial assessment through the dashboard and make corrections as necessary.

[1412] The server stores the modified evaluation results in a database.

[1413] 6. Data accumulation and re-learning phase

[1414] The server accumulates the evaluation results in a database over a long period of time, and uses the accumulated data to retrain the generative AI model and improve the accuracy of the evaluation.

[1415] Specific examples

[1416] Example of the initial setup phase

[1417] The user (factory manager) logs in to the dedicated dashboard, selects the "Work Settings" tab, and enters and saves evaluation criteria such as "Efficiency," "Accuracy," and "Flexibility." After authenticating the manager's login, the server displays the work settings tab and saves the entered evaluation criteria in the database.

[1418] A concrete example of the data collection phase

[1419] The user (robot) inputs "Completed assembly of part A and adjusted the position of part B" into the work log interface. The terminal sends the work log interface to the server in real time. The server receives the work log and sensor data and stores them in a database.

[1420] A concrete example of the emotion data collection phase

[1421] The device captures the robot's movements as it assembles parts, and the emotion engine analyzes them to detect emotions such as "stress" or "relaxation." The detected emotion data is then sent to a server, which then links the received emotion data with work logs and sensor data and stores it.

[1422] A concrete example of the data analysis phase

[1423] The server passes the received work log, "Assembly of part A completed, position of part B adjusted," to the generative AI model. The generative AI model analyzes the content and detects "assembly" and "adjustment." It then makes corrections based on the emotional data, generating a primary evaluation, for example, "Efficiency: High, Accuracy: Medium, Flexibility: Medium," and stores this in the database.

[1424] Example of the evaluation confirmation phase

[1425] The user (factory manager) accesses the dashboard and checks the primary evaluation results. The manager makes any necessary corrections to the results. The server saves the corrected evaluation results in the database.

[1426] Specific example of data accumulation and re-learning phase

[1427] After the end of the semester, the server aggregates all evaluation data. The aggregated data is used to retrain the generative AI model. The model with the new evaluation criteria is then applied to the next semester's class evaluations.

[1428] Prompt Sentence Examples

[1429] "Based on the following work log, rate the robot's efficiency, accuracy, and flexibility.

[1430] Work log: [Log data]

[1431] Sensor Data: [Sensor Data]"

[1432] In this way, the present invention can highly evaluate the thinking ability and judgment ability of students and the working efficiency of robots.

[1433] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1434] Step 1: Initial Setup Phase

[1435] The server provides an interface for the administrator to log in and performs authentication. After successful login, it displays a form for the administrator to input evaluation criteria (efficiency, accuracy, flexibility, etc.). It receives the evaluation criteria entered by the administrator and stores them in a database.

[1436] Input: Administrator login information and evaluation criteria

[1437] Output: Authentication results and stored metrics data

[1438] Step 2: Data collection phase

[1439] The terminal provides an interface for students to input notes and an interface for robots to input work logs. Students and robots input information into the interface, and the terminal transmits this information in real time to a server. The server stores the received data in a database.

[1440] Input: Student notes, robot work log

[1441] Output: Input data stored in a database

[1442] Step 3: Emotional data collection phase

[1443] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data. The emotional data is then sent to a server, which links it to notes and work logs and stores it in a database.

[1444] Input: Student's facial expressions, voice tone, robot's movement data

[1445] Output: Emotion data and linked notes and work logs

[1446] Step 4: Data analysis phase

[1447] The server passes the notes, comments, work logs, sensor data, and emotion data stored in the database to the generative AI model for analysis. The generative AI model extracts important keywords and evaluation criteria from this data and generates a primary evaluation. The evaluation results are stored in the database.

[1448] Input: Note writing, speech content, work log, sensor data, emotion data

[1449] Output: Analysis results and primary evaluation by the generative AI model

[1450] Step 5: Evaluation confirmation phase

[1451] The terminal provides a dashboard for teachers and factory managers to check the evaluation results. Users can check the primary evaluation results through the dashboard and make corrections as necessary. The server then stores the corrected evaluation results back in the database.

[1452] Input: Primary assessment results, teacher and administrator feedback

[1453] Output: Corrected evaluation result

[1454] Step 6: Data accumulation and retraining phase

[1455] The server accumulates the evaluation results in a database over a long period of time. The accumulated data is used to retrain the generative AI model, improving the model's evaluation accuracy. The model with the new evaluation criteria is used in the next cycle.

[1456] Input: Accumulated evaluation data

[1457] Output: The retrained generative AI model and its evaluation criteria

[1458] By using the above steps, the present invention can highly evaluate the thinking ability and judgment ability of students and the work efficiency of robots.

[1459] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1460] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1461] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1462] [Third embodiment]

[1463] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1464] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1465] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1466] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1467] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1468] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1469] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1470] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1471] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1472] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1473] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1474] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1475] System Overview

[1476] The present invention is a system that records students' note taking and comments made during class, analyzes them using a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities. The following describes in detail an embodiment of this system.

[1477] 1. Initial Setup Phase

[1478] server:

[1479] It provides an interface for teachers to log in and performs login authentication.

[1480] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[1481] Save important keywords in a database.

[1482] Device:

[1483] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[1484] User (Teacher):

[1485] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[1486] 2. Data Collection Phase

[1487] server:

[1488] Students' notes and comments are received in real time and stored in a database.

[1489] Device:

[1490] Provides an interface for students to enter notes.

[1491] Provides the ability to record student comments and convert them into text.

[1492] User (student):

[1493] Type notes into an electronic device during class.

[1494] Make a statement and record what you say.

[1495] 3. Data analysis phase

[1496] server:

[1497] The received notes and statements are analyzed using a generative AI model.

[1498] Important keywords are extracted and students' thinking and judgment skills are evaluated based on these.

[1499] The primary evaluation results are stored in a database.

[1500] 4. Evaluation confirmation phase

[1501] Device:

[1502] Provide a dashboard for teachers to view assessment results.

[1503] User (Teacher):

[1504] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[1505] server:

[1506] The corrected evaluation results are stored in the database.

[1507] 5. Data accumulation and re-learning phase

[1508] server:

[1509] The evaluation results will be accumulated in a database over a long period of time.

[1510] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[1511] Specific examples

[1512] Example of the initial setup phase

[1513] User (Teacher):

[1514] 1. Teachers log in to their dedicated dashboard.

[1515] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[1516] server:

[1517] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[1518] 2. Save the entered keywords in the database.

[1519] A concrete example of the data collection phase

[1520] User (student):

[1521] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[1522] 2. Speaking at specific class times.

[1523] Device:

[1524] 1. Send note input contents to the server in real time.

[1525] 2. Record the speech as audio, convert it into text, and send it to the server.

[1526] server:

[1527] 1. Receive note entries and comments and store them in a database.

[1528] A concrete example of the data analysis phase

[1529] server:

[1530] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[1531] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[1532] 3. Generate a primary evaluation score of "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and save it in the database.

[1533] Example of the evaluation confirmation phase

[1534] User (Teacher):

[1535] 1. Teachers access the dashboard and check the primary assessment results.

[1536] 2. The teacher makes the necessary corrections to the results.

[1537] server:

[1538] 1. Save the corrected evaluation results in the database.

[1539] Specific example of data accumulation and re-learning phase

[1540] server:

[1541] 1. Compile all evaluation data at the end of the semester.

[1542] 2. Retrain the generative AI model using the aggregated data.

[1543] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[1544] In this way, a system for qualitatively evaluating students' thinking and judgment abilities is constructed, realizing efficient and fair evaluation.

[1545] The processing flow will be explained below.

[1546] Step 1:

[1547] User (Teacher):

[1548] Teachers log in to their dedicated dashboard.

[1549] The teacher selects the "Class Settings" tab.

[1550] server:

[1551] Log in and authenticate to start the teacher session.

[1552] Displays the class settings tab and provides a form for entering important keywords.

[1553] Step 2:

[1554] User (Teacher):

[1555] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[1556] server:

[1557] The entered keywords are received and stored in a database.

[1558] Step 3:

[1559] User (student):

[1560] During class, students take notes using a dedicated application.

[1561] Speak up at specific times in class.

[1562] Device:

[1563] It provides a note-taking interface and sends inputs to a server in real time.

[1564] It provides a function to record speech, convert it into text, and send it to the server.

[1565] Step 4:

[1566] server:

[1567] Receive notes and comments sent from the device in real time.

[1568] The received content is temporarily saved and prepared for analysis.

[1569] Step 5:

[1570] server:

[1571] The note descriptions and speech content are passed to a generative AI model for analysis.

[1572] A generative AI model analyzes the content and extracts important keywords.

[1573] A primary evaluation is generated based on the extracted keywords and the results are stored in a database.

[1574] Step 6:

[1575] User (Teacher):

[1576] Teachers access the dashboard to check assessments.

[1577] Device:

[1578] Display the teacher dashboard and provide primary assessment results.

[1579] Step 7:

[1580] User (Teacher):

[1581] Check the results of the initial evaluation and make corrections as necessary.

[1582] Once the corrections are complete, the final evaluation is sent to the server.

[1583] server:

[1584] The corrected evaluation results are received and stored in a database.

[1585] Step 8:

[1586] server:

[1587] All evaluation data will be compiled at the end of the semester.

[1588] The generative AI model is retrained based on the aggregated data.

[1589] Step 9:

[1590] server:

[1591] Prepare to apply the retrained generative AI model to assessment in the next class.

[1592] Implement a model with new evaluation criteria into the system.

[1593] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and an efficient and fair evaluation is realized.

[1594] Example 1

[1595] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1596] To effectively evaluate students' thinking and judgment skills, a system is needed that can analyze notes and comments taken during class in real time and accurately reflect the evaluation results. However, conventional systems have difficulty consistently recording and analyzing notes and comments, which means that fairness and efficiency of evaluation cannot be ensured. Furthermore, there is a lack of a way for teachers to easily check evaluation results and make corrections as necessary. Technology that can solve these issues is needed.

[1597] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1598] In this invention, the server includes means for recording students' notes and comments during class, means for receiving the recorded notes and comments in real time and saving them in a database, means for using a generative AI model that analyzes the recorded notes and comments to extract important keywords, means for inputting prompts to the generative AI model and analyzing the received data, means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model, means for saving the evaluation results in a database and using them for subsequent evaluation and learning, and means for teachers to check and correct the evaluation results. This allows students' learning activities to be recorded and analyzed in real time, enabling fair and efficient evaluation.

[1599] "Notes" are text data that students input into their electronic devices during class to record important information and personal thoughts.

[1600] "Comments" refer to the verbal expressions of opinions and questions that students make during class, and the recordings are then converted into text.

[1601] The "database" is an information storage system for uniformly managing and storing recorded notes, comments, and evaluation results.

[1602] A "generative AI model" is a model that uses artificial intelligence to analyze text data, extract important keywords, and evaluate thinking ability and judgment.

[1603] A "prompt sentence" is a text sentence entered into a generative AI model to instruct it to perform a specific process or analysis.

[1604] The "evaluation results" are the results of a qualitative assessment of students' thinking and judgment abilities based on data analyzed by the generative AI model.

[1605] "Means" refers to specific methods or functions for achieving a certain purpose.

[1606] A "server" is a computer system that processes, stores, and manages data, and has functions such as receiving, analyzing, evaluating, and storing note-taking and comment content.

[1607] A "terminal" is a device that a user directly operates to input and confirm data.

[1608] "User" refers to a person such as a teacher or student who uses this system.

[1609] This invention is a system that records students' notes and comments during class and uses a generative AI model to qualitatively evaluate their thinking and judgment abilities. This system consists of three parties: a server, a terminal, and a user, each of which processes and calculates data using specific hardware and software.

[1610] server

[1611] The server performs the following main roles:

[1612] 1. Login authentication and dashboard provision: Provides an interface for teachers to log in to a dedicated dashboard and authenticates the login information. If login is successful, displays the dashboard for lesson settings and evaluation confirmation.

[1613] 2. Data reception and storage: Students' notes and comments are received in real time and stored in a database.

[1614] 3. Data analysis: Prompt sentences are input into the generative AI model, which analyzes the notes and speech content to extract important keywords.

[1615] 4. Generating and saving evaluations: Based on the extracted keywords, students' thinking and judgment skills are evaluated and the results are saved in the database. If the evaluation results are revised, they are also saved.

[1616] 5. Retraining: Retrain the generative AI model using accumulated evaluation data to improve the accuracy of the evaluation.

[1617] Terminal

[1618] The terminal performs the following main roles:

[1619] 1. Login and settings interface provided: An interface is provided for teachers to log in and enter and save important keywords from the lesson settings tab.

[1620] 2. Note taking and speech recording: Provides an interface for students to take notes and has a function to record speech. This data is sent to the server in real time.

[1621] 3. Displaying assessment results: Provide a dashboard for teachers to view assessment results.

[1622] User

[1623] Users (teachers and students) have the following roles:

[1624] 1. Teachers: Log in to the dedicated dashboard and enter important keywords such as originality and critical thinking in the "Class Settings" tab. Also, check the evaluation results and make corrections as necessary.

[1625] 2. Students: Use electronic devices to take notes and speak during class.

[1626] Hardware and Software Examples

[1627] Server: Lincoln Central Server (Generic Cloud Server), Database System (MySQL)

[1628] Devices: Teacher and student laptops or tablets

[1629] Generative AI models: text analysis software (e.g., OpenAI GPT-3)

[1630] Specific examples and prompts

[1631] Teacher operation example

[1632] 1. Teachers log in to their dedicated dashboard.

[1633] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[1634] 3. Teachers access the dashboard to review the primary assessment results and make corrections as necessary.

[1635] Student operation example

[1636] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[1637] 2. Students will speak during class, and their comments will be recorded audio and converted into text.

[1638] Prompt Sentence Examples

[1639] "Evaluate the students' thinking and judgment based on the text below."

[1640] By implementing such a system, students' learning activities can be recorded and analyzed in real time, enabling fair and efficient evaluation.

[1641] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1642] Step 1:

[1643] User (Teacher):

[1644] Teachers log in to their dedicated dashboard.

[1645] Specific behavior:

[1646] The teacher uses an electronic device to enter a username and password and clicks the login button.

[1647] input:

[1648] Username, Password

[1649] output:

[1650] If the login authentication is successful, the dashboard will be displayed.

[1651] Step 2:

[1652] server:

[1653] Authenticate your login details and view the teacher dashboard.

[1654] Specific behavior:

[1655] The server checks the received username and password against the information in its database. If authentication is successful, the class settings tab is displayed.

[1656] input:

[1657] Username, Password

[1658] output:

[1659] Upon successful login, a dashboard containing the Course Settings tab will be displayed.

[1660] Step 3:

[1661] User (Teacher):

[1662] Select the lesson settings tab, enter important keywords, and save.

[1663] Specific behavior:

[1664] Teachers enter key keywords such as originality, critical thinking, trial and error, and click the save button.

[1665] input:

[1666] Important keywords (e.g., "originality," "critical thinking," "trial and error")

[1667] output:

[1668] A save confirmation message will be displayed.

[1669] Step 4:

[1670] server:

[1671] The entered important keywords are saved in the database.

[1672] Specific behavior:

[1673] The server receives the important keywords entered by the teacher and stores them in a database.

[1674] input:

[1675] Important Keywords

[1676] output:

[1677] Notification that saving to the database is complete.

[1678] Step 5:

[1679] User (student):

[1680] Take notes during class.

[1681] Specific behavior:

[1682] Students use their electronic devices to enter text into a note-taking interface, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[1683] input:

[1684] Note contents

[1685] output:

[1686] The note contents will be displayed on the device.

[1687] Step 6:

[1688] Device:

[1689] The note description input contents are sent to the server in real time.

[1690] Specific behavior:

[1691] The contents entered in the note entry interface are periodically sent to the server.

[1692] input:

[1693] Note contents

[1694] output:

[1695] Data sent to the server.

[1696] Step 7:

[1697] User (student):

[1698] Speak during class.

[1699] Specific behavior:

[1700] Students speak aloud during class.

[1701] input:

[1702] Voice remarks

[1703] output:

[1704] The speech is recorded as audio data.

[1705] Step 8:

[1706] Device:

[1707] Speech is recorded as audio, converted into text and sent to the server.

[1708] Specific behavior:

[1709] The recorded voice data is converted into text using a text conversion engine and sent to the server.

[1710] input:

[1711] Audio data

[1712] output:

[1713] The speech converted to text.

[1714] Step 9:

[1715] server:

[1716] The received note contents and speech content are passed to the generation AI model.

[1717] Specific behavior:

[1718] The server reads the received note descriptions and speech text from the database and prepares them to be passed to the generative AI model.

[1719] input:

[1720] Note writing, speech text

[1721] output:

[1722] Text data input into a generative AI model.

[1723] Step 10:

[1724] server:

[1725] The prompt sentence is fed into a generative AI model to analyze the data.

[1726] Specific behavior:

[1727] The server passes a prompt statement, for example, "Please evaluate the student's thinking ability and judgment based on the text below," to the generative AI model and begins analysis.

[1728] input:

[1729] Note description, speech text, prompt text

[1730] output:

[1731] Important keywords as analysis results.

[1732] Step 11:

[1733] server:

[1734] Students' thinking and judgment skills are evaluated based on the extracted keywords, and the evaluation results are stored in a database.

[1735] Specific behavior:

[1736] Based on the keywords extracted by the generative AI model, an evaluation score is generated and stored in a database.

[1737] input:

[1738] Important Keywords

[1739] output:

[1740] Evaluation scores are stored in a database.

[1741] Step 12:

[1742] User (Teacher):

[1743] Check the assessment results on the dashboard and make corrections as necessary.

[1744] Specific behavior:

[1745] Teachers can access the dashboard, check the assessment results, make corrections if necessary, and click the save button.

[1746] input:

[1747] Evaluation results

[1748] output:

[1749] Corrected evaluation results.

[1750] Step 13:

[1751] server:

[1752] The corrected evaluation results are stored in the database.

[1753] Specific behavior:

[1754] The server receives the teacher-corrected evaluation results and stores them in a database.

[1755] input:

[1756] Corrected evaluation results

[1757] output:

[1758] Notification that saving to the database is complete.

[1759] Step 14:

[1760] server:

[1761] The generative AI model is retrained using accumulated evaluation data.

[1762] Specific behavior:

[1763] The server aggregates the evaluation data at the end of the semester and builds a dataset for retraining the generative AI model. The retraining of the generative AI model is then performed, and the new model is applied to the next semester's evaluations.

[1764] input:

[1765] Evaluation Data

[1766] output:

[1767] Retrained generative AI model.

[1768] The above are the specific processing steps of this system.

[1769] (Application example 1)

[1770] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1771] Factory workplaces require accurate evaluation of worker efficiency and skill levels, and the provision of appropriate feedback and training. However, conventional evaluation methods require cumbersome recording and evaluation criteria, making efficient and objective evaluation difficult. Additionally, there is a lack of means to evaluate skills and efficiency based on worker comments, making comprehensive evaluation difficult.

[1772] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1773] In this invention, the server includes a means for recording work records and speech contents, a means for using a generative AI model that analyzes the recorded records and speech contents to extract important keywords, a means for qualitatively evaluating work efficiency and skill level based on the keywords extracted by the generative AI model, and a means for storing the evaluation results in a database and using them for subsequent evaluation and learning, thereby enabling a comprehensive evaluation of work efficiency and skill level.

[1774] "Work records" are data that records in detail the procedures and results of work performed by workers in a factory.

[1775] "Speech content" refers to data recorded in text format, including verbal communications, questions, and responses made by workers while they were working.

[1776] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze text data, extract important keywords, and understand the text.

[1777] "Key words" are important words or phrases related to specific evaluation metrics extracted by the generative AI model.

[1778] "Work efficiency" is an indicator of how effectively and quickly a worker can perform a given task.

[1779] "Skill level" is an indicator of the degree of technical ability possessed by a worker.

[1780] "Qualitative evaluation" refers to evaluation based on text data and important keywords, rather than on quantified data.

[1781] A "database" is a data storage system that systematically stores collected records and evaluation results so that they can be reused as needed.

[1782] "Future evaluation and learning" refers to future re-evaluation based on the stored data and re-learning to improve the generative AI model.

[1783] MODE FOR CARRYING OUT THE INVENTION

[1784] One embodiment of the present invention provides a system for evaluating the efficiency and skill level of factory workers. This system records work logs and speech, analyzes them using a generative AI model, and stores the evaluation results in a database.

[1785] 1. Initial Setup Phase

[1786] server

[1787] The server provides an interface for administrators to log in, performs login authentication, provides a work setting tab, and displays a form for administrators to input key evaluation indicators, which are then stored in a database.

[1788] Terminal

[1789] The terminal provides an interface for administrators to log in and enter and save evaluation indicators from the work settings tab.

[1790] User (Administrator)

[1791] Administrators log in to a dedicated dashboard and enter and save key evaluation indicators such as "operation accuracy," "efficiency," and "adaptability" in the "Work Settings" tab.

[1792] 2. Data Collection Phase

[1793] server

[1794] The server receives the workers' work records and comments in real time and stores them in a database.

[1795] Terminal

[1796] The terminal provides an interface for workers to input their work, and also provides the function of recording what the workers say and converting it into text.

[1797] User (worker)

[1798] While working, workers input their work details into the electronic device, and also make statements, which are then recorded.

[1799] 3. Data analysis phase

[1800] server

[1801] The server analyzes the received work records and comments using a generative AI model. It extracts important evaluation indicators and evaluates work efficiency and skill level based on these. The evaluation results are stored in a database.

[1802] 4. Evaluation confirmation phase

[1803] Terminal

[1804] The terminal provides a dashboard for administrators to view evaluation results.

[1805] User (Administrator)

[1806] Administrators can check the evaluation results through the dashboard and make corrections as necessary.

[1807] server

[1808] The server stores the modified evaluation results in a database.

[1809] 5. Data accumulation and re-learning phase

[1810] server

[1811] The server accumulates the evaluation results in a database over a long period of time and uses the accumulated data to retrain the generative AI model, thereby improving the accuracy of the evaluation.

[1812] Hardware and software used

[1813] This system uses the following hardware and software:

[1814] Hardware: Factory robots, voice input devices

[1815] Software: databases (e.g., MySQL), speech-to-text tools (e.g., Google Speech-to-Text API), generative AI models (e.g., OpenAI GPT-4)

[1816] Specific examples

[1817] For example, consider a situation where a worker in a particular factory needs to learn how to operate a new machine. The robot records the worker's machine operation logs and conversations, which are then analyzed by a generative AI model. For example, the evaluation result may be that "the efficiency of the new machine operation has improved, but more skill is needed." In this case, the manager can plan additional training based on this. The manager can view the evaluation results in real time on a dashboard.

[1818] Prompt Sentence Examples

[1819] Work log data:

[1820] "The workers operated the CNC machines, adjusted the settings, and completed the assembly of the parts."

[1821] Speech data:

[1822] "Is there a way to tweak this setting to improve the quality of the finished result?"

[1823] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1824] Step 1:

[1825] The administrator logs in.

[1826] The administrator accesses the dedicated dashboard and enters their login information. The server receives the information and performs authentication. If authentication is successful, the dashboard is displayed to the administrator. The input is the administrator's login information, and the output is the administrator's dashboard.

[1827] Step 2:

[1828] Managers set key metrics.

[1829] The administrator selects the "Work Settings" tab on the dashboard, enters evaluation indicators such as "operation accuracy," "efficiency," and "adaptability," and clicks the save button. The terminal receives the input and sends it to the server, which stores the information in a database. The input is the evaluation indicator entered by the administrator, and the output is the evaluation indicator stored in the database.

[1830] Step 3:

[1831] The worker enters the work record.

[1832] While working, workers enter details of the machines they operated and the work they performed into a terminal. The terminal sends the entered data in real time to a server, which stores the data in a database. The input is the work record entered by the worker, and the output is the work record stored in the database.

[1833] Step 4:

[1834] The worker records what is said.

[1835] Conversations and questions asked by workers while they are working are recorded using a voice input device. The device converts the voice data into text and sends it to a server. The server stores the data in a database. The input is the worker's voice data, and the output is the text of what was said.

[1836] Step 5:

[1837] The server passes the data to the generative AI model.

[1838] The server extracts work records and speech content from the database and inputs them into the generative AI model. The generative AI model analyzes the data, extracts important keywords, and evaluates work efficiency and skill level. The inputs are work records and speech content, and the output is the extracted keywords and evaluation results.

[1839] Step 6:

[1840] The evaluation results are stored in a database.

[1841] The server stores the evaluation results obtained from the generative AI model in a database. The evaluation results include indicators such as "operation accuracy," "efficiency," and "adaptability." The input is the evaluation results from the generative AI model, and the output is the evaluation results stored in the database.

[1842] Step 7:

[1843] The administrator checks and corrects the evaluation results.

[1844] Administrators can access the dashboard to check the evaluation results and make corrections as necessary. The corrections made by the administrator are sent from the terminal to the server and stored in the database. The input is the corrections made by the administrator, and the output is the corrected evaluation results.

[1845] Step 8:

[1846] Evaluation data is stored for a long period of time and re-learned.

[1847] The server stores the evaluation results in a database for a long period of time. The accumulated data is used to periodically retrain the generative AI model. The input is the accumulated evaluation data, and the output is the retrained generative AI model.

[1848] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1849] System Overview

[1850] The present invention provides a system that records students' notes and comments during class, analyzes them with a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities, and also provides a system that combines an emotion engine. A specific embodiment of this system will be described below.

[1851] 1. Initial Setup Phase

[1852] server:

[1853] It provides an interface for teachers to log in and performs login authentication.

[1854] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[1855] Save important keywords in a database.

[1856] Device:

[1857] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[1858] User (Teacher):

[1859] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[1860] 2. Data Collection Phase

[1861] server:

[1862] Students' notes and comments are received in real time and stored in a database.

[1863] Device:

[1864] Provides an interface for students to enter notes.

[1865] Provides the ability to record student comments and convert them into text.

[1866] User (student):

[1867] Type notes into an electronic device during class.

[1868] Make a statement and record what you say.

[1869] 3. Emotional data collection phase

[1870] Device:

[1871] Students' facial expressions and vocal tones are analyzed in real time using an emotion engine to obtain emotional data.

[1872] The emotion data is sent to the server.

[1873] server:

[1874] Emotion data sent from the device is received and saved in a linked manner with the note description and the content of the remarks.

[1875] 4. Data analysis phase

[1876] server:

[1877] The received notes and statements are passed to a generative AI model for analysis.

[1878] A generative AI model analyzes the content and extracts important keywords.

[1879] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[1880] 5. Evaluation confirmation phase

[1881] Device:

[1882] Provide a dashboard for teachers to view assessment results.

[1883] User (Teacher):

[1884] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[1885] server:

[1886] The corrected evaluation results are stored in the database.

[1887] 6. Data accumulation and re-learning phase

[1888] server:

[1889] The evaluation results will be accumulated in a database over a long period of time.

[1890] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[1891] Specific examples

[1892] Example of the initial setup phase

[1893] User (Teacher):

[1894] 1. Teachers log in to their dedicated dashboard.

[1895] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[1896] server:

[1897] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[1898] 2. Save the entered keywords in the database.

[1899] A concrete example of the data collection phase

[1900] User (student):

[1901] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[1902] 2. Speaking at specific class times.

[1903] Device:

[1904] 1. Send note input contents to the server in real time.

[1905] 2. Record the speech as audio, convert it into text, and send it to the server.

[1906] server:

[1907] 1. Receive note entries and comments and store them in a database.

[1908] A concrete example of the emotion data collection phase

[1909] Device:

[1910] 1. The camera captures students' facial expressions as they speak.

[1911] 2. The emotion engine analyzes this and detects emotions such as "tension," "excitement," and "anxiety."

[1912] 3. The detected emotion data is sent to the server.

[1913] server:

[1914] 1. The received emotion data is saved by linking it to the note description and the speech content.

[1915] A concrete example of the data analysis phase

[1916] server:

[1917] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[1918] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[1919] 3. Correct the result based on the emotional data, and generate a primary evaluation such as "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and store it in the database.

[1920] Example of the evaluation confirmation phase

[1921] User (Teacher):

[1922] 1. Teachers access the dashboard and check the primary assessment results.

[1923] 2. The teacher makes the necessary corrections to the results.

[1924] server:

[1925] 1. Save the corrected evaluation results in the database.

[1926] Specific example of data accumulation and re-learning phase

[1927] server:

[1928] 1. After the semester ends, all evaluation data will be compiled.

[1929] 2. Retrain the generative AI model using the aggregated data.

[1930] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[1931] In this way, by qualitatively evaluating students' thinking and judgment abilities and taking emotional data into consideration, more efficient and fair evaluation becomes possible.

[1932] The processing flow will be explained below.

[1933] Step 1:

[1934] User (Teacher):

[1935] Teachers log in to their dedicated dashboard.

[1936] The teacher selects the "Class Settings" tab.

[1937] server:

[1938] Log in and authenticate to start the teacher session.

[1939] Displays the class settings tab and provides a form for entering important keywords.

[1940] Step 2:

[1941] User (Teacher):

[1942] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[1943] server:

[1944] The entered keywords are received and stored in a database.

[1945] Step 3:

[1946] User (student):

[1947] During class, students take notes using a dedicated application.

[1948] Speak up at specific times in class.

[1949] Device:

[1950] It provides a note-taking interface and sends inputs to a server in real time.

[1951] It provides a function to record speech, convert it into text, and send it to the server.

[1952] Step 4:

[1953] server:

[1954] Receive notes and comments sent from the device in real time.

[1955] The received content is temporarily saved and prepared for analysis.

[1956] Step 5:

[1957] Device:

[1958] The camera captures students' facial expressions and activates an emotion engine that analyzes their voice tone in real time.

[1959] The emotion engine analyzes facial expressions and vocal tone to generate emotion data (e.g., "nervous," "excited," "anxious").

[1960] The emotion data is sent to the server.

[1961] server:

[1962] Emotion data transmitted from the terminal is received.

[1963] Save the notes and link them to the comments.

[1964] Step 6:

[1965] server:

[1966] The note description, speech content, and linked emotional data are passed to a generative AI model for analysis.

[1967] A generative AI model extracts important keywords.

[1968] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[1969] Step 7:

[1970] User (Teacher):

[1971] Teachers access the dashboard to check assessments.

[1972] Device:

[1973] Display the teacher dashboard and provide primary assessment results.

[1974] Step 8:

[1975] User (Teacher):

[1976] Check the results of the initial evaluation and make corrections as necessary.

[1977] Once the corrections are complete, the final evaluation is sent to the server.

[1978] server:

[1979] The corrected evaluation results are received and stored in a database.

[1980] Step 9:

[1981] server:

[1982] All evaluation data will be compiled at the end of the semester.

[1983] The generative AI model is retrained using the aggregated data.

[1984] Step 10:

[1985] server:

[1986] Prepare to apply the retrained generative AI model to assessment in the next class.

[1987] Implement a model with new evaluation criteria into the system.

[1988] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and by taking emotional data into consideration, more efficient and fair evaluation is realized.

[1989] Example 2

[1990] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1991] Conventional student evaluation systems evaluate students based solely on their notes and comments, making it difficult to accurately assess their thinking and judgment abilities. Furthermore, they do not take emotional data into account, making it difficult to provide fair evaluations that reflect students' psychological state and emotional changes. For these reasons, there has been a demand for a more accurate and fair evaluation method.

[1992] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1993] In this invention, the server includes a means for recording students' notes and comments during class, a means for analyzing the recorded notes and comments and using a generative AI model to extract important keywords, a means for analyzing the students' facial expressions and voice tones during evaluation to obtain emotional data, and a means for qualitatively evaluating the students' thinking ability and judgment ability based on the keywords extracted by the generative AI model. This allows for more accurate evaluation of students' thinking ability and judgment ability, and by taking emotional data into consideration, enables fair and comprehensive evaluation.

[1994] "Note writing" is text data that students input using electronic devices during class.

[1995] "Speech" refers to the verbal content of a student's speech during class, and the content is recorded as audio or converted into text data.

[1996] A "generative AI model" is an artificial intelligence model that analyzes text data and extracts important keywords. For example, it includes models that use natural language processing technology.

[1997] "Important keywords" are particularly noteworthy words and phrases extracted by the generative AI model from students' notes and comments.

[1998] "Thinking ability" refers to the logical thinking skills that students use to solve problems, and specifically includes originality and critical thinking.

[1999] "Judgment" is a student's ability to take appropriate action or make appropriate decisions depending on the situation.

[2000] "Emotional data" refers to data that indicates the emotional state of a student, obtained by analyzing their facial expressions and vocal tones. For example, it includes information on tension, excitement, anxiety, etc.

[2001] A "database" is a system for centrally managing and storing recorded data.

[2002] "Retraining" is the process of using accumulated data to improve the accuracy of a generative AI model.

[2003] "Evaluation results" are evaluation data on students' thinking and judgment abilities generated based on the generative AI model and emotional data.

[2004] A "teacher" is a person who is responsible for teaching and assessing students at an educational institution.

[2005] The present invention is a system that records students' notes and comments in class in real time, extracts important keywords using a generative AI model, and further combines them with emotional data to qualitatively evaluate students' thinking and judgment abilities. An embodiment of this system will be described in detail.

[2006] Hardware and software used

[2007] Server: Database management system, generative AI model (e.g., model using natural language processing technology), login authentication system, evaluation result management system

[2008] Device: Camera, microphone, note input interface, facial expression analysis engine (e.g. emotion engine), voice recognition API

[2009] Users: Electronic devices (tablets and PCs) used by teachers and students

[2010] Specific operation of the system

[2011] Initial Setup Phase

[2012] User (Teacher):

[2013] 1. The teacher logs in to the dedicated dashboard and enters their username and password for authentication.

[2014] 2. After logging in, select the "Class Settings" tab, enter the keywords you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the Save button.

[2015] server:

[2016] Receives the teacher's authentication information and verifies the login via the authentication server. If the login is successful, it provides the lesson settings tab interface.

[2017] Important keywords entered by the teacher are saved in a database.

[2018] Data Collection Phase

[2019] User (student):

[2020] Students typing notes into electronic devices during class.

[2021] Students will speak during class and their comments will be recorded.

[2022] Device:

[2023] It provides an interface for note entry and sends the note contents entered by students to the server in real time.

[2024] It uses a speech recognition API to record what you say and convert it into text, which is then sent to the server.

[2025] server:

[2026] The received note contents and speech contents are saved in a database.

[2027] Emotional data collection phase

[2028] Device:

[2029] A camera captures students' facial expressions as they speak.

[2030] The emotion engine is used to analyze facial expressions and voice tones to detect emotion data (e.g., "tension," "excitement," "anxiety"), and transmit the detected emotion data to the server.

[2031] server:

[2032] The received emotion data is saved by linking it to the note description and the content of the speech.

[2033] Data analysis phase

[2034] server:

[2035] The received note content and speech content are passed to a generative AI model, which uses natural language processing technology such as OpenAI's GPT-3.

[2036] Send the following prompt to the generative AI model: "Analyze the student's notebook entry, 'We held a discussion about the impact of global warming on society, taking into account new perspectives,' and extract key keywords. Additionally, take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error."

[2037] The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database.

[2038] Evaluation confirmation phase

[2039] User (Teacher):

[2040] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page.

[2041] Check the results of the primary evaluation and make any necessary corrections. Once the corrections are complete, click the Save button.

[2042] server:

[2043] The corrections made by the teacher are saved in a database.

[2044] Data accumulation and re-learning phase

[2045] server:

[2046] After each semester, all evaluation data is compiled and stored in a database.

[2047] The generated AI model will be retrained using the accumulated data to improve the accuracy of the evaluation algorithm, and the retrained model will be applied to class evaluations from the next semester.

[2048] Specific examples

[2049] In the initial setup phase, the teacher inputs and saves key keywords such as "originality," "critical thinking," and "trial and error." In the data collection phase, students write in their notebooks, "We held a discussion that considered new perspectives on the impact of global warming on society." In the emotion data collection phase, the students' facial expressions while speaking are captured and analyzed with an emotion engine to detect "tension." In the data analysis phase, the generative AI model extracts "new perspectives" and "discussion" and generates an initial evaluation of "originality: high, critical thinking: medium, trial and error: medium." In the evaluation confirmation phase, the teacher checks the results of the initial evaluation and makes any necessary corrections.

[2050] The present invention can be implemented according to the specific steps set forth in the above detailed description.

[2051] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2052] The specific processing flow of this system program

[2053] Step 1:

[2054] User (Teacher):

[2055] The teacher logs in to the dedicated dashboard. Enter the username and password and click the login button.

[2056] server:

[2057] Receives the teacher's authentication information and verifies the login via the authentication server (input: username, password, output: authentication result). If the login is successful, the lesson settings tab interface is provided.

[2058] Step 2:

[2059] User (Teacher):

[2060] Select the "Class Settings" tab, enter keywords that you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the save button.

[2061] Device:

[2062] It provides a keyword input interface and sends the input keywords to the server (input: important keywords, output: transmission signal).

[2063] server:

[2064] Receives keyword input from teachers and stores it in a database (input: important keywords, output: stored results).

[2065] Step 3:

[2066] User (student):

[2067] A student takes notes on an electronic device during class, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[2068] Device:

[2069] It provides a note input interface and transmits the input note content to the server in real time (input: note content, output: transmission signal).

[2070] server:

[2071] Save the received note contents to the database (input: note contents, output: saved result).

[2072] Step 4:

[2073] User (student):

[2074] He speaks up during class, saying things like, "We should think about environmental issues from a new policy perspective."

[2075] Device:

[2076] The speech is recorded as audio and converted into text using a speech recognition API. The converted text is sent to the server (input: audio data, output: transmission signal).

[2077] server:

[2078] Save the received comment content in the database (input: comment content, output: saved result).

[2079] Step 5:

[2080] Device:

[2081] When a student speaks, the camera captures their facial expressions, and the emotion engine analyzes their facial expressions and vocal tone to detect emotional data (e.g., "nervous," "excited," "anxious") and transmits it to the server (input: facial expressions, vocal data, output: emotional data).

[2082] server:

[2083] The received emotion data is linked to the note description and the speech content and saved (input: emotion data, output: saved result).

[2084] Step 6:

[2085] server:

[2086] The received note content and speech content are passed to the generative AI model (input: note content, speech content, output: prompt). For example, a prompt such as "Analyze the student's note, 'We held a discussion on the impact of global warming on society, taking into account new perspectives,' and extract important keywords. In addition, please take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error" is sent to the generative AI model. The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database (input: generative AI model analysis results, emotional data, output: primary evaluation).

[2087] Step 7:

[2088] User (Teacher):

[2089] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page. Check the primary evaluation results and make corrections as necessary (input: primary evaluation results, corrections, output: final evaluation results).

[2090] server:

[2091] The teacher's corrections are saved in the database (input: final evaluation result, output: saved result).

[2092] Step 8:

[2093] server:

[2094] After each semester, all evaluation data is aggregated and stored in a database (input: each evaluation data, output: stored data). The stored data is used to retrain the generative AI model (input: stored data, output: retrained model). The retrained model is applied to class evaluations from the next semester onwards (input: retrained model, output: application results).

[2095] The above is the specific processing flow of this system.

[2096] (Application example 2)

[2097] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2098] In addition to a system for efficiently evaluating students' thinking and judgment skills, there is a need for a method for qualitatively evaluating the performance and processing capabilities of factory robots by collecting and analyzing their work logs and sensor data in real time. This will enable fairer and more efficient evaluations in both educational and production settings.

[2099] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2100] In this invention, the server includes: means for recording students' notes and comments during class; means for using a generative AI model to analyze the recorded notes and comments and extract important keywords; means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model; means for storing the evaluation results in a database and using them for subsequent evaluation and learning; means for collecting robot work logs and sensor data in real time and evaluating them using the generative AI model; and means for qualitatively evaluating the robot's performance and processing capabilities based on the sensor data analyzed in real time. This makes it possible to highly evaluate both student learning effectiveness and robot work efficiency.

[2101] "Note taking" refers to handwritten or electronic notes or records that students make during class.

[2102] "Content of remarks" refers to the words and opinions expressed by students during class.

[2103] A "generative AI model" is an artificial intelligence model that extracts and analyzes useful information from input data.

[2104] "Important keywords" refer to words or phrases that are particularly meaningful in the recorded notes or statements.

[2105] "Thinking ability" refers to the intellectual abilities necessary for problem-solving and generating new ideas.

[2106] "Judgment" is the ability to choose the best option depending on the situation.

[2107] "Evaluation results" are the evaluation values ​​of the student's or robot's performance after analysis using the generative AI model.

[2108] A "database" is a computer system for systematically storing, retrieving, and managing digital information.

[2109] A "work log" is a detailed record of the work a robot performs.

[2110] "Sensor data" refers to digital data obtained from sensors that measure the operating conditions of a robot.

[2111] "Performance" refers to the efficiency and accuracy with which a robot carries out a task.

[2112] "Processing capacity" refers to how effectively a robot can perform multiple tasks.

[2113] System Overview

[2114] This invention is a system that qualitatively evaluates students' thinking and judgment skills, as well as robot performance and processing capabilities, by collecting students' note-taking and comments during class, factory robot work logs, and sensor data in real time and analyzing them with a generative AI model.

[2115] 1. Initial Setup Phase

[2116] The server provides an interface for teachers and factory managers to log in and performs login authentication. After login authentication is complete, it displays the lesson settings tab and work settings tab, and provides a form for entering important keywords and evaluation criteria (e.g., efficiency, accuracy, flexibility). The entered evaluation criteria are saved in a database.

[2117] The terminal provides an interface for teachers and factory managers to log in and input and save evaluation criteria.

[2118] Users (teachers or factory managers) log in to a dedicated dashboard, enter important keywords and evaluation criteria in the "Class Settings" or "Work Settings" tab, and save them.

[2119] 2. Data Collection Phase

[2120] The server receives students' notes and comments, as well as the robot's work logs and sensor data in real time, stores them in a database, and prepares them as input for the generative AI model.

[2121] The device provides an interface for students to input notes and a function to record the robot's work log. It also has the ability to record student comments and the robot's operation status and convert them into text as needed.

[2122] During class, users (students or robots) take notes and make comments on electronic devices. The robots perform assigned tasks and send their work logs and sensor data to the terminal.

[2123] 3. Emotional data collection phase

[2124] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data, which is then sent to a server.

[2125] The server receives the emotional data sent from the device and stores it, linking it with note descriptions, statements, the robot's work log, and sensor data.

[2126] 4. Data analysis phase

[2127] The server passes the received note entries, comments, robot work logs, and sensor data to the generative AI model for analysis. The generative AI model analyzes the content and extracts important keywords and evaluation criteria. It generates a primary evaluation based on the extracted keywords and criteria and stores the results in a database.

[2128] 5. Evaluation confirmation phase

[2129] The devices provide a dashboard for teachers and factory managers to view assessment results.

[2130] Users (teachers and factory managers) can check the results of the initial assessment through the dashboard and make corrections as necessary.

[2131] The server stores the modified evaluation results in a database.

[2132] 6. Data accumulation and re-learning phase

[2133] The server accumulates the evaluation results in a database over a long period of time, and uses the accumulated data to retrain the generative AI model and improve the accuracy of the evaluation.

[2134] Specific examples

[2135] Example of the initial setup phase

[2136] The user (factory manager) logs in to the dedicated dashboard, selects the "Work Settings" tab, and enters and saves evaluation criteria such as "Efficiency," "Accuracy," and "Flexibility." After authenticating the manager's login, the server displays the work settings tab and saves the entered evaluation criteria in the database.

[2137] A concrete example of the data collection phase

[2138] The user (robot) inputs "Completed assembly of part A and adjusted the position of part B" into the work log interface. The terminal sends the work log interface to the server in real time. The server receives the work log and sensor data and stores them in a database.

[2139] A concrete example of the emotion data collection phase

[2140] The device captures the robot's movements as it assembles parts, and the emotion engine analyzes them to detect emotions such as "stress" or "relaxation." The detected emotion data is then sent to a server, which then links the received emotion data with work logs and sensor data and stores it.

[2141] A concrete example of the data analysis phase

[2142] The server passes the received work log, "Assembly of part A completed, position of part B adjusted," to the generative AI model. The generative AI model analyzes the content and detects "assembly" and "adjustment." It then makes corrections based on the emotional data, generating a primary evaluation, for example, "Efficiency: High, Accuracy: Medium, Flexibility: Medium," and stores this in the database.

[2143] Example of the evaluation confirmation phase

[2144] The user (factory manager) accesses the dashboard and checks the primary evaluation results. The manager makes any necessary corrections to the results. The server saves the corrected evaluation results in the database.

[2145] Specific example of data accumulation and re-learning phase

[2146] After the end of the semester, the server aggregates all evaluation data. The aggregated data is used to retrain the generative AI model. The model with the new evaluation criteria is then applied to the next semester's class evaluations.

[2147] Prompt Sentence Examples

[2148] "Based on the following work log, rate the robot's efficiency, accuracy, and flexibility.

[2149] Work log: [Log data]

[2150] Sensor Data: [Sensor Data]"

[2151] In this way, the present invention can highly evaluate the thinking ability and judgment ability of students and the working efficiency of robots.

[2152] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2153] Step 1: Initial Setup Phase

[2154] The server provides an interface for the administrator to log in and performs authentication. After successful login, it displays a form for the administrator to input evaluation criteria (efficiency, accuracy, flexibility, etc.). It receives the evaluation criteria entered by the administrator and stores them in a database.

[2155] Input: Administrator login information and evaluation criteria

[2156] Output: Authentication results and stored metrics data

[2157] Step 2: Data collection phase

[2158] The terminal provides an interface for students to input notes and an interface for robots to input work logs. Students and robots input information into the interface, and the terminal transmits this information in real time to a server. The server stores the received data in a database.

[2159] Input: Student notes, robot work log

[2160] Output: Input data stored in a database

[2161] Step 3: Emotional data collection phase

[2162] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data. The emotional data is then sent to a server, which links it to notes and work logs and stores it in a database.

[2163] Input: Student's facial expressions, voice tone, robot's movement data

[2164] Output: Emotion data and linked notes and work logs

[2165] Step 4: Data analysis phase

[2166] The server passes the notes, comments, work logs, sensor data, and emotion data stored in the database to the generative AI model for analysis. The generative AI model extracts important keywords and evaluation criteria from this data and generates a primary evaluation. The evaluation results are stored in the database.

[2167] Input: Note writing, speech content, work log, sensor data, emotion data

[2168] Output: Analysis results and primary evaluation by the generative AI model

[2169] Step 5: Evaluation confirmation phase

[2170] The terminal provides a dashboard for teachers and factory managers to check the evaluation results. Users can check the primary evaluation results through the dashboard and make corrections as necessary. The server then stores the corrected evaluation results back in the database.

[2171] Input: Primary assessment results, teacher and administrator feedback

[2172] Output: Corrected evaluation result

[2173] Step 6: Data accumulation and retraining phase

[2174] The server accumulates the evaluation results in a database over a long period of time. The accumulated data is used to retrain the generative AI model, improving the model's evaluation accuracy. The model with the new evaluation criteria is used in the next cycle.

[2175] Input: Accumulated evaluation data

[2176] Output: The retrained generative AI model and its evaluation criteria

[2177] By using the above steps, the present invention can highly evaluate the thinking ability and judgment ability of students and the work efficiency of robots.

[2178] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2179] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2180] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2181] [Fourth embodiment]

[2182] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2183] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2184] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2185] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2186] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2187] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2188] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2189] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2190] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2191] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2192] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2193] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2194] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2195] System Overview

[2196] The present invention is a system that records students' note taking and comments made during class, analyzes them using a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities. The following describes in detail an embodiment of this system.

[2197] 1. Initial Setup Phase

[2198] server:

[2199] It provides an interface for teachers to log in and performs login authentication.

[2200] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[2201] Save important keywords in a database.

[2202] Device:

[2203] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[2204] User (Teacher):

[2205] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[2206] 2. Data Collection Phase

[2207] server:

[2208] Students' notes and comments are received in real time and stored in a database.

[2209] Device:

[2210] Provides an interface for students to enter notes.

[2211] Provides the ability to record student comments and convert them into text.

[2212] User (student):

[2213] Type notes into an electronic device during class.

[2214] Make a statement and record what you say.

[2215] 3. Data analysis phase

[2216] server:

[2217] The received notes and statements are analyzed using a generative AI model.

[2218] Important keywords are extracted and students' thinking and judgment skills are evaluated based on these.

[2219] The primary evaluation results are stored in a database.

[2220] 4. Evaluation confirmation phase

[2221] Device:

[2222] Provide a dashboard for teachers to view assessment results.

[2223] User (Teacher):

[2224] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[2225] server:

[2226] The corrected evaluation results are stored in the database.

[2227] 5. Data accumulation and re-learning phase

[2228] server:

[2229] The evaluation results will be accumulated in a database over a long period of time.

[2230] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[2231] Specific examples

[2232] Example of the initial setup phase

[2233] User (Teacher):

[2234] 1. Teachers log in to their dedicated dashboard.

[2235] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[2236] server:

[2237] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[2238] 2. Save the entered keywords in the database.

[2239] A concrete example of the data collection phase

[2240] User (student):

[2241] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[2242] 2. Speaking at specific class times.

[2243] Device:

[2244] 1. Send note input contents to the server in real time.

[2245] 2. Record the speech as audio, convert it into text, and send it to the server.

[2246] server:

[2247] 1. Receive note entries and comments and store them in a database.

[2248] A concrete example of the data analysis phase

[2249] server:

[2250] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[2251] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[2252] 3. Generate a primary evaluation score of "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and save it in the database.

[2253] Example of the evaluation confirmation phase

[2254] User (Teacher):

[2255] 1. Teachers access the dashboard and check the primary assessment results.

[2256] 2. The teacher makes the necessary corrections to the results.

[2257] server:

[2258] 1. Save the corrected evaluation results in the database.

[2259] Specific example of data accumulation and re-learning phase

[2260] server:

[2261] 1. Compile all evaluation data at the end of the semester.

[2262] 2. Retrain the generative AI model using the aggregated data.

[2263] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[2264] In this way, a system for qualitatively evaluating students' thinking and judgment abilities is constructed, realizing efficient and fair evaluation.

[2265] The processing flow will be explained below.

[2266] Step 1:

[2267] User (Teacher):

[2268] Teachers log in to their dedicated dashboard.

[2269] The teacher selects the "Class Settings" tab.

[2270] server:

[2271] Log in and authenticate to start the teacher session.

[2272] Displays the class settings tab and provides a form for entering important keywords.

[2273] Step 2:

[2274] User (Teacher):

[2275] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[2276] server:

[2277] The entered keywords are received and stored in a database.

[2278] Step 3:

[2279] User (student):

[2280] During class, students take notes using a dedicated application.

[2281] Speak up at specific times in class.

[2282] Device:

[2283] It provides a note-taking interface and sends inputs to a server in real time.

[2284] It provides a function to record speech, convert it into text, and send it to the server.

[2285] Step 4:

[2286] server:

[2287] Receive notes and comments sent from the device in real time.

[2288] The received content is temporarily saved and prepared for analysis.

[2289] Step 5:

[2290] server:

[2291] The note descriptions and speech content are passed to a generative AI model for analysis.

[2292] A generative AI model analyzes the content and extracts important keywords.

[2293] A primary evaluation is generated based on the extracted keywords and the results are stored in a database.

[2294] Step 6:

[2295] User (Teacher):

[2296] Teachers access the dashboard to check assessments.

[2297] Device:

[2298] Display the teacher dashboard and provide primary assessment results.

[2299] Step 7:

[2300] User (Teacher):

[2301] Check the results of the initial evaluation and make corrections as necessary.

[2302] Once the corrections are complete, the final evaluation is sent to the server.

[2303] server:

[2304] The corrected evaluation results are received and stored in a database.

[2305] Step 8:

[2306] server:

[2307] All evaluation data will be compiled at the end of the semester.

[2308] The generative AI model is retrained based on the aggregated data.

[2309] Step 9:

[2310] server:

[2311] Prepare to apply the retrained generative AI model to assessment in the next class.

[2312] Implement a model with new evaluation criteria into the system.

[2313] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and an efficient and fair evaluation is realized.

[2314] Example 1

[2315] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2316] To effectively evaluate students' thinking and judgment skills, a system is needed that can analyze notes and comments taken during class in real time and accurately reflect the evaluation results. However, conventional systems have difficulty consistently recording and analyzing notes and comments, which means that fairness and efficiency of evaluation cannot be ensured. Furthermore, there is a lack of a way for teachers to easily check evaluation results and make corrections as necessary. Technology that can solve these issues is needed.

[2317] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2318] In this invention, the server includes means for recording students' notes and comments during class, means for receiving the recorded notes and comments in real time and saving them in a database, means for using a generative AI model that analyzes the recorded notes and comments to extract important keywords, means for inputting prompts to the generative AI model and analyzing the received data, means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model, means for saving the evaluation results in a database and using them for subsequent evaluation and learning, and means for teachers to check and correct the evaluation results. This allows students' learning activities to be recorded and analyzed in real time, enabling fair and efficient evaluation.

[2319] "Notes" are text data that students input into their electronic devices during class to record important information and personal thoughts.

[2320] "Comments" refer to the verbal expressions of opinions and questions that students make during class, and the recordings are then converted into text.

[2321] The "database" is an information storage system for uniformly managing and storing recorded notes, comments, and evaluation results.

[2322] A "generative AI model" is a model that uses artificial intelligence to analyze text data, extract important keywords, and evaluate thinking ability and judgment.

[2323] A "prompt sentence" is a text sentence entered into a generative AI model to instruct it to perform a specific process or analysis.

[2324] The "evaluation results" are the results of a qualitative assessment of students' thinking and judgment abilities based on data analyzed by the generative AI model.

[2325] "Means" refers to specific methods or functions for achieving a certain purpose.

[2326] A "server" is a computer system that processes, stores, and manages data, and has functions such as receiving, analyzing, evaluating, and storing note-taking and comment content.

[2327] A "terminal" is a device that a user directly operates to input and confirm data.

[2328] "User" refers to a person such as a teacher or student who uses this system.

[2329] This invention is a system that records students' notes and comments during class and uses a generative AI model to qualitatively evaluate their thinking and judgment abilities. This system consists of three parties: a server, a terminal, and a user, each of which processes and calculates data using specific hardware and software.

[2330] server

[2331] The server performs the following main roles:

[2332] 1. Login authentication and dashboard provision: Provides an interface for teachers to log in to a dedicated dashboard and authenticates the login information. If login is successful, displays the dashboard for lesson settings and evaluation confirmation.

[2333] 2. Data reception and storage: Students' notes and comments are received in real time and stored in a database.

[2334] 3. Data analysis: Prompt sentences are input into the generative AI model, which analyzes the notes and speech content to extract important keywords.

[2335] 4. Generating and saving evaluations: Based on the extracted keywords, students' thinking and judgment skills are evaluated and the results are saved in the database. If the evaluation results are revised, they are also saved.

[2336] 5. Retraining: Retrain the generative AI model using accumulated evaluation data to improve the accuracy of the evaluation.

[2337] Terminal

[2338] The terminal performs the following main roles:

[2339] 1. Login and settings interface provided: An interface is provided for teachers to log in and enter and save important keywords from the lesson settings tab.

[2340] 2. Note taking and speech recording: Provides an interface for students to take notes and has a function to record speech. This data is sent to the server in real time.

[2341] 3. Displaying assessment results: Provide a dashboard for teachers to view assessment results.

[2342] User

[2343] Users (teachers and students) have the following roles:

[2344] 1. Teachers: Log in to the dedicated dashboard and enter important keywords such as originality and critical thinking in the "Class Settings" tab. Also, check the evaluation results and make corrections as necessary.

[2345] 2. Students: Use electronic devices to take notes and speak during class.

[2346] Hardware and Software Examples

[2347] Server: Lincoln Central Server (Generic Cloud Server), Database System (MySQL)

[2348] Devices: Teacher and student laptops or tablets

[2349] Generative AI models: text analysis software (e.g., OpenAI GPT-3)

[2350] Specific examples and prompts

[2351] Teacher operation example

[2352] 1. Teachers log in to their dedicated dashboard.

[2353] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[2354] 3. Teachers access the dashboard to review the primary assessment results and make corrections as necessary.

[2355] Student operation example

[2356] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[2357] 2. Students will speak during class, and their comments will be recorded audio and converted into text.

[2358] Prompt Sentence Examples

[2359] "Evaluate the students' thinking and judgment based on the text below."

[2360] By implementing such a system, students' learning activities can be recorded and analyzed in real time, enabling fair and efficient evaluation.

[2361] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2362] Step 1:

[2363] User (Teacher):

[2364] Teachers log in to their dedicated dashboard.

[2365] Specific behavior:

[2366] The teacher uses an electronic device to enter a username and password and clicks the login button.

[2367] input:

[2368] Username, Password

[2369] output:

[2370] If the login authentication is successful, the dashboard will be displayed.

[2371] Step 2:

[2372] server:

[2373] Authenticate your login details and view the teacher dashboard.

[2374] Specific behavior:

[2375] The server checks the received username and password against the information in its database. If authentication is successful, the class settings tab is displayed.

[2376] input:

[2377] Username, Password

[2378] output:

[2379] Upon successful login, a dashboard containing the Course Settings tab will be displayed.

[2380] Step 3:

[2381] User (Teacher):

[2382] Select the lesson settings tab, enter important keywords, and save.

[2383] Specific behavior:

[2384] Teachers enter key keywords such as originality, critical thinking, trial and error, and click the save button.

[2385] input:

[2386] Important keywords (e.g., "originality," "critical thinking," "trial and error")

[2387] output:

[2388] A save confirmation message will be displayed.

[2389] Step 4:

[2390] server:

[2391] The entered important keywords are saved in the database.

[2392] Specific behavior:

[2393] The server receives the important keywords entered by the teacher and stores them in a database.

[2394] input:

[2395] Important Keywords

[2396] output:

[2397] Notification that saving to the database is complete.

[2398] Step 5:

[2399] User (student):

[2400] Take notes during class.

[2401] Specific behavior:

[2402] Students use their electronic devices to enter text into a note-taking interface, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[2403] input:

[2404] Note contents

[2405] output:

[2406] The note contents will be displayed on the device.

[2407] Step 6:

[2408] Device:

[2409] The note description input contents are sent to the server in real time.

[2410] Specific behavior:

[2411] The contents entered in the note entry interface are periodically sent to the server.

[2412] input:

[2413] Note contents

[2414] output:

[2415] Data sent to the server.

[2416] Step 7:

[2417] User (student):

[2418] Speak during class.

[2419] Specific behavior:

[2420] Students speak aloud during class.

[2421] input:

[2422] Voice remarks

[2423] output:

[2424] The speech is recorded as audio data.

[2425] Step 8:

[2426] Device:

[2427] Speech is recorded as audio, converted into text and sent to the server.

[2428] Specific behavior:

[2429] The recorded voice data is converted into text using a text conversion engine and sent to the server.

[2430] input:

[2431] Audio data

[2432] output:

[2433] The speech converted to text.

[2434] Step 9:

[2435] server:

[2436] The received note contents and speech content are passed to the generation AI model.

[2437] Specific behavior:

[2438] The server reads the received note descriptions and speech text from the database and prepares them to be passed to the generative AI model.

[2439] input:

[2440] Note writing, speech text

[2441] output:

[2442] Text data input into a generative AI model.

[2443] Step 10:

[2444] server:

[2445] The prompt sentence is fed into a generative AI model to analyze the data.

[2446] Specific behavior:

[2447] The server passes a prompt statement, for example, "Please evaluate the student's thinking ability and judgment based on the text below," to the generative AI model and begins analysis.

[2448] input:

[2449] Note description, speech text, prompt text

[2450] output:

[2451] Important keywords as analysis results.

[2452] Step 11:

[2453] server:

[2454] Students' thinking and judgment skills are evaluated based on the extracted keywords, and the evaluation results are stored in a database.

[2455] Specific behavior:

[2456] Based on the keywords extracted by the generative AI model, an evaluation score is generated and stored in a database.

[2457] input:

[2458] Important Keywords

[2459] output:

[2460] Evaluation scores are stored in a database.

[2461] Step 12:

[2462] User (Teacher):

[2463] Check the assessment results on the dashboard and make corrections as necessary.

[2464] Specific behavior:

[2465] Teachers can access the dashboard, check the assessment results, make corrections if necessary, and click the save button.

[2466] input:

[2467] Evaluation results

[2468] output:

[2469] Corrected evaluation results.

[2470] Step 13:

[2471] server:

[2472] The corrected evaluation results are stored in the database.

[2473] Specific behavior:

[2474] The server receives the teacher-corrected evaluation results and stores them in a database.

[2475] input:

[2476] Corrected evaluation results

[2477] output:

[2478] Notification that saving to the database is complete.

[2479] Step 14:

[2480] server:

[2481] The generative AI model is retrained using accumulated evaluation data.

[2482] Specific behavior:

[2483] The server aggregates the evaluation data at the end of the semester and builds a dataset for retraining the generative AI model. The retraining of the generative AI model is then performed, and the new model is applied to the next semester's evaluations.

[2484] input:

[2485] Evaluation Data

[2486] output:

[2487] Retrained generative AI model.

[2488] The above are the specific processing steps of this system.

[2489] (Application example 1)

[2490] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2491] Factory workplaces require accurate evaluation of worker efficiency and skill levels, and the provision of appropriate feedback and training. However, conventional evaluation methods require cumbersome recording and evaluation criteria, making efficient and objective evaluation difficult. Additionally, there is a lack of means to evaluate skills and efficiency based on worker comments, making comprehensive evaluation difficult.

[2492] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2493] In this invention, the server includes a means for recording work records and speech contents, a means for using a generative AI model that analyzes the recorded records and speech contents to extract important keywords, a means for qualitatively evaluating work efficiency and skill level based on the keywords extracted by the generative AI model, and a means for storing the evaluation results in a database and using them for subsequent evaluation and learning, thereby enabling a comprehensive evaluation of work efficiency and skill level.

[2494] "Work records" are data that records in detail the procedures and results of work performed by workers in a factory.

[2495] "Speech content" refers to data recorded in text format, including verbal communications, questions, and responses made by workers while they were working.

[2496] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze text data, extract important keywords, and understand the text.

[2497] "Key words" are important words or phrases related to specific evaluation metrics extracted by the generative AI model.

[2498] "Work efficiency" is an indicator of how effectively and quickly a worker can perform a given task.

[2499] "Skill level" is an indicator of the degree of technical ability possessed by a worker.

[2500] "Qualitative evaluation" refers to evaluation based on text data and important keywords, rather than on quantified data.

[2501] A "database" is a data storage system that systematically stores collected records and evaluation results so that they can be reused as needed.

[2502] "Future evaluation and learning" refers to future re-evaluation based on the stored data and re-learning to improve the generative AI model.

[2503] MODE FOR CARRYING OUT THE INVENTION

[2504] One embodiment of the present invention provides a system for evaluating the efficiency and skill level of factory workers. This system records work logs and speech, analyzes them using a generative AI model, and stores the evaluation results in a database.

[2505] 1. Initial Setup Phase

[2506] server

[2507] The server provides an interface for administrators to log in, performs login authentication, provides a work setting tab, and displays a form for administrators to input key evaluation indicators, which are then stored in a database.

[2508] Terminal

[2509] The terminal provides an interface for administrators to log in and enter and save evaluation indicators from the work settings tab.

[2510] User (Administrator)

[2511] Administrators log in to a dedicated dashboard and enter and save key evaluation indicators such as "operation accuracy," "efficiency," and "adaptability" in the "Work Settings" tab.

[2512] 2. Data Collection Phase

[2513] server

[2514] The server receives the workers' work records and comments in real time and stores them in a database.

[2515] Terminal

[2516] The terminal provides an interface for workers to input their work, and also provides the function of recording what the workers say and converting it into text.

[2517] User (worker)

[2518] While working, workers input their work details into the electronic device, and also make statements, which are then recorded.

[2519] 3. Data analysis phase

[2520] server

[2521] The server analyzes the received work records and comments using a generative AI model. It extracts important evaluation indicators and evaluates work efficiency and skill level based on these. The evaluation results are stored in a database.

[2522] 4. Evaluation confirmation phase

[2523] Terminal

[2524] The terminal provides a dashboard for administrators to view evaluation results.

[2525] User (Administrator)

[2526] Administrators can check the evaluation results through the dashboard and make corrections as necessary.

[2527] server

[2528] The server stores the modified evaluation results in a database.

[2529] 5. Data accumulation and re-learning phase

[2530] server

[2531] The server accumulates the evaluation results in a database over a long period of time and uses the accumulated data to retrain the generative AI model, thereby improving the accuracy of the evaluation.

[2532] Hardware and software used

[2533] This system uses the following hardware and software:

[2534] Hardware: Factory robots, voice input devices

[2535] Software: databases (e.g., MySQL), speech-to-text tools (e.g., Google Speech-to-Text API), generative AI models (e.g., OpenAI GPT-4)

[2536] Specific examples

[2537] For example, consider a situation where a worker in a particular factory needs to learn how to operate a new machine. The robot records the worker's machine operation logs and conversations, which are then analyzed by a generative AI model. For example, the evaluation result may be that "the efficiency of the new machine operation has improved, but more skill is needed." In this case, the manager can plan additional training based on this. The manager can view the evaluation results in real time on a dashboard.

[2538] Prompt Sentence Examples

[2539] Work log data:

[2540] "The workers operated the CNC machines, adjusted the settings, and completed the assembly of the parts."

[2541] Speech data:

[2542] "Is there a way to tweak this setting to improve the quality of the finished result?"

[2543] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2544] Step 1:

[2545] The administrator logs in.

[2546] The administrator accesses the dedicated dashboard and enters their login information. The server receives the information and performs authentication. If authentication is successful, the dashboard is displayed to the administrator. The input is the administrator's login information, and the output is the administrator's dashboard.

[2547] Step 2:

[2548] Managers set key metrics.

[2549] The administrator selects the "Work Settings" tab on the dashboard, enters evaluation indicators such as "operation accuracy," "efficiency," and "adaptability," and clicks the save button. The terminal receives the input and sends it to the server, which stores the information in a database. The input is the evaluation indicator entered by the administrator, and the output is the evaluation indicator stored in the database.

[2550] Step 3:

[2551] The worker enters the work record.

[2552] While working, workers enter details of the machines they operated and the work they performed into a terminal. The terminal sends the entered data in real time to a server, which stores the data in a database. The input is the work record entered by the worker, and the output is the work record stored in the database.

[2553] Step 4:

[2554] The worker records what is said.

[2555] Conversations and questions asked by workers while they are working are recorded using a voice input device. The device converts the voice data into text and sends it to a server. The server stores the data in a database. The input is the worker's voice data, and the output is the text of what was said.

[2556] Step 5:

[2557] The server passes the data to the generative AI model.

[2558] The server extracts work records and speech content from the database and inputs them into the generative AI model. The generative AI model analyzes the data, extracts important keywords, and evaluates work efficiency and skill level. The inputs are work records and speech content, and the output is the extracted keywords and evaluation results.

[2559] Step 6:

[2560] The evaluation results are stored in a database.

[2561] The server stores the evaluation results obtained from the generative AI model in a database. The evaluation results include indicators such as "operation accuracy," "efficiency," and "adaptability." The input is the evaluation results from the generative AI model, and the output is the evaluation results stored in the database.

[2562] Step 7:

[2563] The administrator checks and corrects the evaluation results.

[2564] Administrators can access the dashboard to check the evaluation results and make corrections as necessary. The corrections made by the administrator are sent from the terminal to the server and stored in the database. The input is the corrections made by the administrator, and the output is the corrected evaluation results.

[2565] Step 8:

[2566] Evaluation data is stored for a long period of time and re-learned.

[2567] The server stores the evaluation results in a database for a long period of time. The accumulated data is used to periodically retrain the generative AI model. The input is the accumulated evaluation data, and the output is the retrained generative AI model.

[2568] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2569] System Overview

[2570] The present invention provides a system that records students' notes and comments during class, analyzes them with a generative AI model, and qualitatively evaluates the students' thinking and judgment abilities, and also provides a system that combines an emotion engine. A specific embodiment of this system will be described below.

[2571] 1. Initial Setup Phase

[2572] server:

[2573] It provides an interface for teachers to log in and performs login authentication.

[2574] It provides a lesson settings tab and displays a form for teachers to enter important keywords.

[2575] Save important keywords in a database.

[2576] Device:

[2577] It provides an interface for teachers to log in and enter and save important keywords from the lesson settings tab.

[2578] User (Teacher):

[2579] Log in to the dedicated dashboard, enter important keywords in the "Class Settings" tab, and save.

[2580] 2. Data Collection Phase

[2581] server:

[2582] Students' notes and comments are received in real time and stored in a database.

[2583] Device:

[2584] Provides an interface for students to enter notes.

[2585] Provides the ability to record student comments and convert them into text.

[2586] User (student):

[2587] Type notes into an electronic device during class.

[2588] Make a statement and record what you say.

[2589] 3. Emotional data collection phase

[2590] Device:

[2591] Students' facial expressions and vocal tones are analyzed in real time using an emotion engine to obtain emotional data.

[2592] The emotion data is sent to the server.

[2593] server:

[2594] Emotion data sent from the device is received and saved in a linked manner with the note description and the content of the remarks.

[2595] 4. Data analysis phase

[2596] server:

[2597] The received notes and statements are passed to a generative AI model for analysis.

[2598] A generative AI model analyzes the content and extracts important keywords.

[2599] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[2600] 5. Evaluation confirmation phase

[2601] Device:

[2602] Provide a dashboard for teachers to view assessment results.

[2603] User (Teacher):

[2604] Check the results of the initial assessment through the dashboard and make corrections as necessary.

[2605] server:

[2606] The corrected evaluation results are stored in the database.

[2607] 6. Data accumulation and re-learning phase

[2608] server:

[2609] The evaluation results will be accumulated in a database over a long period of time.

[2610] The generative AI model is retrained using accumulated data to improve the accuracy of evaluation.

[2611] Specific examples

[2612] Example of the initial setup phase

[2613] User (Teacher):

[2614] 1. Teachers log in to their dedicated dashboard.

[2615] 2. Select the "Class Settings" tab, enter important keywords such as "originality," "critical thinking," and "trial and error," and click the Save button.

[2616] server:

[2617] 1. After authenticating the teacher's login, the lesson settings tab will be displayed.

[2618] 2. Save the entered keywords in the database.

[2619] A concrete example of the data collection phase

[2620] User (student):

[2621] 1. A student types into the note-taking interface, "We had a discussion about the impact of global warming on society, taking into account new perspectives."

[2622] 2. Speaking at specific class times.

[2623] Device:

[2624] 1. Send note input contents to the server in real time.

[2625] 2. Record the speech as audio, convert it into text, and send it to the server.

[2626] server:

[2627] 1. Receive note entries and comments and store them in a database.

[2628] A concrete example of the emotion data collection phase

[2629] Device:

[2630] 1. The camera captures students' facial expressions as they speak.

[2631] 2. The emotion engine analyzes this and detects emotions such as "tension," "excitement," and "anxiety."

[2632] 3. The detected emotion data is sent to the server.

[2633] server:

[2634] 1. The received emotion data is saved by linking it to the note description and the speech content.

[2635] A concrete example of the data analysis phase

[2636] server:

[2637] 1. The received note content, "We held a discussion that took into account new perspectives on the impact of global warming on society," is passed to the generative AI model.

[2638] 2. The generative AI model analyzes the content and detects "new perspectives" and "discussions."

[2639] 3. Correct the result based on the emotional data, and generate a primary evaluation such as "Originality: High, Critical Thinking: Medium, Trial and Error: Medium" and store it in the database.

[2640] Example of the evaluation confirmation phase

[2641] User (Teacher):

[2642] 1. Teachers access the dashboard and check the primary assessment results.

[2643] 2. The teacher makes the necessary corrections to the results.

[2644] server:

[2645] 1. Save the corrected evaluation results in the database.

[2646] Specific example of data accumulation and re-learning phase

[2647] server:

[2648] 1. After the semester ends, all evaluation data will be compiled.

[2649] 2. Retrain the generative AI model using the aggregated data.

[2650] 3. Apply the model with the new evaluation criteria to the next semester's course evaluations.

[2651] In this way, by qualitatively evaluating students' thinking and judgment abilities and taking emotional data into consideration, more efficient and fair evaluation becomes possible.

[2652] The processing flow will be explained below.

[2653] Step 1:

[2654] User (Teacher):

[2655] Teachers log in to their dedicated dashboard.

[2656] The teacher selects the "Class Settings" tab.

[2657] server:

[2658] Log in and authenticate to start the teacher session.

[2659] Displays the class settings tab and provides a form for entering important keywords.

[2660] Step 2:

[2661] User (Teacher):

[2662] Enter important keywords (e.g., "originality," "critical thinking," "trial and error") and click the Save button.

[2663] server:

[2664] The entered keywords are received and stored in a database.

[2665] Step 3:

[2666] User (student):

[2667] During class, students take notes using a dedicated application.

[2668] Speak up at specific times in class.

[2669] Device:

[2670] It provides a note-taking interface and sends inputs to a server in real time.

[2671] It provides a function to record speech, convert it into text, and send it to the server.

[2672] Step 4:

[2673] server:

[2674] Receive notes and comments sent from the device in real time.

[2675] The received content is temporarily saved and prepared for analysis.

[2676] Step 5:

[2677] Device:

[2678] The camera captures students' facial expressions and activates an emotion engine that analyzes their voice tone in real time.

[2679] The emotion engine analyzes facial expressions and vocal tone to generate emotion data (e.g., "nervous," "excited," "anxious").

[2680] The emotion data is sent to the server.

[2681] server:

[2682] Emotion data transmitted from the terminal is received.

[2683] Save the notes and link them to the comments.

[2684] Step 6:

[2685] server:

[2686] The note description, speech content, and linked emotional data are passed to a generative AI model for analysis.

[2687] A generative AI model extracts important keywords.

[2688] A primary evaluation is generated based on the extracted keywords and emotion data, and the results are stored in a database.

[2689] Step 7:

[2690] User (Teacher):

[2691] Teachers access the dashboard to check assessments.

[2692] Device:

[2693] Display the teacher dashboard and provide primary assessment results.

[2694] Step 8:

[2695] User (Teacher):

[2696] Check the results of the initial evaluation and make corrections as necessary.

[2697] Once the corrections are complete, the final evaluation is sent to the server.

[2698] server:

[2699] The corrected evaluation results are received and stored in a database.

[2700] Step 9:

[2701] server:

[2702] All evaluation data will be compiled at the end of the semester.

[2703] The generative AI model is retrained using the aggregated data.

[2704] Step 10:

[2705] server:

[2706] Prepare to apply the retrained generative AI model to assessment in the next class.

[2707] Implement a model with new evaluation criteria into the system.

[2708] Through the above processing steps, students' thinking and judgment abilities are qualitatively evaluated, and by taking emotional data into consideration, more efficient and fair evaluation is realized.

[2709] Example 2

[2710] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2711] Conventional student evaluation systems evaluate students based solely on their notes and comments, making it difficult to accurately assess their thinking and judgment abilities. Furthermore, they do not take emotional data into account, making it difficult to provide fair evaluations that reflect students' psychological state and emotional changes. For these reasons, there has been a demand for a more accurate and fair evaluation method.

[2712] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2713] In this invention, the server includes a means for recording students' notes and comments during class, a means for analyzing the recorded notes and comments and using a generative AI model to extract important keywords, a means for analyzing the students' facial expressions and voice tones during evaluation to obtain emotional data, and a means for qualitatively evaluating the students' thinking ability and judgment ability based on the keywords extracted by the generative AI model. This allows for more accurate evaluation of students' thinking ability and judgment ability, and by taking emotional data into consideration, enables fair and comprehensive evaluation.

[2714] "Note writing" is text data that students input using electronic devices during class.

[2715] "Speech" refers to the verbal content of a student's speech during class, and the content is recorded as audio or converted into text data.

[2716] A "generative AI model" is an artificial intelligence model that analyzes text data and extracts important keywords. For example, it includes models that use natural language processing technology.

[2717] "Important keywords" are particularly noteworthy words and phrases extracted by the generative AI model from students' notes and comments.

[2718] "Thinking ability" refers to the logical thinking skills that students use to solve problems, and specifically includes originality and critical thinking.

[2719] "Judgment" is a student's ability to take appropriate action or make appropriate decisions depending on the situation.

[2720] "Emotional data" refers to data that indicates the emotional state of a student, obtained by analyzing their facial expressions and vocal tones. For example, it includes information on tension, excitement, anxiety, etc.

[2721] A "database" is a system for centrally managing and storing recorded data.

[2722] "Retraining" is the process of using accumulated data to improve the accuracy of a generative AI model.

[2723] "Evaluation results" are evaluation data on students' thinking and judgment abilities generated based on the generative AI model and emotional data.

[2724] A "teacher" is a person who is responsible for teaching and assessing students at an educational institution.

[2725] The present invention is a system that records students' notes and comments in class in real time, extracts important keywords using a generative AI model, and further combines them with emotional data to qualitatively evaluate students' thinking and judgment abilities. An embodiment of this system will be described in detail.

[2726] Hardware and software used

[2727] Server: Database management system, generative AI model (e.g., model using natural language processing technology), login authentication system, evaluation result management system

[2728] Device: Camera, microphone, note input interface, facial expression analysis engine (e.g. emotion engine), voice recognition API

[2729] Users: Electronic devices (tablets and PCs) used by teachers and students

[2730] Specific operation of the system

[2731] Initial Setup Phase

[2732] User (Teacher):

[2733] 1. The teacher logs in to the dedicated dashboard and enters their username and password for authentication.

[2734] 2. After logging in, select the "Class Settings" tab, enter the keywords you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the Save button.

[2735] server:

[2736] Receives the teacher's authentication information and verifies the login via the authentication server. If the login is successful, it provides the lesson settings tab interface.

[2737] Important keywords entered by the teacher are saved in a database.

[2738] Data Collection Phase

[2739] User (student):

[2740] Students typing notes into electronic devices during class.

[2741] Students will speak during class and their comments will be recorded.

[2742] Device:

[2743] It provides an interface for note entry and sends the note contents entered by students to the server in real time.

[2744] It uses a speech recognition API to record what you say and convert it into text, which is then sent to the server.

[2745] server:

[2746] The received note contents and speech contents are saved in a database.

[2747] Emotional data collection phase

[2748] Device:

[2749] A camera captures students' facial expressions as they speak.

[2750] The emotion engine is used to analyze facial expressions and voice tones to detect emotion data (e.g., "tension," "excitement," "anxiety"), and transmit the detected emotion data to the server.

[2751] server:

[2752] The received emotion data is saved by linking it to the note description and the content of the speech.

[2753] Data analysis phase

[2754] server:

[2755] The received note content and speech content are passed to a generative AI model, which uses natural language processing technology such as OpenAI's GPT-3.

[2756] Send the following prompt to the generative AI model: "Analyze the student's notebook entry, 'We held a discussion about the impact of global warming on society, taking into account new perspectives,' and extract key keywords. Additionally, take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error."

[2757] The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database.

[2758] Evaluation confirmation phase

[2759] User (Teacher):

[2760] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page.

[2761] Check the results of the primary evaluation and make any necessary corrections. Once the corrections are complete, click the Save button.

[2762] server:

[2763] The corrections made by the teacher are saved in a database.

[2764] Data accumulation and re-learning phase

[2765] server:

[2766] After each semester, all evaluation data is compiled and stored in a database.

[2767] The generated AI model will be retrained using the accumulated data to improve the accuracy of the evaluation algorithm, and the retrained model will be applied to class evaluations from the next semester.

[2768] Specific examples

[2769] In the initial setup phase, the teacher inputs and saves key keywords such as "originality," "critical thinking," and "trial and error." In the data collection phase, students write in their notebooks, "We held a discussion that considered new perspectives on the impact of global warming on society." In the emotion data collection phase, the students' facial expressions while speaking are captured and analyzed with an emotion engine to detect "tension." In the data analysis phase, the generative AI model extracts "new perspectives" and "discussion" and generates an initial evaluation of "originality: high, critical thinking: medium, trial and error: medium." In the evaluation confirmation phase, the teacher checks the results of the initial evaluation and makes any necessary corrections.

[2770] The present invention can be implemented according to the specific steps set forth in the above detailed description.

[2771] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2772] The specific processing flow of this system program

[2773] Step 1:

[2774] User (Teacher):

[2775] The teacher logs in to the dedicated dashboard. Enter the username and password and click the login button.

[2776] server:

[2777] Receives the teacher's authentication information and verifies the login via the authentication server (input: username, password, output: authentication result). If the login is successful, the lesson settings tab interface is provided.

[2778] Step 2:

[2779] User (Teacher):

[2780] Select the "Class Settings" tab, enter keywords that you consider important (e.g., "originality," "critical thinking," "trial and error"), and click the save button.

[2781] Device:

[2782] It provides a keyword input interface and sends the input keywords to the server (input: important keywords, output: transmission signal).

[2783] server:

[2784] Receives keyword input from teachers and stores it in a database (input: important keywords, output: stored results).

[2785] Step 3:

[2786] User (student):

[2787] A student takes notes on an electronic device during class, for example, "We discussed the impact of global warming on society, taking into account new perspectives."

[2788] Device:

[2789] It provides a note input interface and transmits the input note content to the server in real time (input: note content, output: transmission signal).

[2790] server:

[2791] Save the received note contents to the database (input: note contents, output: saved result).

[2792] Step 4:

[2793] User (student):

[2794] He speaks up during class, saying things like, "We should think about environmental issues from a new policy perspective."

[2795] Device:

[2796] The speech is recorded as audio and converted into text using a speech recognition API. The converted text is sent to the server (input: audio data, output: transmission signal).

[2797] server:

[2798] Save the received comment content in the database (input: comment content, output: saved result).

[2799] Step 5:

[2800] Device:

[2801] When a student speaks, the camera captures their facial expressions, and the emotion engine analyzes their facial expressions and vocal tone to detect emotional data (e.g., "nervous," "excited," "anxious") and transmits it to the server (input: facial expressions, vocal data, output: emotional data).

[2802] server:

[2803] The received emotion data is linked to the note description and the speech content and saved (input: emotion data, output: saved result).

[2804] Step 6:

[2805] server:

[2806] The received note content and speech content are passed to the generative AI model (input: note content, speech content, output: prompt). For example, a prompt such as "Analyze the student's note, 'We held a discussion on the impact of global warming on society, taking into account new perspectives,' and extract important keywords. In addition, please take into account emotional data (tension, excitement, anxiety), and evaluate originality, critical thinking, and trial and error" is sent to the generative AI model. The analysis results from the generative AI model are obtained, and a primary evaluation is generated based on the extracted keywords and emotional data, which is then stored in a database (input: generative AI model analysis results, emotional data, output: primary evaluation).

[2807] Step 7:

[2808] User (Teacher):

[2809] The teacher logs in to the dedicated dashboard and opens the primary evaluation results page. Check the primary evaluation results and make corrections as necessary (input: primary evaluation results, corrections, output: final evaluation results).

[2810] server:

[2811] The teacher's corrections are saved in the database (input: final evaluation result, output: saved result).

[2812] Step 8:

[2813] server:

[2814] After each semester, all evaluation data is aggregated and stored in a database (input: each evaluation data, output: stored data). The stored data is used to retrain the generative AI model (input: stored data, output: retrained model). The retrained model is applied to class evaluations from the next semester onwards (input: retrained model, output: application results).

[2815] The above is the specific processing flow of this system.

[2816] (Application example 2)

[2817] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2818] In addition to a system for efficiently evaluating students' thinking and judgment skills, there is a need for a method for qualitatively evaluating the performance and processing capabilities of factory robots by collecting and analyzing their work logs and sensor data in real time. This will enable fairer and more efficient evaluations in both educational and production settings.

[2819] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2820] In this invention, the server includes: means for recording students' notes and comments during class; means for using a generative AI model to analyze the recorded notes and comments and extract important keywords; means for qualitatively evaluating students' thinking and judgment abilities based on the keywords extracted by the generative AI model; means for storing the evaluation results in a database and using them for subsequent evaluation and learning; means for collecting robot work logs and sensor data in real time and evaluating them using the generative AI model; and means for qualitatively evaluating the robot's performance and processing capabilities based on the sensor data analyzed in real time. This makes it possible to highly evaluate both student learning effectiveness and robot work efficiency.

[2821] "Note taking" refers to handwritten or electronic notes or records that students make during class.

[2822] "Content of remarks" refers to the words and opinions expressed by students during class.

[2823] A "generative AI model" is an artificial intelligence model that extracts and analyzes useful information from input data.

[2824] "Important keywords" refer to words or phrases that are particularly meaningful in the recorded notes or statements.

[2825] "Thinking ability" refers to the intellectual abilities necessary for problem-solving and generating new ideas.

[2826] "Judgment" is the ability to choose the best option depending on the situation.

[2827] "Evaluation results" are the evaluation values ​​of the student's or robot's performance after analysis using the generative AI model.

[2828] A "database" is a computer system for systematically storing, retrieving, and managing digital information.

[2829] A "work log" is a detailed record of the work a robot performs.

[2830] "Sensor data" refers to digital data obtained from sensors that measure the operating conditions of a robot.

[2831] "Performance" refers to the efficiency and accuracy with which a robot carries out a task.

[2832] "Processing capacity" refers to how effectively a robot can perform multiple tasks.

[2833] System Overview

[2834] This invention is a system that qualitatively evaluates students' thinking and judgment skills, as well as robot performance and processing capabilities, by collecting students' note-taking and comments during class, factory robot work logs, and sensor data in real time and analyzing them with a generative AI model.

[2835] 1. Initial Setup Phase

[2836] The server provides an interface for teachers and factory managers to log in and performs login authentication. After login authentication is complete, it displays the lesson settings tab and work settings tab, and provides a form for entering important keywords and evaluation criteria (e.g., efficiency, accuracy, flexibility). The entered evaluation criteria are saved in a database.

[2837] The terminal provides an interface for teachers and factory managers to log in and input and save evaluation criteria.

[2838] Users (teachers or factory managers) log in to a dedicated dashboard, enter important keywords and evaluation criteria in the "Class Settings" or "Work Settings" tab, and save them.

[2839] 2. Data Collection Phase

[2840] The server receives students' notes and comments, as well as the robot's work logs and sensor data in real time, stores them in a database, and prepares them as input for the generative AI model.

[2841] The device provides an interface for students to input notes and a function to record the robot's work log. It also has the ability to record student comments and the robot's operation status and convert them into text as needed.

[2842] During class, users (students or robots) take notes and make comments on electronic devices. The robots perform assigned tasks and send their work logs and sensor data to the terminal.

[2843] 3. Emotional data collection phase

[2844] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data, which is then sent to a server.

[2845] The server receives the emotional data sent from the device and stores it, linking it with note descriptions, statements, the robot's work log, and sensor data.

[2846] 4. Data analysis phase

[2847] The server passes the received note entries, comments, robot work logs, and sensor data to the generative AI model for analysis. The generative AI model analyzes the content and extracts important keywords and evaluation criteria. It generates a primary evaluation based on the extracted keywords and criteria and stores the results in a database.

[2848] 5. Evaluation confirmation phase

[2849] The devices provide a dashboard for teachers and factory managers to view assessment results.

[2850] Users (teachers and factory managers) can check the results of the initial assessment through the dashboard and make corrections as necessary.

[2851] The server stores the modified evaluation results in a database.

[2852] 6. Data accumulation and re-learning phase

[2853] The server accumulates the evaluation results in a database over a long period of time, and uses the accumulated data to retrain the generative AI model and improve the accuracy of the evaluation.

[2854] Specific examples

[2855] Example of the initial setup phase

[2856] The user (factory manager) logs in to the dedicated dashboard, selects the "Work Settings" tab, and enters and saves evaluation criteria such as "Efficiency," "Accuracy," and "Flexibility." After authenticating the manager's login, the server displays the work settings tab and saves the entered evaluation criteria in the database.

[2857] A concrete example of the data collection phase

[2858] The user (robot) inputs "Completed assembly of part A and adjusted the position of part B" into the work log interface. The terminal sends the work log interface to the server in real time. The server receives the work log and sensor data and stores them in a database.

[2859] A concrete example of the emotion data collection phase

[2860] The device captures the robot's movements as it assembles parts, and the emotion engine analyzes them to detect emotions such as "stress" or "relaxation." The detected emotion data is then sent to a server, which then links the received emotion data with work logs and sensor data and stores it.

[2861] A concrete example of the data analysis phase

[2862] The server passes the received work log, "Assembly of part A completed, position of part B adjusted," to the generative AI model. The generative AI model analyzes the content and detects "assembly" and "adjustment." It then makes corrections based on the emotional data, generating a primary evaluation, for example, "Efficiency: High, Accuracy: Medium, Flexibility: Medium," and stores this in the database.

[2863] Example of the evaluation confirmation phase

[2864] The user (factory manager) accesses the dashboard and checks the primary evaluation results. The manager makes any necessary corrections to the results. The server saves the corrected evaluation results in the database.

[2865] Specific example of data accumulation and re-learning phase

[2866] After the end of the semester, the server aggregates all evaluation data. The aggregated data is used to retrain the generative AI model. The model with the new evaluation criteria is then applied to the next semester's class evaluations.

[2867] Prompt Sentence Examples

[2868] "Based on the following work log, rate the robot's efficiency, accuracy, and flexibility.

[2869] Work log: [Log data]

[2870] Sensor Data: [Sensor Data]"

[2871] In this way, the present invention can highly evaluate the thinking ability and judgment ability of students and the working efficiency of robots.

[2872] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2873] Step 1: Initial Setup Phase

[2874] The server provides an interface for the administrator to log in and performs authentication. After successful login, it displays a form for the administrator to input evaluation criteria (efficiency, accuracy, flexibility, etc.). It receives the evaluation criteria entered by the administrator and stores them in a database.

[2875] Input: Administrator login information and evaluation criteria

[2876] Output: Authentication results and stored metrics data

[2877] Step 2: Data collection phase

[2878] The terminal provides an interface for students to input notes and an interface for robots to input work logs. Students and robots input information into the interface, and the terminal transmits this information in real time to a server. The server stores the received data in a database.

[2879] Input: Student notes, robot work log

[2880] Output: Input data stored in a database

[2881] Step 3: Emotional data collection phase

[2882] The device uses an emotion engine to analyze students' facial expressions, voice tones, and the force and mechanical sounds of the robot in real time to obtain emotional data. The emotional data is then sent to a server, which links it to notes and work logs and stores it in a database.

[2883] Input: Student's facial expressions, voice tone, robot's movement data

[2884] Output: Emotion data and linked notes and work logs

[2885] Step 4: Data analysis phase

[2886] The server passes the notes, comments, work logs, sensor data, and emotion data stored in the database to the generative AI model for analysis. The generative AI model extracts important keywords and evaluation criteria from this data and generates a primary evaluation. The evaluation results are stored in the database.

[2887] Input: Note writing, speech content, work log, sensor data, emotion data

[2888] Output: Analysis results and primary evaluation by the generative AI model

[2889] Step 5: Evaluation confirmation phase

[2890] The terminal provides a dashboard for teachers and factory managers to check the evaluation results. Users can check the primary evaluation results through the dashboard and make corrections as necessary. The server then stores the corrected evaluation results back in the database.

[2891] Input: Primary assessment results, teacher and administrator feedback

[2892] Output: Corrected evaluation result

[2893] Step 6: Data accumulation and retraining phase

[2894] The server accumulates the evaluation results in a database over a long period of time. The accumulated data is used to retrain the generative AI model, improving the model's evaluation accuracy. The model with the new evaluation criteria is used in the next cycle.

[2895] Input: Accumulated evaluation data

[2896] Output: The retrained generative AI model and its evaluation criteria

[2897] By using the above steps, the present invention can highly evaluate the thinking ability and judgment ability of students and the work efficiency of robots.

[2898] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2899] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2900] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2901] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2902] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentr...

Claims

1. A means of recording students' notes and comments during class, One is to use a generative AI model that analyzes recorded notes and statements to extract important keywords, and the other is to A means to qualitatively evaluate students' thinking and judgment abilities based on keywords extracted by the generative AI model; and A means to store the evaluation results in a database and use them for future evaluation and learning. A system including:

2. The system according to claim 1 , further comprising means for a teacher to check the evaluation results and make corrections as necessary.

3. The system of claim 1 , further comprising means for accumulating long-term evaluation data and retraining the generative AI model.

Images