System

Abstract

To provide a system.SOLUTION: A system includes: means for generating a test regarding a predetermined document by using a regenerative AI model; means for receiving an examinee's answer to the generated test; means for recognizing the examinee's feeling on the basis of the received answer; and means for providing feedback to the examinee on the basis of the received answer and feeling data showing the recognized feeling of the examinee.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When assessing the level of understanding of an operational manual, it is difficult to accurately grasp the differences in understanding levels among employees and provide immediate feedback. [Means for solving the problem]

[0005] We propose a system that uses AI to generate business manual comprehension tests and provides instant feedback based on test takers' answers. This system generates customizable tests based on the content of business manuals, enabling the effectiveness of educational programs to be measured. [Brief explanation of the drawings]

[0006] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, the AI-based means for generating business manual comprehension tests analyzes the contents of the business manual and generates questions based on the contents. Specifically, the text data of the business manual is used as input, and important keywords and concepts are extracted using natural language processing technology, and questions are generated based on these.

[0029] "Example 2"

[0030] In addition, as a means for accepting answers from test takers, the company provides an interface for test takers to answer the generated tests. Specifically, the company provides a web-based test platform that allows test takers to log in and answer the tests.

[0031] "Example 3"

[0032] Furthermore, as a means of providing immediate feedback based on the answers received, AI technology is used to evaluate the examinee's answers and provide the results to the examinee immediately. Specifically, the examinee's answers are analyzed using natural language processing technology to evaluate their accuracy. Based on the evaluation results, immediate feedback is provided to the examinee.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Enter the text data of the business manual into the system.

[0036] Step 2: The system uses natural language processing techniques to extract important keywords and concepts from the text data.

[0037] Step 3: Based on the extracted keywords and concepts, the system generates comprehension test questions.

[0038] "Example 2"

[0039] Step 1: The system provides a web-based testing platform.

[0040] Step 2: Candidates log in to the platform and answer the generated test.

[0041] "Example 3"

[0042] Step 1: The system accepts the test taker's answers and analyzes them using natural language processing technology.

[0043] Step 2: The system evaluates the accuracy of the answer based on the analysis results.

[0044] Step 3: Based on the evaluation results, the system provides immediate feedback to the test-taker.

[0045] Example 1

[0046] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0047] Conventional business manual comprehension tests require manual question creation, which is time-consuming and labor-intensive. Furthermore, they do not provide immediate feedback on test takers' answers, which can reduce learning effectiveness. Furthermore, there was a lack of a way to automatically generate customizable tests based on the content of business manuals.

[0048] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0049] In this invention, the server includes means for receiving text data of a business manual as input, means for analyzing the received text data, means for extracting important keywords and concepts from the analyzed text data, means for generating questions based on the extracted keywords and concepts, means for outputting the generated questions, means for accepting answers from examinees to the generated test, and means for providing immediate feedback based on the accepted answers. This enables the automatic generation of customizable comprehension tests based on the contents of the business manual and immediate feedback to examinees on their answers.

[0050] A "business manual" is a document created by a company or organization to explain business procedures and rules to employees.

[0051] "Text data" is data that represents character information in digital form.

[0052] "Analysis" is the process of breaking down data and understanding its structure and meaning.

[0053] "Keywords" are words or phrases that are particularly important in the text data.

[0054] A "concept" is an abstract idea or theme expressed in text data.

[0055] "Questions" are questions presented to test takers to gauge their understanding.

[0056] "Generation" means creating new data or information using specific algorithms or models.

[0057] "Output" means providing the processing results to the outside.

[0058] "Test taker" means a person taking a comprehension test.

[0059] An "answer" is an answer provided by a test-taker to a question.

[0060] "Immediate feedback" refers to evaluations and comments provided immediately to test takers on their answers.

[0061] A "system" is a structure in which multiple elements work together to achieve a specific function.

[0062] MODE FOR CARRYING OUT THE INVENTION

[0063] This invention is a system that automatically generates comprehension tests based on the contents of a business manual and provides immediate feedback to test takers on their answers. A specific embodiment of this system will be described below.

[0064] 1. Program Generation

[0065] The server creates a program to generate a business manual comprehension test. This program has the function of analyzing the contents of the business manual and generating questions based on that content.

[0066] 2. Program Processing

[0067] The server receives the text data of the business manual as input. This text data can come from a file uploaded by the user or retrieved from a database. For example, if a user uploads a PDF file of the business manual, the server converts the PDF into text data.

[0068] Next, the server uses natural language processing technology to analyze the text data. For this analysis, it uses libraries such as Python's "NLTK" and "spaCy." This divides the text data into sentences and words and analyzes their grammatical structure.

[0069] The server extracts important keywords and concepts from the analyzed text data. This process uses techniques such as TF-IDF (Term Frequency-Inverse Document Frequency) and Word2Vec. For example, keywords such as "customer service," "quick," "polite," "accurate information," and "problem solving" are extracted.

[0070] Next, the server generates questions based on the extracted keywords and concepts. To generate these questions, it uses a generative AI model (for example, GPT-3 (registered trademark) from OpenAI (registered trademark)). The server inputs a prompt sentence to the generative AI model, instructing it to generate an appropriate question. For example, the server inputs the following prompt sentence:

[0071] "Please explain the basics of customer service. Why is prompt and courteous service required?"

[0072] The generative AI model generates questions based on this prompt.

[0073] 3. Specific Examples

[0074] For example, consider a scenario where a user wants to upload a PDF file of a business manual. The user uses a terminal to access the system interface and clicks the "Select File" button. Next, the user selects the PDF file of the business manual and clicks the "Upload" button. The server receives this PDF file and converts it into text data.

[0075] The server then analyzes the text data and extracts important keywords and concepts, such as "customer service," "speed," "polite," "accurate information," and "problem resolution."

[0076] The server then inputs a prompt to the generative AI model to generate a question, such as:

[0077] "What are the basics of customer service? Explain why prompt and courteous service is required."

[0078] Finally, the server provides the generated questions to the user, who then obtains a list of questions to use as comprehension tests. In this way, it is possible to automatically generate customizable comprehension tests based on the contents of the business manual and provide immediate feedback to the examinee's answers.

[0079] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0080] Step 1:

[0081] The server receives the text data of the business manual as input. When a user uploads a PDF file of the business manual using a terminal, the server converts the PDF file into text data. This conversion uses OCR (Optical Character Recognition) technology. The input is a PDF file, and the output is text data.

[0082] Step 2:

[0083] The server analyzes the received text data. This analysis uses natural language processing libraries such as Python's "NLTK" or "spaCy." Specifically, the text data is divided into sentences and words, and the grammatical structure is analyzed. The input is the text data, and the output is a list of analyzed sentences and words.

[0084] Step 3:

[0085] The server extracts important keywords and concepts from the analyzed text data. This process uses technologies such as TF-IDF (Term Frequency-Inverse Document Frequency) and Word2Vec. For example, keywords such as "customer service," "quick," "polite," "accurate information," and "problem solving" are extracted. The input is a list of analyzed sentences and words, and the output is a list of important keywords and concepts.

[0086] Step 4:

[0087] The server generates questions based on the extracted keywords and concepts. To generate these questions, it uses a generative AI model (for example, OpenAI's GPT-3). The server inputs a prompt to the generative AI model, instructing it to generate an appropriate question. For example, the server inputs the following prompt:

[0088] "Please explain the basics of customer service. Why is prompt and courteous service required?"

[0089] The generative AI model generates a question based on this prompt: the input is the prompt and the output is the generated question.

[0090] Step 5:

[0091] The server provides the generated questions to the user. The user can use a terminal to review the generated questions and make corrections or additions as needed. Finally, the user gets a list of questions to use as a comprehension test. The input is the generated questions, and the output is the final list of questions.

[0092] Step 6:

[0093] The server accepts the examinee's answers to the generated test. The examinee answers the test using a terminal and sends the answers to the server. The input is the examinee's answers and the output is the accepted answers.

[0094] Step 7:

[0095] The server provides instant feedback based on the received answers. The server uses a generative AI model to evaluate the answers and generate appropriate feedback, such as determining whether the answer is correct or incorrect and suggesting areas for improvement. The input is the received answers, and the output is instant feedback.

[0096] (Application example 1)

[0097] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Conventional business manual comprehension tests require manual creation, which is time-consuming and labor-intensive, and the quality of the tests is inconsistent. It is also difficult to immediately assess the test taker's level of understanding and provide feedback, making it difficult to measure the effectiveness of educational programs. Furthermore, there is a need for more efficient training for factory robot operators and engineers.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0100] In this invention, the server includes a means for generating an AI-based business manual comprehension test, a means for analyzing the text data of the business manual and extracting important keywords and concepts, a means for generating questions based on the extracted keywords and concepts, a means for presenting the generated questions to robot operators and engineers, a means for collecting response data and evaluating comprehension, and a means for providing feedback on the evaluation results and suggesting additional training. This enables the automatic generation of high-quality comprehension tests based on the contents of the business manual, and allows for the instant evaluation of the test taker's comprehension and the provision of feedback. It also improves the efficiency of training for factory robot operators and engineers.

[0101] "AI" is an abbreviation for artificial intelligence, a technology that allows computers to imitate human intelligence to learn and reason.

[0102] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[0103] A "generator" is a method or device for generating data or information for a particular purpose.

[0104] "Text data" is data that represents character information in digital form.

[0105] "Analysis" is the process of examining data or information in detail to understand its structure and meaning.

[0106] A "keyword" is a word or phrase that has significant meaning in a particular document or piece of data.

[0107] A "concept" represents an abstract understanding or recognition of a particular event or phenomenon.

[0108] A "question" is a question asked to elicit specific information.

[0109] A "robot operator" is an engineer or worker who operates factory robots.

[0110] An "engineer" is a person who has specialized knowledge and skills in a particular technical field.

[0111] "Response data" refers to information about responses provided by test takers to questions.

[0112] "Comprehension" refers to the degree to which a particular piece of content or information is accurately understood.

[0113] "Evaluation" is the process of judging the value or performance of an object based on specific criteria.

[0114] "Feedback" means providing a response or evaluation of a particular action or result.

[0115] "Training" means training or education to acquire specific skills or knowledge.

[0116] As an embodiment of the present invention, the following system is constructed.

[0117] System Configuration

[0118] The system consists of a server, factory robots, tablet terminals, and a network. The server plays a central role in generating, analyzing, and evaluating work manual comprehension tests. The factory robots present the generated tests to operators and engineers and collect their responses. The tablet terminals are used as an interface for users to take the tests.

[0119] Hardware and software used

[0120] Hardware: Servers, factory robots, tablets

[0121] Software: Python, TENSORFLOW (registered trademark), NLTK (Natural Language Toolkit), Flask (web framework)

[0122] Data processing and calculation

[0123] 1. Analysis of the business manual:

[0124] The server reads the text data of the business manual using Python and uses NLTK to tokenize and extract keywords, thereby extracting important keywords and concepts.

[0125] 2. Question generation:

[0126] The server generates questions using a generative AI model (e.g., GPT-3) based on the extracted keywords and concepts. The generated questions are used to measure comprehension and are in the form of multiple-choice, written answers, or practical tests.

[0127] 3. Present the test:

[0128] The factory robot then presents the generated questions to the robot operator or engineer, who then answers the questions via a tablet device.

[0129] 4. Collecting and Evaluating Responses:

[0130] The server collects the response data sent from the tablet device and runs a scoring algorithm to evaluate the level of comprehension. The evaluation results are immediately fed back to the user.

[0131] 5. Additional training suggestions:

[0132] Based on the evaluation results, the server will suggest additional training as needed, thereby improving the user's skills.

[0133] Specific examples

[0134] Contents of the operations manual: "Robot maintenance procedures"

[0135] Generated question: "What is the first task that should be performed in the robot maintenance procedure?"

[0136] Prompt Sentence Examples

[0137] Below are some example prompts to input to the generative AI model:

[0138] Based on the contents of the operations manual below, generate questions to gauge comprehension.

[0139] Operation Manual: Robot Maintenance Procedures

[0140] 1. Turn off the power

[0141] 2. Remove the outer cover

[0142] 3. Clean the interior

[0143] 4. Inspect the parts

[0144] Generate a question.

[0145] In this way, the system can automatically generate high-quality comprehension tests based on the contents of the operation manual, instantly evaluate the test taker's level of understanding, and provide feedback, thereby improving the efficiency of training for factory robot operators and engineers.

[0146] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0147] Step 1:

[0148] The server receives the text data of the business manual. It reads the text data of the business manual as input and analyzes the text data using Python. Specifically, it uses NLTK to tokenize the text and extract important keywords and concepts. The output is a list of the extracted keywords and concepts.

[0149] Step 2:

[0150] The server generates questions using a generative AI model (e.g., GPT-3) based on the extracted keywords and concepts. It receives a list of keywords and concepts as input and inputs a prompt sentence into the generative AI model. Specifically, it sends the prompt sentence to the generative AI model to generate questions. The output is a list of generated questions.

[0151] Step 3:

[0152] The server sends the generated questions to the factory robot. It receives the list of generated questions as input and sends them to the factory robot. Specifically, it transfers the question data to the factory robot via the network. The output is the question data sent to the factory robot.

[0153] Step 4:

[0154] The factory robot presents the generated questions to the robot operator or engineer. It receives question data sent from the server as input and displays it to the user via a tablet device. Specifically, it displays the question on the tablet device screen so that the user can answer it. The output is the user's answer data.

[0155] Step 5:

[0156] The server collects the response data sent from the tablet device and evaluates the level of understanding. It receives the user's response data as input and executes a scoring algorithm. Specifically, it analyzes the response data and evaluates the accuracy rate and level of understanding. The output is the evaluation result data.

[0157] Step 6:

[0158] The server feeds back the evaluation results to the user. It receives the evaluation result data as input and provides it to the user. Specifically, it displays the evaluation results on a tablet device and provides feedback to the user. The output is the feedback data provided to the user.

[0159] Step 7:

[0160] The server proposes additional training based on the evaluation results. It receives the evaluation result data as input and generates additional training content as needed. Specifically, it analyzes the evaluation results, identifies the user's weaknesses and areas for improvement, and proposes an appropriate training program. The output is the data of the proposed training program.

[0161] Example 2

[0162] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0163] In conventional testing systems, test question generation is done manually, which is time-consuming and labor-intensive. It is also difficult to immediately evaluate test takers' answers and provide feedback. Furthermore, it is difficult to generate customizable tests based on specific educational content. This poses a challenge in measuring the effectiveness of educational programs.

[0164] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0165] In this invention, the server includes means for generating test questions using a generative AI model, means for accepting answers from examinees to the generated test questions, means for storing the accepted answers in a database, and means for evaluating and displaying results based on the stored answers. This enables automatic generation and immediate evaluation of test questions, and enables the generation of customizable tests based on specific educational content and the measurement of the effectiveness of educational programs.

[0166] A "generative AI model" is an algorithm that uses artificial intelligence technology to automatically generate test questions based on specific prompts.

[0167] "Test questions" are questions that test takers answer and are generated by a generative AI model.

[0168] An "examinee" is a user who answers test questions.

[0169] "Means for accepting answers" refers to an interface that allows the system to receive and process answers entered by the examinee.

[0170] A "database" is a system for storing data such as test taker answers and generated test questions.

[0171] "Means for evaluating and displaying results" refers to a function that determines whether the test taker's answers are correct or incorrect and provides feedback on the results to the test taker.

[0172] A "customizable test" is a test whose content can be modified to suit specific educational content or purposes.

[0173] An "educational program" is a set of educational activities and materials designed to achieve specific learning objectives.

[0174] "Effectiveness measurement" is the process of evaluating the impact and results that an educational program has on test takers.

[0175] This invention is a system that uses a generative AI model to generate test questions, accepts answers from test takers, and evaluates them. Specific embodiments of this system are described below.

[0176] System configuration

[0177] server

[0178] The server hosts a web-based testing platform. The server uses the following hardware and software:

[0179] Hardware: High-performance server machine (e.g., general-purpose server)

[0180] Software: web server (e.g., Apache (registered trademark) HTTP Server), database (e.g., MySQL (registered trademark)), programming language (e.g., Python, JavaScript (registered trademark))

[0181] The server generates test questions using a generative AI model, which are then stored in a database for test takers to access.

[0182] Terminal

[0183] The terminal provides the interface for test takers to complete the test. The terminal uses the following hardware and software:

[0184] Hardware: PCs, tablets, smartphones

[0185] Software: Web browser (e.g., GOOGLE CHROME®)

[0186] The device displays the test questions received from the server and allows the test taker to enter their answers, which are then sent to the server in real time and stored.

[0187] User

[0188] The user (candidate) logs into the web-based testing platform and answers the provided test questions. The user follows these steps:

[0189] 1. Open a web browser and access the test platform URL.

[0190] 2. Enter your username and password on the login screen to log in.

[0191] 3. Enter your answers to the test questions that appear and click the submit button.

[0192] Specific examples

[0193] For example, if a test taker takes a math test, the server uses a generative AI model to generate questions like this:

[0194] Question 1: "Solve the following equation: 2x + 3 = 7"

[0195] Question 2: "Find the next term in the following sequence: 1, 4, 9, 16, ?"

[0196] The examinee enters the answers to these questions and submits them.

[0197] Prompt Sentence Examples

[0198] Enter the following prompt for the generative AI model:

[0199] "Generate math test questions. The questions should be at a middle school level and include equations and sequences."

[0200] Using this prompt, the generative AI model generates appropriate test questions.

[0201] The above is a specific embodiment for carrying out the present invention. This system enables automatic generation and instant evaluation of test questions, and enables the creation of customizable tests based on specific educational content and the measurement of the effectiveness of educational programs.

[0202] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0203] Step 1:

[0204] User Login

[0205] Input: Username, Password

[0206] Server: The server receives the username and password entered by the user on the login screen and checks them against the authentication information stored in the database. If authentication is successful, the server issues the user a session ID and redirects them to the dashboard screen.

[0207] Output: Authentication result, session ID

[0208] What happens: The server securely sends and receives the user's authentication information using SSL / TLS encryption. If authentication is successful, it starts the user's session and returns the URL of the dashboard screen.

[0209] Step 2:

[0210] Generate test questions

[0211] Input: Prompt statement (e.g., "Generate a middle school level math test question.")

[0212] Server: The server sends prompts to the generative AI model to generate test questions, which are then stored in a database.

[0213] Output: Generated test questions

[0214] Specific operation: The server calls the API of the generative AI model and sends a prompt. The generative AI model generates test questions based on the prompt and returns them to the server. The server stores the generated test questions in a database.

[0215] Step 3:

[0216] View test questions

[0217] Input: User request (request to start test)

[0218] Server: The server retrieves the generated test questions from the database and sends them to the user's device.

[0219] Terminal: The terminal displays the test questions received from the server to the user.

[0220] Output: Test question display screen

[0221] Specific operation: The server receives a user request, retrieves the corresponding test questions from the database, and sends the retrieved test questions in HTML format to the terminal, which then displays them.

[0222] Step 4:

[0223] User answer input

[0224] Input: User answer (e.g. "x = 2")

[0225] Terminal: The terminal provides an input field to accept user-entered answers and temporarily stores the data as the user enters the answers.

[0226] Output: User response data

[0227] What it does: The device validates your answers in real time as you enter them, providing feedback to help prevent typing errors. Your answers are temporarily stored in the device's memory.

[0228] Step 5:

[0229] Submit and save your answers

[0230] Input: User response data

[0231] Terminal: The terminal sends the answers entered by the user to the server.

[0232] Server: The server stores the received answer in a database and sends a confirmation message to the user if the save was successful.

[0233] Output: Save confirmation message

[0234] Specific operation: When the user clicks the send button, the device sends the answer data to the server. The server saves the received answer data in the database and returns a message to the device indicating that the save was successful.

[0235] Step 6:

[0236] Displaying the results

[0237] Input: Saved response data

[0238] Server: The server evaluates the user's answers and generates a result, e.g., whether the answer is correct or incorrect and calculates a score.

[0239] Terminal: The terminal displays the results received from the server to the user.

[0240] Output: Evaluation results, scores

[0241] Specific operation: The server retrieves the user's answer data from the database and uses an evaluation algorithm to determine whether it is correct or incorrect. It calculates the evaluation results and score and sends them to the terminal. The terminal then displays the received evaluation results and score to the user.

[0242] The above are the specific processing steps of the program of this system.

[0243] (Application example 2)

[0244] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0245] Conventional business manual comprehension testing systems have the problem that test generation is done manually, which takes time and effort. Also, there is a possibility that learning effectiveness may be reduced because immediate feedback on test takers' answers is not provided. Furthermore, because the test content is fixed, there is also the issue that it is difficult to customize according to the comprehension level of each individual test taker.

[0246] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0247] In this invention, the server includes means for generating a business manual comprehension test using a generative AI model, means for accepting answers from examinees to the test generated by the generation means, means for providing immediate feedback based on the accepted answers, means for generating a test by sending prompts to the generative AI model, means for providing the generated test content to the user, and means for the user to answer the test and evaluate the answer content. This enables automatic test generation and provision of immediate feedback, making it possible to provide a test customized according to the examinee's level of understanding.

[0248] A "generative AI model" is an algorithm that uses artificial intelligence to generate an output for a specific task.

[0249] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[0250] A "generator" is a function for automatically creating tests for a specific purpose.

[0251] "Test taker" means a person taking a test.

[0252] "Means for accepting answers" refers to the function that allows the system to receive answers entered by test takers for the test.

[0253] "Immediate feedback" is a feature that provides evaluations and comments on test takers' answers immediately after they answer a test.

[0254] A "prompt" is an instruction entered into a generative AI model to cause it to generate a specific output.

[0255] "User" refers to a person who uses the system.

[0256] "Means of evaluation" refers to a function for determining the accuracy and level of understanding of the test taker's answers.

[0257] The embodiments of the present invention will be described in detail below.

[0258] First, the server uses a generative AI model to generate a business manual comprehension test. The generation method involves sending a prompt to the generative AI model, which has the ability to automatically create a specific test. An example of a prompt is "Please generate basic math problems."

[0259] The server then provides a means to accept test takers' responses to the generated tests. Test takers can log in to a web-based testing platform using devices such as smartphones or PCs and take the tests.

[0260] When a test taker answers the test, the server immediately evaluates the answer and provides feedback. The evaluation method has a function to determine the accuracy and level of understanding based on the test taker's answer. For example, if a test taker answers "2 + 2 = 4," the server immediately evaluates whether the answer is correct and provides feedback that "That's correct" if it is.

[0261] The implementation of this system uses the following hardware and software:

[0262] Hardware: Servers, smartphones, PCs

[0263] Software: Flask (Python web framework), OpenAI API (generative AI model)

[0264] For example, if a user inputs the prompt "Generate basic math problems," the generative AI model will generate math problems and provide them to the user. The user answers the generated questions, and the answers are sent to the server. The server evaluates the answers and provides immediate feedback.

[0265] In this way, tests can be automatically generated and immediate feedback can be provided, allowing tests to be customized according to the examinee's level of understanding.

[0266] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0267] Step 1:

[0268] A user logs into a web-based testing platform using a terminal.

[0269] Input: User login information (username, password)

[0270] Output: Login success or failure message

[0271] Specific operation: The user enters a username and password into the login form and sends it to the server, which checks it against the database and returns the authentication result.

[0272] Step 2:

[0273] The user inputs a prompt statement into the generative AI model to request test generation.

[0274] Input: Prompt sentence (e.g. "Generate a basic math problem")

[0275] Output: Generated test content

[0276] Specific operation: The user enters a prompt into the test generation form and sends it to the server. The server then sends the prompt to the generative AI model and receives the generated test content.

[0277] Step 3:

[0278] The server provides the generated test content to the user.

[0279] Input: Generated test content

[0280] Output: Test results displayed on the user's device

[0281] Specific operation: The server sends the generated test content to the user's terminal and displays it on the user's screen.

[0282] Step 4:

[0283] The user answers the quiz.

[0284] Input: User's answer

[0285] Output: Response data sent to the server

[0286] Specific operation: The user enters answers to each question in the test and sends the answers to the server.

[0287] Step 5:

[0288] The server evaluates the user's answers.

[0289] Input: User response data

[0290] Output: Evaluation results (correct answers, incorrect answers, score, etc.)

[0291] Specific operation: The server analyzes the user's answer data, compares it with the correct answer, and generates an evaluation result.

[0292] Step 6:

[0293] The server provides the evaluation results to the user as real-time feedback.

[0294] Input: Evaluation result

[0295] Output: A feedback message that is displayed on the user's terminal.

[0296] Specific operation: The server sends the evaluation results to the user's device and displays a feedback message on the user's screen.

[0297] Example 3

[0298] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0299] Conventional business manual comprehension tests have the problem of reducing learning effectiveness because test takers' answers are not evaluated and feedback is not provided immediately. In addition, tests are generated in a fixed format, making it difficult to customize them to fit the content of the business manual. Furthermore, because feedback is not directly linked to measuring the effectiveness of the educational program, there is a lack of data to improve the quality of education.

[0300] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0301] In this invention, the server includes a means for generating a business manual comprehension test using a generative AI model, a means for accepting test takers' answers, a means for analyzing the accepted answers using natural language processing technology and evaluating their accuracy, and a means for providing immediate feedback based on the evaluation results. This allows for immediate evaluation and feedback of the test takers' answers, improving learning effectiveness. Furthermore, generating customizable tests based on the contents of the business manual allows for more appropriate evaluation. Furthermore, the immediate feedback enables the effectiveness of educational programs to be measured, allowing for the collection of data to improve the quality of education.

[0302] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate answers or data for specific tasks.

[0303] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[0304] A "generation means" is a method or device for generating data or information for a specific purpose.

[0305] "Candidate" refers to a person taking a test or examination.

[0306] "Means for accepting answers" refers to a method or device by which the system receives answers entered by the examinee.

[0307] "Natural language processing technology" is a technology that allows computers to understand and analyze human language.

[0308] An "evaluation means" is a method or device that uses received data or information to determine its accuracy or appropriateness.

[0309] "Immediate feedback" refers to providing test takers with prompt evaluation results and advice in response to their answers.

[0310] A "customizable test" is a test whose content can be modified to suit specific conditions or needs.

[0311] An "educational program" is a set of learning activities and materials designed to acquire specific knowledge or skills.

[0312] "Effectiveness measurement" refers to assessing the success of a particular activity or program.

[0313] This invention is a system that uses a generative AI model to generate a business manual comprehension test, analyzes the test taker's answers, and provides immediate feedback. Specific embodiments of this system are described below.

[0314] Hardware and software used

[0315] Hardware: Servers, terminals

[0316] Software: Generative AI models, Natural Language Processing (NLP)

[0317] System Overview

[0318] 1. The user enters the answer

[0319] The user inputs a response into an input form on the device. For example, in response to the question "What color is the Earth?", the user inputs "blue."

[0320] 2. The device sends the answer to the server

[0321] The terminal sends the answer entered by the user to the server. Specifically, the terminal sends the answer data to the server using an HTTP request.

[0322] 3. The server receives the answer

[0323] The server receives the HTTP request sent from the device and extracts the response data. For example, it verifies that the received data is "blue."

[0324] 4. The server parses the answer

[0325] The server analyzes the received response data using natural language processing (NLP). Specifically, it inputs a prompt sentence into the generative AI model to perform the analysis. An example of a prompt sentence is "Question: What color is the Earth? Answer: Blue."

[0326] 5. The server generates the evaluation result

[0327] The server evaluates the accuracy of the answer based on the analysis results from the generative AI model. For example, if the generative AI model determines that the answer "blue" is accurate, it generates that evaluation result.

[0328] 6. The server generates feedback

[0329] The server generates feedback for the user based on the evaluation result, for example, a feedback message such as "You're right. The Earth is blue."

[0330] 7. The server sends feedback to the device

[0331] The server sends the generated feedback message to the terminal. Specifically, the server sends the feedback data to the terminal using an HTTP response.

[0332] 8. The device displays feedback to the user

[0333] The device displays the feedback message received from the server to the user, for example, a message such as "You're right! The Earth is blue."

[0334] Specific examples

[0335] For example, when a user answers "blue" to the question "What color is the Earth?", the specific operation is as follows.

[0336] 1. The user enters the answer

[0337] The user enters "blue" into the input form on the device.

[0338] 2. The device sends the answer to the server

[0339] The device uses an HTTP request to send the answer data "blue" to the server.

[0340] 3. The server receives the answer

[0341] The server receives the HTTP request and extracts the response data "blue."

[0342] 4. The server parses the answer

[0343] The server inputs the prompt sentence "Question: What color is the Earth? Answer: Blue" into the generative AI model and performs analysis.

[0344] 5. The server generates the evaluation result

[0345] Based on the analysis results from the generative AI model, the server evaluates that the answer "blue" is accurate.

[0346] 6. The server generates feedback

[0347] The server generates a feedback message saying "Correct. The Earth is blue."

[0348] 7. The server sends feedback to the device

[0349] The server sends a feedback message to the device using an HTTP response.

[0350] 8. The device displays feedback to the user

[0351] The device will display the message "Correct. The Earth is blue." on the screen.

[0352] In this way, the user can receive instant feedback, thereby improving the learning effect. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0353] Step 1:

[0354] The user enters the answer.

[0355] Input: A user types a response into a device input form. For example, the user types "blue" in response to the question "What color is the Earth?"

[0356] Output: The entered answer data is saved on the device.

[0357] What happens: The user uses the keyboard to enter an answer and clicks the submit button.

[0358] Step 2:

[0359] The device sends the response to the server.

[0360] Input: The answer data entered by the user.

[0361] Output: The response data is sent to the server as an HTTP request.

[0362] Specific operation: The terminal sends the answer data entered by the user to the server in the form of an HTTP request.

[0363] Step 3:

[0364] The server receives the response.

[0365] Input: The HTTP request sent from the device.

[0366] Output: The extracted response data.

[0367] Specific operation: The server receives the HTTP request and extracts the response data. For example, it obtains the response data "blue."

[0368] Step 4:

[0369] The server parses the response.

[0370] Input: The extracted response data.

[0371] Output: Analysis results.

[0372] Specific operation: The server inputs a prompt sentence into the generative AI model and performs analysis. For example, the prompt sentence "Question: What color is the Earth? Answer: Blue" is input into the generative AI model and the analysis result is obtained.

[0373] Step 5:

[0374] The server generates the evaluation results.

[0375] Input: Analysis results from a generative AI model.

[0376] Output: Evaluation result.

[0377] Specific operation: The server evaluates the accuracy of the answer based on the analysis results from the generative AI model. For example, it determines that the answer "blue" is accurate and generates an evaluation result.

[0378] Step 6:

[0379] The server generates the feedback.

[0380] Input: Evaluation result.

[0381] Output: The feedback message.

[0382] Specific operation: The server generates feedback for the user based on the evaluation result. For example, it generates a feedback message such as "You're right. The Earth is blue."

[0383] Step 7:

[0384] The server sends the feedback to the device.

[0385] Input: The feedback message.

[0386] Output: The feedback message is sent to the device as an HTTP response.

[0387] Specific operation: The server sends the generated feedback message to the terminal in the form of an HTTP response.

[0388] Step 8:

[0389] The device displays the feedback to the user.

[0390] Input: The feedback message sent by the server.

[0391] Output: Feedback messages displayed on the screen.

[0392] Specific operation: The device displays the feedback message received from the server on the screen, for example, "You're right. The Earth is blue."

[0393] (Application example 3)

[0394] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0395] Conventional business manual comprehension test systems rarely provide immediate feedback on test takers' answers, which reduces learning effectiveness. Furthermore, in factories and other workplaces, errors in work procedures can have a negative impact on production efficiency and quality, so real-time evaluation and feedback are required. To solve these issues, an immediate feedback system using AI technology is needed.

[0396] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0397] In this invention, the server includes means for generating an AI-based business manual comprehension test, means for accepting test takers' answers to the test generated by the generating means, means for providing immediate feedback based on the accepted answers, means for evaluating work procedures in real time, and means for providing immediate feedback to workers based on the evaluation results. This increases the learning effect of test takers and reduces errors in work procedures at work sites such as factories, thereby enabling improved production efficiency and quality.

[0398] "A means for generating business manual comprehension tests using AI" is a function that uses artificial intelligence technology to automatically create comprehension tests based on the contents of business manuals.

[0399] The "means for accepting answers from examinees" is a function for receiving answers given by examinees to comprehension tests as input.

[0400] "Means for providing immediate feedback based on accepted answers" refers to a function that analyzes the answers of test takers and provides feedback in real time based on the results.

[0401] The "means for evaluating work procedures in real time" is a function that monitors the work procedures performed by workers in real time and evaluates their accuracy and efficiency.

[0402] The "means for providing immediate feedback to workers based on the evaluation results" is a function for providing feedback to workers in real time based on the evaluation results of work procedures.

[0403] A system for implementing this invention is configured as follows: First, the server is equipped with a means for generating a business manual comprehension test using AI. This generation means has the function of automatically creating a comprehension test based on the contents of the business manual. Specifically, it uses natural language processing technology to analyze the text of the business manual and generate appropriate questions.

[0404] Next, the server is provided with a means for accepting answers from examinees. This means has a function for receiving answers given by examinees to comprehension tests as input. Examinees can input their answers using devices such as smartphones and tablets.

[0405] Furthermore, the server is equipped with a means for providing immediate feedback based on the answers received. This means has the function of analyzing the examinee's answers and providing real-time feedback based on the results. For example, if the examinee gives an incorrect answer, the correct answer and the reason for it are immediately displayed.

[0406] The server also has a means for evaluating work procedures in real time. This means has the function of monitoring the work procedures performed by workers in real time and evaluating their accuracy and efficiency. Specifically, the work procedures are monitored using cameras and sensors installed on factory robots, and evaluations are made using AI models.

[0407] Finally, the server is equipped with a means for providing immediate feedback to workers based on the evaluation results. This means has the function of providing real-time feedback to workers based on the evaluation results of their work procedures. For example, if a worker performs an incorrect procedure, the correct procedure will be instructed via voice or display.

[0408] The hardware used includes factory robots (e.g., robotic arms), smartphones, and tablets. The software used is Python, spaCy, and the Transformers library. Combining these technologies will enhance the learning effect of test takers and reduce errors in work procedures in factories and other workplaces, thereby improving production efficiency and quality.

[0409] As a specific example, if a worker performs a task in the following order, "Attach part A. Tighten the screws. Attach part B," the system will determine that the "tighten the screws" step is incorrect and provide feedback such as, "Step 2 is incorrect. Correct step: Attach part B."

[0410] Examples of prompt sentences include the following:

[0411] "Create a system that evaluates work procedures and provides feedback if there are any errors. It takes the text of the work procedures as input, compares it with the correct procedure, and generates a feedback message pointing out the errors. The techniques used are natural language processing and AI models."

[0412] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0413] Step 1:

[0414] The server receives the text data of the business manual as input. It analyzes the text data using natural language processing technology and generates comprehension test questions. Specifically, it uses the spaCy library to divide the text into sentences and extract important keywords and phrases. Based on this, it uses a generative AI model to create appropriate questions. The output is a list of generated comprehension test questions.

[0415] Step 2:

[0416] The terminal receives as input the examinee's answers to the comprehension test. The examinee enters the answers using a smartphone or tablet. The entered answers are sent to the server. The output is the examinee's answer data.

[0417] Step 3:

[0418] The server receives and analyzes the test taker's response data. Specifically, it uses a generative AI model to evaluate the accuracy of the answers. Based on the evaluation results, it generates an immediate feedback message. For example, it creates a feedback message that includes the correct answer and the reason for it. The output is the feedback message.

[0419] Step 4:

[0420] The server sends a feedback message to the terminal. The terminal displays the feedback message to the test taker. The test taker can check the feedback and improve their understanding. The output is the feedback message displayed to the test taker.

[0421] Step 5:

[0422] The server receives real-time data on work procedures from cameras and sensors installed on factory robots. Specifically, it monitors the work procedures performed by workers and collects data. The input is data from cameras and sensors.

[0423] Step 6:

[0424] The server analyzes the collected work procedure data and evaluates its accuracy and efficiency. Specifically, it uses an AI model to evaluate the work procedures and, if there are any errors, identifies the errors. The output is the evaluation result of the work procedures.

[0425] Step 7:

[0426] The server provides immediate feedback to the worker based on the evaluation results. Specifically, it instructs the correct procedure through voice or display. For example, it provides feedback such as "Procedure 2 is incorrect. Correct procedure: Install part B." The output is a feedback message provided to the worker.

[0427] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0428] "Example 1"

[0429] One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the examinee's answers and recognizes their emotional state. Specifically, the emotion engine estimates the examinee's emotional state from the examinee's typing speed, mouse movements, and eye movements as they answer the test. This information is used to evaluate not only the examinee's level of comprehension, but also their learning progress and psychological state.

[0430] "Example 2"

[0431] The emotion engine also recognizes emotions based on the examinee's answers and reflects the results in the feedback. Specifically, it analyzes how the examinee feels about the questions and how those emotions change over time, and provides the results as feedback to the examinee. This allows the examinee to gain a deeper understanding of their own learning situation. "Format Example 3"

[0432] Furthermore, the emotion engine tracks changes in the examinee's emotions and uses the results to measure the effectiveness of the educational program. Specifically, it records changes in emotions such as joy, confusion, excitement, and anxiety that the examinee experiences as they progress through their studies, and reflects these changes in the effectiveness of the educational program. This allows for a more accurate evaluation of the program's effectiveness.

[0433] The processing flow of each embodiment will be described below.

[0434] "Example 1"

[0435] Step 1: The test taker begins answering the test.

[0436] Step 2: The emotion engine analyzes the test taker's keyboard typing speed, mouse movements, eye movements, etc.

[0437] Step 3: The emotion engine uses this information to estimate the test-taker's emotional state.

[0438] Step 4: This emotional state information is used to assess the test taker's comprehension, as well as their learning situation and psychological state.

[0439] "Example 2"

[0440] Step 1: The candidate answers the test.

[0441] Step 2: The emotion engine recognizes emotions based on the test taker's answers.

[0442] Step 3: The emotion engine analyzes how test takers feel about the question and how their emotions change over time.

[0443] Step 4: Provide feedback to the test taker on the results of this analysis.

[0444] "Example 3"

[0445] Step 1: Candidates progress in their studies.

[0446] Step 2: The emotion engine tracks changes in the test taker's emotions.

[0447] Step 3: The emotion engine records changes in emotions such as joy, confusion, excitement, and anxiety as the test-taker progresses through the study.

[0448] Step 4: Use this record to measure the effectiveness of the educational program.

[0449] Example 1

[0450] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0451] In conventional business manual comprehension tests, questions to assess the test taker's level of comprehension are often generated manually, which is inefficient. In addition, the test taker's emotional and psychological state is not taken into account in the assessment, making it difficult to accurately grasp the test taker's true level of comprehension and learning situation.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0453] In this invention, the server includes means for inputting text data of a business manual, means for analyzing the input text data and extracting important keywords and concepts, means for generating questions based on the extracted keywords and concepts, means for collecting data when examinees answer tests, means for analyzing the collected data and estimating the examinees' emotional states, and means for evaluating the examinees' levels of comprehension and emotional states based on the results of an analysis of the examinees' answers to the generated questions and their emotional states. This enables efficient question generation based on the contents of the business manual and accurate evaluation of the examinees' levels of comprehension that takes into account the examinees' emotional states.

[0454] A "business manual" is a document that describes the procedures and rules for a company or organization to carry out its business.

[0455] "Text data" is data that represents character information in digital form.

[0456] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[0457] A "keyword" is a word or phrase that has a particularly important meaning in text data.

[0458] A "concept" is an abstract idea or way of thinking about a particular phenomenon or thing.

[0459] "Question generation" is the process of creating a question based on specific information.

[0460] "Candidate" refers to a person taking a test or examination.

[0461] "Data collection" is the process of gathering information needed for a specific purpose.

[0462] "Emotional state" refers to an individual's psychological state or mood.

[0463] "Evaluation" is the process of judging the value or performance of an object based on specific criteria.

[0464] MODE FOR CARRYING OUT THE INVENTION

[0465] The present invention provides a system for efficiently generating comprehension tests for business manuals and evaluating test takers while taking into account their emotional state. Specific embodiments of this system are described below.

[0466] Inputting text data for business manuals

[0467] The user uploads the text data of the business manual to the system. Specifically, the user selects the file of the business manual and clicks the upload button, which sends the text data to the server.

[0468] Text data analysis and keyword extraction

[0469] The server analyzes the text data of the received business manual. Specifically, it uses the Python natural language processing library "spaCy" to extract important keywords and concepts from the text data. For example, keywords such as "customer service," "basic procedures," and "satisfaction level" are extracted.

[0470] Question Generation

[0471] The server generates questions based on the extracted keywords and concepts. This process uses the generative AI model "GPT-3." Specifically, the extracted keywords are input into GPT-3 as prompts, and an appropriate question is generated. For example, the question generated might be, "What is the first step in the basic customer service procedure?"

[0472] Collection of test taker response data

[0473] The device collects data when test takers answer questions, such as keyboard typing speed, mouse movements, and eye movements on the screen. This data is acquired using hardware such as eye tracking devices and input devices.

[0474] Emotional state analysis

[0475] The server analyzes the data sent from the device and estimates the test-taker's emotional state. Specifically, it uses an emotion recognition algorithm to estimate the test-taker's emotional state from their keyboard typing speed, mouse movements, and eye movements. This analysis uses an emotion recognition model using "OpenCV" and "TensorFlow."

[0476] Assessment of comprehension and emotional state

[0477] The server evaluates the examinee's level of understanding and emotional state based on their answers to the generated questions and the results of an analysis of their emotional state. Specifically, it comprehensively evaluates the accuracy of the answers, the response time, and the emotional state data to determine the examinee's learning situation and psychological state.

[0478] Examples of specific examples and prompts

[0479] As a concrete example, if the business manual describes "basic procedures for dealing with customers," the server generates the question "What is the first step in the basic procedures for dealing with customers?"

[0480] Below are some example prompts to input to a generative AI model:

[0481] 1. Generate comprehension test questions based on the contents of the following operations manual: Operations manual: 'The basic procedure for dealing with customers begins with accurately listening to the customer's request. Next, you propose an appropriate response, and finally you confirm the customer's satisfaction.'

[0482] 2. "Please estimate the test taker's emotional state from their keyboard typing speed, mouse movements, and eye movements. The following data was collected: 'Key typing speed: fast, mouse movements: frequent, eye movements: focused'"

[0483] This allows the system to generate questions based on the contents of the job manual and analyze the test taker's emotional state.

[0484] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0485] Program processing flow

[0486] Step 1: Enter the text data for the business manual

[0487] The user uploads the text data of the business manual to the system. Specifically, the user selects the file of the business manual and clicks the upload button, which sends the text data to the server.

[0488] Input: Business manual text file

[0489] Output: Text data sent to the server

[0490] Step 2: Analyzing text data and extracting keywords

[0491] The server analyzes the text data of the received business manuals, specifically using the Python natural language processing library "spaCy" to extract important keywords and concepts from the text data.

[0492] Input: Text data of the business manual

[0493] Data processing: Extracting keywords and concepts using natural language processing technology

[0494] Output: Extracted keywords and concepts

[0495] Step 3: Generate questions

[0496] The server generates questions based on the extracted keywords and concepts. This process uses the generative AI model "GPT-3." Specifically, the extracted keywords are input into GPT-3 as prompts, and appropriate questions are generated.

[0497] Input: Extracted keywords and concepts

[0498] Data Computing: Generating Questions Using Generative AI Models

[0499] Output: Generated questions

[0500] Step 4: Collect test-taker response data

[0501] The device collects data when test takers answer questions, such as keyboard typing speed, mouse movements, and eye movements on the screen. This data is acquired using hardware such as eye tracking devices and input devices.

[0502] Input: Test taker response behavior data (typing speed, mouse movement, eye movement)

[0503] Output: Collected behavioral data

[0504] Step 5: Analyze emotional state

[0505] The server analyzes the data sent from the device and estimates the test-taker's emotional state. Specifically, it uses an emotion recognition algorithm to estimate the test-taker's emotional state from their keyboard typing speed, mouse movements, and eye movements. This analysis uses an emotion recognition model using "OpenCV" and "TensorFlow."

[0506] Input: Collected behavioral data

[0507] Data Computing: Estimating emotional states using emotion recognition algorithms

[0508] Output: Estimated emotional state

[0509] Step 6: Assess your comprehension and emotional state

[0510] The server evaluates the examinee's level of understanding and emotional state based on their answers to the generated questions and the results of an analysis of their emotional state. Specifically, it comprehensively evaluates the accuracy of the answers, the response time, and the emotional state data to determine the examinee's learning situation and psychological state.

[0511] Input: test taker's answers, estimated emotional state

[0512] Data calculation: Comprehensive evaluation of answer accuracy, response time, and emotional state

[0513] Output: Evaluation results of the test taker's comprehension and emotional state

[0514] (Application example 1)

[0515] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0516] Conventional business manual comprehension test systems not only evaluate the examinee's level of understanding, but also have difficulty in comprehensively evaluating the examinee's emotional state and learning situation. Furthermore, they are unable to provide feedback that takes the examinee's psychological state into account, making it difficult to provide appropriate guidance to maximize learning effectiveness. Furthermore, they are unable to generate comprehension tests specifically for factory robot operation manuals, making it difficult to efficiently support the improvement of operator skills.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0518] In this invention, the server includes means for generating an AI-based business manual comprehension test, means for accepting examinees' answers to the test generated by the generation means, means for providing immediate feedback based on the accepted answers, means for analyzing the examinee's keyboard typing speed, pointing device movement, and eye movement when answering, means for estimating the examinee's emotional state based on the analyzed data, and means for evaluating the examinee's learning situation and psychological state based on the estimated emotional state. This makes it possible to comprehensively evaluate not only the examinee's level of understanding but also their emotional state and learning situation and provide appropriate feedback.

[0519] "A means for generating business manual comprehension tests using AI" is a means for analyzing the contents of business manuals and automatically generating comprehension tests based on those contents.

[0520] The "means for accepting answers from examinees" refers to a means for accepting, as input, answers from examinees to the generated comprehension test.

[0521] A "means for providing immediate feedback" is a means for providing real-time feedback based on the test-taker's responses.

[0522] "Keyboard typing speed" refers to the typing speed when the examinee operates the keyboard.

[0523] "Pointing device movements" refers to the movements made by the test taker when operating a pointing device such as a mouse or touchpad.

[0524] "Eye movement" refers to how the examinee's eyes move across the screen.

[0525] The "analyzing means" refers to a means for analyzing data such as keyboard typing speed, pointing device movement, and eye movement.

[0526] The "means for estimating an emotional state" is a means for estimating the emotional state of the test taker based on the analyzed data.

[0527] "Means for assessing learning situation and psychological state" refers to means for assessing the learning situation and psychological state of the examinee based on the estimated emotional state.

[0528] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The following explains how each element functions.

[0529] server

[0530] The server includes the following means:

[0531] 1. AI-based method for generating business manual comprehension tests:

[0532] The server analyzes the text data of the business manual and extracts important keywords and concepts using natural language processing technology. This analysis uses a natural language processing model (e.g., spaCy).

[0533] Based on the extracted keywords and concepts, comprehension test questions are generated, which are automatically created using a generative AI model.

[0534] 2. How to receive test taker responses:

[0535] The server receives the examinee's answers sent from the terminal, which are sent to the server in text format.

[0536] 3. Means of providing immediate feedback:

[0537] The server generates real-time feedback based on the examinee's answers and sends it to the device, allowing the effectiveness of the educational program to be measured.

[0538] 4. Methods for analyzing keyboard typing speed, pointing device movements, and eye movements:

[0539] The server analyzes data sent from the device, such as keyboard typing speed, pointing device movement, and eye movement, using libraries such as OpenCV.

[0540] 5. Means of estimating emotional state:

[0541] The server estimates the test-taker's emotional state based on the analyzed data, using an emotion analysis model (e.g., transformers pipeline).

[0542] 6. Methods for assessing learning and psychological status:

[0543] The server evaluates the test-taker's learning progress and psychological state based on the estimated emotional state and provides appropriate feedback.

[0544] Terminal

[0545] The terminal is a device that allows users to take the comprehension test. The terminal has the following functions:

[0546] 1. Show test:

[0547] The terminal displays the comprehension test questions sent from the server.

[0548] 2. Enter your answer:

[0549] The user uses a terminal to input test answers, which are then sent to the server in text format.

[0550] 3. Recording user behavior:

[0551] The terminal records the user's keyboard typing speed, pointing device movements, eye movements, etc. and sends them to the server.

[0552] User

[0553] A user is an individual who takes a comprehension test. A user takes a test by following these steps:

[0554] 1. Start the test:

[0555] The user starts the comprehension test using the terminal.

[0556] 2. Answers to questions:

[0557] The user inputs answers to the questions displayed.

[0558] 3. Receiving feedback:

[0559] The user receives real-time feedback sent from the server.

[0560] Specific examples

[0561] For example, parse the operating manual for a factory robot and generate questions like:

[0562] "Please explain how to activate the robot."

[0563] "Please explain how to maintain the robot."

[0564] If the user answers, "Starting the robot is easy," the server analyzes this response and estimates the user's emotional state. Furthermore, it evaluates the user's learning status and psychological state based on data such as keyboard typing speed and eye movement.

[0565] Prompt Sentence Examples

[0566] "Analyze the following text and extract key keywords: This is an operation manual for a factory robot. It describes how to start, stop, and maintain the robot."

[0567] "Please analyze the emotional state of the text below: Activating the robot is easy."

[0568] In this way, a system is realized in which the server, terminal, and user work together to conduct comprehension tests and provide comprehensive evaluations and feedback.

[0569] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0570] Step 1:

[0571] The server receives the text data of the business manual. It receives the text data of the business manual as input and uses natural language processing technology to extract important keywords and concepts. Specifically, it uses a natural language processing model (e.g., spaCy) to analyze the text and extract keywords. The output is the extracted keywords and concepts.

[0572] Step 2:

[0573] The server generates comprehension test questions based on the extracted keywords and concepts. It receives the extracted keywords and concepts as input and automatically creates questions using a generative AI model. Specifically, it inputs a prompt sentence into the generative AI model and generates a question. The output is the generated question.

[0574] Step 3:

[0575] The server sends the generated question to the terminal. It receives the generated question as input and sends it to the terminal. Specifically, it sends the question to the terminal via the network. The output is the question displayed on the terminal.

[0576] Step 4:

[0577] The terminal displays the question sent from the server to the user. It receives the question sent from the server as input and displays it to the user. Specifically, it displays the question on the terminal's display. The output is the question displayed to the user.

[0578] Step 5:

[0579] The user inputs an answer to a displayed question. The user receives the displayed question as input and inputs an answer. Specifically, the answer is input using a keyboard or a touch screen. The output is the input answer.

[0580] Step 6:

[0581] The terminal sends the user's answer to the server. It receives the answer entered by the user as input and sends it to the server. Specifically, it sends the answer to the server via the network. The output is the answer sent to the server.

[0582] Step 7:

[0583] The server analyzes the received answers and generates immediate feedback. The server analyzes the answers received as input and generates appropriate feedback. Specifically, it analyzes the content of the answers and measures the effectiveness of the educational program. The output is the generated feedback.

[0584] Step 8:

[0585] The server transmits the generated feedback to the terminal. The server receives the generated feedback as input and transmits it to the terminal. Specifically, the server transmits the feedback to the terminal via a network. The output is the feedback displayed on the terminal.

[0586] Step 9:

[0587] The terminal displays the feedback sent from the server to the user. The terminal receives the feedback sent from the server as input and displays it to the user. Specifically, the terminal displays the feedback on the display of the terminal. The output is the feedback displayed to the user.

[0588] Step 10:

[0589] The server analyzes the user's keyboard typing speed, pointing device movement, and eye movement. It receives and analyzes the user's behavioral data sent from the device as input. Specifically, it analyzes the data using libraries such as OpenCV. The output is the analyzed behavioral data.

[0590] Step 11:

[0591] The server estimates the user's emotional state based on the analyzed behavioral data. It receives the analyzed behavioral data as input and estimates the emotional state using an emotion analysis model. Specifically, it estimates the emotional state using a pipeline of transformers. The output is the estimated emotional state.

[0592] Step 12:

[0593] The server evaluates the user's learning situation and psychological state based on the estimated emotional state. It receives the estimated emotional state as input and evaluates the learning situation and psychological state. Specifically, it performs a comprehensive evaluation using an evaluation algorithm. The output is the evaluation result.

[0594] Example 2

[0595] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0596] Conventional test systems rarely provide immediate feedback on test takers' answers, and in particular lack feedback that takes test takers' emotions into account. This can prevent test takers from fully understanding their own learning situation and emotional changes, potentially reducing their learning effectiveness. Another issue is the difficulty of generating customizable tests based on work manuals.

[0597] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0598] In this invention, the server includes a test generation unit that utilizes a generative AI model, a unit that accepts examinee responses to the generated test, a unit that recognizes emotions based on the accepted answers, and a unit that provides immediate feedback based on the recognized emotions. This allows examinees to understand their own learning status and emotional changes in real time, which is expected to improve learning effectiveness. It also makes it easy to generate customizable tests based on work manuals.

[0599] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to automatically generate test questions and content.

[0600] "Test generation means" refers to the function or process that uses generative AI models to create test questions.

[0601] "Test taker" means an individual or group who takes a test and answers questions.

[0602] "Means for accepting answers" refers to the functions and processes by which the system receives and stores answers entered by test takers.

[0603] "Means for recognizing emotions" refers to algorithms or systems that analyze test-taker responses and identify the emotions contained within them.

[0604] "Means for providing immediate feedback" refers to the functionality or process of generating and providing feedback in real time based on test-taker responses and perceived sentiment.

[0605] A "customizable test" is a test whose content can be tailored based on specific business manuals or learning content.

[0606] "Educational program effectiveness measurement" refers to the process and means for evaluating the impact and effectiveness of an educational program on test takers.

[0607] This invention is a system that generates tests using a generative AI model, accepts answers from test takers, recognizes emotions based on the answers, and provides immediate feedback. Specific embodiments of this system are described below.

[0608] System configuration

[0609] This system consists of three main components: a server, a terminal, and a user.

[0610] server

[0611] The server has the following functions:

[0612] 1. Test generation using generative AI models: The server uses generative AI models (e.g., GPT-3) to generate customizable tests based on the business manual and learning content. The generated tests are provided to test takers.

[0613] 2. Means for accepting answers: The server receives and stores the answers entered by the examinee. The answer data is used for subsequent processing.

[0614] 3. Means of emotion recognition: The server sends the received response data to the emotion engine, which analyzes the response content and recognizes the emotion. The emotion engine is implemented in Python and uses emotion analysis libraries such as NLTK and TextBlob.

[0615] 4. Means for providing immediate feedback: The server generates feedback messages based on the analysis results from the emotion engine and provides them to the examinee. The feedback is intended to understand the examinee's learning status and emotional changes in real time.

[0616] Terminal

[0617] The terminal provides an interface for test takers to answer questions. Specifically, it has the following functions:

[0618] 1. Displaying the login screen: The terminal displays a screen for the examinee to log in. The login screen is built using HTML, CSS, and JavaScript.

[0619] 2. Displaying test questions: The terminal displays the test questions received from the server. The examinee enters answers to the displayed test questions.

[0620] 3. Displaying feedback: The terminal displays the feedback message received from the server to the examinee.

[0621] User

[0622] Users (test takers) use the system by following the steps below:

[0623] 1. Login: The user accesses the test platform using a web browser and logs in by entering their username and password.

[0624] 2. Taking the test: The user enters answers to the test questions displayed and clicks the submit button.

[0625] 3. Checking feedback: The user checks the feedback messages provided by the server to understand their own learning status and changes in their emotions.

[0626] Specific examples

[0627] Here's a concrete example of how it works:

[0628] A user accesses the test platform and logs in by entering the username "test_user" and password "password123".

[0629] The server verifies the user's credentials and allows the login.

[0630] The server uses a generative AI model (e.g., GPT-3) to generate an "English grammar test" and provide it to the user.

[0631] The user answers the test question "The cat is on the mat." by typing "The cat is on the mat." and clicks the submit button.

[0632] The server receives the user's response and sends it to the emotion engine.

[0633] The emotion engine analyzes the user's responses and recognizes positive emotions.

[0634] The server receives the results from the emotion engine and generates a feedback message saying, "Your answer is accurate and you feel a positive emotion."

[0635] The terminal displays a feedback message to the user.

[0636] Prompt Sentence Examples

[0637] "Analyze the following text and recognize the emotion: 'I am very happy with my progress.'"

[0638] "Please provide emotional feedback based on the following response: 'I found the test difficult and frustrating.'"

[0639] In this way, the system accepts the test-taker's answers, analyzes their emotions, and provides feedback, allowing the test-taker to gain a deeper understanding of their learning progress.

[0640] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0641] Step 1:

[0642] A user accesses the test platform and logs in.

[0643] Input: Username and Password

[0644] Specific operation: The user accesses the URL of the test platform using a web browser, enters the username and password in the login screen, and clicks the login button.

[0645] Output: Authentication information is sent to the server.

[0646] Step 2:

[0647] The server verifies the user's credentials and allows the login.

[0648] Input: User credentials (username and password)

[0649] Specific operation: The server checks the entered authentication information against the user information stored in the database. If authentication is successful, the user is logged in.

[0650] Output: A login successful message is sent to the terminal.

[0651] Step 3:

[0652] The server uses the generative AI model to generate test questions and provides them to the user.

[0653] Input: User login information, work manuals and learning contents

[0654] Specific operation: The server calls a generative AI model (e.g., GPT-3) and generates customized test questions based on the business manual and learning content.

[0655] Output: The generated test questions are sent to the terminal.

[0656] Step 4:

[0657] The terminal displays the generated test questions.

[0658] Input: Test questions sent from the server

[0659] Specific operation: The device displays the received test questions using HTML, CSS, and JavaScript. The user checks the test questions.

[0660] Output: The test questions are ready for the user to review.

[0661] Step 5:

[0662] The user answers the test questions.

[0663] Input: Test questions, user answers

[0664] Specific operation: The user enters answers to the test questions and clicks the submit button.

[0665] Output: The user's answer is sent from the device to the server.

[0666] Step 6:

[0667] The server receives the user's response and sends it to the emotion engine.

[0668] Input: User response data

[0669] Specific operation: The server sends the received response data to the emotion engine, which is implemented in Python and uses emotion analysis libraries such as NLTK and TextBlob.

[0670] Output: The response data sent to the sentiment engine.

[0671] Step 7:

[0672] The emotion engine analyzes the user's responses and recognizes their emotions.

[0673] Input: User response data

[0674] What it does: The sentiment engine analyzes the response text and recognizes sentiment categories such as positive, negative, and neutral.

[0675] Output: The recognized emotion data is sent to the server.

[0676] Step 8:

[0677] The server receives the results from the emotion engine and provides feedback to the user.

[0678] Input: Emotion data from the emotion engine

[0679] Specific behavior: The server generates a feedback message based on the emotion data, such as "Your answer was accurate and you felt a positive emotion."

[0680] Output: The generated feedback message is sent to the terminal.

[0681] Step 9:

[0682] The terminal displays a feedback message to the user.

[0683] Input: Feedback message sent by the server

[0684] Specific operation: The device displays the received feedback message. The user checks the feedback and understands their own learning status and emotional changes.

[0685] Output: The feedback is available to the user.

[0686] (Application example 2)

[0687] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0688] Conventional business manual comprehension test systems provide instant feedback on test takers' answers, but do not take into account the test taker's emotional state. This makes it difficult to optimize test takers' learning efficiency and motivation. Furthermore, there is no way to grasp the test taker's emotional state, making it difficult to gain a deep understanding of their learning situation. This makes it difficult to measure and improve the effectiveness of educational programs.

[0689] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0690] In this invention, the server includes a means for generating an AI-based business manual comprehension test, a means for accepting test takers' answers, a means for providing immediate feedback based on the accepted answers, a means for recognizing emotions based on the test takers' answers, and a means for providing feedback based on the recognized emotions. This enables feedback that takes into account the test takers' emotional state, optimizing learning efficiency and motivation. Furthermore, understanding test takers' emotions makes it easier to measure and improve the effectiveness of educational programs.

[0691] "AI" is an abbreviation for artificial intelligence, a technology that enables computer systems to mimic human intelligence by learning, reasoning, and self-correcting.

[0692] The "Business Manual Comprehension Test" is a test to evaluate the examinee's level of understanding based on the contents of the business manual.

[0693] A "generator" is a method or device for automatically creating tests and data for a specific purpose.

[0694] "Examinee" refers to the person taking the test, who is evaluated on whether or not they understand the contents of the business manual.

[0695] A "means for accepting responses" is a method or device for collecting and recording responses given by test takers to a test.

[0696] "Immediate feedback" means providing test takers with evaluation and advice on their answers immediately after they answer a test.

[0697] An "emotion recognition means" is a method or device for identifying the test taker's current emotional state based on their answers and behavior.

[0698] A "means for providing feedback" is a method or device for providing appropriate evaluation or advice based on the test taker's answers and emotional state.

[0699] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The following explains how each element functions.

[0700] server

[0701] The server has a means for generating business manual comprehension tests using AI. Specifically, a program for generating customizable tests based on the contents of the business manual is installed on the server. This program uses a generative AI model to analyze the contents of the business manual and generate appropriate test questions.

[0702] The server also has a means for accepting answers from test takers. When test takers submit their answers via their devices, the answer data is stored on the server. The server also has a means for providing immediate feedback based on the accepted answers. The feedback is customized based not only on whether the answers are correct or incorrect, but also on the test taker's emotional state.

[0703] Terminal

[0704] The terminal provides the interface for test takers. Specifically, devices such as smartphones and tablets are used. An application for communicating with the server is installed on the terminal, and test takers answer questions through this application.

[0705] The device also has a means for recognizing emotions based on the examinee's answers. For emotion recognition, an emotion recognition library is used. This library analyzes the examinee's answer data and identifies the examinee's emotional state at that time.

[0706] User

[0707] The user is a test taker who takes a business manual comprehension test. The user answers the test using a terminal, and the answer data is sent to the server. The server provides immediate feedback based on the test taker's answers and also provides customized feedback based on the emotion recognition results.

[0708] Specific examples

[0709] For example, if a user answers "This problem is difficult," the device's emotion recognition software will recognize the emotion as "frustrated." The server will receive this emotion data and generate feedback for the user, such as "Take a short break and refresh yourself."

[0710] Prompt Sentence Examples

[0711] User Answer: "This problem is difficult"

[0712] Emotion engine output: "frustrated"

[0713] Feedback: "Take a short break and refresh yourself."

[0714] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0715] Step 1:

[0716] The server generates customizable tests based on the contents of the business manual. It uses a generative AI model to analyze the contents of the business manual and generate appropriate test questions. The input is the text data of the business manual, and the output is the generated test questions.

[0717] Step 2:

[0718] Users log in to the server using a terminal to access the generated tests. Through the terminal application, users receive test questions. The input is the user's login information, and the output is the test questions.

[0719] Step 3:

[0720] The user answers the test using a terminal. The answer data is sent from the terminal to the server. The input is the user's answer data, and the output is the answer data sent to the server.

[0721] Step 4:

[0722] The server analyzes the received answer data and generates real-time feedback. The feedback is generated based on whether the answer is correct or incorrect. The input is the user's answer data, and the output is real-time feedback.

[0723] Step 5:

[0724] The terminal inputs the user's response data into emotion recognition software to recognize emotions. The input is the user's response data, and the output is the recognized emotion data.

[0725] Step 6:

[0726] The server generates customized feedback based on the recognized emotion data, where the input is the emotion data and the output is the customized feedback.

[0727] Step 7:

[0728] The terminal displays the feedback received from the server to the user. The input is the feedback data from the server, and the output is the feedback displayed to the user.

[0729] Step 8:

[0730] The user receives the feedback and moves on to the next test question or continues learning. The input is the feedback and the output is the user's next action.

[0731] Example 3

[0732] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0733] Conventional business manual comprehension tests have the problem of reducing learning effectiveness because test takers do not receive immediate evaluation or feedback on their answers. Also, there is a lack of a way to track changes in test takers' emotions and reflect them in measuring the effectiveness of educational programs.

[0734] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0735] In this invention, the server includes means for generating a business manual comprehension test using a generative AI model, means for accepting examinees' answers to the test generated by the generation means, means for analyzing the accepted answers using natural language processing technology, means for evaluating the answers based on the analysis results, means for providing immediate feedback based on the evaluation results, means for tracking changes in the examinees' emotions, and means for using the emotion data to measure the effectiveness of the educational program. This enables immediate feedback on the examinees' answers, thereby improving learning effectiveness and measuring the effectiveness of the educational program.

[0736] A "generative AI model" is an algorithm or system that uses artificial intelligence techniques to generate data for a specific task.

[0737] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[0738] "Test taker" means an individual or group taking the business manual comprehension test.

[0739] "Natural language processing technology" is a technology that allows computers to analyze, understand, and generate human language.

[0740] "Analysis results" refers to data and information obtained as a result of analyzing test takers' responses using natural language processing technology.

[0741] "Evaluation results" are data and information obtained as a result of evaluating the examinee's answers based on the analysis results.

[0742] "Immediate feedback" refers to evaluation and advice provided in real time to test takers' answers.

[0743] An "emotion engine" is a system or algorithm that tracks and analyzes changes in test-takers' emotions.

[0744] "Emotional data" refers to data regarding changes in the test taker's emotions tracked using the emotion engine.

[0745] An "educational program" is a systematic program for learning specific knowledge or skills.

[0746] "Effectiveness measurement" refers to the process and methods for evaluating the effectiveness of educational programs.

[0747] This invention is a system that includes a means for generating a business manual comprehension test using a generative AI model, a means for accepting answers from test takers, a means for analyzing the answers using natural language processing technology, a means for evaluating the answers based on the analysis results, a means for providing immediate feedback based on the evaluation results, a means for tracking changes in the test takers' emotions, and a means for using the emotional data to measure the effectiveness of an educational program.

[0748] The server first generates a business manual comprehension test using a generative AI model, such as OpenAI's GPT-4 (registered trademark) or Google's BERT (registered trademark). The generated test is customized based on the content of the business manual.

[0749] Users use devices (PCs, tablets, smartphones, etc.) to input answers to the generated tests. For example, to the question "Explain the causes of World War II," users input answers based on their own knowledge.

[0750] The terminal sends the user's inputted answer to the server, which then analyzes the received answer using natural language processing technology, which includes grammar checking, content understanding, and keyword extraction.

[0751] The server evaluates the answers based on the analysis results. Evaluation criteria include the accuracy, logic, and specificity of the answers. For example, the server evaluates whether the answers are based on historical facts and whether they are logically structured.

[0752] Based on the evaluation results, the server generates instant feedback, including good points and areas for improvement, as well as additional advice. For example, "Your answer is based on historical facts and is logically structured. However, it would be better if you provided more specific examples."

[0753] The server sends the generated feedback to the terminal, which then displays the feedback received from the server to the user. The user can then review the displayed feedback to understand what was good about their answer and what needs to be improved.

[0754] Furthermore, the server uses an emotion engine to track changes in the user's emotions. The emotion engine analyzes changes in facial expressions and voice as the user enters their answers, recording emotions such as joy, confusion, excitement, and anxiety. This data is used to measure the effectiveness of the educational program.

[0755] As a specific example, in response to the question "Explain the causes of World War II," a user inputs the answer, "The causes of World War II were economic hardship due to the treaties following World War I and the rise of a particular political party." The device sends this answer to the server, which then analyzes the answer using GPT-4. The analysis results confirm that the answer is based on historical facts and is logically constructed.

[0756] The server generates feedback such as, "Your answer is based on historical facts and is logically structured. However, it would be better if you provided more specific examples," and sends it to the device. The device displays this feedback to the user, allowing them to understand what was good about their answer and what needs to be improved.

[0757] An example of a prompt sentence is, "Please explain the causes of World War II. We will evaluate the accuracy and logic of your answer. We will also record any changes in your emotions while you are answering." The flow of the identification process in the third embodiment will be described with reference to FIG. 21.

[0758] Step 1:

[0759] The user uses the terminal to enter the answer.

[0760] Users use devices (PCs, tablets, smartphones, etc.) to input answers to questions presented to the system. For example, in response to the question "Explain the causes of World War II," users input answers based on their own knowledge.

[0761] Input: User's answer

[0762] Output: Answers typed into the terminal

[0763] Step 2:

[0764] The terminal transmits the inputted answer to the server.

[0765] The terminal transmits the answers entered by the user to the server in real time over the Internet.

[0766] Input: Answer typed into the terminal

[0767] Output: The answer sent to the server

[0768] Step 3:

[0769] The server analyzes the received response using natural language processing technology.

[0770] The server analyzes the received answers using natural language processing techniques (e.g., OpenAI's GPT-4 or Google's BERT), which includes grammar checks, content understanding, and keyword extraction.

[0771] Input: The answer sent to the server

[0772] Output: Analysis results (grammar check, content understanding, keyword extraction)

[0773] Step 4:

[0774] The server evaluates the answer based on the analysis results.

[0775] The server evaluates the answers based on the analysis results. Evaluation criteria include the accuracy, logic, and specificity of the answers. For example, the server evaluates whether the answers are based on historical facts and whether they are logically structured.

[0776] Input: Analysis results

[0777] Output: Evaluation results (evaluation of accuracy, logic, and concreteness)

[0778] Step 5:

[0779] The server generates feedback based on the evaluation results.

[0780] The server generates feedback based on the evaluation results, including good points and areas for improvement, as well as additional advice. For example, "Your answer is based on historical facts and is logically structured. However, it would be better if you provided more specific examples."

[0781] Input: Evaluation result

[0782] Output: Feedback

[0783] Step 6:

[0784] The server sends the feedback to the device.

[0785] The server then transmits the generated feedback to the device, which is also transmitted in real time via the Internet.

[0786] Input: Feedback

[0787] Output: Feedback sent to the terminal

[0788] Step 7:

[0789] The device displays the feedback to the user.

[0790] The terminal displays the feedback received from the server to the user, who can then check the displayed feedback to understand what was good about their answer and what needs to be improved.

[0791] Input: Feedback sent to the device

[0792] Output: Feedback displayed to the user

[0793] Step 8:

[0794] The server uses an emotion engine to track changes in the user's emotions.

[0795] The server tracks the user's emotional changes using an emotion engine, which analyzes facial and vocal changes as the user enters answers and records emotions such as joy, confusion, excitement, and anxiety.

[0796] Input: User's facial expressions and voice data

[0797] Output: Emotion data

[0798] Step 9:

[0799] The server uses the emotional data to measure the effectiveness of educational programs.

[0800] The server uses the tracked emotion data to measure the effectiveness of the educational program, allowing for a more accurate assessment of which parts of the program were effective for the user and which parts need improvement.

[0801] Input: Emotion data

[0802] Output: Results of measuring the effectiveness of the educational program

[0803] (Application example 3)

[0804] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0805] Conventional business manual comprehension testing systems have the problem of reducing learning effectiveness because they do not provide immediate feedback on test takers' answers. They also lack a means to measure the effectiveness of educational programs by tracking changes in test takers' emotions. Furthermore, they lack a means to instantly evaluate customer feedback in virtual stores and collect data to improve the quality of the shopping experience. A new system is needed to solve these issues.

[0806] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0807] In this invention, the server includes means for generating an AI-based business manual comprehension test, means for accepting test takers' answers to the test generated by the generation means, means for providing instant feedback based on the accepted answers, means for tracking changes in the test takers' emotions, means for using the emotional changes to measure the effectiveness of an educational program, means for instantly evaluating customer feedback in a virtual store, means for tracking changes in customer emotions and collecting data for improving the quality of the shopping experience, means for analyzing the feedback using a generative AI model, and means for evaluating the feedback using prompt sentences. This makes it possible to improve the test takers' learning effectiveness, accurately measure the effectiveness of an educational program, and improve the customer experience in the virtual store.

[0808] "AI" is an abbreviation for artificial intelligence, a technology that enables computer systems to mimic human intelligence by learning, reasoning, and self-correcting.

[0809] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[0810] A "generation means" is a method or device for generating data or information for a specific purpose.

[0811] "Candidate" refers to a person taking a test or examination.

[0812] "Means for accepting answers" refers to the method or device for receiving and recording answers submitted by test takers.

[0813] "Immediate feedback" refers to providing immediate evaluation and comments on the test taker's answers.

[0814] "Means for tracking emotional changes" refers to a method or device for continuously monitoring and recording changes in the emotions of test takers or customers.

[0815] "Educational program effectiveness measurement" is a method for evaluating how effective an educational program is.

[0816] "Virtual Store" means a virtual store operated on the Internet where customers can browse and purchase products online.

[0817] A "means for instantly evaluating customer feedback" is a method or device for instantly analyzing and evaluating customer opinions and comments.

[0818] A "means for collecting data to improve the quality of shopping experience" is a method or device for collecting data necessary to improve the customer's shopping experience.

[0819] A "generative AI model" is a model of artificial intelligence that is trained to perform a specific task.

[0820] A "prompt" is an instruction entered into a generative AI model to make it perform a specific task.

[0821] The following system configuration will be described as an embodiment of the present invention.

[0822] System Configuration

[0823] Hardware

[0824] Server: Use a server with high-performance computing power, specifically a server for running AI models and processing data.

[0825] Terminal: Uses user devices such as smartphones and head-mounted displays. These terminals act as a user interface and accept input from the user.

[0826] Camera: Equipped with a camera that can be used to track user emotions.

[0827] software

[0828] Generative AI model: Use OpenAI's API to run a generative AI model to analyze user feedback.

[0829] Emotion Recognition Library: Use the emotion recognition library to track user emotions.

[0830] Feedback evaluation module: This module analyzes user feedback and provides instant evaluation results.

[0831] Processing flow

[0832] 1. Accepting Feedback:

[0833] The user inputs feedback through the device, for example, "This product is very easy to use, but I think it would be better if it were a little cheaper."

[0834] 2. Feedback analysis:

[0835] The server analyzes the feedback using a generative AI model, specifically using OpenAI's API to evaluate the content of the feedback.

[0836] An example prompt would be "Please rate the following feedback: This product is very easy to use, but I think it would be better if it were a little cheaper."

[0837] 3. Providing immediate feedback:

[0838] The server generates real-time feedback based on the analysis results and provides it to the user, for example, "This feedback is very useful. We will take your opinion on the price into consideration."

[0839] 4. Emotion Tracking:

[0840] The camera is used to track changes in the user's emotions. An emotion recognition library is used to analyze the user's facial expressions and tone of voice and record changes in emotions.

[0841] 5. Data Collection and Use:

[0842] The server collects the tracked emotional data and uses it to measure the effectiveness of educational programs and improve the quality of the shopping experience.

[0843] Specific examples

[0844] Example feedback: "This product is very easy to use, but I think it would be better if it were a little cheaper."

[0845] Example prompt: "Please rate the following feedback: This product is very easy to use, but I think it would be better if it were a little cheaper."

[0846] In this way, it is possible to improve the learning effect of test takers, accurately measure the effectiveness of educational programs, and improve the customer experience in virtual stores.

[0847] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0848] Step 1:

[0849] The user inputs feedback through the terminal. For example, the user may input feedback such as, "This product is very easy to use, but I think it would be better if it were a little cheaper." The input feedback is sent from the terminal to the server.

[0850] Step 2:

[0851] The server inputs the received feedback into the generative AI model. Specifically, it analyzes the content of the feedback using OpenAI's API. The prompt sentence used is "Please rate the following feedback: This product is very easy to use, but I think it would be better if it were a little cheaper." The generative AI model analyzes the content of the feedback and generates an evaluation result.

[0852] Step 3:

[0853] The server generates immediate feedback based on the evaluation results obtained from the generative AI model. For example, it generates feedback such as, "This feedback is very useful. We will take your opinion on the price into consideration." The generated feedback is sent from the server to the device.

[0854] Step 4:

[0855] The terminal displays the instant feedback received from the server to the user, who can then confirm the displayed feedback.

[0856] Step 5:

[0857] The device uses a camera to track changes in the user's emotions. The device captures the user's facial expressions and tone of voice with the camera and analyzes them using an emotion recognition library. The analysis results show changes in the user's emotions.

[0858] Step 6:

[0859] The server collects and stores the tracked emotional data in a database, which is used to measure the effectiveness of educational programs and improve the quality of shopping experiences.

[0860] Step 7:

[0861] The server measures the effectiveness of the educational program based on the collected emotional data and feedback evaluation results. Specifically, it integrates the emotional data and feedback evaluation results to identify areas for improvement in the educational program.

[0862] Step 8:

[0863] The server generates a new training program based on the improvements made to the training program, and the generated training program is used for the next test or training.

[0864] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0865] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0866] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0867] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0868] [Second embodiment]

[0869] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0870] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0871] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure.

[0872] The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0873] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0874] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0875] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0876] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0877] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0878] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0879] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0880] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0881] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0882] "Example 1"

[0883] In one embodiment of the present invention, the AI-based means for generating business manual comprehension tests analyzes the contents of the business manual and generates questions based on the contents. Specifically, the text data of the business manual is used as input, and important keywords and concepts are extracted using natural language processing technology, and questions are generated based on these.

[0884] "Example 2"

[0885] In addition, as a means for accepting answers from test takers, the company provides an interface for test takers to answer the generated tests. Specifically, the company provides a web-based test platform that allows test takers to log in and answer the tests.

[0886] "Example 3"

[0887] Furthermore, as a means of providing immediate feedback based on the answers received, AI technology is used to evaluate the examinee's answers and provide the results to the examinee immediately. Specifically, the examinee's answers are analyzed using natural language processing technology to evaluate their accuracy. Based on the evaluation results, immediate feedback is provided to the examinee.

[0888] The processing flow of each embodiment will be described below.

[0889] "Example 1"

[0890] Step 1: Enter the text data of the business manual into the system.

[0891] Step 2: The system uses natural language processing techniques to extract important keywords and concepts from the text data.

[0892] Step 3: Based on the extracted keywords and concepts, the system generates comprehension test questions.

[0893] "Example 2"

[0894] Step 1: The system provides a web-based testing platform.

[0895] Step 2: Candidates log in to the platform and answer the generated test.

[0896] "Example 3"

[0897] Step 1: The system accepts the test taker's answers and analyzes them using natural language processing technology.

[0898] Step 2: The system evaluates the accuracy of the answer based on the analysis results.

[0899] Step 3: Based on the evaluation results, the system provides immediate feedback to the test-taker.

[0900] Example 1

[0901] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] Conventional business manual comprehension tests require manual question creation, which is time-consuming and labor-intensive. Furthermore, they do not provide immediate feedback on test takers' answers, which can reduce learning effectiveness. Furthermore, there was a lack of a way to automatically generate customizable tests based on the content of business manuals.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0904] In this invention, the server includes means for receiving text data of a business manual as input, means for analyzing the received text data, means for extracting important keywords and concepts from the analyzed text data, means for generating questions based on the extracted keywords and concepts, means for outputting the generated questions, means for accepting answers from examinees to the generated test, and means for providing immediate feedback based on the accepted answers. This enables the automatic generation of customizable comprehension tests based on the contents of the business manual and immediate feedback to examinees on their answers.

[0905] A "business manual" is a document created by a company or organization to explain business procedures and rules to employees.

[0906] "Text data" is data that represents character information in digital form.

[0907] "Analysis" is the process of breaking down data and understanding its structure and meaning.

[0908] "Keywords" are words or phrases that are particularly important in the text data.

[0909] A "concept" is an abstract idea or theme expressed in text data.

[0910] "Questions" are questions presented to test takers to gauge their understanding.

[0911] "Generation" means creating new data or information using specific algorithms or models.

[0912] "Output" means providing the processing results to the outside.

[0913] "Test taker" means a person taking a comprehension test.

[0914] An "answer" is an answer provided by a test-taker to a question.

[0915] "Immediate feedback" refers to evaluations and comments provided immediately to test takers on their answers.

[0916] A "system" is a structure in which multiple elements work together to achieve a specific function.

[0917] MODE FOR CARRYING OUT THE INVENTION

[0918] This invention is a system that automatically generates comprehension tests based on the contents of a business manual and provides immediate feedback to test takers on their answers. A specific embodiment of this system will be described below.

[0919] 1. Program Generation

[0920] The server creates a program to generate a business manual comprehension test. This program has the function of analyzing the contents of the business manual and generating questions based on that content.

[0921] 2. Program Processing

[0922] The server receives the text data of the business manual as input. This text data can come from a file uploaded by the user or retrieved from a database. For example, if a user uploads a PDF file of the business manual, the server converts the PDF into text data.

[0923] The server then uses natural language processing techniques to analyze the text data, using Python libraries such as NLTK and spaCy to split the text data into sentences and words and analyze their grammatical structure.

[0924] The server extracts important keywords and concepts from the analyzed text data. This process uses techniques such as TF-IDF (Term Frequency-Inverse Document Frequency) and Word2Vec. For example, keywords such as "customer service," "quick," "polite," "accurate information," and "problem solving" are extracted.

[0925] Next, the server generates questions based on the extracted keywords and concepts. To generate these questions, it uses a generative AI model (for example, OpenAI's GPT-3). The server inputs a prompt to the generative AI model, instructing it to generate an appropriate question. For example, the server inputs the following prompt:

[0926] "Please explain the basics of customer service. Why is prompt and courteous service required?"

[0927] The generative AI model generates questions based on this prompt.

[0928] 3. Specific Examples

[0929] For example, consider a scenario where a user wants to upload a PDF file of a business manual. The user uses a terminal to access the system interface and clicks the "Select File" button. Next, the user selects the PDF file of the business manual and clicks the "Upload" button. The server receives this PDF file and converts it into text data.

[0930] The server then analyzes the text data and extracts important keywords and concepts, such as "customer service," "speed," "polite," "accurate information," and "problem resolution."

[0931] The server then inputs a prompt to the generative AI model to generate a question, such as:

[0932] "What are the basics of customer service? Explain why prompt and courteous service is required."

[0933] Finally, the server provides the generated questions to the user, who then obtains a list of questions to use as comprehension tests. In this way, it is possible to automatically generate customizable comprehension tests based on the contents of the business manual and provide immediate feedback to the examinee's answers.

[0934] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0935] Step 1:

[0936] The server receives the text data of the business manual as input. When a user uploads a PDF file of the business manual using a terminal, the server converts the PDF file into text data. This conversion uses OCR (Optical Character Recognition) technology. The input is a PDF file, and the output is text data.

[0937] Step 2:

[0938] The server analyzes the received text data. This analysis uses natural language processing libraries such as Python's "NLTK" or "spaCy." Specifically, the text data is divided into sentences and words, and the grammatical structure is analyzed. The input is the text data, and the output is a list of analyzed sentences and words.

[0939] Step 3:

[0940] The server extracts important keywords and concepts from the analyzed text data. This process uses technologies such as TF-IDF (Term Frequency-Inverse Document Frequency) and Word2Vec. For example, keywords such as "customer service," "quick," "polite," "accurate information," and "problem solving" are extracted. The input is a list of analyzed sentences and words, and the output is a list of important keywords and concepts.

[0941] Step 4:

[0942] The server generates questions based on the extracted keywords and concepts. To generate these questions, it uses a generative AI model (for example, OpenAI's GPT-3). The server inputs a prompt to the generative AI model, instructing it to generate an appropriate question. For example, the server inputs the following prompt:

[0943] "Please explain the basics of customer service. Why is prompt and courteous service required?"

[0944] The generative AI model generates a question based on this prompt: the input is the prompt and the output is the generated question.

[0945] Step 5:

[0946] The server provides the generated questions to the user. The user can use a terminal to review the generated questions and make corrections or additions as needed. Finally, the user gets a list of questions to use as a comprehension test. The input is the generated questions, and the output is the final list of questions.

[0947] Step 6:

[0948] The server accepts the examinee's answers to the generated test. The examinee answers the test using a terminal and sends the answers to the server. The input is the examinee's answers and the output is the accepted answers.

[0949] Step 7:

[0950] The server provides instant feedback based on the received answers. The server uses a generative AI model to evaluate the answers and generate appropriate feedback, such as determining whether the answer is correct or incorrect and suggesting areas for improvement. The input is the received answers, and the output is instant feedback.

[0951] (Application example 1)

[0952] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0953] Conventional business manual comprehension tests require manual creation, which is time-consuming and labor-intensive, and the quality of the tests is inconsistent. It is also difficult to immediately assess the test taker's level of understanding and provide feedback, making it difficult to measure the effectiveness of educational programs. Furthermore, there is a need for more efficient training for factory robot operators and engineers.

[0954] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0955] In this invention, the server includes a means for generating an AI-based business manual comprehension test, a means for analyzing the text data of the business manual and extracting important keywords and concepts, a means for generating questions based on the extracted keywords and concepts, a means for presenting the generated questions to robot operators and engineers, a means for collecting response data and evaluating comprehension, and a means for providing feedback on the evaluation results and suggesting additional training. This enables the automatic generation of high-quality comprehension tests based on the contents of the business manual, and allows for the instant evaluation of the test taker's comprehension and the provision of feedback. It also improves the efficiency of training for factory robot operators and engineers.

[0956] "AI" is an abbreviation for artificial intelligence, a technology that allows computers to imitate human intelligence to learn and reason.

[0957] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[0958] A "generator" is a method or device for generating data or information for a particular purpose.

[0959] "Text data" is data that represents character information in digital form.

[0960] "Analysis" is the process of examining data or information in detail to understand its structure and meaning.

[0961] A "keyword" is a word or phrase that has significant meaning in a particular document or piece of data.

[0962] A "concept" represents an abstract understanding or recognition of a particular event or phenomenon.

[0963] A "question" is a question asked to elicit specific information.

[0964] A "robot operator" is an engineer or worker who operates factory robots.

[0965] An "engineer" is a person who has specialized knowledge and skills in a particular technical field.

[0966] "Response data" refers to information about responses provided by test takers to questions.

[0967] "Comprehension" refers to the degree to which a particular piece of content or information is accurately understood.

[0968] "Evaluation" is the process of judging the value or performance of an object based on specific criteria.

[0969] "Feedback" means providing a response or evaluation of a particular action or result.

[0970] "Training" means training or education to acquire specific skills or knowledge.

[0971] As an embodiment of the present invention, the following system is constructed.

[0972] System Configuration

[0973] The system consists of a server, factory robots, tablet terminals, and a network. The server plays a central role in generating, analyzing, and evaluating work manual comprehension tests. The factory robots present the generated tests to operators and engineers and collect their responses. The tablet terminals are used as an interface for users to take the tests.

[0974] Hardware and software used

[0975] Hardware: Servers, factory robots, tablets

[0976] Software: Python, TensorFlow, NLTK (Natural Language Toolkit), Flask (web framework)

[0977] Data processing and calculation

[0978] 1. Analysis of the business manual:

[0979] The server reads the text data of the business manual using Python and uses NLTK to tokenize and extract keywords, thereby extracting important keywords and concepts.

[0980] 2. Question generation:

[0981] The server generates questions using a generative AI model (e.g., GPT-3) based on the extracted keywords and concepts. The generated questions are used to measure comprehension and are in the form of multiple-choice, written answers, or practical tests.

[0982] 3. Present the test:

[0983] The factory robot then presents the generated questions to the robot operator or engineer, who then answers the questions via a tablet device.

[0984] 4. Collecting and Evaluating Responses:

[0985] The server collects the response data sent from the tablet device and runs a scoring algorithm to evaluate the level of comprehension. The evaluation results are immediately fed back to the user.

[0986] 5. Additional training suggestions:

[0987] Based on the evaluation results, the server will suggest additional training as needed, thereby improving the user's skills.

[0988] Specific examples

[0989] Contents of the operations manual: "Robot maintenance procedures"

[0990] Generated question: "What is the first task that should be performed in the robot maintenance procedure?"

[0991] Prompt Sentence Examples

[0992] Below are some example prompts to input to the generative AI model:

[0993] Based on the contents of the operations manual below, generate questions to gauge comprehension.

[0994] Operation Manual: Robot Maintenance Procedures

[0995] 1. Turn off the power

[0996] 2. Remove the outer cover

[0997] 3. Clean the interior

[0998] 4. Inspect the parts

[0999] Generate a question.

[1000] In this way, the system can automatically generate high-quality comprehension tests based on the contents of the operation manual, instantly evaluate the test taker's level of understanding, and provide feedback, thereby improving the efficiency of training for factory robot operators and engineers.

[1001] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1002] Step 1:

[1003] The server receives the text data of the business manual. It reads the text data of the business manual as input and analyzes the text data using Python. Specifically, it uses NLTK to tokenize the text and extract important keywords and concepts. The output is a list of the extracted keywords and concepts.

[1004] Step 2:

[1005] The server generates questions using a generative AI model (e.g., GPT-3) based on the extracted keywords and concepts. It receives a list of keywords and concepts as input and inputs a prompt sentence into the generative AI model. Specifically, it sends the prompt sentence to the generative AI model to generate questions. The output is a list of generated questions.

[1006] Step 3:

[1007] The server sends the generated questions to the factory robot. It receives the list of generated questions as input and sends them to the factory robot. Specifically, it transfers the question data to the factory robot via the network. The output is the question data sent to the factory robot.

[1008] Step 4:

[1009] The factory robot presents the generated questions to the robot operator or engineer. It receives question data sent from the server as input and displays it to the user via a tablet device. Specifically, it displays the question on the tablet device screen so that the user can answer it. The output is the user's answer data.

[1010] Step 5:

[1011] The server collects the response data sent from the tablet device and evaluates the level of understanding. It receives the user's response data as input and executes a scoring algorithm. Specifically, it analyzes the response data and evaluates the accuracy rate and level of understanding. The output is the evaluation result data.

[1012] Step 6:

[1013] The server feeds back the evaluation results to the user. It receives the evaluation result data as input and provides it to the user. Specifically, it displays the evaluation results on a tablet device and provides feedback to the user. The output is the feedback data provided to the user.

[1014] Step 7:

[1015] The server proposes additional training based on the evaluation results. It receives the evaluation result data as input and generates additional training content as needed. Specifically, it analyzes the evaluation results, identifies the user's weaknesses and areas for improvement, and proposes an appropriate training program. The output is the data of the proposed training program.

[1016] Example 2

[1017] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1018] In conventional testing systems, test question generation is done manually, which is time-consuming and labor-intensive. It is also difficult to immediately evaluate test takers' answers and provide feedback. Furthermore, it is difficult to generate customizable tests based on specific educational content. This poses a challenge in measuring the effectiveness of educational programs.

[1019] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1020] In this invention, the server includes means for generating test questions using a generative AI model, means for accepting answers from examinees to the generated test questions, means for storing the accepted answers in a database, and means for evaluating and displaying results based on the stored answers. This enables automatic generation and immediate evaluation of test questions, and enables the generation of customizable tests based on specific educational content and the measurement of the effectiveness of educational programs.

[1021] A "generative AI model" is an algorithm that uses artificial intelligence technology to automatically generate test questions based on specific prompts.

[1022] "Test questions" are questions that test takers answer and are generated by a generative AI model.

[1023] An "examinee" is a user who answers test questions.

[1024] "Means for accepting answers" refers to an interface that allows the system to receive and process answers entered by the examinee.

[1025] A "database" is a system for storing data such as test taker answers and generated test questions.

[1026] "Means for evaluating and displaying results" refers to a function that determines whether the test taker's answers are correct or incorrect and provides feedback on the results to the test taker.

[1027] A "customizable test" is a test whose content can be modified to suit specific educational content or purposes.

[1028] An "educational program" is a set of educational activities and materials designed to achieve specific learning objectives.

[1029] "Effectiveness measurement" is the process of evaluating the impact and results that an educational program has on test takers.

[1030] This invention is a system that uses a generative AI model to generate test questions, accepts answers from test takers, and evaluates them. Specific embodiments of this system are described below.

[1031] System configuration

[1032] server

[1033] The server hosts a web-based testing platform. The server uses the following hardware and software:

[1034] Hardware: High-performance server machine (e.g., general-purpose server)

[1035] Software: web server (e.g., Apache HTTP Server), database (e.g., MySQL), programming language (e.g., Python, JavaScript)

[1036] The server generates test questions using a generative AI model, which are then stored in a database for test takers to access.

[1037] Terminal

[1038] The terminal provides the interface for test takers to complete the test. The terminal uses the following hardware and software:

[1039] Hardware: PCs, tablets, smartphones

[1040] Software: Web browser (e.g. Google Chrome)

[1041] The device displays the test questions received from the server and allows the test taker to enter their answers, which are then sent to the server in real time and stored.

[1042] User

[1043] The user (candidate) logs into the web-based testing platform and answers the provided test questions. The user follows these steps:

[1044] 1. Open a web browser and access the test platform URL.

[1045] 2. Enter your username and password on the login screen to log in.

[1046] 3. Enter your answers to the test questions that appear and click the submit button.

[1047] Specific examples

[1048] For example, if a test taker takes a math test, the server uses a generative AI model to generate questions like this:

[1049] Question 1: "Solve the following equation: 2x + 3 = 7"

[1050] Question 2: "Find the next term in the following sequence: 1, 4, 9, 16, ?"

[1051] The examinee enters the answers to these questions and submits them.

[1052] Prompt Sentence Examples

[1053] Enter the following prompt for the generative AI model:

[1054] "Generate math test questions. The questions should be at a middle school level and include equations and sequences."

[1055] Using this prompt, the generative AI model generates appropriate test questions.

[1056] The above is a specific embodiment for carrying out the present invention. This system enables automatic generation and instant evaluation of test questions, and enables the creation of customizable tests based on specific educational content and the measurement of the effectiveness of educational programs.

[1057] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1058] Step 1:

[1059] User Login

[1060] Input: Username, Password

[1061] Server: The server receives the username and password entered by the user on the login screen and checks them against the authentication information stored in the database. If authentication is successful, the server issues the user a session ID and redirects them to the dashboard screen.

[1062] Output: Authentication result, session ID

[1063] What happens: The server securely sends and receives the user's authentication information using SSL / TLS encryption. If authentication is successful, it starts the user's session and returns the URL of the dashboard screen.

[1064] Step 2:

[1065] Generate test questions

[1066] Input: Prompt statement (e.g., "Generate a middle school level math test question.")

[1067] Server: The server sends prompts to the generative AI model to generate test questions, which are then stored in a database.

[1068] Output: Generated test questions

[1069] Specific operation: The server calls the API of the generative AI model and sends a prompt. The generative AI model generates test questions based on the prompt and returns them to the server. The server stores the generated test questions in a database.

[1070] Step 3:

[1071] View test questions

[1072] Input: User request (request to start test)

[1073] Server: The server retrieves the generated test questions from the database and sends them to the user's device.

[1074] Terminal: The terminal displays the test questions received from the server to the user.

[1075] Output: Test question display screen

[1076] Specific operation: The server receives a user request, retrieves the corresponding test questions from the database, and sends the retrieved test questions in HTML format to the terminal, which then displays them.

[1077] Step 4:

[1078] User answer input

[1079] Input: User answer (e.g. "x = 2")

[1080] Terminal: The terminal provides an input field to accept user-entered answers and temporarily stores the data as the user enters the answers.

[1081] Output: User response data

[1082] What it does: The device validates your answers in real time as you enter them, providing feedback to help prevent typing errors. Your answers are temporarily stored in the device's memory.

[1083] Step 5:

[1084] Submit and save your answers

[1085] Input: User response data

[1086] Terminal: The terminal sends the answers entered by the user to the server.

[1087] Server: The server stores the received answer in a database and sends a confirmation message to the user if the save is successful.

[1088] Output: Save confirmation message

[1089] Specific operation: When the user clicks the send button, the device sends the answer data to the server. The server saves the received answer data in the database and returns a message to the device indicating that the save was successful.

[1090] Step 6:

[1091] Displaying the results

[1092] Input: Saved response data

[1093] Server: The server evaluates the user's answers and generates a result, e.g., whether the answer is correct or incorrect and calculates a score.

[1094] Terminal: The terminal displays the results received from the server to the user.

[1095] Output: Evaluation results, scores

[1096] Specific operation: The server retrieves the user's answer data from the database and uses an evaluation algorithm to determine whether it is correct or incorrect. It calculates the evaluation results and score and sends them to the terminal. The terminal then displays the received evaluation results and score to the user.

[1097] The above are the specific processing steps of the program of this system.

[1098] (Application example 2)

[1099] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1100] Conventional business manual comprehension testing systems have the problem that test generation is done manually, which takes time and effort. Also, there is a possibility that learning effectiveness may be reduced because immediate feedback on test takers' answers is not provided. Furthermore, because the test content is fixed, there is also the issue that it is difficult to customize according to the comprehension level of each individual test taker.

[1101] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1102] In this invention, the server includes means for generating a business manual comprehension test using a generative AI model, means for accepting answers from examinees to the test generated by the generation means, means for providing immediate feedback based on the accepted answers, means for generating a test by sending prompts to the generative AI model, means for providing the generated test content to the user, and means for the user to answer the test and evaluate the answer content. This enables automatic test generation and provision of immediate feedback, making it possible to provide a test customized according to the examinee's level of understanding.

[1103] A "generative AI model" is an algorithm that uses artificial intelligence to generate an output for a specific task.

[1104] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[1105] A "generator" is a function for automatically creating tests for a specific purpose.

[1106] "Test taker" means a person taking a test.

[1107] "Means for accepting answers" refers to the function that allows the system to receive answers entered by test takers for the test.

[1108] "Immediate feedback" is a feature that provides evaluations and comments on test takers' answers immediately after they answer a test.

[1109] A "prompt" is an instruction entered into a generative AI model to cause it to generate a specific output.

[1110] "User" refers to a person who uses the system.

[1111] "Means of evaluation" refers to a function for determining the accuracy and level of understanding of the test taker's answers.

[1112] The embodiments of the present invention will be described in detail below.

[1113] First, the server uses a generative AI model to generate a business manual comprehension test. The generation method involves sending a prompt to the generative AI model, which has the ability to automatically create a specific test. An example of a prompt is "Please generate basic math problems."

[1114] The server then provides a means to accept test takers' responses to the generated tests. Test takers can log in to a web-based testing platform using devices such as smartphones or PCs and take the tests.

[1115] When a test taker answers the test, the server immediately evaluates the answer and provides feedback. The evaluation method has a function to determine the accuracy and level of understanding based on the test taker's answer. For example, if a test taker answers "2 + 2 = 4," the server immediately evaluates whether the answer is correct and provides feedback that "That's correct" if it is.

[1116] The implementation of this system uses the following hardware and software:

[1117] Hardware: Servers, smartphones, PCs

[1118] Software: Flask (Python web framework), OpenAI API (generative AI model)

[1119] For example, if a user inputs the prompt "Generate basic math problems," the generative AI model will generate math problems and provide them to the user. The user answers the generated questions, and the answers are sent to the server. The server evaluates the answers and provides immediate feedback.

[1120] In this way, tests can be automatically generated and immediate feedback can be provided, allowing tests to be customized according to the examinee's level of understanding.

[1121] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1122] Step 1:

[1123] A user logs into a web-based testing platform using a terminal.

[1124] Input: User login information (username, password)

[1125] Output: Login success or failure message

[1126] Specific operation: The user enters a username and password into the login form and sends it to the server, which checks it against the database and returns the authentication result.

[1127] Step 2:

[1128] The user inputs a prompt statement into the generative AI model to request test generation.

[1129] Input: Prompt sentence (e.g. "Generate a basic math problem")

[1130] Output: Generated test content

[1131] Specific operation: The user enters a prompt into the test generation form and sends it to the server. The server then sends the prompt to the generative AI model and receives the generated test content.

[1132] Step 3:

[1133] The server provides the generated test content to the user.

[1134] Input: Generated test content

[1135] Output: Test results displayed on the user's device

[1136] Specific operation: The server sends the generated test content to the user's terminal and displays it on the user's screen.

[1137] Step 4:

[1138] The user answers the quiz.

[1139] Input: User's answer

[1140] Output: Response data sent to the server

[1141] Specific operation: The user enters answers to each question in the test and sends the answers to the server.

[1142] Step 5:

[1143] The server evaluates the user's answers.

[1144] Input: User response data

[1145] Output: Evaluation results (correct answers, incorrect answers, score, etc.)

[1146] Specific operation: The server analyzes the user's answer data, compares it with the correct answer, and generates an evaluation result.

[1147] Step 6:

[1148] The server provides the evaluation results to the user as real-time feedback.

[1149] Input: Evaluation result

[1150] Output: A feedback message that is displayed on the user's terminal.

[1151] Specific operation: The server sends the evaluation results to the user's device and displays a feedback message on the user's screen.

[1152] Example 3

[1153] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1154] Conventional business manual comprehension tests have the problem of reducing learning effectiveness because test takers' answers are not evaluated and feedback is not provided immediately. In addition, tests are generated in a fixed format, making it difficult to customize them to fit the content of the business manual. Furthermore, because feedback is not directly linked to measuring the effectiveness of the educational program, there is a lack of data to improve the quality of education.

[1155] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1156] In this invention, the server includes a means for generating a business manual comprehension test using a generative AI model, a means for accepting test takers' answers, a means for analyzing the accepted answers using natural language processing technology and evaluating their accuracy, and a means for providing immediate feedback based on the evaluation results. This allows for immediate evaluation and feedback of the test takers' answers, improving learning effectiveness. Furthermore, generating customizable tests based on the contents of the business manual allows for more appropriate evaluation. Furthermore, the immediate feedback enables the effectiveness of educational programs to be measured, allowing for the collection of data to improve the quality of education.

[1157] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate answers or data for specific tasks.

[1158] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[1159] A "generation means" is a method or device for generating data or information for a specific purpose.

[1160] "Candidate" refers to a person taking a test or examination.

[1161] "Means for accepting answers" refers to a method or device by which the system receives answers entered by the examinee.

[1162] "Natural language processing technology" is a technology that allows computers to understand and analyze human language.

[1163] An "evaluation means" is a method or device that uses received data or information to determine its accuracy or appropriateness.

[1164] "Immediate feedback" refers to providing test takers with prompt evaluation results and advice in response to their answers.

[1165] A "customizable test" is a test whose content can be modified to suit specific conditions or needs.

[1166] An "educational program" is a set of learning activities and materials designed to acquire specific knowledge or skills.

[1167] "Effectiveness measurement" refers to assessing the success of a particular activity or program.

[1168] This invention is a system that uses a generative AI model to generate a business manual comprehension test, analyzes the test taker's answers, and provides immediate feedback. Specific embodiments of this system are described below.

[1169] Hardware and software used

[1170] Hardware: Servers, terminals

[1171] Software: Generative AI models, Natural Language Processing (NLP)

[1172] System Overview

[1173] 1. The user enters the answer

[1174] The user inputs a response into an input form on the device. For example, in response to the question "What color is the Earth?", the user inputs "blue."

[1175] 2. The device sends the answer to the server

[1176] The terminal sends the answer entered by the user to the server. Specifically, the terminal sends the answer data to the server using an HTTP request.

[1177] 3. The server receives the answer

[1178] The server receives the HTTP request sent from the device and extracts the response data. For example, it verifies that the received data is "blue."

[1179] 4. The server parses the answer

[1180] The server analyzes the received response data using natural language processing (NLP). Specifically, it inputs a prompt sentence into the generative AI model to perform the analysis. An example of a prompt sentence is "Question: What color is the Earth? Answer: Blue."

[1181] 5. The server generates the evaluation result

[1182] The server evaluates the accuracy of the answer based on the analysis results from the generative AI model. For example, if the generative AI model determines that the answer "blue" is accurate, it generates that evaluation result.

[1183] 6. The server generates feedback

[1184] The server generates feedback for the user based on the evaluation result, for example, a feedback message such as "You're right. The Earth is blue."

[1185] 7. The server sends feedback to the device

[1186] The server sends the generated feedback message to the terminal. Specifically, the server sends the feedback data to the terminal using an HTTP response.

[1187] 8. The device displays feedback to the user

[1188] The device displays the feedback message received from the server to the user, for example, a message such as "You're right! The Earth is blue."

[1189] Specific examples

[1190] For example, when a user answers "blue" to the question "What color is the Earth?", the specific operation is as follows.

[1191] 1. The user enters the answer

[1192] The user enters "blue" into the input form on the device.

[1193] 2. The device sends the answer to the server

[1194] The device uses an HTTP request to send the answer data "blue" to the server.

[1195] 3. The server receives the answer

[1196] The server receives the HTTP request and extracts the response data "blue."

[1197] 4. The server parses the answer

[1198] The server inputs the prompt sentence "Question: What color is the Earth? Answer: Blue" into the generative AI model and performs analysis.

[1199] 5. The server generates the evaluation result

[1200] Based on the analysis results from the generative AI model, the server evaluates that the answer "blue" is accurate.

[1201] 6. The server generates feedback

[1202] The server generates a feedback message saying "Correct. The Earth is blue."

[1203] 7. The server sends feedback to the device

[1204] The server sends a feedback message to the device using an HTTP response.

[1205] 8. The device displays feedback to the user

[1206] The device will display the message "Correct. The Earth is blue." on the screen.

[1207] In this way, the user can receive instant feedback, thereby improving the learning effect. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1208] Step 1:

[1209] The user enters the answer.

[1210] Input: A user types a response into a device input form. For example, the user types "blue" in response to the question "What color is the Earth?"

[1211] Output: The entered answer data is saved on the device.

[1212] What happens: The user uses the keyboard to enter an answer and clicks the submit button.

[1213] Step 2:

[1214] The device sends the response to the server.

[1215] Input: The answer data entered by the user.

[1216] Output: The response data is sent to the server as an HTTP request.

[1217] Specific operation: The terminal sends the answer data entered by the user to the server in the form of an HTTP request.

[1218] Step 3:

[1219] The server receives the response.

[1220] Input: The HTTP request sent from the device.

[1221] Output: The extracted response data.

[1222] Specific operation: The server receives the HTTP request and extracts the response data. For example, it obtains the response data "blue."

[1223] Step 4:

[1224] The server parses the response.

[1225] Input: The extracted response data.

[1226] Output: Analysis results.

[1227] Specific operation: The server inputs a prompt sentence into the generative AI model and performs analysis. For example, the prompt sentence "Question: What color is the Earth? Answer: Blue" is input into the generative AI model and the analysis result is obtained.

[1228] Step 5:

[1229] The server generates the evaluation results.

[1230] Input: Analysis results from a generative AI model.

[1231] Output: Evaluation result.

[1232] Specific operation: The server evaluates the accuracy of the answer based on the analysis results from the generative AI model. For example, it determines that the answer "blue" is accurate and generates an evaluation result.

[1233] Step 6:

[1234] The server generates the feedback.

[1235] Input: Evaluation result.

[1236] Output: The feedback message.

[1237] Specific operation: The server generates feedback for the user based on the evaluation result. For example, it generates a feedback message such as "You're right. The Earth is blue."

[1238] Step 7:

[1239] The server sends the feedback to the device.

[1240] Input: The feedback message.

[1241] Output: The feedback message is sent to the device as an HTTP response.

[1242] Specific operation: The server sends the generated feedback message to the terminal in the form of an HTTP response.

[1243] Step 8:

[1244] The device displays the feedback to the user.

[1245] Input: The feedback message sent by the server.

[1246] Output: Feedback messages displayed on the screen.

[1247] Specific operation: The device displays the feedback message received from the server on the screen, for example, "You're right. The Earth is blue."

[1248] (Application example 3)

[1249] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1250] Conventional business manual comprehension test systems rarely provide immediate feedback on test takers' answers, which reduces learning effectiveness. Furthermore, in factories and other workplaces, errors in work procedures can have a negative impact on production efficiency and quality, so real-time evaluation and feedback are required. To solve these issues, an immediate feedback system using AI technology is needed.

[1251] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1252] In this invention, the server includes means for generating an AI-based business manual comprehension test, means for accepting test takers' answers to the test generated by the generating means, means for providing immediate feedback based on the accepted answers, means for evaluating work procedures in real time, and means for providing immediate feedback to workers based on the evaluation results. This increases the learning effect of test takers and reduces errors in work procedures at work sites such as factories, thereby enabling improved production efficiency and quality.

[1253] "A means for generating business manual comprehension tests using AI" is a function that uses artificial intelligence technology to automatically create comprehension tests based on the contents of business manuals.

[1254] The "means for accepting answers from examinees" is a function for receiving answers given by examinees to comprehension tests as input.

[1255] "Means for providing immediate feedback based on accepted answers" refers to a function that analyzes the answers of test takers and provides feedback in real time based on the results.

[1256] The "means for evaluating work procedures in real time" is a function that monitors the work procedures performed by workers in real time and evaluates their accuracy and efficiency.

[1257] The "means for providing immediate feedback to workers based on the evaluation results" is a function for providing feedback to workers in real time based on the evaluation results of work procedures.

[1258] A system for implementing this invention is configured as follows: First, the server is equipped with a means for generating a business manual comprehension test using AI. This generation means has the function of automatically creating a comprehension test based on the contents of the business manual. Specifically, it uses natural language processing technology to analyze the text of the business manual and generate appropriate questions.

[1259] Next, the server is provided with a means for accepting answers from examinees. This means has a function for receiving answers given by examinees to comprehension tests as input. Examinees can input their answers using devices such as smartphones and tablets.

[1260] Furthermore, the server is equipped with a means for providing immediate feedback based on the answers received. This means has the function of analyzing the examinee's answers and providing real-time feedback based on the results. For example, if the examinee gives an incorrect answer, the correct answer and the reason for it are immediately displayed.

[1261] The server also has a means for evaluating work procedures in real time. This means has the function of monitoring the work procedures performed by workers in real time and evaluating their accuracy and efficiency. Specifically, the work procedures are monitored using cameras and sensors installed on factory robots, and evaluations are made using AI models.

[1262] Finally, the server is equipped with a means for providing immediate feedback to workers based on the evaluation results. This means has the function of providing real-time feedback to workers based on the evaluation results of their work procedures. For example, if a worker performs an incorrect procedure, the correct procedure will be instructed via voice or display.

[1263] The hardware used includes factory robots (e.g., robotic arms), smartphones, and tablets. The software used is Python, spaCy, and the Transformers library. Combining these technologies will enhance the learning effect of test takers and reduce errors in work procedures in factories and other workplaces, thereby improving production efficiency and quality.

[1264] As a specific example, if a worker performs a task in the following order, "Attach part A. Tighten the screws. Attach part B," the system will determine that the "tighten the screws" step is incorrect and provide feedback such as, "Step 2 is incorrect. Correct step: Attach part B."

[1265] Examples of prompt sentences include the following:

[1266] "Create a system that evaluates work procedures and provides feedback if there are any errors. It takes the text of the work procedures as input, compares it with the correct procedure, and generates a feedback message pointing out the errors. The techniques used are natural language processing and AI models."

[1267] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1268] Step 1:

[1269] The server receives the text data of the business manual as input. It analyzes the text data using natural language processing technology and generates comprehension test questions. Specifically, it uses the spaCy library to divide the text into sentences and extract important keywords and phrases. Based on this, it uses a generative AI model to create appropriate questions. The output is a list of generated comprehension test questions.

[1270] Step 2:

[1271] The terminal receives as input the examinee's answers to the comprehension test. The examinee enters the answers using a smartphone or tablet. The entered answers are sent to the server. The output is the examinee's answer data.

[1272] Step 3:

[1273] The server receives and analyzes the test taker's response data. Specifically, it uses a generative AI model to evaluate the accuracy of the answers. Based on the evaluation results, it generates an immediate feedback message. For example, it creates a feedback message that includes the correct answer and the reason for it. The output is the feedback message.

[1274] Step 4:

[1275] The server sends a feedback message to the terminal. The terminal displays the feedback message to the test taker. The test taker can check the feedback and improve their understanding. The output is the feedback message displayed to the test taker.

[1276] Step 5:

[1277] The server receives real-time data on work procedures from cameras and sensors installed on factory robots. Specifically, it monitors the work procedures performed by workers and collects data. The input is data from cameras and sensors.

[1278] Step 6:

[1279] The server analyzes the collected work procedure data and evaluates its accuracy and efficiency. Specifically, it uses an AI model to evaluate the work procedures and, if there are any errors, identifies the errors. The output is the evaluation result of the work procedures.

[1280] Step 7:

[1281] The server provides immediate feedback to the worker based on the evaluation results. Specifically, it instructs the correct procedure through voice or display. For example, it provides feedback such as "Procedure 2 is incorrect. Correct procedure: Install part B." The output is a feedback message provided to the worker.

[1282] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1283] "Example 1"

[1284] One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the examinee's answers and recognizes their emotional state. Specifically, the emotion engine estimates the examinee's emotional state from the examinee's typing speed, mouse movements, and eye movements as they answer the test. This information is used to evaluate not only the examinee's level of comprehension, but also their learning progress and psychological state.

[1285] "Example 2"

[1286] The emotion engine also recognizes emotions based on the examinee's answers and reflects the results in the feedback. Specifically, it analyzes how the examinee feels about the questions and how those emotions change over time, and provides the results as feedback to the examinee. This allows the examinee to gain a deeper understanding of their own learning situation. "Format Example 3"

[1287] Furthermore, the emotion engine tracks changes in the examinee's emotions and uses the results to measure the effectiveness of the educational program. Specifically, it records changes in emotions such as joy, confusion, excitement, and anxiety that the examinee experiences as they progress through their studies, and reflects these changes in the effectiveness of the educational program. This allows for a more accurate evaluation of the program's effectiveness.

[1288] The processing flow of each embodiment will be described below.

[1289] "Example 1"

[1290] Step 1: The test taker begins answering the test.

[1291] Step 2: The emotion engine analyzes the test taker's keyboard typing speed, mouse movements, eye movements, etc.

[1292] Step 3: The emotion engine uses this information to estimate the test-taker's emotional state.

[1293] Step 4: This emotional state information is used to assess the test taker's comprehension, as well as their learning situation and psychological state.

[1294] "Example 2"

[1295] Step 1: The candidate answers the test.

[1296] Step 2: The emotion engine recognizes emotions based on the test taker's answers.

[1297] Step 3: The emotion engine analyzes how test takers feel about the question and how their emotions change over time.

[1298] Step 4: Provide feedback to the test taker on the results of this analysis.

[1299] "Example 3"

[1300] Step 1: Candidates progress in their studies.

[1301] Step 2: The emotion engine tracks changes in the test taker's emotions.

[1302] Step 3: The emotion engine records changes in emotions such as joy, confusion, excitement, and anxiety as the test-taker progresses through the study.

[1303] Step 4: Use this record to measure the effectiveness of the educational program.

[1304] Example 1

[1305] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1306] In conventional business manual comprehension tests, questions to assess the test taker's level of comprehension are often generated manually, which is inefficient. In addition, the test taker's emotional and psychological state is not taken into account in the assessment, making it difficult to accurately grasp the test taker's true level of comprehension and learning situation.

[1307] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1308] In this invention, the server includes means for inputting text data of a business manual, means for analyzing the input text data and extracting important keywords and concepts, means for generating questions based on the extracted keywords and concepts, means for collecting data when examinees answer tests, means for analyzing the collected data and estimating the examinees' emotional states, and means for evaluating the examinees' levels of comprehension and emotional states based on the results of an analysis of the examinees' answers to the generated questions and their emotional states. This enables efficient question generation based on the contents of the business manual and accurate evaluation of the examinees' levels of comprehension that takes into account the examinees' emotional states.

[1309] A "business manual" is a document that describes the procedures and rules for a company or organization to carry out its business.

[1310] "Text data" is data that represents character information in digital form.

[1311] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[1312] A "keyword" is a word or phrase that has a particularly important meaning in text data.

[1313] A "concept" is an abstract idea or way of thinking about a particular phenomenon or thing.

[1314] "Question generation" is the process of creating a question based on specific information.

[1315] "Candidate" refers to a person taking a test or examination.

[1316] "Data collection" is the process of gathering information needed for a specific purpose.

[1317] "Emotional state" refers to an individual's psychological state or mood.

[1318] "Evaluation" is the process of judging the value or performance of an object based on specific criteria.

[1319] MODE FOR CARRYING OUT THE INVENTION

[1320] The present invention provides a system for efficiently generating comprehension tests for business manuals and evaluating test takers while taking into account their emotional state. Specific embodiments of this system are described below.

[1321] Inputting text data for business manuals

[1322] The user uploads the text data of the business manual to the system. Specifically, the user selects the file of the business manual and clicks the upload button, which sends the text data to the server.

[1323] Text data analysis and keyword extraction

[1324] The server analyzes the text data of the received business manual. Specifically, it uses the Python natural language processing library "spaCy" to extract important keywords and concepts from the text data. For example, keywords such as "customer service," "basic procedures," and "satisfaction level" are extracted.

[1325] Question Generation

[1326] The server generates questions based on the extracted keywords and concepts. This process uses the generative AI model "GPT-3." Specifically, the extracted keywords are input into GPT-3 as prompts, and an appropriate question is generated. For example, the question generated might be, "What is the first step in the basic customer service procedure?"

[1327] Collection of test taker response data

[1328] The device collects data when test takers answer questions, such as keyboard typing speed, mouse movements, and eye movements on the screen. This data is acquired using hardware such as eye tracking devices and input devices.

[1329] Emotional state analysis

[1330] The server analyzes the data sent from the device and estimates the test-taker's emotional state. Specifically, it uses an emotion recognition algorithm to estimate the test-taker's emotional state from their keyboard typing speed, mouse movements, and eye movements. This analysis uses an emotion recognition model using "OpenCV" and "TensorFlow."

[1331] Assessment of comprehension and emotional state

[1332] The server evaluates the examinee's level of understanding and emotional state based on their answers to the generated questions and the results of an analysis of their emotional state. Specifically, it comprehensively evaluates the accuracy of the answers, the response time, and the emotional state data to determine the examinee's learning situation and psychological state.

[1333] Examples of concrete examples and prompts

[1334] As a concrete example, if the business manual describes "basic procedures for dealing with customers," the server generates the question "What is the first step in the basic procedures for dealing with customers?"

[1335] Below are some example prompts to input to a generative AI model:

[1336] 1. Generate comprehension test questions based on the contents of the following operations manual: Operations manual: 'The basic procedure for dealing with customers begins with accurately listening to the customer's request. Next, you propose an appropriate response, and finally you confirm the customer's satisfaction.'

[1337] 2. "Please estimate the test taker's emotional state from their keyboard typing speed, mouse movements, and eye movements. The following data was collected: 'Key typing speed: fast, mouse movements: frequent, eye movements: focused'"

[1338] This allows the system to generate questions based on the contents of the job manual and analyze the test taker's emotional state.

[1339] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1340] Program processing flow

[1341] Step 1: Enter the text data for the business manual

[1342] The user uploads the text data of the business manual to the system. Specifically, the user selects the file of the business manual and clicks the upload button, which sends the text data to the server.

[1343] Input: Business manual text file

[1344] Output: Text data sent to the server

[1345] Step 2: Analyzing text data and extracting keywords

[1346] The server analyzes the text data of the received business manuals, specifically using the Python natural language processing library "spaCy" to extract important keywords and concepts from the text data.

[1347] Input: Text data of the business manual

[1348] Data processing: Extracting keywords and concepts using natural language processing technology

[1349] Output: Extracted keywords and concepts

[1350] Step 3: Generate questions

[1351] The server generates questions based on the extracted keywords and concepts. This process uses the generative AI model "GPT-3." Specifically, the extracted keywords are input into GPT-3 as prompts, and appropriate questions are generated.

[1352] Input: Extracted keywords and concepts

[1353] Data Computing: Generating Questions Using Generative AI Models

[1354] Output: Generated questions

[1355] Step 4: Collect test-taker response data

[1356] The device collects data when test takers answer questions, such as keyboard typing speed, mouse movements, and eye movements on the screen. This data is acquired using hardware such as eye tracking devices and input devices.

[1357] Input: Test taker response behavior data (typing speed, mouse movement, eye movement)

[1358] Output: Collected behavioral data

[1359] Step 5: Analyze emotional state

[1360] The server analyzes the data sent from the device and estimates the test-taker's emotional state. Specifically, it uses an emotion recognition algorithm to estimate the test-taker's emotional state from their keyboard typing speed, mouse movements, and eye movements. This analysis uses an emotion recognition model using "OpenCV" and "TensorFlow."

[1361] Input: Collected behavioral data

[1362] Data Computing: Estimating emotional states using emotion recognition algorithms

[1363] Output: Estimated emotional state

[1364] Step 6: Assess your comprehension and emotional state

[1365] The server evaluates the examinee's level of understanding and emotional state based on their answers to the generated questions and the results of an analysis of their emotional state. Specifically, it comprehensively evaluates the accuracy of the answers, the response time, and the emotional state data to determine the examinee's learning situation and psychological state.

[1366] Input: test taker's answers, estimated emotional state

[1367] Data calculation: Comprehensive evaluation of answer accuracy, response time, and emotional state

[1368] Output: Evaluation results of the test taker's comprehension and emotional state

[1369] (Application example 1)

[1370] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1371] Conventional business manual comprehension test systems not only evaluate the examinee's level of understanding, but also have difficulty in comprehensively evaluating the examinee's emotional state and learning situation. Furthermore, they are unable to provide feedback that takes the examinee's psychological state into account, making it difficult to provide appropriate guidance to maximize learning effectiveness. Furthermore, they are unable to generate comprehension tests specifically for factory robot operation manuals, making it difficult to efficiently support the improvement of operator skills.

[1372] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1373] In this invention, the server includes means for generating an AI-based business manual comprehension test, means for accepting examinees' answers to the test generated by the generation means, means for providing immediate feedback based on the accepted answers, means for analyzing the examinee's keyboard typing speed, pointing device movement, and eye movement when answering, means for estimating the examinee's emotional state based on the analyzed data, and means for evaluating the examinee's learning situation and psychological state based on the estimated emotional state. This makes it possible to comprehensively evaluate not only the examinee's level of understanding but also their emotional state and learning situation and provide appropriate feedback.

[1374] "A means for generating business manual comprehension tests using AI" is a means for analyzing the contents of business manuals and automatically generating comprehension tests based on those contents.

[1375] The "means for accepting answers from examinees" refers to a means for accepting, as input, answers from examinees to the generated comprehension test.

[1376] A "means for providing immediate feedback" is a means for providing real-time feedback based on the test-taker's responses.

[1377] "Keyboard typing speed" refers to the typing speed when the examinee operates the keyboard.

[1378] "Pointing device movements" refers to the movements made by the test taker when operating a pointing device such as a mouse or touchpad.

[1379] "Eye movement" refers to how the examinee's eyes move across the screen.

[1380] The "analyzing means" refers to a means for analyzing data such as keyboard typing speed, pointing device movement, and eye movement.

[1381] The "means for estimating an emotional state" is a means for estimating the emotional state of the test taker based on the analyzed data.

[1382] "Means for assessing learning situation and psychological state" refers to means for assessing the learning situation and psychological state of the examinee based on the estimated emotional state.

[1383] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The following explains how each element functions.

[1384] server

[1385] The server includes the following means:

[1386] 1. AI-based method for generating business manual comprehension tests:

[1387] The server analyzes the text data of the business manual and extracts important keywords and concepts using natural language processing technology. This analysis uses a natural language processing model (e.g., spaCy).

[1388] Based on the extracted keywords and concepts, comprehension test questions are generated, which are automatically created using a generative AI model.

[1389] 2. How to receive test taker responses:

[1390] The server receives the examinee's answers sent from the terminal, which are sent to the server in text format.

[1391] 3. Means of providing immediate feedback:

[1392] The server generates real-time feedback based on the examinee's answers and sends it to the device, allowing the effectiveness of the educational program to be measured.

[1393] 4. Methods for analyzing keyboard typing speed, pointing device movements, and eye movements:

[1394] The server analyzes data sent from the device, such as keyboard typing speed, pointing device movement, and eye movement, using libraries such as OpenCV.

[1395] 5. Means of estimating emotional state:

[1396] The server estimates the test-taker's emotional state based on the analyzed data, using an emotion analysis model (e.g., transformers pipeline).

[1397] 6. Methods for assessing learning and psychological status:

[1398] The server evaluates the test-taker's learning progress and psychological state based on the estimated emotional state and provides appropriate feedback.

[1399] Terminal

[1400] The terminal is a device that allows users to take the comprehension test. The terminal has the following functions:

[1401] 1. Show test:

[1402] The terminal displays the comprehension test questions sent from the server.

[1403] 2. Enter your answer:

[1404] The user uses a terminal to input test answers, which are then sent to the server in text format.

[1405] 3. Recording user behavior:

[1406] The terminal records the user's keyboard typing speed, pointing device movements, eye movements, etc. and sends them to the server.

[1407] User

[1408] A user is an individual who takes a comprehension test. A user takes a test by following these steps:

[1409] 1. Start the test:

[1410] The user starts the comprehension test using the terminal.

[1411] 2. Answers to questions:

[1412] The user inputs answers to the questions displayed.

[1413] 3. Receiving feedback:

[1414] The user receives real-time feedback sent from the server.

[1415] Specific examples

[1416] For example, parse the operating manual for a factory robot and generate questions like:

[1417] "Please explain how to activate the robot."

[1418] "Please explain how to maintain the robot."

[1419] If the user answers, "Starting the robot is easy," the server analyzes this response and estimates the user's emotional state. Furthermore, it evaluates the user's learning status and psychological state based on data such as keyboard typing speed and eye movement.

[1420] Prompt Sentence Examples

[1421] "Analyze the following text and extract key keywords: This is an operation manual for a factory robot. It describes how to start, stop, and maintain the robot."

[1422] "Please analyze the emotional state of the text below: Activating the robot is easy."

[1423] In this way, a system is realized in which the server, terminal, and user work together to conduct comprehension tests and provide comprehensive evaluations and feedback.

[1424] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1425] Step 1:

[1426] The server receives the text data of the business manual. It receives the text data of the business manual as input and uses natural language processing technology to extract important keywords and concepts. Specifically, it uses a natural language processing model (e.g., spaCy) to analyze the text and extract keywords. The output is the extracted keywords and concepts.

[1427] Step 2:

[1428] The server generates comprehension test questions based on the extracted keywords and concepts. It receives the extracted keywords and concepts as input and automatically creates questions using a generative AI model. Specifically, it inputs a prompt sentence into the generative AI model and generates a question. The output is the generated question.

[1429] Step 3:

[1430] The server sends the generated question to the terminal. It receives the generated question as input and sends it to the terminal. Specifically, it sends the question to the terminal via the network. The output is the question displayed on the terminal.

[1431] Step 4:

[1432] The terminal displays the question sent from the server to the user. It receives the question sent from the server as input and displays it to the user. Specifically, it displays the question on the terminal's display. The output is the question displayed to the user.

[1433] Step 5:

[1434] The user inputs an answer to a displayed question. The user receives the displayed question as input and inputs an answer. Specifically, the answer is input using a keyboard or a touch screen. The output is the input answer.

[1435] Step 6:

[1436] The terminal sends the user's answer to the server. It receives the answer entered by the user as input and sends it to the server. Specifically, it sends the answer to the server via the network. The output is the answer sent to the server.

[1437] Step 7:

[1438] The server analyzes the received answers and generates immediate feedback. The server analyzes the answers received as input and generates appropriate feedback. Specifically, it analyzes the content of the answers and measures the effectiveness of the educational program. The output is the generated feedback.

[1439] Step 8:

[1440] The server transmits the generated feedback to the terminal. The server receives the generated feedback as input and transmits it to the terminal. Specifically, the server transmits the feedback to the terminal via a network. The output is the feedback displayed on the terminal.

[1441] Step 9:

[1442] The terminal displays the feedback sent from the server to the user. The terminal receives the feedback sent from the server as input and displays it to the user. Specifically, the terminal displays the feedback on the display of the terminal. The output is the feedback displayed to the user.

[1443] Step 10:

[1444] The server analyzes the user's keyboard typing speed, pointing device movement, and eye movement. It receives and analyzes the user's behavioral data sent from the device as input. Specifically, it analyzes the data using libraries such as OpenCV. The output is the analyzed behavioral data.

[1445] Step 11:

[1446] The server estimates the user's emotional state based on the analyzed behavioral data. It receives the analyzed behavioral data as input and estimates the emotional state using an emotion analysis model. Specifically, it estimates the emotional state using a pipeline of transformers. The output is the estimated emotional state.

[1447] Step 12:

[1448] The server evaluates the user's learning situation and psychological state based on the estimated emotional state. It receives the estimated emotional state as input and evaluates the learning situation and psychological state. Specifically, it performs a comprehensive evaluation using an evaluation algorithm. The output is the evaluation result.

[1449] Example 2

[1450] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1451] Conventional test systems rarely provide immediate feedback on test takers' answers, and in particular lack feedback that takes test takers' emotions into account. This can prevent test takers from fully understanding their own learning situation and emotional changes, potentially reducing their learning effectiveness. Another issue is the difficulty of generating customizable tests based on work manuals.

[1452] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1453] In this invention, the server includes a test generation unit that utilizes a generative AI model, a unit that accepts examinee responses to the generated test, a unit that recognizes emotions based on the accepted answers, and a unit that provides immediate feedback based on the recognized emotions. This allows examinees to understand their own learning status and emotional changes in real time, which is expected to improve learning effectiveness. It also makes it easy to generate customizable tests based on work manuals.

[1454] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to automatically generate test questions and content.

[1455] "Test generation means" refers to the function or process that uses generative AI models to create test questions.

[1456] "Test taker" means an individual or group who takes a test and answers questions.

[1457] "Means for accepting answers" refers to the functions and processes by which the system receives and stores answers entered by test takers.

[1458] "Means for recognizing emotions" refers to algorithms or systems that analyze test-taker responses and identify the emotions contained within them.

[1459] "Means for providing immediate feedback" refers to the functionality or process of generating and providing feedback in real time based on test-taker responses and perceived sentiment.

[1460] A "customizable test" is a test whose content can be tailored based on specific business manuals or learning content.

[1461] "Educational program effectiveness measurement" refers to the process and means for evaluating the impact and effectiveness of an educational program on test takers.

[1462] This invention is a system that generates tests using a generative AI model, accepts answers from test takers, recognizes emotions based on the answers, and provides immediate feedback. Specific embodiments of this system are described below.

[1463] System configuration

[1464] This system consists of three main components: a server, a terminal, and a user.

[1465] server

[1466] The server has the following functions:

[1467] 1. Test generation using generative AI models: The server uses generative AI models (e.g., GPT-3) to generate customizable tests based on the business manual and learning content. The generated tests are provided to test takers.

[1468] 2. Means for accepting answers: The server receives and stores the answers entered by the examinee. The answer data is used for subsequent processing.

[1469] 3. Means of emotion recognition: The server sends the received answer data to the emotion engine, which analyzes the answer content and recognizes the emotion. The emotion engine is implemented in Python and uses emotion analysis libraries such as NLTK and TextBlob.

[1470] 4. Means for providing immediate feedback: The server generates feedback messages based on the analysis results from the emotion engine and provides them to the test-taker. The feedback is intended to understand the test-taker's learning status and emotional changes in real time.

[1471] Terminal

[1472] The terminal provides an interface for test takers to answer questions. Specifically, it has the following functions:

[1473] 1. Displaying the login screen: The terminal displays a screen for the examinee to log in. The login screen is built using HTML, CSS, and JavaScript.

[1474] 2. Displaying test questions: The terminal displays the test questions received from the server. The examinee enters answers to the displayed test questions.

[1475] 3. Displaying feedback: The terminal displays the feedback message received from the server to the examinee.

[1476] User

[1477] Users (test takers) use the system by following the steps below:

[1478] 1. Login: The user accesses the test platform using a web browser and logs in by entering their username and password.

[1479] 2. Taking the test: The user enters answers to the test questions displayed and clicks the submit button.

[1480] 3. Checking feedback: The user checks the feedback messages provided by the server to understand their own learning status and changes in their emotions.

[1481] Specific examples

[1482] Here's a concrete example of how it works:

[1483] A user accesses the test platform and logs in by entering the username "test_user" and password "password123".

[1484] The server verifies the user's credentials and allows the login.

[1485] The server uses a generative AI model (e.g., GPT-3) to generate an "English grammar test" and provide it to the user.

[1486] The user answers the test question "The cat is on the mat." by typing "The cat is on the mat." and clicks the submit button.

[1487] The server receives the user's response and sends it to the emotion engine.

[1488] The emotion engine analyzes the user's responses and recognizes positive emotions.

[1489] The server receives the results from the emotion engine and generates a feedback message saying, "Your answer is accurate and you feel a positive emotion."

[1490] The terminal displays a feedback message to the user.

[1491] Prompt Sentence Examples

[1492] "Analyze the following text and recognize the emotion: 'I am very happy with my progress.'"

[1493] "Please provide emotional feedback based on the following response: 'I found the test difficult and frustrating.'"

[1494] In this way, the system accepts the test-taker's answers, analyzes their emotions, and provides feedback, allowing the test-taker to gain a deeper understanding of their learning progress.

[1495] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1496] Step 1:

[1497] A user accesses the test platform and logs in.

[1498] Input: Username and Password

[1499] Specific operation: The user accesses the URL of the test platform using a web browser, enters the username and password in the login screen, and clicks the login button.

[1500] Output: Authentication information is sent to the server.

[1501] Step 2:

[1502] The server verifies the user's credentials and allows the login.

[1503] Input: User credentials (username and password)

[1504] Specific operation: The server checks the entered authentication information against the user information stored in the database. If authentication is successful, the user is logged in.

[1505] Output: A login successful message is sent to the terminal.

[1506] Step 3:

[1507] The server uses the generative AI model to generate test questions and provides them to the user.

[1508] Input: User login information, work manuals and learning contents

[1509] Specific operation: The server calls a generative AI model (e.g., GPT-3) and generates customized test questions based on the business manual and learning content.

[1510] Output: The generated test questions are sent to the terminal.

[1511] Step 4:

[1512] The terminal displays the generated test questions.

[1513] Input: Test questions sent from the server

[1514] Specific operation: The device displays the received test questions using HTML, CSS, and JavaScript. The user checks the test questions.

[1515] Output: The test questions are ready for the user to review.

[1516] Step 5:

[1517] The user answers the test questions.

[1518] Input: Test questions, user answers

[1519] Specific operation: The user enters answers to the test questions and clicks the submit button.

[1520] Output: The user's answer is sent from the device to the server.

[1521] Step 6:

[1522] The server receives the user's response and sends it to the emotion engine.

[1523] Input: User response data

[1524] Specific operation: The server sends the received response data to the emotion engine, which is implemented in Python and uses emotion analysis libraries such as NLTK and TextBlob.

[1525] Output: The response data sent to the sentiment engine.

[1526] Step 7:

[1527] The emotion engine analyzes the user's responses and recognizes their emotions.

[1528] Input: User response data

[1529] What it does: The sentiment engine analyzes the response text and recognizes sentiment categories such as positive, negative, and neutral.

[1530] Output: The recognized emotion data is sent to the server.

[1531] Step 8:

[1532] The server receives the results from the emotion engine and provides feedback to the user.

[1533] Input: Emotion data from the emotion engine

[1534] Specific behavior: The server generates a feedback message based on the emotion data, such as "Your answer was accurate and you felt a positive emotion."

[1535] Output: The generated feedback message is sent to the terminal.

[1536] Step 9:

[1537] The terminal displays a feedback message to the user.

[1538] Input: Feedback message sent by the server

[1539] Specific operation: The device displays the received feedback message. The user checks the feedback and understands their own learning status and emotional changes.

[1540] Output: The feedback is available to the user.

[1541] (Application example 2)

[1542] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1543] Conventional business manual comprehension test systems provide instant feedback on test takers' answers, but do not take into account the test taker's emotional state. This makes it difficult to optimize test takers' learning efficiency and motivation. Furthermore, there is no way to grasp the test taker's emotional state, making it difficult to gain a deep understanding of their learning situation. This makes it difficult to measure and improve the effectiveness of educational programs.

[1544] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1545] In this invention, the server includes a means for generating an AI-based business manual comprehension test, a means for accepting test takers' answers, a means for providing immediate feedback based on the accepted answers, a means for recognizing emotions based on the test takers' answers, and a means for providing feedback based on the recognized emotions. This enables feedback that takes into account the test takers' emotional state, optimizing learning efficiency and motivation. Furthermore, understanding test takers' emotions makes it easier to measure and improve the effectiveness of educational programs.

[1546] "AI" is an abbreviation for artificial intelligence, a technology that enables computer systems to mimic human intelligence by learning, reasoning, and self-correcting.

[1547] The "Business Manual Comprehension Test" is a test to evaluate the examinee's level of understanding based on the contents of the business manual.

[1548] A "generator" is a method or device for automatically generating tests and data for a specific purpose.

[1549] "Examinee" refers to the person taking the test, who is evaluated on whether or not they understand the contents of the business manual.

[1550] A "means for accepting responses" is a method or device for collecting and recording responses given by test takers to a test.

[1551] "Immediate feedback" means providing test takers with evaluation and advice on their answers immediately after they answer a test.

[1552] An "emotion recognition means" is a method or device for identifying the test taker's current emotional state based on their answers and behavior.

[1553] A "means for providing feedback" is a method or device for providing appropriate evaluation or advice based on the test taker's answers and emotional state.

[1554] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The following explains how each element functions.

[1555] server

[1556] The server has a means for generating business manual comprehension tests using AI. Specifically, a program for generating customizable tests based on the contents of the business manual is installed on the server. This program uses a generative AI model to analyze the contents of the business manual and generate appropriate test questions.

[1557] The server also has a means for accepting answers from test takers. When test takers submit their answers via their devices, the answer data is stored on the server. The server also has a means for providing immediate feedback based on the accepted answers. The feedback is customized based not only on whether the answers are correct or incorrect, but also on the test taker's emotional state.

[1558] Terminal

[1559] The terminal provides the interface for test takers. Specifically, devices such as smartphones and tablets are used. An application for communicating with the server is installed on the terminal, and test takers answer questions through this application.

[1560] The device also has a means for recognizing emotions based on the examinee's answers. For emotion recognition, an emotion recognition library is used. This library analyzes the examinee's answer data and identifies the examinee's emotional state at that time.

[1561] User

[1562] The user is a test taker who takes a business manual comprehension test. The user answers the test using a terminal, and the answer data is sent to the server. The server provides immediate feedback based on the test taker's answers and also provides customized feedback based on the emotion recognition results.

[1563] Specific examples

[1564] For example, if a user answers "This problem is difficult," the device's emotion recognition software will recognize the emotion as "frustrated." The server will receive this emotion data and generate feedback for the user, such as "Take a short break and refresh yourself."

[1565] Prompt Sentence Examples

[1566] User Answer: "This problem is difficult"

[1567] Emotion engine output: "frustrated"

[1568] Feedback: "Take a short break and refresh yourself."

[1569] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1570] Step 1:

[1571] The server generates customizable tests based on the contents of the business manual. It uses a generative AI model to analyze the contents of the business manual and generate appropriate test questions. The input is the text data of the business manual, and the output is the generated test questions.

[1572] Step 2:

[1573] Users log in to the server using a terminal to access the generated tests. Through the terminal application, users receive test questions. The input is the user's login information, and the output is the test questions.

[1574] Step 3:

[1575] The user answers the test using a terminal. The answer data is sent from the terminal to the server. The input is the user's answer data, and the output is the answer data sent to the server.

[1576] Step 4:

[1577] The server analyzes the received answer data and generates real-time feedback. The feedback is generated based on whether the answer is correct or incorrect. The input is the user's answer data, and the output is real-time feedback.

[1578] Step 5:

[1579] The terminal inputs the user's response data into emotion recognition software to recognize emotions. The input is the user's response data, and the output is the recognized emotion data.

[1580] Step 6:

[1581] The server generates customized feedback based on the recognized emotion data, where the input is the emotion data and the output is the customized feedback.

[1582] Step 7:

[1583] The terminal displays the feedback received from the server to the user. The input is the feedback data from the server, and the output is the feedback displayed to the user.

[1584] Step 8:

[1585] The user receives the feedback and moves on to the next test question or continues learning. The input is the feedback and the output is the user's next action.

[1586] Example 3

[1587] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1588] Conventional business manual comprehension tests have the problem of reducing learning effectiveness because test takers do not receive immediate evaluation or feedback on their answers. Also, there is a lack of a way to track changes in test takers' emotions and reflect them in measuring the effectiveness of educational programs.

[1589] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1590] In this invention, the server includes means for generating a business manual comprehension test using a generative AI model, means for accepting examinees' answers to the test generated by the generation means, means for analyzing the accepted answers using natural language processing technology, means for evaluating the answers based on the analysis results, means for providing immediate feedback based on the evaluation results, means for tracking changes in the examinees' emotions, and means for using the emotion data to measure the effectiveness of the educational program. This enables immediate feedback on the examinees' answers, thereby improving learning effectiveness and measuring the effectiveness of the educational program.

[1591] A "generative AI model" is an algorithm or system that uses artificial intelligence techniques to generate data for a specific task.

[1592] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[1593] "Test taker" means an individual or group taking the business manual comprehension test.

[1594] "Natural language processing technology" is a technology that allows computers to analyze, understand, and generate human language.

[1595] "Analysis results" refers to data and information obtained as a result of analyzing test takers' responses using natural language processing technology.

[1596] "Evaluation results" are data and information obtained as a result of evaluating the examinee's answers based on the analysis results.

[1597] "Immediate feedback" refers to evaluation and advice provided in real time to test takers' answers.

[1598] An "emotion engine" is a system or algorithm that tracks and analyzes changes in test-takers' emotions.

[1599] "Emotional data" refers to data regarding changes in the test taker's emotions tracked using the emotion engine.

[1600] An "educational program" is a systematic program for learning specific knowledge or skills.

[1601] "Effectiveness measurement" refers to the process and methods for evaluating the effectiveness of educational programs.

[1602] This invention is a system that includes a means for generating a business manual comprehension test using a generative AI model, a means for accepting answers from test takers, a means for analyzing the answers using natural language processing technology, a means for evaluating the answers based on the analysis results, a means for providing immediate feedback based on the evaluation results, a means for tracking changes in the test takers' emotions, and a means for using the emotional data to measure the effectiveness of an educational program.

[1603] The server first generates a business manual comprehension test using a generative AI model, such as OpenAI's GPT-4 or Google's BERT. The generated test is customized based on the content of the business manual.

[1604] Users use devices (PCs, tablets, smartphones, etc.) to input answers to the generated tests. For example, to the question "Explain the causes of World War II," users input answers based on their own knowledge.

[1605] The terminal sends the user's inputted answer to the server, which then analyzes the received answer using natural language processing technology, which includes grammar check, content understanding, and keyword extraction.

[1606] The server evaluates the answers based on the analysis results. Evaluation criteria include the accuracy, logic, and specificity of the answers. For example, the server evaluates whether the answers are based on historical facts and whether they are logically structured.

[1607] Based on the evaluation results, the server generates instant feedback, including good points and areas for improvement, as well as additional advice. For example, "Your answer is based on historical facts and is logically structured. However, it would be better if you provided more specific examples."

[1608] The server sends the generated feedback to the terminal, which then displays the feedback received from the server to the user. The user can then review the displayed feedback to understand what was good about their answer and what needs to be improved.

[1609] Furthermore, the server uses an emotion engine to track changes in the user's emotions. The emotion engine analyzes changes in facial expressions and voice as the user enters their answers, recording emotions such as joy, confusion, excitement, and anxiety. This data is used to measure the effectiveness of the educational program.

[1610] As a specific example, in response to the question "Explain the causes of World War II," a user inputs the answer, "The causes of World War II were economic hardship due to the treaties following World War I and the rise of a particular political party." The device sends this answer to the server, which then analyzes the answer using GPT-4. The analysis results confirm that the answer is based on historical facts and is logically constructed.

[1611] The server generates feedback such as, "Your answer is based on historical facts and is logically structured. However, it would be better if you provided more specific examples," and sends it to the device. The device displays this feedback to the user, allowing them to understand what was good about their answer and what needs to be improved.

[1612] An example of a prompt sentence is, "Please explain the causes of World War II. We will evaluate the accuracy and logic of your answer. We will also record any changes in your emotions while you are answering." The flow of the identification process in the third embodiment will be described with reference to FIG. 21.

[1613] Step 1:

[1614] The user uses the terminal to enter the answer.

[1615] Users use devices (PCs, tablets, smartphones, etc.) to input answers to questions presented to the system. For example, in response to the question "Explain the causes of World War II," users input answers based on their own knowledge.

[1616] Input: User's answer

[1617] Output: Answers typed into the terminal

[1618] Step 2:

[1619] The terminal transmits the inputted answer to the server.

[1620] The terminal transmits the answers entered by the user to the server in real time over the Internet.

[1621] Input: Answer typed into the terminal

[1622] Output: The answer sent to the server

[1623] Step 3:

[1624] The server analyzes the received response using natural language processing technology.

[1625] The server analyzes the received answers using natural language processing techniques (e.g., OpenAI's GPT-4 or Google's BERT), which includes grammar checks, content understanding, and keyword extraction.

[1626] Input: The answer sent to the server

[1627] Output: Analysis results (grammar check, content understanding, keyword extraction)

[1628] Step 4:

[1629] The server evaluates the answer based on the analysis results.

[1630] The server evaluates the answers based on the analysis results. Evaluation criteria include the accuracy, logic, and specificity of the answers. For example, the server evaluates whether the answers are based on historical facts and whether they are logically structured.

[1631] Input: Analysis results

[1632] Output: Evaluation results (evaluation of accuracy, logic, and concreteness)

[1633] Step 5:

[1634] The server generates feedback based on the evaluation results.

[1635] The server generates feedback based on the evaluation results, including good points and areas for improvement, as well as additional advice. For example, "Your answer is based on historical facts and is logically structured. However, it would be better if you provided more specific examples."

[1636] Input: Evaluation result

[1637] Output: Feedback

[1638] Step 6:

[1639] The server sends the feedback to the device.

[1640] The server then transmits the generated feedback to the device, which is also transmitted in real time via the Internet.

[1641] Input: Feedback

[1642] Output: Feedback sent to the terminal

[1643] Step 7:

[1644] The device displays the feedback to the user.

[1645] The terminal displays the feedback received from the server to the user, who can then check the displayed feedback to understand what was good about their answer and what needs to be improved.

[1646] Input: Feedback sent to the device

[1647] Output: Feedback displayed to the user

[1648] Step 8:

[1649] The server uses an emotion engine to track changes in the user's emotions.

[1650] The server tracks the user's emotional changes using an emotion engine, which analyzes facial and vocal changes as the user enters answers and records emotions such as joy, confusion, excitement, and anxiety.

[1651] Input: User's facial expressions and voice data

[1652] Output: Emotion data

[1653] Step 9:

[1654] The server uses the emotional data to measure the effectiveness of educational programs.

[1655] The server uses the tracked emotion data to measure the effectiveness of the educational program, allowing for a more accurate assessment of which parts of the program were effective for the user and which parts need improvement.

[1656] Input: Emotion data

[1657] Output: Results of measuring the effectiveness of the educational program

[1658] (Application example 3)

[1659] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1660] Conventional business manual comprehension testing systems have the problem of reducing learning effectiveness because they do not provide immediate feedback on test takers' answers. They also lack a means to measure the effectiveness of educational programs by tracking changes in test takers' emotions. Furthermore, they lack a means to instantly evaluate customer feedback in virtual stores and collect data to improve the quality of the shopping experience. A new system is needed to solve these issues.

[1661] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1662] In this invention, the server includes means for generating an AI-based business manual comprehension test, means for accepting test takers' answers to the test generated by the generation means, means for providing instant feedback based on the accepted answers, means for tracking changes in the test takers' emotions, means for using the emotional changes to measure the effectiveness of an educational program, means for instantly evaluating customer feedback in a virtual store, means for tracking changes in customer emotions and collecting data for improving the quality of the shopping experience, means for analyzing the feedback using a generative AI model, and means for evaluating the feedback using prompt sentences. This makes it possible to improve the test takers' learning effectiveness, accurately measure the effectiveness of an educational program, and improve the customer experience in the virtual store.

[1663] "AI" is an abbreviation for artificial intelligence, a technology that enables computer systems to mimic human intelligence by learning, reasoning, and self-correcting.

[1664] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[1665] A "generation means" is a method or device for generating data or information for a specific purpose.

[1666] "Candidate" refers to a person taking a test or examination.

[1667] "Means for accepting answers" refers to the method or device for receiving and recording answers submitted by test takers.

[1668] "Immediate feedback" refers to providing immediate evaluation and comments on the test taker's answers.

[1669] "Means for tracking emotional changes" refers to a method or device for continuously monitoring and recording changes in the emotions of test takers or customers.

[1670] "Educational program effectiveness measurement" is a method for evaluating how effective an educational program is.

[1671] "Virtual Store" means a virtual store operated on the Internet where customers can browse and purchase products online.

[1672] A "means for instantly evaluating customer feedback" is a method or device for instantly analyzing and evaluating customer opinions and comments.

[1673] A "means for collecting data to improve the quality of shopping experience" is a method or device for collecting data necessary to improve the customer's shopping experience.

[1674] A "generative AI model" is a model of artificial intelligence that is trained to perform a specific task.

[1675] A "prompt" is an instruction entered into a generative AI model to make it perform a specific task.

[1676] The following system configuration will be described as an embodiment of the present invention.

[1677] System Configuration

[1678] Hardware

[1679] Server: Use a server with high-performance computing power, specifically a server for running AI models and processing data.

[1680] Terminal: Uses user devices such as smartphones and head-mounted displays. These terminals act as a user interface and accept input from the user.

[1681] Camera: Equipped with a camera that can be used to track user emotions.

[1682] software

[1683] Generative AI model: Use OpenAI's API to run a generative AI model to analyze user feedback.

[1684] Emotion Recognition Library: Use the emotion recognition library to track user emotions.

[1685] Feedback evaluation module: This module analyzes user feedback and provides instant evaluation results.

[1686] Processing flow

[1687] 1. Accepting Feedback:

[1688] The user inputs feedback through the device, for example, "This product is very easy to use, but I think it would be better if it were a little cheaper."

[1689] 2. Feedback analysis:

[1690] The server analyzes the feedback using a generative AI model, specifically using OpenAI's API to evaluate the content of the feedback.

[1691] An example prompt would be "Please rate the following feedback: This product is very easy to use, but I think it would be better if it were a little cheaper."

[1692] 3. Providing immediate feedback:

[1693] The server generates real-time feedback based on the analysis results and provides it to the user, for example, "This feedback is very useful. We will take your opinion on the price into consideration."

[1694] 4. Emotion Tracking:

[1695] The camera is used to track changes in the user's emotions. An emotion recognition library is used to analyze the user's facial expressions and tone of voice and record changes in emotions.

[1696] 5. Data Collection and Use:

[1697] The server collects the tracked emotional data and uses it to measure the effectiveness of educational programs and improve the quality of the shopping experience.

[1698] Specific examples

[1699] Example feedback: "This product is very easy to use, but I think it would be better if it were a little cheaper."

[1700] Example prompt: "Please rate the following feedback: This product is very easy to use, but I think it would be better if it were a little cheaper."

[1701] In this way, it is possible to improve the learning effect of test takers, accurately measure the effectiveness of educational programs, and improve the customer experience in virtual stores.

[1702] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1703] Step 1:

[1704] The user inputs feedback through the terminal. For example, the user may input feedback such as, "This product is very easy to use, but I think it would be better if it were a little cheaper." The input feedback is sent from the terminal to the server.

[1705] Step 2:

[1706] The server inputs the received feedback into the generative AI model. Specifically, it analyzes the content of the feedback using OpenAI's API. The prompt sentence used is "Please rate the following feedback: This product is very easy to use, but I think it would be better if it were a little cheaper." The generative AI model analyzes the content of the feedback and generates an evaluation result.

[1707] Step 3:

[1708] The server generates immediate feedback based on the evaluation results obtained from the generative AI model. For example, it generates feedback such as, "This feedback is very useful. We will take your opinion on the price into consideration." The generated feedback is sent from the server to the device.

[1709] Step 4:

[1710] The terminal displays the instant feedback received from the server to the user, who can then confirm the displayed feedback.

[1711] Step 5:

[1712] The device uses a camera to track changes in the user's emotions. The device captures the user's facial expressions and tone of voice with the camera and analyzes them using an emotion recognition library. The analysis results show changes in the user's emotions.

[1713] Step 6:

[1714] The server collects and stores the tracked emotional data in a database, which is used to measure the effectiveness of educational programs and improve the quality of shopping experiences.

[1715] Step 7:

[1716] The server measures the effectiveness of the educational program based on the collected emotional data and feedback evaluation results. Specifically, it integrates the emotional data and feedback evaluation results to identify areas for improvement in the educational program.

[1717] Step 8:

[1718] The server generates a new training program based on the improvements made to the training program, and the generated training program is used for the next test or training.

[1719] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1720] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1721] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1722] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1723] [Third embodiment]

[1724] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1725] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1726] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1727] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1728] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1729] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1730] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1731] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1732] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1733] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1734] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1735] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1736] "Example 1"

[1737] In one embodiment of the present invention, the AI-based means for generating business manual comprehension tests analyzes the contents of the business manual and generates questions based on the contents. Specifically, the text data of the business manual is used as input, and important keywords and concepts are extracted using natural language processing technology, and questions are generated based on these.

[1738] "Example 2"

[1739] In addition, as a means for accepting answers from test takers, the company provides an interface for test takers to answer the generated tests. Specifically, the company provides a web-based test platform that allows test takers to log in and answer the tests.

[1740] "Example 3"

[1741] Furthermore, as a means of providing immediate feedback based on the answers received, AI technology is used to evaluate the examinee's answers and provide the results to the examinee immediately. Specifically, the examinee's answers are analyzed using natural language processing technology to evaluate their accuracy. Based on the evaluation results, immediate feedback is provided to the examinee.

[1742] The processing flow of each embodiment will be described below.

[1743] "Example 1"

[1744] Step 1: Enter the text data of the business manual into the system.

[1745] Step 2: The system uses natural language processing techniques to extract important keywords and concepts from the text data.

[1746] Step 3: Based on the extracted keywords and concepts, the system generates comprehension test questions.

[1747] "Example 2"

[1748] Step 1: The system provides a web-based testing platform.

[1749] Step 2: Candidates log in to the platform and answer the generated test.

[1750] "Example 3"

[1751] Step 1: The system accepts the test taker's answers and analyzes them using natural language processing technology.

[1752] Step 2: The system evaluates the accuracy of the answer based on the analysis results.

[1753] Step 3: Based on the evaluation results, the system provides immediate feedback to the test-taker.

[1754] Example 1

[1755] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1756] Conventional business manual comprehension tests require manual question creation, which is time-consuming and labor-intensive. Furthermore, they do not provide immediate feedback on test takers' answers, which can reduce learning effectiveness. Furthermore, there was a lack of a way to automatically generate customizable tests based on the content of business manuals.

[1757] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1758] In this invention, the server includes means for receiving text data of a business manual as input, means for analyzing the received text data, means for extracting important keywords and concepts from the analyzed text data, means for generating questions based on the extracted keywords and concepts, means for outputting the generated questions, means for accepting answers from examinees to the generated test, and means for providing immediate feedback based on the accepted answers. This enables the automatic generation of customizable comprehension tests based on the contents of the business manual and immediate feedback to examinees on their answers.

[1759] A "business manual" is a document created by a company or organization to explain business procedures and rules to employees.

[1760] "Text data" is data that represents character information in digital form.

[1761] "Analysis" is the process of breaking down data and understanding its structure and meaning.

[1762] "Keywords" are words or phrases that are particularly important in the text data.

[1763] A "concept" is an abstract idea or theme expressed in text data.

[1764] "Questions" are questions presented to test takers to gauge their understanding.

[1765] "Generation" means creating new data or information using specific algorithms or models.

[1766] "Output" means providing the processing results to the outside.

[1767] "Test taker" means a person taking a comprehension test.

[1768] An "answer" is an answer provided by a test-taker to a question.

[1769] "Immediate feedback" refers to evaluations and comments provided immediately to test takers on their answers.

[1770] A "system" is a structure in which multiple elements work together to achieve a specific function.

[1771] MODE FOR CARRYING OUT THE INVENTION

[1772] This invention is a system that automatically generates comprehension tests based on the contents of a business manual and provides immediate feedback to test takers on their answers. A specific embodiment of this system will be described below.

[1773] 1. Program Generation

[1774] The server creates a program to generate a business manual comprehension test. This program has the function of analyzing the contents of the business manual and generating questions based on that content.

[1775] 2. Program Processing

[1776] The server receives the text data of the business manual as input. This text data can come from a file uploaded by the user or retrieved from a database. For example, if a user uploads a PDF file of the business manual, the server converts the PDF into text data.

[1777] The server then uses natural language processing techniques to analyze the text data, using Python libraries such as NLTK and spaCy to split the text data into sentences and words and analyze their grammatical structure.

[1778] The server extracts important keywords and concepts from the analyzed text data. This process uses techniques such as TF-IDF (Term Frequency-Inverse Document Frequency) and Word2Vec. For example, keywords such as "customer service," "quick," "polite," "accurate information," and "problem solving" are extracted.

[1779] Next, the server generates questions based on the extracted keywords and concepts. To generate these questions, it uses a generative AI model (for example, OpenAI's GPT-3). The server inputs a prompt to the generative AI model, instructing it to generate an appropriate question. For example, the server inputs the following prompt:

[1780] "Please explain the basics of customer service. Why is prompt and courteous service required?"

[1781] The generative AI model generates questions based on this prompt.

[1782] 3. Specific Examples

[1783] For example, consider a scenario where a user wants to upload a PDF file of a business manual. The user uses a terminal to access the system interface and clicks the "Select File" button. Next, the user selects the PDF file of the business manual and clicks the "Upload" button. The server receives this PDF file and converts it into text data.

[1784] The server then analyzes the text data and extracts important keywords and concepts, such as "customer service," "speed," "polite," "accurate information," and "problem resolution."

[1785] The server then inputs a prompt to the generative AI model to generate a question, such as:

[1786] "What are the basics of customer service? Explain why prompt and courteous service is required."

[1787] Finally, the server provides the generated questions to the user, who then obtains a list of questions to use as comprehension tests. In this way, it is possible to automatically generate customizable comprehension tests based on the contents of the business manual and provide immediate feedback to the examinee's answers.

[1788] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1789] Step 1:

[1790] The server receives the text data of the business manual as input. When a user uploads a PDF file of the business manual using a terminal, the server converts the PDF file into text data. This conversion uses OCR (Optical Character Recognition) technology. The input is a PDF file, and the output is text data.

[1791] Step 2:

[1792] The server analyzes the received text data. This analysis uses natural language processing libraries such as Python's "NLTK" or "spaCy." Specifically, the text data is divided into sentences and words, and the grammatical structure is analyzed. The input is the text data, and the output is a list of analyzed sentences and words.

[1793] Step 3:

[1794] The server extracts important keywords and concepts from the analyzed text data. This process uses technologies such as TF-IDF (Term Frequency-Inverse Document Frequency) and Word2Vec. For example, keywords such as "customer service," "quick," "polite," "accurate information," and "problem solving" are extracted. The input is a list of analyzed sentences and words, and the output is a list of important keywords and concepts.

[1795] Step 4:

[1796] The server generates questions based on the extracted keywords and concepts. To generate these questions, it uses a generative AI model (for example, OpenAI's GPT-3). The server inputs a prompt to the generative AI model, instructing it to generate an appropriate question. For example, the server inputs the following prompt:

[1797] "Please explain the basics of customer service. Why is prompt and courteous service required?"

[1798] The generative AI model generates a question based on this prompt: the input is the prompt and the output is the generated question.

[1799] Step 5:

[1800] The server provides the generated questions to the user. The user can use a terminal to review the generated questions and make corrections or additions as needed. Finally, the user gets a list of questions to use as a comprehension test. The input is the generated questions, and the output is the final list of questions.

[1801] Step 6:

[1802] The server accepts the examinee's answers to the generated test. The examinee answers the test using a terminal and sends the answers to the server. The input is the examinee's answers and the output is the accepted answers.

[1803] Step 7:

[1804] The server provides instant feedback based on the received answers. The server uses a generative AI model to evaluate the answers and generate appropriate feedback, such as determining whether the answer is correct or incorrect and suggesting areas for improvement. The input is the received answers, and the output is instant feedback.

[1805] (Application example 1)

[1806] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1807] Conventional business manual comprehension tests require manual creation, which is time-consuming and labor-intensive, and the quality of the tests is inconsistent. It is also difficult to immediately assess the test taker's level of understanding and provide feedback, making it difficult to measure the effectiveness of educational programs. Furthermore, there is a need for more efficient training for factory robot operators and engineers.

[1808] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1809] In this invention, the server includes a means for generating an AI-based business manual comprehension test, a means for analyzing the text data of the business manual and extracting important keywords and concepts, a means for generating questions based on the extracted keywords and concepts, a means for presenting the generated questions to robot operators and engineers, a means for collecting response data and evaluating comprehension, and a means for providing feedback on the evaluation results and suggesting additional training. This enables the automatic generation of high-quality comprehension tests based on the contents of the business manual, and allows for the instant evaluation of the test taker's comprehension and the provision of feedback. It also improves the efficiency of training for factory robot operators and engineers.

[1810] "AI" is an abbreviation for artificial intelligence, a technology that allows computers to imitate human intelligence to learn and reason.

[1811] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[1812] A "generator" is a method or device for generating data or information for a particular purpose.

[1813] "Text data" is data that represents character information in digital form.

[1814] "Analysis" is the process of examining data or information in detail to understand its structure and meaning.

[1815] A "keyword" is a word or phrase that has significant meaning in a particular document or piece of data.

[1816] A "concept" represents an abstract understanding or recognition of a particular event or phenomenon.

[1817] A "question" is a question asked to elicit specific information.

[1818] A "robot operator" is an engineer or worker who operates factory robots.

[1819] An "engineer" is a person who has specialized knowledge and skills in a particular technical field.

[1820] "Response data" refers to information about responses provided by test takers to questions.

[1821] "Comprehension" refers to the degree to which a particular piece of content or information is accurately understood.

[1822] "Evaluation" is the process of judging the value or performance of an object based on specific criteria.

[1823] "Feedback" means providing a response or evaluation of a particular action or result.

[1824] "Training" means training or education to acquire specific skills or knowledge.

[1825] As an embodiment of the present invention, the following system is constructed.

[1826] System Configuration

[1827] The system consists of a server, factory robots, tablet terminals, and a network. The server plays a central role in generating, analyzing, and evaluating work manual comprehension tests. The factory robots present the generated tests to operators and engineers and collect their responses. The tablet terminals are used as an interface for users to take the tests.

[1828] Hardware and software used

[1829] Hardware: Servers, factory robots, tablets

[1830] Software: Python, TensorFlow, NLTK (Natural Language Toolkit), Flask (web framework)

[1831] Data processing and calculation

[1832] 1. Analysis of the business manual:

[1833] The server reads the text data of the business manual using Python and uses NLTK to tokenize and extract keywords, thereby extracting important keywords and concepts.

[1834] 2. Question generation:

[1835] The server generates questions using a generative AI model (e.g., GPT-3) based on the extracted keywords and concepts. The generated questions are used to measure comprehension and are in the form of multiple-choice, written answers, or practical tests.

[1836] 3. Present the test:

[1837] The factory robot then presents the generated questions to the robot operator or engineer, who then answers the questions via a tablet device.

[1838] 4. Collecting and Evaluating Responses:

[1839] The server collects the response data sent from the tablet device and runs a scoring algorithm to evaluate the level of comprehension. The evaluation results are immediately fed back to the user.

[1840] 5. Additional training suggestions:

[1841] Based on the evaluation results, the server will suggest additional training as needed, thereby improving the user's skills.

[1842] Specific examples

[1843] Contents of the operations manual: "Robot maintenance procedures"

[1844] Generated question: "What is the first task that should be performed in the robot maintenance procedure?"

[1845] Prompt Sentence Examples

[1846] Below are some example prompts to input to the generative AI model:

[1847] Based on the contents of the operations manual below, generate questions to gauge comprehension.

[1848] Operation Manual: Robot Maintenance Procedures

[1849] 1. Turn off the power

[1850] 2. Remove the outer cover

[1851] 3. Clean the interior

[1852] 4. Inspect the parts

[1853] Generate a question.

[1854] In this way, the system can automatically generate high-quality comprehension tests based on the contents of the operation manual, instantly evaluate the test taker's level of understanding, and provide feedback, thereby improving the efficiency of training for factory robot operators and engineers.

[1855] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1856] Step 1:

[1857] The server receives the text data of the business manual. It reads the text data of the business manual as input and analyzes the text data using Python. Specifically, it uses NLTK to tokenize the text and extract important keywords and concepts. The output is a list of the extracted keywords and concepts.

[1858] Step 2:

[1859] The server generates questions using a generative AI model (e.g., GPT-3) based on the extracted keywords and concepts. It receives a list of keywords and concepts as input and inputs a prompt sentence into the generative AI model. Specifically, it sends the prompt sentence to the generative AI model to generate questions. The output is a list of generated questions.

[1860] Step 3:

[1861] The server sends the generated questions to the factory robot. It receives the list of generated questions as input and sends them to the factory robot. Specifically, it transfers the question data to the factory robot via the network. The output is the question data sent to the factory robot.

[1862] Step 4:

[1863] The factory robot presents the generated questions to the robot operator or engineer. It receives question data sent from the server as input and displays it to the user via a tablet device. Specifically, it displays the question on the tablet device screen so that the user can answer it. The output is the user's answer data.

[1864] Step 5:

[1865] The server collects the response data sent from the tablet device and evaluates the level of understanding. It receives the user's response data as input and executes a scoring algorithm. Specifically, it analyzes the response data and evaluates the accuracy rate and level of understanding. The output is the evaluation result data.

[1866] Step 6:

[1867] The server feeds back the evaluation results to the user. It receives the evaluation result data as input and provides it to the user. Specifically, it displays the evaluation results on a tablet device and provides feedback to the user. The output is the feedback data provided to the user.

[1868] Step 7:

[1869] The server proposes additional training based on the evaluation results. It receives the evaluation result data as input and generates additional training content as needed. Specifically, it analyzes the evaluation results, identifies the user's weaknesses and areas for improvement, and proposes an appropriate training program. The output is the data of the proposed training program.

[1870] Example 2

[1871] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1872] In conventional testing systems, test question generation is done manually, which is time-consuming and labor-intensive. It is also difficult to immediately evaluate test takers' answers and provide feedback. Furthermore, it is difficult to generate customizable tests based on specific educational content. This poses a challenge in measuring the effectiveness of educational programs.

[1873] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1874] In this invention, the server includes means for generating test questions using a generative AI model, means for accepting answers from examinees to the generated test questions, means for storing the accepted answers in a database, and means for evaluating and displaying results based on the stored answers. This enables automatic generation and immediate evaluation of test questions, and enables the generation of customizable tests based on specific educational content and the measurement of the effectiveness of educational programs.

[1875] A "generative AI model" is an algorithm that uses artificial intelligence technology to automatically generate test questions based on specific prompts.

[1876] "Test questions" are questions that test takers answer and are generated by a generative AI model.

[1877] An "examinee" is a user who answers test questions.

[1878] "Means for accepting answers" refers to an interface that allows the system to receive and process answers entered by the examinee.

[1879] A "database" is a system for storing data such as test taker answers and generated test questions.

[1880] "Means for evaluating and displaying results" refers to a function that determines whether the test taker's answers are correct or incorrect and provides feedback on the results to the test taker.

[1881] A "customizable test" is a test whose content can be modified to suit specific educational content or purposes.

[1882] An "educational program" is a set of educational activities and materials designed to achieve specific learning objectives.

[1883] "Effectiveness measurement" is the process of evaluating the impact and results that an educational program has on test takers.

[1884] This invention is a system that uses a generative AI model to generate test questions, accepts answers from test takers, and evaluates them. Specific embodiments of this system are described below.

[1885] System configuration

[1886] server

[1887] The server hosts a web-based testing platform. The server uses the following hardware and software:

[1888] Hardware: High-performance server machine (e.g., general-purpose server)

[1889] Software: web server (e.g., Apache HTTP Server), database (e.g., MySQL), programming language (e.g., Python, JavaScript)

[1890] The server generates test questions using a generative AI model, which are then stored in a database for test takers to access.

[1891] Terminal

[1892] The terminal provides the interface for test takers to complete the test. The terminal uses the following hardware and software:

[1893] Hardware: PCs, tablets, smartphones

[1894] Software: Web browser (e.g. Google Chrome)

[1895] The device displays the test questions received from the server and allows the test taker to enter their answers, which are then sent to the server in real time and stored.

[1896] User

[1897] The user (candidate) logs into the web-based testing platform and answers the provided test questions. The user follows these steps:

[1898] 1. Open a web browser and access the test platform URL.

[1899] 2. Enter your username and password on the login screen to log in.

[1900] 3. Enter your answers to the test questions that appear and click the submit button.

[1901] Specific examples

[1902] For example, if a test taker takes a math test, the server uses a generative AI model to generate questions like this:

[1903] Question 1: "Solve the following equation: 2x + 3 = 7"

[1904] Question 2: "Find the next term in the following sequence: 1, 4, 9, 16, ?"

[1905] The examinee enters the answers to these questions and submits them.

[1906] Prompt Sentence Examples

[1907] Enter the following prompt for the generative AI model:

[1908] "Generate math test questions. The questions should be at a middle school level and include equations and sequences."

[1909] Using this prompt, the generative AI model generates appropriate test questions.

[1910] The above is a specific embodiment for carrying out the present invention. This system enables automatic generation and instant evaluation of test questions, and enables the creation of customizable tests based on specific educational content and the measurement of the effectiveness of educational programs.

[1911] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1912] Step 1:

[1913] User Login

[1914] Input: Username, Password

[1915] Server: The server receives the username and password entered by the user on the login screen and checks them against the authentication information stored in the database. If authentication is successful, the server issues the user a session ID and redirects them to the dashboard screen.

[1916] Output: Authentication result, session ID

[1917] What happens: The server securely sends and receives the user's authentication information using SSL / TLS encryption. If authentication is successful, it starts the user's session and returns the URL of the dashboard screen.

[1918] Step 2:

[1919] Generate test questions

[1920] Input: Prompt statement (e.g., "Generate a middle school level math test question.")

[1921] Server: The server sends prompts to the generative AI model to generate test questions, which are then stored in a database.

[1922] Output: Generated test questions

[1923] Specific operation: The server calls the API of the generative AI model and sends a prompt. The generative AI model generates test questions based on the prompt and returns them to the server. The server stores the generated test questions in a database.

[1924] Step 3:

[1925] View test questions

[1926] Input: User request (request to start test)

[1927] Server: The server retrieves the generated test questions from the database and sends them to the user's device.

[1928] Terminal: The terminal displays the test questions received from the server to the user.

[1929] Output: Test question display screen

[1930] Specific operation: The server receives a user request, retrieves the corresponding test questions from the database, and sends the retrieved test questions in HTML format to the terminal, which then displays them.

[1931] Step 4:

[1932] User answer input

[1933] Input: User answer (e.g. "x = 2")

[1934] Terminal: The terminal provides an input field to accept user-entered answers and temporarily stores the data as the user enters the answers.

[1935] Output: User response data

[1936] What it does: The device validates your answers in real time as you enter them, providing feedback to help prevent typing errors. Your answers are temporarily stored in the device's memory.

[1937] Step 5:

[1938] Submit and save your answers

[1939] Input: User response data

[1940] Terminal: The terminal sends the answers entered by the user to the server.

[1941] Server: The server stores the received answer in a database and sends a confirmation message to the user if the save was successful.

[1942] Output: Save confirmation message

[1943] Specific operation: When the user clicks the send button, the device sends the answer data to the server. The server saves the received answer data in the database and returns a message to the device indicating that the save was successful.

[1944] Step 6:

[1945] Displaying the results

[1946] Input: Saved response data

[1947] Server: The server evaluates the user's answers and generates a result, e.g., whether the answer is correct or incorrect and calculates a score.

[1948] Terminal: The terminal displays the results received from the server to the user.

[1949] Output: Evaluation results, scores

[1950] Specific operation: The server retrieves the user's answer data from the database and uses an evaluation algorithm to determine whether it is correct or incorrect. It calculates the evaluation results and score and sends them to the terminal. The terminal then displays the received evaluation results and score to the user.

[1951] The above are the specific processing steps of the program of this system.

[1952] (Application example 2)

[1953] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1954] Conventional business manual comprehension testing systems have the problem that test generation is done manually, which takes time and effort. Also, there is a possibility that learning effectiveness may be reduced because immediate feedback on test takers' answers is not provided. Furthermore, because the test content is fixed, there is also the issue that it is difficult to customize according to the comprehension level of each individual test taker.

[1955] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1956] In this invention, the server includes means for generating a business manual comprehension test using a generative AI model, means for accepting answers from examinees to the test generated by the generation means, means for providing immediate feedback based on the accepted answers, means for generating a test by sending prompts to the generative AI model, means for providing the generated test content to the user, and means for the user to answer the test and evaluate the answer content. This enables automatic test generation and provision of immediate feedback, making it possible to provide a test customized according to the examinee's level of understanding.

[1957] A "generative AI model" is an algorithm that uses artificial intelligence to generate an output for a specific task.

[1958] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[1959] A "generator" is a function for automatically creating tests for a specific purpose.

[1960] "Test taker" means a person taking a test.

[1961] "Means for accepting answers" refers to the function that allows the system to receive answers entered by test takers for the test.

[1962] "Immediate feedback" is a feature that provides evaluations and comments on test takers' answers immediately after they answer a test.

[1963] A "prompt" is an instruction entered into a generative AI model to cause it to generate a specific output.

[1964] "User" refers to a person who uses the system.

[1965] "Means of evaluation" refers to a function for determining the accuracy and level of understanding of the test taker's answers.

[1966] The embodiments of the present invention will be described in detail below.

[1967] First, the server uses a generative AI model to generate a business manual comprehension test. The generation method involves sending a prompt to the generative AI model, which has the ability to automatically create a specific test. An example of a prompt is "Please generate basic math problems."

[1968] The server then provides a means to accept test takers' responses to the generated tests. Test takers can log in to a web-based testing platform using devices such as smartphones or PCs and take the tests.

[1969] When a test taker answers the test, the server immediately evaluates the answer and provides feedback. The evaluation method has a function to determine the accuracy and level of understanding based on the test taker's answer. For example, if a test taker answers "2 + 2 = 4," the server immediately evaluates whether the answer is correct and provides feedback that "That's correct" if it is.

[1970] The implementation of this system uses the following hardware and software:

[1971] Hardware: Servers, smartphones, PCs

[1972] Software: Flask (Python web framework), OpenAI API (generative AI model)

[1973] For example, if a user inputs the prompt "Generate basic math problems," the generative AI model will generate math problems and provide them to the user. The user answers the generated questions, and the answers are sent to the server. The server evaluates the answers and provides immediate feedback.

[1974] In this way, tests can be automatically generated and immediate feedback can be provided, allowing tests to be customized according to the examinee's level of understanding.

[1975] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1976] Step 1:

[1977] A user logs into a web-based testing platform using a terminal.

[1978] Input: User login information (username, password)

[1979] Output: Login success or failure message

[1980] Specific operation: The user enters a username and password into the login form and sends it to the server, which checks it against the database and returns the authentication result.

[1981] Step 2:

[1982] The user inputs a prompt statement into the generative AI model to request test generation.

[1983] Input: Prompt sentence (e.g. "Generate a basic math problem")

[1984] Output: Generated test content

[1985] Specific operation: The user enters a prompt into the test generation form and sends it to the server. The server then sends the prompt to the generative AI model and receives the generated test content.

[1986] Step 3:

[1987] The server provides the generated test content to the user.

[1988] Input: Generated test content

[1989] Output: Test results displayed on the user's device

[1990] Specific operation: The server sends the generated test content to the user's terminal and displays it on the user's screen.

[1991] Step 4:

[1992] The user answers the quiz.

[1993] Input: User's answer

[1994] Output: Response data sent to the server

[1995] Specific operation: The user enters answers to each question in the test and sends the answers to the server.

[1996] Step 5:

[1997] The server evaluates the user's answers.

[1998] Input: User response data

[1999] Output: Evaluation results (correct answers, incorrect answers, score, etc.)

[2000] Specific operation: The server analyzes the user's answer data, compares it with the correct answer, and generates an evaluation result.

[2001] Step 6:

[2002] The server provides the evaluation results to the user as real-time feedback.

[2003] Input: Evaluation result

[2004] Output: A feedback message that is displayed on the user's terminal.

[2005] Specific operation: The server sends the evaluation results to the user's device and displays a feedback message on the user's screen.

[2006] Example 3

[2007] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2008] Conventional business manual comprehension tests have the problem of reducing learning effectiveness because test takers' answers are not evaluated and feedback is not provided immediately. In addition, tests are generated in a fixed format, making it difficult to customize them to fit the content of the business manual. Furthermore, because feedback is not directly linked to measuring the effectiveness of the educational program, there is a lack of data to improve the quality of education.

[2009] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[2010] In this invention, the server includes a means for generating a business manual comprehension test using a generative AI model, a means for accepting test takers' answers, a means for analyzing the accepted answers using natural language processing technology and evaluating their accuracy, and a means for providing immediate feedback based on the evaluation results. This allows for immediate evaluation and feedback of the test takers' answers, improving learning effectiveness. Furthermore, generating customizable tests based on the contents of the business manual allows for more appropriate evaluation. Furthermore, the immediate feedback enables the effectiveness of educational programs to be measured, allowing for the collection of data to improve the quality of education.

[2011] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate answers or data for specific tasks.

[2012] The "Business Manual Comprehension Test" is a test to evaluate whether or not the contents of the business manual are understood.

[2013] A "generation means" is a method or device for generating data or information for a specific purpose.

[2014] "Candidate" refers to a person taking a test or examination.

[2015] "Means for accepting answers" refers to a method or device by which the system receives answers entered by the examinee.

[2016] "Natural language processing technology" is a technology that allows computers to understand and analyze human language.

[2017] An "evaluation means" is a method or device that uses received data or information to determine its accuracy or appropriateness.

[2018] "Immediate feedback" refers to providing test takers with prompt evaluation results and advice in response to their answers.

[2019] A "customizable test" is a test whose content can be modified to suit specific conditions or needs.

[2020] An "educational program" is a set of learning activities and materials designed to acquire specific knowledge or skills.

[2021] "Effectiveness measurement" refers to assessing the success of a particular activity or program.

[2022] This invention is a system that uses a generative AI model to generate a business manual comprehension test, analyzes the test taker's answers, and provides immediate feedback. Specific embodiments of this system are described below.

[2023] Hardware and software used

[2024] Hardware: Servers, terminals

[2025] Software: Generative AI models, Natural Language Processing (NLP)

[2026] System Overview

[2027] 1. The user enters the answer

[2028] The user inputs a response into an input form on the device. For example, in response to the question "What color is the Earth?", the user inputs "blue."

[2029] 2. The device sends the answer to the server

[2030] The terminal sends the answer entered by the user to the server. Specifically, the terminal sends the answer data to the server using an HTTP request.

[2031] 3. The server receives the answer

[2032] The server receives the HTTP request sent from the device and extracts the response data. For example, it verifies that the received data is "blue."

[2033] 4. The server parses the answer

[2034] The server analyzes the received response data using natural language processing (NLP). Specifically, it inputs a prompt sentence into the generative AI model to perform the analysis. An example of a prompt sentence is "Question: What color is the Earth? Answer: Blue."

[2035] 5. The server generates the evaluation result

[2036] The server evaluates the accuracy of the answer based on the analysis results from the generative AI model. For example, if the generative AI model determines that the answer "blue" is accurate, it generates that evaluation result.

[2037] 6. The server generates feedback

[2038] The server generates feedback for the user based on the evaluation result, for example, a feedback message such as "You're right. The Earth is blue."

[2039] 7. The server sends feedback to the device

[2040] The server sends the generated feedback message to the terminal. Specifically, the server sends the feedback data to the terminal using an HTTP response.

[2041] 8. The device displays feedback to the user

[2042] The device displays the feedback message received from the server to the user, for example, a message such as "You're right! The Earth is blue."

[2043] Specific examples

[2044] For example, when a user answers "blue" to the question "What color is the Earth?", the specific operation is as follows.

[2045] 1. The user enters the answer

[2046] The user enters "blue" into the input form on the device.

[2047] 2. The device sends the answer to the server

[2048] The device uses an HTTP request to send the answer data "blue" to the server.

[2049] 3. The server receives the answer

[2050] The server receives the HTTP request and extracts the response data "blue."

[2051] 4. The server parses the answer

[2052] The server inputs the prompt sentence "Question: What color is the Earth? Answer: Blue" into the generative AI model and performs analysis.

[2053] 5. The server generates the evaluation result

[2054] Based on the analysis results from the generative AI model, the server evaluates that the answer "blue" is accurate.

[2055] 6. The server generates feedback

[2056] The server generates a feedback message saying "Correct. The Earth is blue."

[2057] 7. The server sends feedback to the device

[2058] The server sends a feedback message to the device using an HTTP response.

[2059] 8. The device displays feedback to the user

[2060] The device will display the message "Correct. The Earth is blue." on the screen.

[2061] In this way, the user can receive instant feedback, thereby improving the learning effect. The flow of the identification process in the third embodiment will be described with reference to FIG.

[2062] Step 1:

[2063] The user enters the answer.

[2064] Input: A user types a response into a device input form. For example, the user types "blue" in response to the question "What color is the Earth?"

[2065] Output: The entered answer data is saved on the device.

[2066] What happens: The user uses the keyboard to enter an answer and clicks the submit button.

[2067] Step 2:

[2068] The device sends the response to the server.

[2069] Input: The answer data entered by the user.

[2070] Output: The response data is sent to the server as an HTTP request.

[2071] Specific operation: The terminal sends the answer data entered by the user to the server in the form of an HTTP request.

[2072] Step 3:

[2073] The server receives the response.

[2074] Input: The HTTP request sent from the device.

[2075] Output: The extracted response data.

[2076] Specific operation: The server receives the HTTP request and extracts the response data. For example, it obtains the response data "blue."

[2077] Step 4:

[2078] The server parses the response.

[2079] Input: The extracted response data.

[2080] Output: Analysis results.

[2081] Specific operation: The server inputs a prompt sentence into the generative AI model and performs analysis. For example, the prompt sentence "Question: What color is the Earth? Answer: Blue" is input into the generative AI model and the analysis result is obtained.

[2082] Step 5:

[2083] The server generates the evaluation results.

[2084] Input: Analysis results from a generative AI model.

[2085] Output: Evaluation result.

[2086] Specific operation: The server evaluates the accuracy of the answer based on the analysis results from the generative AI model. For example, it determines that the answer "blue" is accurate and generates an evaluation result.

[2087] Step 6:

[2088] The server generates the feedback.

[2089] Input: Evaluation result.

[2090] Output: The feedback message.

[2091] Specific operation: The server generates feedback for the user based on the evaluation result. For example, it generates a feedback message such as "You're right. The Earth is blue."

[2092] Step 7:

[2093] The server sends the feedback to the device.

[2094] Input: The feedback message.

[2095] Output: The feedback message is sent to the device as an HTTP response.

[2096] Specific operation: The server sends the generated feedback message to the terminal in the form of an HTTP response.

[2097] Step 8:

[2098] The device displays the feedback to the user.

[2099] Input: The feedback message sent by the server.

[2100] Output: Feedback messages displayed on the screen.

[2101] Specific operation: The device displays the feedback message received from the server on the screen, for example, "You're right. The Earth is blue."

[2102] (Application example 3)

[2103] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2104] Conventional business manual comprehension test systems rarely provide immediate feedback on test takers' answers, ...

Claims

1. means for generating tests for a given document using a generative AI model; means for accepting test taker responses to the generated test; means for recognizing emotions of the examinee based on the received answers; means for providing feedback to the test-taker based on the received answers and emotion data indicative of the recognized emotion of the test-taker; A system including:

2. A means for analyzing the received response using natural language processing technology; A means for evaluating the answers based on an analysis result of the answers; The providing means provides the feedback based on an evaluation result of the answer. The system of claim 1 .

3. means for reflecting said emotion data in an effect measurement of said test; The system of claim 1 .

Images