System

Abstract

An object of the system according to the embodiment is to provide a problem corresponding to a skill level or a weak point of a user and support effective programming learning.SOLUTION: A system includes a question generation part, an answer matching part, an explanation providing part, a weak point specification part, and a moving image analysis part. The problem generation unit generates a problem according to the skill level of the user. The answer matching unit analyzes the answer to the question generated by the question generating unit. The explanation providing unit provides an explanation based on the answer analyzed by the answer matching unit. The weak point specifying unit specifies a weak point of the user. The moving image analysis unit analyzes a state of coding of a user in a moving image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it was difficult to provide problems that suited the user's skill level and weaknesses when learning programming, making effective learning difficult.

[0005] The system according to the embodiment aims to provide problems that correspond to the user's skill level and weaknesses, and to support effective programming learning. [Means for solving the problem]

[0006] The system according to the embodiment includes a question generation unit, an answer checking unit, an explanation providing unit, a weakness identification unit, and a video analysis unit. The question generation unit generates questions according to the user's skill level. The answer checking unit analyzes the answers to the questions generated by the question generation unit. The explanation providing unit provides explanations based on the answers analyzed by the answer checking unit. The weakness identification unit identifies the user's weaknesses. The video analysis unit analyzes video of the user's coding. [Effects of the Invention]

[0007] The system according to the embodiment can provide problems that correspond to the user's skill level and weaknesses, and can support effective programming learning. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The fully automated programming school according to an embodiment of the present invention is a system that automatically generates problems according to a user's skill level, provides answers and explanations, customizes problems based on the user's weaknesses, supports all programming languages, and analyzes coding processes using video. This allows users to learn effectively at their own pace without the stress of interacting with others and overcome their weaknesses.

[0029] A fully automated programming school according to an embodiment includes a problem generation unit, an answer checking unit, an explanation providing unit, a weakness identification unit, and a video analysis unit. The problem generation unit generates problems according to a user's skill level. For example, the generation AI analyzes the user's skill level and automatically generates programming problems according to that level. The generation AI generates problems using a text generation AI (e.g., LLM). The generation AI can also generate problem content using a multimodal generation AI. The generation AI generates problems based on information related to the user's skill level. The answer checking unit analyzes the answers to problems generated by the problem generation unit. For example, the generation AI analyzes the answers to problems solved by the user and determines whether they are correct. The generation AI analyzes code submitted by the user and checks the answers. The explanation providing unit provides explanations based on the answers analyzed by the answer checking unit. For example, the generation AI provides a detailed explanation of the reason if the answer is correct, and a detailed explanation of the incorrect part if the answer is incorrect. If there is an error in the code written by the user, the generation AI specifically explains the cause of the error and how to correct it. The weakness identification unit identifies the user's weaknesses. For example, the generation AI analyzes the user's past answer history and coding behavior to identify the user's weaknesses. The generation AI identifies weaknesses based on the user's answer history and coding behavior. The video analysis unit analyzes the user's coding behavior using video. For example, the generation AI analyzes videos of the user coding to identify the user's weaknesses. The generation AI analyzes the user's coding videos to identify the user's weaknesses. This allows the fully automated programming school according to the embodiment to allow users to effectively progress through learning at their own pace without interpersonal stress and overcome their weaknesses. For example, the user can correct errors in their code based on the explanations provided by the generation AI and move on to the next problem. Furthermore, the user can intensively study specific skills based on the weaknesses identified by the generation AI. Furthermore, the user can improve their coding style based on the coding videos analyzed by the generation AI.

[0030] The question generation unit can analyze the learning history, compare skill levels, and generate optimal questions. In the question generation unit, for example, the generation AI analyzes the user's past learning history and grasps the progress of the learning content. For example, the current skill level is evaluated based on the topics learned in the past and the accuracy rate of questions answered, and questions are generated accordingly. In addition, the question generation unit compares the user's skill level and generates optimal questions. For example, the generation AI evaluates the user's skill level based on past test results and self-assessment, and generates questions accordingly. In this way, the user's learning history can be analyzed and optimal questions can be generated according to the skill level.

[0031] The question generation unit can analyze a user's learning style and provide a question format that corresponds to that. For example, the question generation unit uses a generation AI to analyze a user's learning style and provide questions that make extensive use of diagrams and graphs to users who prefer visual learning. For example, it generates questions that include flowcharts that visually show the flow of an algorithm. The question generation unit also analyzes a user's learning style and provides questions with audio explanations to users who prefer auditory learning. For example, it provides audio explanations for questions. The question generation unit also analyzes a user's learning style and provides interactive questions to users who prefer tactile learning. For example, it provides questions that require users to actually input and execute code. This makes it possible to provide question formats that correspond to the user's learning style.

[0032] The question generation unit can integrate data from different learning platforms to generate comprehensive questions. For example, the question generation unit uses a generation AI to collect data from online courses, books, and tutorials and integrate them to generate comprehensive questions. For example, it provides questions that promote comprehensive understanding based on knowledge obtained from multiple resources. The question generation unit also integrates data from different learning platforms to generate comprehensive questions. For example, it provides questions that combine the content of an online course and the content of a book. The question generation unit also integrates data from different learning platforms to support user learning. For example, it provides questions based on the content of a tutorial. This allows data from different learning platforms to be integrated to generate comprehensive questions.

[0033] The question generation unit can generate questions including related themes and scenarios based on the user's interests and hobbies. For example, the question generation unit uses a generation AI to analyze the user's interests and hobbies and generate questions including related themes and scenarios based on the analysis. For example, a user who is interested in game development is provided with questions based on game scenarios. The question generation unit also analyzes the user's interests and hobbies and generates questions including related themes and scenarios based on the analysis. For example, a user who is interested in music is provided with questions based on music scenarios. The question generation unit also analyzes the user's interests and hobbies and generates questions including related themes and scenarios based on the analysis. For example, a user who is interested in sports is provided with questions based on sports scenarios. In this way, questions including related themes and scenarios can be generated based on the user's interests and hobbies.

[0034] The answer checking section can present multiple answer patterns for the answer and explain the advantages and disadvantages of each. In the answer checking section, for example, the generation AI analyzes the user's answer and presents multiple answer patterns. For example, it shows example answers using different algorithms or data structures and explains the advantages and disadvantages of each. In the answer checking section, the generation AI analyzes the user's answer and presents multiple answer patterns. For example, it shows example answers using different algorithms or data structures and explains the advantages and disadvantages of each. In the answer checking section, the generation AI analyzes the user's answer and presents multiple answer patterns. For example, it shows example answers using different algorithms or data structures and explains the advantages and disadvantages of each. In this way, by presenting multiple answer patterns for the user's answer and explaining the advantages and disadvantages of each, the user's understanding can be deepened.

[0035] The answer checking section can analyze the answer history and compare past errors with current answers to evaluate progress. In the answer checking section, for example, the generation AI analyzes the user's answer history and compares past errors with current answers. For example, it evaluates changes in the accuracy and efficiency of answers to the same type of problem. In addition, the answer checking section can analyze the user's answer history and compare past errors with current answers. For example, it evaluates changes in the accuracy and efficiency of answers to the same type of problem. In addition, the answer checking section can analyze the user's answer history and compare past errors with current answers. For example, it evaluates changes in the accuracy and efficiency of answers to the same type of problem. In this way, the user's growth can be visualized by analyzing the user's answer history and comparing past errors with current answers to evaluate progress.

[0036] The answer checking section can provide example answers in different programming languages, allowing the user to understand from multiple perspectives. In the answer checking section, for example, the generation AI analyzes the user's answer and provides example answers in different programming languages. For example, it converts an answer written in Python into an answer in Java or C++ and explains the differences between them. In addition, the answer checking section can analyze the user's answer and provide example answers in different programming languages. For example, it converts an answer written in Python into an answer in Java or C++ and explains the differences between them. In addition, the answer checking section can analyze the user's answer and provide example answers in different programming languages. For example, it converts an answer written in Python into an answer in Java or C++ and explains the differences between them. In this way, by providing example answers in different programming languages, the user can understand from multiple perspectives.

[0037] The answer checking section can automatically suggest related documents and resources for the answer, promoting a deeper understanding. In the answer checking section, for example, the generation AI analyzes the user's answer and automatically suggests related documents and resources. For example, detailed explanations and tutorials about specific algorithms are provided. In addition, the answer checking section can analyze the user's answer and automatically suggest related documents and resources. For example, detailed explanations and tutorials about specific algorithms are provided. In addition, the answer checking section can analyze the user's answer and automatically suggest related documents and resources. For example, detailed explanations and tutorials about specific algorithms are provided. In this way, automatic suggestions of related documents and resources for the user's answer can promote a deeper understanding.

[0038] The weakness identification unit can analyze the user's weaknesses in detail and generate questions that focus on a specific skill set. In the weakness identification unit, for example, the generation AI analyzes the user's past answer history and identifies weaknesses in a specific skill set. For example, a user who is not good at loop structures is provided with many questions related to loops. In addition, the weakness identification unit analyzes the user's past answer history and identifies weaknesses in a specific skill set. For example, a user who is not good at loop structures is provided with many questions related to loops. In addition, the weakness identification unit analyzes the user's past answer history and identifies weaknesses in a specific skill set. For example, a user who is not good at loop structures is provided with many questions related to loops. In this way, the user's weaknesses are analyzed in detail and questions that focus on a specific skill set are generated, thereby providing effective learning.

[0039] The weakness identification unit can propose improvements to the user's weaknesses and provide problems based on those improvements. For example, the generation AI analyzes the user's weaknesses and proposes improvements. For example, if the user lacks understanding of a specific algorithm, it provides problems that will teach that algorithm from the basics to its applications. The weakness identification unit can also analyze the user's weaknesses and propose improvements. For example, if the user lacks understanding of a specific algorithm, it provides problems that will teach that algorithm from the basics to its applications. The weakness identification unit can also analyze the user's weaknesses and propose improvements. For example, if the user lacks understanding of a specific algorithm, it provides problems that will teach that algorithm from the basics to its applications. In this way, by proposing improvements to the user's weaknesses and providing problems based on those improvements, effective learning can be provided.

[0040] The weakness identification unit can increase motivation by presenting success stories of other users related to the user's weaknesses. In the weakness identification unit, for example, the generation AI collects success stories of other users related to the user's weaknesses and presents them. For example, it introduces the experiences and success stories of users who have overcome the same weaknesses. In addition, the weakness identification unit can increase motivation by presenting success stories of other users related to the user's weaknesses. For example, it introduces the experiences and success stories of users who have overcome the same weaknesses. In addition, the weakness identification unit can increase motivation by presenting success stories of other users related to the user's weaknesses.

[0041] The weakness identification unit can suggest group learning sessions that address the user's weaknesses and provide opportunities for the user to study cooperatively with other users. In the weakness identification unit, for example, the generation AI identifies the user's weaknesses and suggests a group learning session based on the weaknesses. For example, a session is provided in which users with the same weaknesses can study cooperatively. In addition, the weakness identification unit can suggest group learning sessions that address the user's weaknesses and suggest a group learning session based on the weaknesses. For example, a session is provided in which users with the same weaknesses can study cooperatively. In addition, the weakness identification unit can suggest group learning sessions that address the user's weaknesses and suggest a group learning session based on the weaknesses. For example, a session is provided in which users with the same weaknesses can study cooperatively. In this way, effective learning can be provided by suggesting group learning sessions that address the user's weaknesses and providing opportunities for learning cooperatively with other users.

[0042] The question generation unit can analyze the characteristics of each programming language and provide the optimal learning path for each language. In the question generation unit, for example, a generation AI analyzes the characteristics of each programming language and provides the optimal learning path for each language. For example, a learning path that starts with basic grammar is provided for beginners of Python. In addition, the question generation unit can analyze the characteristics of each programming language and provide the optimal learning path for each language. For example, a learning path that starts with basic grammar is provided for beginners of Python. In addition, the question generation unit can analyze the characteristics of each programming language and provide the optimal learning path for each language. For example, a learning path that starts with basic grammar is provided for beginners of Python. In this way, by analyzing the characteristics of each programming language and providing the optimal learning path for each language, effective learning can be provided.

[0043] The question generation unit can analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, the question generation unit uses a generation AI to analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, a question is provided that asks the user to implement an algorithm learned in Python in Java. The question generation unit also uses a generation AI to analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, a question is provided that asks the user to implement an algorithm learned in Python in Java. The question generation unit also uses a generation AI to analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, a question is provided that asks the user to implement an algorithm learned in Python in Java. In this way, the user's learning progress can be analyzed and questions that promote skill transfer between multiple programming languages ​​can be provided, thereby providing effective learning.

[0044] The question generation unit can integrate feedback from different programming language communities and provide it to the user. For example, the generation AI in the question generation unit collects feedback from different programming language communities, integrates it, and provides it to the user. For example, it provides best practices from the Python community and the Java community. The question generation unit also collects feedback from different programming language communities, integrates it, and provides it to the user. For example, it provides best practices from the Python community and the Java community. The question generation unit also collects feedback from different programming language communities, integrates it, and provides it to the user. For example, it provides best practices from the Python community and the Java community. In this way, feedback from different programming language communities can be integrated and provided to the user, thereby providing effective learning.

[0045] The question generation unit can suggest the most appropriate programming language according to the user's learning goal and provide questions specialized for that language. In the question generation unit, for example, a generation AI analyzes the user's learning goal and suggests the most appropriate programming language accordingly. For example, it suggests Python for a user who wants to learn data science and provides questions specialized for that language. In addition, the question generation unit can analyze the user's learning goal and suggest the most appropriate programming language according to that goal. For example, it suggests Python for a user who wants to learn data science and provides questions specialized for that language. In addition, the question generation unit can analyze the user's learning goal and suggest the most appropriate programming language according to that goal. For example, it suggests Python for a user who wants to learn data science and provides questions specialized for that language. In this way, by suggesting the most appropriate programming language according to the user's learning goal and providing questions specialized for that language, effective learning can be provided.

[0046] The video analysis unit can analyze a user's coding videos and suggest improvements for specific operations or patterns. In the video analysis unit, for example, the generation AI analyzes a user's coding videos and suggests improvements for specific operations or patterns. For example, it suggests efficient methods for operations that users frequently make mistakes in. In addition, the video analysis unit can analyze a user's coding videos and suggest improvements for specific operations or patterns. For example, it suggests efficient methods for operations that users frequently make mistakes in. In addition, the video analysis unit can analyze a user's coding videos and suggest improvements for specific operations or patterns. For example, it suggests efficient methods for operations that users frequently make mistakes in. In this way, by analyzing a user's coding videos and suggesting improvements for specific operations or patterns, it is possible to provide effective learning.

[0047] The video analysis unit can analyze the user's coding style and suggest efficient coding techniques. For example, the video analysis unit allows the generation AI to analyze the user's coding style and suggest efficient coding techniques. For example, if the user is writing redundant code, it shows how to write more concise code. The video analysis unit also allows the generation AI to analyze the user's coding style and suggest efficient coding techniques. For example, if the user is writing redundant code, it shows how to write more concise code. The video analysis unit also allows the generation AI to analyze the user's coding style and suggest efficient coding techniques. For example, if the user is writing redundant code, it shows how to write more concise code. In this way, by analyzing the user's coding style and suggesting efficient coding techniques, effective learning can be provided.

[0048] The video analysis unit can compare the user's coding video with other users and suggest best practices. In the video analysis unit, for example, the generation AI compares the user's coding video with other users and suggests best practices. For example, it shows other users' efficient ways of solving the same problem. In addition, the video analysis unit can compare the user's coding video with other users and suggest best practices. For example, it shows other users' efficient ways of solving the same problem. In addition, the video analysis unit can compare the user's coding video with other users and suggest best practices. For example, it shows other users' efficient ways of solving the same problem. In this way, by comparing the user's coding video with other users and suggesting best practices, effective learning can be provided.

[0049] The video analysis unit can analyze a user's coding videos and suggest the optimal approach for a specific project or task. In the video analysis unit, for example, a generation AI analyzes a user's coding videos and suggests the optimal approach for a specific project or task. For example, it suggests the optimal algorithm for the project the user is working on. In addition, the video analysis unit can analyze a user's coding videos and suggest the optimal approach for a specific project or task. For example, it suggests the optimal algorithm for the project the user is working on. In addition, the video analysis unit can analyze a user's coding videos and suggest the optimal approach for a specific project or task. For example, it suggests the optimal algorithm for the project the user is working on. In this way, by analyzing a user's coding videos and suggesting the optimal approach for a specific project or task, effective learning can be provided.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] The fully automated programming school can analyze a user's learning progress and provide problems that promote skill transfer in a specific programming language. For example, it can provide problems that require users to implement an algorithm learned in Python in Java. It can also recommend solving the same problem in multiple languages ​​to promote skill transfer between different programming languages. Furthermore, it can provide problems that allow users to compare example solutions in different languages ​​to deepen their understanding of a specific language. This allows users to effectively acquire skills in multiple programming languages.

[0052] The fully automated programming school can suggest the most suitable programming language based on the user's learning goals and provide problems specialized for that language. For example, it can suggest Python to a user who wants to learn data science and provide problems specialized for that language. It can also suggest JavaScript to a user aiming to become a web developer and provide problems specialized for that language. It can also suggest C++ to a user interested in game development and provide problems specialized for that language. This allows users to effectively learn the programming language that best suits their learning goals.

[0053] The Fully Automated Programming School can generate questions that include related themes and scenarios based on a user's interests and hobbies. For example, a user interested in game development can be provided with questions based on game scenarios. A user interested in music can also be provided with questions based on music scenarios. Furthermore, a user interested in sports can be provided with questions based on sports scenarios. This allows users to enjoy learning through questions based on their own interests and hobbies.

[0054] The fully automated programming school can integrate data from different learning platforms to generate comprehensive questions. For example, it can collect data from online courses, books, and tutorials and integrate them to generate comprehensive questions. It can also provide questions that promote comprehensive understanding based on knowledge obtained from multiple resources. It can also provide questions that combine the content of online courses and books. This allows users to gain a deeper understanding through comprehensive questions that integrate data from different learning platforms.

[0055] The Fully Automated Programming School can analyze a user's learning style and provide a problem format that suits them. For example, for users who prefer visual learning, it can provide problems that make extensive use of diagrams and graphs. For users who prefer auditory learning, it can also provide problems with audio commentary. Furthermore, it is conceivable that interactive problems can be provided for users who prefer tactile learning. This allows users to study effectively through problem formats that suit their own learning style.

[0056] The processing flow of the first embodiment will be briefly explained below.

[0057] Step 1: The problem generator generates problems according to the user's skill level. For example, the generation AI analyzes the user's skill level and automatically generates programming problems according to that level. The generation AI generates problems using a text generation AI (e.g., LLM). The generation AI can also generate the content of the problems using a multimodal generation AI. The generation AI also generates problems based on information about the user's skill level. Step 2: The answer checking unit analyzes the answers to the problems generated by the problem generation unit. For example, the generation AI analyzes the answers to problems solved by the user and determines whether they are correct. The generation AI analyzes the code submitted by the user and checks the answers. Step 3: The explanation provider provides an explanation based on the answer analyzed by the answer checker. For example, if the answer is correct, the generation AI will explain in detail why, and if it is incorrect, what part is wrong. If there is an error in the code written by the user, the generation AI will specifically explain the cause of the error and how to correct it. Step 4: The weakness identification unit identifies the user's weaknesses. For example, the generation AI analyzes the user's past answer history and coding style to identify the user's weaknesses. The generation AI identifies weaknesses based on the user's answer history and coding style. Step 5: The video analysis unit analyzes the video of the user coding. For example, the generation AI analyzes the video of the user coding and identifies the user's weaknesses. The generation AI analyzes the user's coding video and identifies the user's weaknesses.

[0058] (Example 2) The fully automated programming school according to an embodiment of the present invention is a system that automatically generates problems according to a user's skill level, provides answers and explanations, customizes problems based on the user's weaknesses, supports all programming languages, and analyzes coding processes using video. This allows users to learn effectively at their own pace without the stress of interacting with others and overcome their weaknesses.

[0059] A fully automated programming school according to an embodiment includes a problem generation unit, an answer checking unit, an explanation providing unit, a weakness identification unit, and a video analysis unit. The problem generation unit generates problems according to a user's skill level. For example, the generation AI analyzes the user's skill level and automatically generates programming problems according to that level. The generation AI generates problems using a text generation AI (e.g., LLM). The generation AI can also generate problem content using a multimodal generation AI. The generation AI generates problems based on information related to the user's skill level. The answer checking unit analyzes the answers to problems generated by the problem generation unit. For example, the generation AI analyzes the answers to problems solved by the user and determines whether they are correct. The generation AI analyzes code submitted by the user and checks the answers. The explanation providing unit provides explanations based on the answers analyzed by the answer checking unit. For example, the generation AI provides a detailed explanation of the reason if the answer is correct, and a detailed explanation of the incorrect part if the answer is incorrect. If there is an error in the code written by the user, the generation AI specifically explains the cause of the error and how to correct it. The weakness identification unit identifies the user's weaknesses. For example, the generation AI analyzes the user's past answer history and coding behavior to identify the user's weaknesses. The generation AI identifies weaknesses based on the user's answer history and coding behavior. The video analysis unit analyzes the user's coding behavior using video. For example, the generation AI analyzes videos of the user coding to identify the user's weaknesses. The generation AI analyzes the user's coding videos to identify the user's weaknesses. This allows the fully automated programming school according to the embodiment to allow users to effectively progress through learning at their own pace without interpersonal stress and overcome their weaknesses. For example, the user can correct errors in their code based on the explanations provided by the generation AI and move on to the next problem. Furthermore, the user can intensively study specific skills based on the weaknesses identified by the generation AI. Furthermore, the user can improve their coding style based on the coding videos analyzed by the generation AI.

[0060] The question generation unit can analyze the learning history, compare skill levels, and generate optimal questions. In the question generation unit, for example, the generation AI analyzes the user's past learning history and grasps the progress of the learning content. For example, the current skill level is evaluated based on the topics learned in the past and the accuracy rate of questions answered, and questions are generated accordingly. In addition, the question generation unit compares the user's skill level and generates optimal questions. For example, the generation AI evaluates the user's skill level based on past test results and self-assessment, and generates questions accordingly. In this way, the user's learning history can be analyzed and optimal questions can be generated according to the skill level.

[0061] The question generation unit can analyze a user's learning style and provide a question format that corresponds to that. For example, the question generation unit uses a generation AI to analyze a user's learning style and provide questions that make extensive use of diagrams and graphs to users who prefer visual learning. For example, it generates questions that include flowcharts that visually show the flow of an algorithm. The question generation unit also analyzes a user's learning style and provides questions with audio explanations to users who prefer auditory learning. For example, it provides audio explanations for questions. The question generation unit also analyzes a user's learning style and provides interactive questions to users who prefer tactile learning. For example, it provides questions that require users to actually input and execute code. This makes it possible to provide question formats that correspond to the user's learning style.

[0062] The question generation unit can use the emotion estimation function to analyze the user's emotions during study in real time and generate questions to reduce stress and frustration. For example, the question generation unit uses the emotion estimation function to analyze the user's facial expressions and voice during study to detect stress and frustration. For example, if the user is confused, the question generation unit provides questions with a lower level of difficulty. The question generation unit can also use the emotion estimation function to analyze the user's emotions during study in real time and generate questions to reduce stress and frustration. For example, it can provide easy questions that allow the user to study in a relaxed manner. The question generation unit can also use the emotion estimation function to analyze the user's emotions during study in real time and provide positive feedback. For example, if the user answers correctly, a praising message can be displayed. In this way, the question generation unit can analyze the user's emotions during study and generate questions to reduce stress and frustration.

[0063] The question generation unit can integrate data from different learning platforms to generate comprehensive questions. For example, the question generation unit uses a generation AI to collect data from online courses, books, and tutorials and integrate them to generate comprehensive questions. For example, it provides questions that promote comprehensive understanding based on knowledge obtained from multiple resources. The question generation unit also integrates data from different learning platforms to generate comprehensive questions. For example, it provides questions that combine the content of an online course and the content of a book. The question generation unit also integrates data from different learning platforms to support user learning. For example, it provides questions based on the content of a tutorial. This allows data from different learning platforms to be integrated to generate comprehensive questions.

[0064] The question generation unit can generate questions including related themes and scenarios based on the user's interests and hobbies. For example, the question generation unit uses a generation AI to analyze the user's interests and hobbies and generate questions including related themes and scenarios based on the analysis. For example, a user who is interested in game development is provided with questions based on game scenarios. The question generation unit also analyzes the user's interests and hobbies and generates questions including related themes and scenarios based on the analysis. For example, a user who is interested in music is provided with questions based on music scenarios. The question generation unit also analyzes the user's interests and hobbies and generates questions including related themes and scenarios based on the analysis. For example, a user who is interested in sports is provided with questions based on sports scenarios. In this way, questions including related themes and scenarios can be generated based on the user's interests and hobbies.

[0065] The question generation unit uses the emotion estimation function to generate questions related to the topic in which the user is most interested, thereby increasing motivation to learn. The question generation unit, for example, uses the emotion estimation function to identify the topic in which the user is most interested and generate questions related to the topic. For example, related programming questions are provided based on the topic in which the user has shown interest. The question generation unit also uses the emotion estimation function to identify the topic in which the user is most interested and generate questions related to the topic. For example, related programming questions are provided based on the topic in which the user has shown interest. The question generation unit also uses the emotion estimation function to identify the topic in which the user is most interested and generate questions related to the topic. For example, related programming questions are provided based on the topic in which the user has shown interest. This allows questions related to the topic in which the user is most interested to be generated, thereby increasing motivation to learn.

[0066] The answer checking section can present multiple answer patterns for the answer and explain the advantages and disadvantages of each. In the answer checking section, for example, the generation AI analyzes the user's answer and presents multiple answer patterns. For example, it shows example answers using different algorithms or data structures and explains the advantages and disadvantages of each. In the answer checking section, the generation AI analyzes the user's answer and presents multiple answer patterns. For example, it shows example answers using different algorithms or data structures and explains the advantages and disadvantages of each. In the answer checking section, the generation AI analyzes the user's answer and presents multiple answer patterns. For example, it shows example answers using different algorithms or data structures and explains the advantages and disadvantages of each. In this way, by presenting multiple answer patterns for the user's answer and explaining the advantages and disadvantages of each, the user's understanding can be deepened.

[0067] The answer checking section can analyze the answer history and compare past errors with current answers to evaluate progress. In the answer checking section, for example, the generation AI analyzes the user's answer history and compares past errors with current answers. For example, it evaluates changes in the accuracy and efficiency of answers to the same type of problem. In addition, the answer checking section can analyze the user's answer history and compare past errors with current answers. For example, it evaluates changes in the accuracy and efficiency of answers to the same type of problem. In addition, the answer checking section can analyze the user's answer history and compare past errors with current answers. For example, it evaluates changes in the accuracy and efficiency of answers to the same type of problem. In this way, the user's growth can be visualized by analyzing the user's answer history and comparing past errors with current answers to evaluate progress.

[0068] The answer checking unit uses the emotion estimation function to analyze the emotion the user feels when they make a mistake and provides positive feedback, thereby maintaining their motivation to learn. The answer checking unit, for example, uses the emotion estimation function to analyze the emotion the user feels when they make a mistake and provides positive feedback. For example, if the user is disappointed, an encouraging message is displayed. The answer checking unit also uses the emotion estimation function to analyze the emotion the user feels when they make a mistake and provides positive feedback. For example, if the user is disappointed, an encouraging message is displayed. The answer checking unit also uses the emotion estimation function to analyze the emotion the user feels when they make a mistake and provides positive feedback. For example, if the user is disappointed, an encouraging message is displayed. In this way, the user's emotion when they make a mistake can be analyzed and positive feedback can be provided, thereby maintaining their motivation to learn.

[0069] The answer checking section can provide example answers in different programming languages, allowing the user to understand from multiple perspectives. In the answer checking section, for example, the generation AI analyzes the user's answer and provides example answers in different programming languages. For example, it converts an answer written in Python into an answer in Java or C++ and explains the differences between them. In addition, the answer checking section can analyze the user's answer and provide example answers in different programming languages. For example, it converts an answer written in Python into an answer in Java or C++ and explains the differences between them. In addition, the answer checking section can analyze the user's answer and provide example answers in different programming languages. For example, it converts an answer written in Python into an answer in Java or C++ and explains the differences between them. In this way, by providing example answers in different programming languages, the user can understand from multiple perspectives.

[0070] The answer checking section can automatically suggest related documents and resources for the answer, promoting a deeper understanding. In the answer checking section, for example, the generation AI analyzes the user's answer and automatically suggests related documents and resources. For example, detailed explanations and tutorials about specific algorithms are provided. In addition, the answer checking section can analyze the user's answer and automatically suggest related documents and resources. For example, detailed explanations and tutorials about specific algorithms are provided. In addition, the answer checking section can analyze the user's answer and automatically suggest related documents and resources. For example, detailed explanations and tutorials about specific algorithms are provided. In this way, automatic suggestions of related documents and resources for the user's answer can promote a deeper understanding.

[0071] The answer checking unit can use the emotion estimation function to analyze the emotion the user feels when they get the answer right, and provide additional questions to enhance the sense of accomplishment. The answer checking unit, for example, uses the emotion estimation function to analyze the emotion the user feels when they get the answer right, and provides additional questions to enhance the sense of accomplishment. For example, if the user feels joy, it provides challenging questions to help them move on to the next step. The answer checking unit also uses the emotion estimation function to analyze the emotion the user feels when they get the answer right, and provides additional questions to enhance the sense of accomplishment. For example, if the user feels joy, it provides challenging questions to help them move on to the next step. The answer checking unit also uses the emotion estimation function to analyze the emotion the user feels when they get the answer right, and provides additional questions to enhance the sense of accomplishment. For example, if the user feels joy, it provides challenging questions to help them move on to the next step. In this way, by analyzing the emotion the user feels when they get the answer right and providing additional questions to enhance the sense of accomplishment, it is possible to increase motivation to learn.

[0072] The weakness identification unit can analyze the user's weaknesses in detail and generate questions that focus on a specific skill set. In the weakness identification unit, for example, the generation AI analyzes the user's past answer history and identifies weaknesses in a specific skill set. For example, a user who is not good at loop structures is provided with many questions related to loops. In addition, the weakness identification unit analyzes the user's past answer history and identifies weaknesses in a specific skill set. For example, a user who is not good at loop structures is provided with many questions related to loops. In addition, the weakness identification unit analyzes the user's past answer history and identifies weaknesses in a specific skill set. For example, a user who is not good at loop structures is provided with many questions related to loops. In this way, the user's weaknesses are analyzed in detail and questions that focus on a specific skill set are generated, thereby providing effective learning.

[0073] The weakness identification unit can propose improvements to the user's weaknesses and provide problems based on those improvements. For example, the generation AI analyzes the user's weaknesses and proposes improvements. For example, if the user lacks understanding of a specific algorithm, it provides problems that will teach that algorithm from the basics to its applications. The weakness identification unit can also analyze the user's weaknesses and propose improvements. For example, if the user lacks understanding of a specific algorithm, it provides problems that will teach that algorithm from the basics to its applications. The weakness identification unit can also analyze the user's weaknesses and propose improvements. For example, if the user lacks understanding of a specific algorithm, it provides problems that will teach that algorithm from the basics to its applications. In this way, by proposing improvements to the user's weaknesses and providing problems based on those improvements, effective learning can be provided.

[0074] The weakness identification unit can use the emotion estimation function to generate questions to reduce the anxiety and stress the user feels about their weaknesses. The weakness identification unit, for example, uses the emotion estimation function to analyze the anxiety and stress the user feels about their weaknesses and generate questions to reduce it. For example, it provides easy questions that allow the user to study in a relaxed manner. The weakness identification unit also uses the emotion estimation function to analyze the anxiety and stress the user feels about their weaknesses and generate questions to reduce it. For example, it provides easy questions that allow the user to study in a relaxed manner. The weakness identification unit also uses the emotion estimation function to analyze the anxiety and stress the user feels about their weaknesses and generate questions to reduce it. For example, it provides easy questions that allow the user to study in a relaxed manner. In this way, by generating questions to reduce the anxiety and stress the user feels about their weaknesses, it is possible to provide effective learning.

[0075] The weakness identification unit can increase motivation by presenting success stories of other users related to the user's weaknesses. In the weakness identification unit, for example, the generation AI collects success stories of other users related to the user's weaknesses and presents them. For example, it introduces the experiences and success stories of users who have overcome the same weaknesses. In addition, the weakness identification unit can increase motivation by presenting success stories of other users related to the user's weaknesses. For example, it introduces the experiences and success stories of users who have overcome the same weaknesses. In addition, the weakness identification unit can increase motivation by presenting success stories of other users related to the user's weaknesses.

[0076] The weakness identification unit can suggest group learning sessions that address the user's weaknesses and provide opportunities for the user to study cooperatively with other users. In the weakness identification unit, for example, the generation AI identifies the user's weaknesses and suggests a group learning session based on the weaknesses. For example, a session is provided in which users with the same weaknesses can study cooperatively. In addition, the weakness identification unit can suggest group learning sessions that address the user's weaknesses and suggest a group learning session based on the weaknesses. For example, a session is provided in which users with the same weaknesses can study cooperatively. In addition, the weakness identification unit can suggest group learning sessions that address the user's weaknesses and suggest a group learning session based on the weaknesses. For example, a session is provided in which users with the same weaknesses can study cooperatively. In this way, effective learning can be provided by suggesting group learning sessions that address the user's weaknesses and providing opportunities for learning cooperatively with other users.

[0077] The weakness identification unit can use the emotion estimation function to analyze the user's emotions regarding their weaknesses and provide problems that will elicit positive emotions. The weakness identification unit, for example, uses the emotion estimation function to analyze the user's emotions regarding their weaknesses and provide problems that will elicit positive emotions. For example, easy problems that the user can tackle with confidence are provided. The weakness identification unit can also use the emotion estimation function to analyze the user's emotions regarding their weaknesses and provide problems that will elicit positive emotions. For example, easy problems that the user can tackle with confidence are provided. The weakness identification unit can also use the emotion estimation function to analyze the user's emotions regarding their weaknesses and provide problems that will elicit positive emotions. For example, easy problems that the user can tackle with confidence are provided. In this way, by analyzing the user's emotions regarding their weaknesses and providing problems that will elicit positive emotions, effective learning can be provided.

[0078] The question generation unit can analyze the characteristics of each programming language and provide the optimal learning path for each language. In the question generation unit, for example, a generation AI analyzes the characteristics of each programming language and provides the optimal learning path for each language. For example, a learning path that starts with basic grammar is provided for beginners of Python. In addition, the question generation unit can analyze the characteristics of each programming language and provide the optimal learning path for each language. For example, a learning path that starts with basic grammar is provided for beginners of Python. In addition, the question generation unit can analyze the characteristics of each programming language and provide the optimal learning path for each language. For example, a learning path that starts with basic grammar is provided for beginners of Python. In this way, by analyzing the characteristics of each programming language and providing the optimal learning path for each language, effective learning can be provided.

[0079] The question generation unit can analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, the question generation unit uses a generation AI to analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, a question is provided that asks the user to implement an algorithm learned in Python in Java. The question generation unit also uses a generation AI to analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, a question is provided that asks the user to implement an algorithm learned in Python in Java. The question generation unit also uses a generation AI to analyze the user's learning progress and provide questions that promote skill transfer between multiple programming languages. For example, a question is provided that asks the user to implement an algorithm learned in Python in Java. In this way, the user's learning progress can be analyzed and questions that promote skill transfer between multiple programming languages ​​can be provided, thereby providing effective learning.

[0080] The question generation unit can use the emotion estimation function to provide questions that increase the user's interest and curiosity in a specific programming language. The question generation unit, for example, uses the emotion estimation function to provide questions that increase the user's interest and curiosity in a specific programming language. For example, questions related to a topic in which the user is interested are provided. The question generation unit also uses the emotion estimation function to provide questions that increase the user's interest and curiosity in a specific programming language. For example, questions related to a topic in which the user is interested are provided. The question generation unit also uses the emotion estimation function to provide questions that increase the user's interest and curiosity in a specific programming language. For example, questions related to a topic in which the user is interested are provided. In this way, by providing questions that increase the user's interest and curiosity in a specific programming language, effective learning can be provided.

[0081] The question generation unit can integrate feedback from different programming language communities and provide it to the user. For example, the generation AI in the question generation unit collects feedback from different programming language communities, integrates it, and provides it to the user. For example, it provides best practices from the Python community and the Java community. The question generation unit also collects feedback from different programming language communities, integrates it, and provides it to the user. For example, it provides best practices from the Python community and the Java community. The question generation unit also collects feedback from different programming language communities, integrates it, and provides it to the user. For example, it provides best practices from the Python community and the Java community. In this way, feedback from different programming language communities can be integrated and provided to the user, thereby providing effective learning.

[0082] The question generation unit can suggest the most appropriate programming language according to the user's learning goal and provide questions specialized for that language. In the question generation unit, for example, a generation AI analyzes the user's learning goal and suggests the most appropriate programming language accordingly. For example, it suggests Python for a user who wants to learn data science and provides questions specialized for that language. In addition, the question generation unit can analyze the user's learning goal and suggest the most appropriate programming language according to that goal. For example, it suggests Python for a user who wants to learn data science and provides questions specialized for that language. In addition, the question generation unit can analyze the user's learning goal and suggest the most appropriate programming language according to that goal. For example, it suggests Python for a user who wants to learn data science and provides questions specialized for that language. In this way, by suggesting the most appropriate programming language according to the user's learning goal and providing questions specialized for that language, effective learning can be provided.

[0083] The question generation unit can use the emotion estimation function to provide questions to reduce anxiety when the user learns a new programming language. The question generation unit, for example, uses the emotion estimation function to analyze the anxiety the user feels when learning a new programming language and provide questions to reduce that anxiety. For example, it provides easy questions that allow the user to study in a relaxed manner. The question generation unit can also use the emotion estimation function to analyze the anxiety the user feels when learning a new programming language and provide questions to reduce that anxiety. For example, it provides easy questions that allow the user to study in a relaxed manner. The question generation unit can also use the emotion estimation function to analyze the anxiety the user feels when learning a new programming language and provide questions to reduce that anxiety. For example, it provides easy questions that allow the user to study in a relaxed manner. In this way, by providing questions to reduce anxiety when the user learns a new programming language, effective learning can be provided.

[0084] The video analysis unit can analyze a user's coding videos and suggest improvements for specific operations or patterns. In the video analysis unit, for example, the generation AI analyzes a user's coding videos and suggests improvements for specific operations or patterns. For example, it suggests efficient methods for operations that users frequently make mistakes in. In addition, the video analysis unit can analyze a user's coding videos and suggest improvements for specific operations or patterns. For example, it suggests efficient methods for operations that users frequently make mistakes in. In addition, the video analysis unit can analyze a user's coding videos and suggest improvements for specific operations or patterns. For example, it suggests efficient methods for operations that users frequently make mistakes in. In this way, by analyzing a user's coding videos and suggesting improvements for specific operations or patterns, it is possible to provide effective learning.

[0085] The video analysis unit can analyze the user's coding style and suggest efficient coding techniques. For example, the video analysis unit allows the generation AI to analyze the user's coding style and suggest efficient coding techniques. For example, if the user is writing redundant code, it shows how to write more concise code. The video analysis unit also allows the generation AI to analyze the user's coding style and suggest efficient coding techniques. For example, if the user is writing redundant code, it shows how to write more concise code. The video analysis unit also allows the generation AI to analyze the user's coding style and suggest efficient coding techniques. For example, if the user is writing redundant code, it shows how to write more concise code. In this way, by analyzing the user's coding style and suggesting efficient coding techniques, effective learning can be provided.

[0086] The video analysis unit can use the emotion estimation function to provide advice to reduce stress and frustration felt by the user while coding. The video analysis unit, for example, uses the emotion estimation function to analyze the stress and frustration felt by the user while coding and provide advice to reduce it. For example, it suggests taking a break to help the user relax. The video analysis unit also uses the emotion estimation function to analyze the stress and frustration felt by the user while coding and provide advice to reduce it. For example, it suggests taking a break to help the user relax. The video analysis unit also uses the emotion estimation function to analyze the stress and frustration felt by the user while coding and provide advice to reduce it. For example, it suggests taking a break to help the user relax. In this way, by providing advice to reduce the stress and frustration felt by the user while coding, effective learning can be provided.

[0087] The video analysis unit can compare the user's coding video with other users and suggest best practices. In the video analysis unit, for example, the generation AI compares the user's coding video with other users and suggests best practices. For example, it shows other users' efficient ways of solving the same problem. In addition, the video analysis unit can compare the user's coding video with other users and suggest best practices. For example, it shows other users' efficient ways of solving the same problem. In addition, the video analysis unit can compare the user's coding video with other users and suggest best practices. For example, it shows other users' efficient ways of solving the same problem. In this way, by comparing the user's coding video with other users and suggesting best practices, effective learning can be provided.

[0088] The video analysis unit can analyze a user's coding videos and suggest the optimal approach for a specific project or task. In the video analysis unit, for example, a generation AI analyzes a user's coding videos and suggests the optimal approach for a specific project or task. For example, it suggests the optimal algorithm for the project the user is working on. In addition, the video analysis unit can analyze a user's coding videos and suggest the optimal approach for a specific project or task. For example, it suggests the optimal algorithm for the project the user is working on. In addition, the video analysis unit can analyze a user's coding videos and suggest the optimal approach for a specific project or task. For example, it suggests the optimal algorithm for the project the user is working on. In this way, by analyzing a user's coding videos and suggesting the optimal approach for a specific project or task, effective learning can be provided.

[0089] The video analysis unit can use the emotion estimation function to provide feedback to reinforce the positive emotions felt by the user while coding. For example, the video analysis unit uses the emotion estimation function to analyze the positive emotions felt by the user while coding and provide feedback to reinforce them. For example, a message of praise is displayed when the user feels a sense of accomplishment. The video analysis unit also uses the emotion estimation function to analyze the positive emotions felt by the user while coding and provide feedback to reinforce them. For example, a message of praise is displayed when the user feels a sense of accomplishment. The video analysis unit also uses the emotion estimation function to analyze the positive emotions felt by the user while coding and provide feedback to reinforce them. For example, a message of praise is displayed when the user feels a sense of accomplishment. In this way, by providing feedback to reinforce the positive emotions felt by the user while coding, effective learning can be provided.

[0090] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0091] The fully automated programming school can analyze a user's learning progress and provide problems that promote skill transfer in a specific programming language. For example, it can provide problems that require users to implement an algorithm learned in Python in Java. It can also recommend solving the same problem in multiple languages ​​to promote skill transfer between different programming languages. Furthermore, it can provide problems that allow users to compare example solutions in different languages ​​to deepen their understanding of a specific language. This allows users to effectively acquire skills in multiple programming languages.

[0092] The fully automated programming school can suggest the most suitable programming language based on the user's learning goals and provide problems specialized for that language. For example, it can suggest Python to a user who wants to learn data science and provide problems specialized for that language. It can also suggest JavaScript to a user aiming to become a web developer and provide problems specialized for that language. It can also suggest C++ to a user interested in game development and provide problems specialized for that language. This allows users to effectively learn the programming language that best suits their learning goals.

[0093] The Fully Automated Programming School can generate questions that include related themes and scenarios based on a user's interests and hobbies. For example, a user interested in game development can be provided with questions based on game scenarios. A user interested in music can also be provided with questions based on music scenarios. Furthermore, a user interested in sports can be provided with questions based on sports scenarios. This allows users to enjoy learning through questions based on their own interests and hobbies.

[0094] The fully automated programming school can integrate data from different learning platforms to generate comprehensive questions. For example, it can collect data from online courses, books, and tutorials and integrate them to generate comprehensive questions. It can also provide questions that promote comprehensive understanding based on knowledge obtained from multiple resources. It can also provide questions that combine the content of online courses and books. This allows users to gain a deeper understanding through comprehensive questions that integrate data from different learning platforms.

[0095] The Fully Automated Programming School can analyze a user's learning style and provide a problem format that suits them. For example, for users who prefer visual learning, it can provide problems that make extensive use of diagrams and graphs. For users who prefer auditory learning, it can also provide problems with audio commentary. Furthermore, it is conceivable that interactive problems can be provided for users who prefer tactile learning. This allows users to study effectively through problem formats that suit their own learning style.

[0096] The Fully Automated Programming School uses its emotion estimation function to analyze the user's emotions in real time while studying, and can generate problems designed to reduce stress and frustration. For example, if the user is confused, it can provide them with questions of lower difficulty. It can also provide them with simple questions that allow them to study in a relaxed atmosphere. It can also display a message of praise if the user answers correctly. This allows the user to study effectively through problems designed to reduce stress and frustration based on the results of the analysis of their emotions while studying.

[0097] The Fully Automated Programming School uses emotion estimation to generate questions related to the topics that interest the user most, thereby increasing motivation to learn. For example, it provides related programming questions based on the topics that the user has shown interest in. Providing questions related to topics that interest the user can also increase motivation to learn. It is also possible to provide related programming questions based on the topics that the user has shown interest in. This allows the user to increase motivation to learn through questions related to the topics that interest them most.

[0098] The Fully Automated Programming School uses its emotion estimation function to analyze the emotions felt when a user makes a mistake and provides positive feedback to maintain motivation to learn. For example, if the user is feeling discouraged, an encouraging message can be displayed. The Fully Automated Programming School can also analyze the emotions felt when a user makes a mistake and provide positive feedback to maintain motivation to learn. For example, if the user is feeling discouraged, an encouraging message can be displayed. This allows the user to maintain motivation to learn through positive feedback based on the results of analyzing the emotions felt when a user makes a mistake.

[0099] The Fully Automated Programming School can use its emotion estimation function to analyze the emotions felt by the user when they get a correct answer and provide additional questions to enhance their sense of accomplishment. For example, if the user feels joy, it can provide challenging questions to help them move on to the next step. Furthermore, by analyzing the emotions felt by the user when they get a correct answer and providing additional questions to enhance their sense of accomplishment, it can also increase their motivation to learn. Furthermore, if the user feels joy, it can also be considered to provide challenging questions to help them move on to the next step. This allows the user to increase their motivation to learn through additional questions to enhance their sense of accomplishment based on the results of analyzing their emotions when they get a correct answer.

[0100] The fully automated programming school can use its emotion estimation function to provide problems that reduce the user's anxiety when learning a new programming language. For example, it can provide easy problems that allow the user to study in a relaxed manner. It can also analyze the user's anxiety when learning a new programming language and provide problems that reduce that anxiety, thereby increasing motivation to learn. It is also possible to provide easy problems that allow the user to study in a relaxed manner. This allows the user to effectively progress in their studies through problems that reduce anxiety based on the results of an analysis of the user's anxiety when learning a new programming language.

[0101] The processing flow of the second embodiment will be briefly explained below.

[0102] Step 1: The problem generator generates problems according to the user's skill level. For example, the generation AI analyzes the user's skill level and automatically generates programming problems according to that level. The generation AI generates problems using a text generation AI (e.g., LLM). The generation AI can also generate the content of the problems using a multimodal generation AI. The generation AI also generates problems based on information about the user's skill level. Step 2: The answer checking unit analyzes the answers to the problems generated by the problem generation unit. For example, the generation AI analyzes the answers to problems solved by the user and determines whether they are correct. The generation AI analyzes the code submitted by the user and checks the answers. Step 3: The explanation provider provides an explanation based on the answer analyzed by the answer checker. For example, if the answer is correct, the generation AI will explain in detail why, and if it is incorrect, what part is wrong. If there is an error in the code written by the user, the generation AI will specifically explain the cause of the error and how to correct it. Step 4: The weakness identification unit identifies the user's weaknesses. For example, the generation AI analyzes the user's past answer history and coding style to identify the user's weaknesses. The generation AI identifies weaknesses based on the user's answer history and coding style. Step 5: The video analysis unit analyzes the video of the user coding. For example, the generation AI analyzes the video of the user coding and identifies the user's weaknesses. The generation AI analyzes the user's coding video and identifies the user's weaknesses.

[0103] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0104] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0105] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0106] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0107] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0108] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0109] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0110] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0111] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0112] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0113] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0114] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0115] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0116] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0117] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0118] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0119] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0120] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0121] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0122] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0123] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0124] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0125] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0126] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0127] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0128] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0129] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0130] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0131] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 may also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0132] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0133] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0134] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0135] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0136] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0137] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0138] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0139] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0140] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0141] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0142] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0143] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0144] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0145] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0146] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0147] In the robot 414, the processor 46 performs the identification process. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0148] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0149] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0150] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0151] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0152] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0153] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0154] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0155] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0156] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0157] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0158] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0159] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0160] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0161] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0162] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0163] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0164] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0165] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0166] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0167] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0168] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0169] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0170] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a question generation unit that generates questions according to the skill level of a user; an answer checking unit that analyzes answers to the questions generated by the question generating unit; an explanation providing unit that provides an explanation based on the answer analyzed by the answer checking unit; a weakness identification unit that identifies a weakness of a user; A video analysis unit that analyzes the user's coding status using video. A system characterized by:

2. The question generator Analyzing the learning history, comparing the skill levels, and generating the most suitable questions 2. The system of claim 1.

3. The answer checking unit Present multiple answer patterns for the answer and explain the advantages and disadvantages of each.

2. The system of claim 1.

4. The weak point identification unit Analyzing the user's weaknesses in detail and generating questions that target specific skill sets 2. The system of claim 1.

5. The video analysis unit Analyze the user's coding video and suggest improvements for specific operations or patterns 2. The system of claim 1.

6. The question generator Analyzing the user's emotions during learning in real time and generating the questions to reduce stress and frustration 2. The system of claim 1.

Images