System

Abstract

A system is provided.SOLUTION: A system comprising: means for automatically generating a programming question based on a skill level of a user; means for determining whether an answer submitted by the user is correct or incorrect; means for analyzing a weakness of the user from a behavior log or a coded video of the user; and means for providing a customized programming question based on an analysis result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional programming learning platforms are often limited to specific programming languages ​​and lack the flexibility to handle a wide variety of languages ​​and problems. Furthermore, they lack sufficient explanations and the ability to accurately assess each learner's skill level and weaknesses and provide effective feedback, resulting in poor learning efficiency. For this reason, there is a need for a system that allows learners to effectively progress through learning at their own pace and overcome their weaknesses. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for automatically generating programming problems based on a user's skill level and determining whether the answers submitted by the user are correct, a means for analyzing the user's weaknesses from the user's behavior log or coding videos, and a means for providing customized programming problems based on the analysis results. Furthermore, by including a means for automatically generating explanations for the answers submitted by the user, users can study more effectively. Furthermore, by analyzing users' weaknesses based on an analysis of coding videos, it is possible to identify specific weaknesses of individual users and provide appropriate feedback.

[0006] "User skill level" refers to the degree of knowledge and technical proficiency of a programming learner, and is a concept with stages such as beginner, intermediate, and advanced.

[0007] "Means for automatically generating programming problems" is a function that mechanically creates appropriate programming tasks according to the user's skill level.

[0008] The "means for determining whether the submitted answer is correct" is a function for determining whether the program code submitted by the user as an answer is correct or incorrect for the specified question.

[0009] An "action log" is data that records historical information about operations and activities performed by a user when using a system.

[0010] "Coding video" is video data that records the process of a user creating a program.

[0011] The means for analyzing "user weaknesses" is a function that analyzes user behavior logs and coding videos to identify specific areas or skills in programming that the user is weak at.

[0012] The "means for providing customized programming problems" is a function for creating and presenting to the user programming problems tailored to address the user's weaknesses.

[0013] "Means for automatically generating explanations" is a function that generates sentences and information that explain the content of the answer based on the answer submitted by the user and help understanding. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a system that automatically generates programming problems according to the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze the user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows.

[0036] User registration and login management

[0037] user:

[0038] Users create an account and log in to the system. When creating an account, users register by entering a username and password. When logging in, users enter their authentication information to access the system.

[0039] server:

[0040] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[0041] Determining a user's skill level

[0042] user:

[0043] Users submit sample code to the system for skill evaluation.

[0044] server:

[0045] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[0046] Automatic generation of programming problems

[0047] server:

[0048] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[0049] Automatic answer checking and explanation generation

[0050] user:

[0051] The user submits the answer code to the system.

[0052] server:

[0053] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[0054] User weakness analysis and customization

[0055] server:

[0056] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[0057] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[0058] Coding video analysis

[0059] user:

[0060] Users record themselves coding and upload videos to the platform.

[0061] server:

[0062] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[0063] Specific examples

[0064] Example 1: A beginner user registers and solves a problem

[0065] 1. User Registration

[0066] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[0067] 2. Log in

[0068] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[0069] 3. Skill Judgment

[0070] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[0071] 4. Providing questions

[0072] The server generates "beginner" questions based on skill level and presents them to the user.

[0073] 5. Submitting and checking your answers

[0074] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[0075] 6. Weakness analysis and customization problem provision

[0076] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[0077] This system provides an environment where beginners can effectively learn programming at their own pace. It dramatically improves learning efficiency by providing questions and explanations optimized based on the user's skill level and weaknesses.

[0078] The processing flow will be explained below.

[0079] User registration and login management

[0080] User Registration

[0081] Step 1:

[0082] The user enters a username and password.

[0083] Step 2:

[0084] The terminal sends the entered username and password to the server.

[0085] Step 3:

[0086] The server extracts the username and password from the request data.

[0087] Step 4:

[0088] The server hashes the password using an algorithm such as SHA-256.

[0089] Step 5:

[0090] The server stores the username and hashed password in a database.

[0091] Step 6:

[0092] The server returns a registration success message to the terminal.

[0093] Log in

[0094] Step 1:

[0095] The user enters the registered username and password.

[0096] Step 2:

[0097] The terminal sends the entered username and password to the server.

[0098] Step 3:

[0099] The server extracts the username and password from the request data.

[0100] Step 4:

[0101] The server retrieves the corresponding hashed password from its database.

[0102] Step 5:

[0103] The server hashes the entered password and verifies that it matches the stored hashed password.

[0104] Step 6:

[0105] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[0106] Determining a user's skill level

[0107] Step 1:

[0108] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[0109] Step 2:

[0110] The device sends the sample code to the server.

[0111] Step 3:

[0112] The server receives the submitted code.

[0113] Step 4:

[0114] The server runs the code in a secure environment.

[0115] Step 5:

[0116] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[0117] Step 6:

[0118] The server returns the skill level evaluation result to the terminal.

[0119] Automatic generation of programming problems

[0120] Step 1:

[0121] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[0122] Step 2:

[0123] The server generates programming problems corresponding to skill levels.

[0124] Step 3:

[0125] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[0126] Step 4:

[0127] The server transmits the generated programming questions to the terminal.

[0128] Automatic answer checking and explanation generation

[0129] Step 1:

[0130] The user enters the answer code into the terminal and clicks the submit button.

[0131] Step 2:

[0132] The terminal sends the answer code to the server.

[0133] Step 3:

[0134] The server receives the submitted answer code.

[0135] Step 4:

[0136] The server compares the answer to predefined correct answers.

[0137] Step 5:

[0138] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[0139] Step 6:

[0140] The server sends the generated message and explanation to the terminal.

[0141] User weakness analysis and customization

[0142] Step 1:

[0143] The server analyzes the user's past answer data and behavioral logs.

[0144] Step 2:

[0145] The server identifies user weaknesses (e.g., loops, conditional branches) based on data analysis.

[0146] Step 3:

[0147] The server invokes logic to generate customized questions according to the identified weaknesses.

[0148] Step 4:

[0149] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[0150] Step 5:

[0151] The server transmits the generated customization question to the terminal.

[0152] Coding video analysis

[0153] Step 1:

[0154] Users upload videos of themselves coding to their devices.

[0155] Step 2:

[0156] The device sends the uploaded video to the server.

[0157] Step 3:

[0158] The server analyzes the received video using a machine learning model.

[0159] Step 4:

[0160] The server identifies behaviors and choke points during coding.

[0161] Step 5:

[0162] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[0163] Step 6:

[0164] The server transmits the generated feedback to the terminal.

[0165] As a result, this fully automatic programming learning system helps users to learn effectively at their own pace.

[0166] Example 1

[0167] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0168] Conventional programming problem-solving systems have difficulty automatically generating appropriate problems based on the user's skill level and weaknesses, resulting in reduced learning efficiency. Additionally, there are generally delays in determining whether an answer is correct and in providing feedback, preventing users from checking their learning progress in real time. Furthermore, while analyzing coding videos, rather than relying solely on traditional behavioral log analysis, would enable more detailed and accurate feedback to be provided as a means of analyzing users' weaknesses, few such systems exist.

[0169] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0170] In this invention, the server includes a means for receiving user authentication information and hashing the password to ensure security, a means for automatically generating programming problems based on the user's skill level, and a means for determining whether the answers submitted by the user are correct, thereby enabling the provision of appropriate programming problems according to the user's skill level and fast and accurate feedback.

[0171] "User Credentials" means the username and password or other authentication credentials used to access a system.

[0172] "Password hashing" is the process of encrypting a user-entered password using a specific algorithm to ensure system security.

[0173] "Skill level" is an indicator of a user's programming skill and knowledge. For example, there is a distinction between "beginner" and "intermediate."

[0174] "Automatic programming problem generation" is the process by which the system automatically creates appropriate programming challenges based on the user's skill level.

[0175] The "means for determining whether the answer submitted by the user is correct" is a means for checking whether the program code submitted by the user is executed according to the specifications and determining whether the answer is appropriate.

[0176] An "action log" is a record of the operations and input a user performs on the system.

[0177] "Coding videos" are video data that record the process of a user programming.

[0178] "Weakness analysis" is the process of analyzing user behavior logs and coding videos to identify areas where users are particularly weak or frequently make errors.

[0179] "Customized programming problems" are personalized programming challenges generated based on a user's identified weaknesses.

[0180] This system automatically generates programming problems based on a user's skill level and judges the accuracy of the user's submitted solutions. Furthermore, it can analyze a user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows:

[0181] User registration and login management

[0182] A user creates an account and enters a username and password to log in to the system. The server receives the user's registration information and hashes and stores the password to ensure security. Also, when logging in, the entered password is compared with the hashed password for authentication. This maintains a high level of security.

[0183] Determining a user's skill level

[0184] Users submit sample code for skill assessment. The server runs the submitted code in a sandbox environment and determines the skill level based on whether an error occurs. For example, if no errors occur, the user is assessed as "intermediate," and if an error occurs, the user is assessed as "beginner."

[0185] Automatic generation of programming problems

[0186] The server automatically generates appropriate programming problems using a generative AI model based on the user's skill level. For example, it generates a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners. The generated problems are then presented to the user.

[0187] Automatic answer checking and explanation generation

[0188] The user submits the answer code. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct." If the answer is incorrect, it responds with a specific explanation. This allows the user to receive instant feedback.

[0189] User weakness analysis and customization

[0190] The server analyzes the user's past answer data and behavioral logs to identify the user's weaknesses. For example, if there are many incorrect answers regarding loops or conditional branching, it identifies these as "weaknesses." Based on the analysis results, the server provides customized programming problems to help the user overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates the corresponding programming problem.

[0191] Coding video analysis

[0192] Users upload videos of themselves coding to the platform. The server then uses machine learning models to analyze the uploaded videos and identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the platform identifies the user's weaknesses and provides appropriate feedback.

[0193] Specific examples

[0194] 1. Example of a beginner user registering and solving a problem

[0195] Create an account: The user creates an account by entering a username and password. The server hashes the password using the bcrypt library and stores it in the database along with the user information.

[0196] Login: The user enters a username and password. The server retrieves the hashed password for that username from the database and compares it with the entered password. If authentication is successful, the user is allowed to access the system.

[0197] Skill assessment: Users submit sample code. The server runs the submitted code in a Docker container and checks the error log to determine the skill level. For example, if sample code such as "print(2 + 2)" can be executed without error, the user is assessed as intermediate.

[0198] Problem provision: The server requests a problem for beginners from the AI ​​model and receives a program to add two numbers. The server then presents the generated problem to the user.

[0199] Answer submission and verification: The user submits an answer. The server compares the answer with predefined correct answers in a Docker container. If the answer is correct, the server returns a "correct" message. If the answer is incorrect, the server returns feedback including the exact error location.

[0200] Weakness analysis and provision of customized problems: The server analyzes the user's answer history and behavior log to identify loops as particularly difficult. It then asks the generative AI model to generate customized problems related to loops and provides them to the user. For example, it generates a problem such as "Create a program to calculate the sum of numbers from 1 to 10."

[0201] Prompt Sentence Examples

[0202] Prompt: Please generate a sample problem for a simple Python program for beginners. Also, please provide an explanation for this problem.

[0203] Example response of the generative AI model:

[0204] Question: Write a program that takes two integers as input and calculates and displays their sum.

[0205] Explanation: In this problem, we will use the input() function to read two integers from the user, add them, and display the result using the print() function. See the code below for an example.

[0206] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0207] Step 1: User Registration

[0208] User: A user creates an account by entering a username and password.

[0209] Input: Username and Password

[0210] Server: The server processes the received user information and hashes the password, specifically encrypting it using the bcrypt library.

[0211] Data manipulation: password hashing

[0212] Output: User information including hashed password

[0213] Server: The server stores this information in a database, specifically by inserting the hashed password and username into the database.

[0214] Step 2: Login authentication

[0215] User: The user enters their username and password into a login form.

[0216] Input: Username and Password

[0217] Server: The server retrieves the hashed password for the given username from its database.

[0218] Data manipulation: Obtaining a hashed password corresponding to a username

[0219] Output: Hashed password corresponding to username

[0220] Server: The server compares the entered password with the hashed password retrieved from the database.

[0221] Data calculation: Password verification

[0222] Output: Authentication result

[0223] Server: If authentication is successful, it generates session information and responds to the user. This includes generating a session ID and storing it with the corresponding user information.

[0224] Step 3: Submit your skill evaluation code

[0225] User: The user writes a sample code for skill evaluation and presses the submit button.

[0226] Input: Sample code

[0227] Server: The server receives the submitted code and executes it in a sandbox environment (Docker container).

[0228] Data processing: Running the sample code

[0229] Output: Execution results and error log

[0230] Server: Based on the execution results, if no errors occur, the user is judged as "Intermediate", and if an error occurs, the user is judged as "Beginner". This operation analyzes the error log and judges the skill level.

[0231] Step 4: Problem generation

[0232] Server: The server sends appropriate prompts to the generative AI model based on the user's skill level.

[0233] Input: Skill level and prompt

[0234] Generative AI model: A generative AI model generates programming problems based on prompts.

[0235] Data processing: Generating programming problems

[0236] Output: The generated programming problem

[0237] Server: Presents the generated questions to the user. This includes displaying the generated questions in a user interface.

[0238] Step 5: Submit your answer

[0239] User: The user writes the answer code for the displayed programming problem and presses the submit button.

[0240] Input: Answer code

[0241] Server: The server receives the submitted code and compares it with pre-stored correct answers.

[0242] Data processing: Analysis and comparison of submitted solution codes

[0243] Output: Correct / incorrect result and feedback

[0244] Server: If the answer is correct, it is deemed "correct" and a feedback message is generated and returned. If the answer is incorrect, feedback is provided, including specific error messages and suggestions for improvement. This action includes generating the feedback and responding to the user.

[0245] Step 6: Weakness analysis

[0246] Server: The server analyzes past answer data and behavioral logs to identify typical user error patterns and coding pitfalls.

[0247] Input: Past answer data and behavior log

[0248] Data processing: Error pattern analysis and statistical analysis

[0249] Output: Weakness identification results

[0250] Server: Based on the analysis results, the server requests the generative AI model to create customized questions that address the user's identified weaknesses. This action involves sending prompts to the generative AI model.

[0251] Step 7: Generate customization questions

[0252] Generative AI model: Based on the results of weakness analysis, the generative AI model generates personalized programming challenges for users to overcome.

[0253] Input: Weakness analysis results and prompt text

[0254] Data manipulation: custom problem generation

[0255] Output: Customized programming problems

[0256] Server: Presents the generated customization questions to the user. This includes displaying the customization questions in the user interface.

[0257] Step 8: Upload and analyze your video

[0258] Users: Users record themselves coding and upload videos to the platform.

[0259] Input: Coding video

[0260] Server: The server receives the uploaded video and requests the machine learning model to analyze it.

[0261] Data processing: Video data analysis

[0262] Output: Video analysis results

[0263] Machine learning model: The machine learning model analyzes the video and identifies points where users get stuck and where they frequently make errors.

[0264] Server: Generates specific feedback based on the video analysis results and provides it to the user. This operation includes generating feedback and responding to the user.

[0265] (Application example 1)

[0266] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0267] Modern factories require robot operators to learn robot control programs efficiently and effectively. However, it is difficult to provide customized learning content tailored to each operator's skill level and weaknesses. Furthermore, conventional programming education systems are unable to provide specific feedback or customized problems based on actual code writing or execution results, preventing improvements in learning efficiency. Furthermore, there is a lack of mechanisms for identifying problems operators encounter when operating robots and providing support based on those problems.

[0268] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0269] In this invention, the server includes means for automatically generating programming problems based on the skill level of a user, means for determining the correctness of answers submitted by the user, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, means for generating program problems for robot operators to improve the user's skills, and an application to be installed on robots used in factories. This makes it possible to provide customized program problems and specific feedback according to the skill level and weaknesses of robot operators.

[0270] "User" refers to a person who uses this system to solve programming problems.

[0271] "Skill level" is a standard for evaluating a user's programming ability and proficiency.

[0272] "Programming Problems" are tasks provided by the system to assess or improve programming skills.

[0273] "Automatic generation" refers to the system automatically creating programming problems according to the user's skill level.

[0274] "Solution" refers to the source code or program submitted by a user in response to a programming problem.

[0275] "Correctness determination" means that the system evaluates whether the submitted answer is correct or incorrect.

[0276] An "action log" refers to the operation history and activity record when a user uses the system.

[0277] A "coding video" is a video that records the process of a user creating a program.

[0278] "Weaknesses" are specific techniques or concepts that users have difficulty with or make repeated mistakes with.

[0279] "Analysis" refers to extracting specific information and patterns based on user behavior logs and coding videos.

[0280] "Customized programming problems" are programming problems that are tailored to each user to overcome their specific weaknesses.

[0281] A "robot operator" refers to a person whose job is to operate robots in a factory and control their movements.

[0282] A "factory" is a production facility for manufacturing products.

[0283] An "application" is software that realizes the functions provided by a system.

[0284] This invention is a system that automatically generates programming problems based on a user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze a user's behavioral log and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system is particularly applicable to programming problems for robot operators used in factories. The following describes specific embodiments of the invention.

[0285] System Configuration

[0286] The server manages a system that includes the following elements:

[0287] 1. User registration and login management:

[0288] User: A user creates an account and logs into the system. When creating an account, the user registers by entering a username and password. When logging in, the user enters their authentication information to access the system.

[0289] Server: The server receives the user's registration information and hashes the password to ensure security. When logging in, the server compares the entered password with the hashed password to authenticate.

[0290] 2. Skill Level Determination:

[0291] User: The user submits sample code to the system for skill evaluation.

[0292] Server: The server runs the submitted code and if the code runs correctly it will be judged as "Intermediate"; if an error occurs it will be judged as "Beginner".

[0293] 3. Automatic generation of programming problems:

[0294] Server: The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would generate a "program to add two numbers," and an intermediate would generate a "program to find factorials."

[0295] 4. Automatic answer checking and explanation generation:

[0296] User: The user submits the answer code to the system.

[0297] Server: The server examines the submitted code and compares it with a predefined answer. If it is correct, it marks it as "correct." If it is incorrect, it responds with a specific explanation.

[0298] 5. User weakness analysis and customization solutions:

[0299] Server: The server identifies the user's weak points based on the user's past answer data and the results of analyzing coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weak points."

[0300] Server: Next, it provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[0301] 6. Coding video analysis:

[0302] Users: Users record themselves coding and upload videos to the platform.

[0303] Server: The server analyzes the uploaded video using machine learning models to identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[0304] Computer Systems and Software

[0305] The server uses Flask to implement the web application and bcrypt to hash passwords. It also uses machine learning models to analyze user activity logs and coding videos. These pieces of software run on the robot's control unit and server.

[0306] Adding concrete examples and prompt sentence examples

[0307] As a specific example, there is the following usage.

[0308] 1. User Registration:

[0309] The user creates an account by entering a username and password.

[0310] 2. Login:

[0311] The user enters their login information and the server authenticates them.

[0312] 3. Skill Check:

[0313] Users submit sample code, and the server runs the code to determine skill level.

[0314] 4. Question submission:

[0315] The server generates "beginner" questions based on skill level and presents them to the user.

[0316] 5. Submitting and checking your answers:

[0317] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[0318] 6. Weakness analysis and customization problem provision:

[0319] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[0320] Prompt Sentence Examples

[0321] "Analyze the coding log for weakness analysis and generate optimal customization questions. If there are many errors related to loops or conditional branching, we will provide appropriate questions based on that."

[0322] This allows robot operators to receive program problems and specific feedback that are tailored to their skill level and weaknesses.

[0323] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0324] Step 1: User Registration

[0325] The user enters their username and password using a terminal and sends it to the server. This input data includes the user's identification and authentication information. The server hashes the received password using bcrypt and stores the user information along with the hashed password in the database. This registers a new user.

[0326] Input: Username, Password

[0327] Data processing: password hashing

[0328] Output: User information stored in the database

[0329] Step 2: User Login

[0330] The user enters the registered username and password using the terminal and sends them to the server. The server retrieves the corresponding user information from the database and verifies whether the sent password matches the hashed password. If the authentication is successful, the user can access the system.

[0331] Input: Username, Password

[0332] Data processing: Comparison with hashed passwords

[0333] Output: Login session started or error message

[0334] Step 3: Determine your skill level

[0335] Users submit sample code from their devices to the server for skill assessment. The server executes the submitted code, and if the code runs correctly, the user is judged as "intermediate," but if an error occurs, the user is judged as "beginner." The result of this assessment is saved in a database as the user's level.

[0336] Input: Sample code

[0337] Data processing: Executing code and determining results

[0338] Output: Updated skill level of the user

[0339] Step 4: Automatic generation of programming problems

[0340] The server automatically generates appropriate programming problems based on the user's skill level. For example, it generates a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners. The generated programming problems are provided to the user.

[0341] Input: User's skill level

[0342] Data manipulation: Generating programming problems

[0343] Output: Generated programming problem

[0344] Step 5: Check answers and automatically generate explanations

[0345] The user submits the answer code from their device to the server. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[0346] Input: Answer code

[0347] Data processing: Examining the code and comparing it to the correct answer

[0348] Output: Judgment result and explanation

[0349] Step 6: Analyze user weaknesses and provide customization solutions

[0350] The server identifies the user's weaknesses based on the analysis of the user's past answer data and coding videos. For example, if the user frequently makes mistakes regarding loops, it identifies that as a "weakness." It then generates customized programming problems to help the user overcome the identified weaknesses.

[0351] Input: Answer data, coding video

[0352] Data processing: Data analysis and weakness identification

[0353] Output: Customized Program Questions

[0354] Step 7: Analyzing the coding video

[0355] Users upload videos of themselves coding from their devices to the server. The server then analyzes the videos using machine learning models to identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[0356] Input: Coding video

[0357] Data processing: Video analysis using machine learning models

[0358] Output: Identified weaknesses and feedback

[0359] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0360] This invention is a system that automatically generates programming problems based on the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it analyzes the user's behavioral logs and coding videos to identify individual weaknesses and provides customized problems based on those weaknesses. By combining this with an emotion engine that recognizes the user's emotions, the user's learning experience becomes even more personalized and effective. Specific implementation methods for this system are described below.

[0361] User registration and login management

[0362] user:

[0363] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[0364] server:

[0365] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[0366] Determining a user's skill level

[0367] user:

[0368] Users submit sample code to the system for skill evaluation.

[0369] server:

[0370] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[0371] Automatic generation of programming problems

[0372] server:

[0373] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[0374] Automatic answer checking and explanation generation

[0375] user:

[0376] The user submits the answer code to the system.

[0377] server:

[0378] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[0379] User weakness analysis and customization

[0380] server:

[0381] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[0382] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[0383] Coding video analysis

[0384] user:

[0385] Users record themselves coding and upload videos to the platform.

[0386] server:

[0387] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[0388] Introducing the Emotion Engine

[0389] user:

[0390] As users code, they use a camera and microphone to record their facial expressions and voice.

[0391] server:

[0392] The server sends data acquired from the camera and microphone to the emotion engine, which analyzes the user's facial expressions and voice to recognize emotions in real time. This analysis can be done using, for example, facial recognition technology and voice analysis technology.

[0393] Emotion-Based Feedback

[0394] server:

[0395] The server adjusts the learning program based on the user's emotions, as detected by the emotion engine. For example, if the user is feeling stressed, it will provide easier problems or display encouraging messages.

[0396] Specific examples

[0397] Example 1: A beginner user registers and solves a problem

[0398] 1. User Registration

[0399] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[0400] 2. Log in

[0401] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[0402] 3. Skill Judgment

[0403] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[0404] 4. Providing questions

[0405] The server generates "beginner" questions based on skill level and presents them to the user.

[0406] 5. Submitting and checking your answers

[0407] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[0408] 6. Weakness analysis and customization problem provision

[0409] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[0410] 7. Analysis of coding videos

[0411] Users upload their own coding videos, which are then analyzed by the server using machine learning models. Based on the results, the server identifies the user's weaknesses and provides appropriate feedback.

[0412] 8. Leveraging Emotional Engines

[0413] The server recognizes the user's emotions from their facial expressions and voice and provides appropriate learning programs and feedback. For example, if the user shows signs of frustration, it provides them with questions of lower difficulty.

[0414] This process allows users to learn effectively at their own pace and overcome their weaknesses. The introduction of an emotional engine makes the learning experience more personalized and increases user motivation.

[0415] The processing flow will be explained below.

[0416] User registration and login management

[0417] User Registration

[0418] Step 1:

[0419] The user enters a username and password.

[0420] Step 2:

[0421] The terminal sends the entered username and password to the server.

[0422] Step 3:

[0423] The server extracts the username and password from the request data.

[0424] Step 4:

[0425] The server hashes the password using an algorithm such as SHA-256.

[0426] Step 5:

[0427] The server stores the username and hashed password in a database.

[0428] Step 6:

[0429] The server returns a registration success message to the terminal.

[0430] Log in

[0431] Step 1:

[0432] The user enters the registered username and password.

[0433] Step 2:

[0434] The terminal sends the entered username and password to the server.

[0435] Step 3:

[0436] The server extracts the username and password from the request data.

[0437] Step 4:

[0438] The server retrieves the corresponding hashed password from its database.

[0439] Step 5:

[0440] The server hashes the entered password and verifies that it matches the stored hashed password.

[0441] Step 6:

[0442] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[0443] Determining a user's skill level

[0444] Step 1:

[0445] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[0446] Step 2:

[0447] The device sends the sample code to the server.

[0448] Step 3:

[0449] The server receives the submitted code.

[0450] Step 4:

[0451] The server runs the code in a secure environment.

[0452] Step 5:

[0453] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[0454] Step 6:

[0455] The server returns the skill level evaluation result to the terminal.

[0456] Automatic generation of programming problems

[0457] Step 1:

[0458] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[0459] Step 2:

[0460] The server generates programming problems corresponding to skill levels.

[0461] Step 3:

[0462] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[0463] Step 4:

[0464] The server transmits the generated programming questions to the terminal.

[0465] Automatic answer checking and explanation generation

[0466] Step 1:

[0467] The user enters the answer code into the terminal and clicks the submit button.

[0468] Step 2:

[0469] The terminal sends the answer code to the server.

[0470] Step 3:

[0471] The server receives the submitted answer code.

[0472] Step 4:

[0473] The server compares the answer to predefined correct answers.

[0474] Step 5:

[0475] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[0476] Step 6:

[0477] The server sends the generated message and explanation to the terminal.

[0478] User weakness analysis and customization

[0479] Step 1:

[0480] The server analyzes the user's past answer data and behavioral logs.

[0481] Step 2:

[0482] The server identifies the user's weaknesses based on data analysis.

[0483] Step 3:

[0484] The server invokes logic to generate customized questions according to the identified weaknesses.

[0485] Step 4:

[0486] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[0487] Step 5:

[0488] The server transmits the generated customization question to the terminal.

[0489] Coding video analysis

[0490] Step 1:

[0491] Users upload videos of themselves coding to their devices.

[0492] Step 2:

[0493] The device sends the uploaded video to the server.

[0494] Step 3:

[0495] The server analyzes the received video using a machine learning model.

[0496] Step 4:

[0497] The server identifies behaviors and choke points during coding.

[0498] Step 5:

[0499] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[0500] Step 6:

[0501] The server transmits the generated feedback to the terminal.

[0502] Introducing the Emotion Engine

[0503] Step 1:

[0504] Set up your camera and microphone when you code.

[0505] Step 2:

[0506] The device collects data from the camera and microphone in real time and sends it to the emotion engine.

[0507] Step 3:

[0508] The server uses an emotion engine to analyze the user's facial expressions and voice in real time and recognize their emotions.

[0509] Emotion-Based Feedback

[0510] Step 1:

[0511] The server receives the analysis results from the emotion engine.

[0512] Step 2:

[0513] Based on the analysis results, the server generates customized feedback according to the user's emotions.

[0514] Step 3:

[0515] For example, if a user is feeling stressed, it offers a simple message of encouragement or an easier problem.

[0516] Step 4:

[0517] The server transmits feedback based on the generated emotion to the terminal.

[0518] This invention can improve the learning effect and reduce stress for users by providing real-time feedback and emotional support according to their learning progress.

[0519] Example 2

[0520] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0521] While conventional programmatic learning systems can generate questions and assess correct answers based on the user's skill level, they lack the ability to provide customized feedback based on a detailed analysis of each user's learning status and weaknesses. Furthermore, they are unable to adjust the learning program to take into account the user's emotional state, which means they are unable to adequately maintain learner motivation or reduce stress. Therefore, a new system that can provide a more effective and personalized learning experience is needed.

[0522] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for automatically generating programming problems based on the user's skill level, means for determining whether the answers submitted by the user are correct, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, and means for recognizing the user's emotions and adjusting the learning program. This makes it possible to provide problems and analyze weaknesses according to the user's skill level, and further to provide a personalized and effective learning experience by adjusting the learning based on the user's emotional state.

[0523] "User" refers to an individual or organization that uses this system to submit answers to programming problems and advance their learning.

[0524] "Skill level" is an indicator of a user's programming ability, and is classified as beginner, intermediate, etc. based on the results of executing the submitted sample code.

[0525] "Programming problems" refer to programming challenges that the system automatically generates for users to answer.

[0526] "Correctness determination" is the process of checking whether the program code submitted by the user matches a predefined answer and determining whether it is correct or incorrect.

[0527] "Behavior log" refers to a record of the operations and activities performed by a user on the system.

[0528] "Coding video" refers to video data that records the process of a user creating a program.

[0529] "Weaknesses" refer to areas or concepts where users frequently make mistakes when solving programming problems.

[0530] "Customized programming problems" refer to programming tasks that are individually designed based on the user's analysis results.

[0531] "Emotion" refers to the psychological or emotional state that a user experiences while coding.

[0532] "Adjusting the learning program" refers to changing the questions and feedback provided based on the user's learning progress and emotional state.

[0533] A "generative AI model" refers to an artificial intelligence method or system that automatically generates new questions, explanations, and feedback using natural language generation techniques, etc.

[0534] This invention is a system that automatically generates programming problems based on the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it analyzes the user's behavioral logs and coding videos to identify individual weaknesses and provides customized problems based on those weaknesses. By combining this with an emotion engine that recognizes the user's emotions, the user's learning experience becomes even more personalized and effective. Specific implementation methods for this system are described below.

[0535] User registration and login management

[0536] user:

[0537] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[0538] server:

[0539] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[0540] Determining a user's skill level

[0541] user:

[0542] Users submit sample code to the system for skill evaluation.

[0543] server:

[0544] The server runs the submitted code in an execution environment (e.g., Python execution environment), and if the code runs correctly, the user is judged as "intermediate," but if an error occurs, the user is judged as "beginner."

[0545] Automatic generation of programming problems

[0546] server:

[0547] The server automatically generates appropriate programming problems based on the user's skill level, using a generative AI model to create the problems. For example, a beginner might get a "program to add two numbers," while an intermediate learner might get a "program to find the factorial."

[0548] Automatic answer checking and explanation generation

[0549] user:

[0550] The user submits the answer code to the system.

[0551] server:

[0552] The server compares the submitted code with predefined answers. If the answer is correct, it is judged as "correct." If the answer is incorrect, a specific explanation is generated using a generative AI model and provided to the user.

[0553] User weakness analysis and customization

[0554] server:

[0555] The server analyzes the user's past answer data and behavioral logs to identify their weaknesses. Based on the identified weaknesses, the server then uses a generative AI model to generate appropriate, customized programming problems and provide them to the user. For example, if there are many incorrect answers about loops, the server generates programming problems with a "loop" theme.

[0556] Coding video analysis

[0557] user:

[0558] Users record themselves coding and upload videos to the platform.

[0559] server:

[0560] The server analyzes the uploaded video using machine learning models (e.g., YOLO or OpenCV) to identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[0561] Introducing the Emotion Engine

[0562] user:

[0563] As users code, they use a camera and microphone to record their facial expressions and voice.

[0564] server:

[0565] The server sends data acquired from the camera and microphone to the emotion engine, which analyzes the user's facial expressions and voice to recognize emotions in real time. This analysis uses facial recognition technology (e.g., OpenCV) and voice analysis technology (e.g., Microsoft Azure's Speech to Text).

[0566] Emotion-Based Feedback

[0567] server:

[0568] The server adjusts the learning program based on the user's emotions, as detected by the emotion engine. For example, if the user is feeling stressed, it will provide easier problems or display encouraging messages. This feedback is generated using a generative AI model.

[0569] Specific examples

[0570] Example 1: A beginner user registers and solves a problem:

[0571] 1. User Registration:

[0572] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[0573] 2. Login:

[0574] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[0575] 3. Skill Check:

[0576] Users submit sample code, and the server runs the code in an execution environment such as Python to determine their skill level. If the code runs without errors, they are considered intermediate.

[0577] 4. Question submission:

[0578] The server generates "beginner" questions based on skill level and presents them to the user.

[0579] 5. Submitting and checking your answers:

[0580] The user submits an answer, and the server compares it with predefined correct answers. If the answer is correct, a "correct" message is generated using a generative AI model, and if the answer is incorrect, a specific explanation is generated and provided to the user.

[0581] 6. Weakness analysis and customization problem provision:

[0582] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[0583] 7. Analysis of coding video:

[0584] Users upload their own coding videos, which are then analyzed by the server using machine learning models. Based on the results, the server identifies the user's weaknesses and provides appropriate feedback.

[0585] 8. Leveraging the Emotion Engine:

[0586] The server recognizes the user's emotions from their facial expressions and voice and provides appropriate learning programs and feedback. For example, if the user shows signs of frustration, it provides them with questions of lower difficulty.

[0587] Example prompts to input to the generative AI model

[0588] Example prompt sentence:

[0589] "Generate feedback and explanations for user-submitted code, assuming it's written in Python."

[0590] "If the emotion engine detects stress from the user's facial expression, generate an encouraging message for the user."

[0591] This process allows users to learn effectively at their own pace and overcome their weaknesses. The introduction of an emotional engine makes the learning experience more personalized and increases user motivation.

[0592] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0593] The flow of this system's program processing

[0594] Step 1: User Registration

[0595] Input: Username, Password

[0596] Output: Registration successful or error message

[0597] user:

[0598] Users access the system from their terminal and enter their username and password on the account creation screen.

[0599] server:

[0600] The server receives the username and password you entered.

[0601] The received password is hashed using a hashing algorithm (e.g., SHA-256).

[0602] The server checks the database to see if the entered username already exists, and if not, saves the new user information in the database.

[0603] Registration information such as username is stored in a database along with hashed passwords.

[0604] Step 2: Login authentication

[0605] Input: Username, Password

[0606] Output: Authentication successful or error message

[0607] user:

[0608] The user enters their username and password on the login screen.

[0609] server:

[0610] The server receives the username and password you entered.

[0611] The server retrieves the hashed password for that username from its database.

[0612] The entered password is hashed and compared with the information in the database.

[0613] If they match, the authentication is successful and the user is logged in to the system. If they do not match, an error message is returned.

[0614] Step 3: Skill Level Assessment

[0615] Input: Sample code

[0616] Output: Skill level judgment result (beginner, intermediate)

[0617] user:

[0618] Users submit sample code to the system for skill evaluation.

[0619] server:

[0620] The server receives sample code submitted by the user.

[0621] The server runs the sample code in an execution environment (e.g., Python execution environment).

[0622] Check the execution results and evaluate whether any errors occur.

[0623] If no error occurs, the player is judged as intermediate, and if an error occurs, the player is judged as beginner.

[0624] The result of the judgment is saved in the user's profile.

[0625] Step 4: Automatic generation of programming problems

[0626] Input: Skill Level

[0627] Output: Programming problem

[0628] server:

[0629] The server uses a generative AI model based on the user's skill level to automatically generate appropriate programming problems.

[0630] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[0631] The generated programming problem is presented to the user.

[0632] Step 5: Submit and check your answers

[0633] Input: Answer code

[0634] Output: Answer check result (correct answers, incorrect answers and explanations)

[0635] user:

[0636] The user submits the solution code to the system for the programming problem presented to them.

[0637] server:

[0638] The server receives the answer code submitted by the user.

[0639] The submitted answer code is compared with a predefined correct answer code.

[0640] If the answer is correct, it is judged as "correct," and if the answer is incorrect, a specific explanation is generated using a generative AI model.

[0641] Provide the results to the user.

[0642] Step 6: Analyze user weaknesses and provide customization solutions

[0643] Input: Past answer data, behavior log

[0644] Output: Custom programming questions

[0645] server:

[0646] The server analyzes the user's past answer data and behavioral logs.

[0647] Reinforcement learning algorithms and machine learning are used to identify weaknesses.

[0648] For example, if there are many loop-related errors, the generative AI model will generate customization problems related to the loop.

[0649] Providing users with customized programming problems.

[0650] Step 7: Analyzing the coding video

[0651] Input: Coding video

[0652] Output: Analysis results and feedback

[0653] user:

[0654] Users record themselves coding and upload videos to the platform.

[0655] server:

[0656] The server analyzes the coding videos uploaded by users.

[0657] Machine learning models (e.g., YOLO, OpenCV) are used to identify movements in the video and areas where errors frequently occur.

[0658] Based on the analysis results, we identify the user's weaknesses and provide appropriate feedback.

[0659] Step 8: Implementing the Emotion Engine and Feedback

[0660] Input: Video and audio data from the camera and microphone

[0661] Output: Emotion recognition results and feedback

[0662] user:

[0663] As users code, they use a camera and microphone to record their facial expressions and voice.

[0664] server:

[0665] The server analyzes the video and audio data acquired from the camera and microphone.

[0666] The emotion engine recognizes user emotions in real time using facial recognition technology (e.g., OpenCV) and voice analysis technology (e.g., Microsoft Azure's Speech to Text).

[0667] The learning program can be adjusted based on the perceived emotions, for example, providing easier problems or encouraging messages generated using a generative AI model if the user is feeling stressed.

[0668] (Application example 2)

[0669] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0670] While the demand for programming education is increasing, providing optimal learning programs tailored to each user's skill level is challenging. Furthermore, conventional systems are insufficient in assessing users' learning progress, identifying individual weaknesses, and providing customized feedback. Furthermore, to improve the user's learning experience, it is necessary to understand the user's emotional state in real time and provide appropriate support accordingly.

[0671] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0672] In this invention, the server includes means for automatically generating programming problems based on the user's skill level, means for determining whether the answers submitted by the user are correct, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, and means for recognizing the user's emotions and adjusting the learning program. This makes it possible to automatically generate optimal programming problems according to the user's skill, identify the user's weaknesses, and provide customized feedback. Furthermore, by analyzing the user's emotional state in real time, the learning experience can be personalized and effective support can be provided.

[0673] "User skill level" refers to the user's current level of programming knowledge and ability.

[0674] "Automatic generation of programming problems" means automatically creating appropriate programming tasks based on the user's skill level.

[0675] "Means for determining whether the answer is correct" refers to the function of checking the correctness of the program submitted by the user and providing feedback on the results.

[0676] An "action log" is a record of a user's operation history and behavioral data when using the system.

[0677] A "coding video" is a video recording of a user writing a program.

[0678] "Means for analyzing weaknesses" refers to a function that analyzes user behavior logs and coding video data to identify areas in which the user is weak.

[0679] "Customized programming problems" refer to programming tasks that are specifically designed to overcome a user's analyzed weaknesses.

[0680] "Means of recognizing emotions" refers to technology for reading a user's psychological state from their facial expressions and voice.

[0681] "Means for adjusting the learning program" refers to the ability to change the learning content and difficulty level provided in real time depending on the user's emotional state and skill level.

[0682] In this invention, a system is constructed in which a server, a user terminal, and an emotion engine work together. The server automatically generates programming problems according to the user's skill level, judges whether the user's answers are correct, analyzes behavior logs and coding videos, provides customized problems, and adjusts the learning program based on the user's emotional state.

[0683] The server uses the following software to perform its processing:

[0684] Flask: A lightweight framework for providing server-side web services.

[0685] werkzeug.security: A security library used for password hashing and authentication.

[0686] emotion_recognition: A technology for identifying user emotions using an emotion analysis engine.

[0687] skill_evaluation: A module that determines skill levels and evaluates submitted program code.

[0688] video_analysis: A machine learning model that analyzes users' coding videos to identify behavioral trends and weaknesses.

[0689] The server analyzes data obtained from the user's submitted program code, behavioral logs, and coding videos to identify the user's weaknesses and learning progress. It also recognizes the user's emotional state in real time through an emotion engine, making it possible to adjust the difficulty of the learning program and provide appropriate feedback.

[0690] As a concrete example, the process of a user registering an account on a smartphone and solving a programming problem is shown below. The user uses a smartphone or smart glasses to submit programming code by voice input or keyboard input. The server then analyzes the submitted code, determines the user's skill level, and provides programming problems of an appropriate level of difficulty. The user answers the problem and submits it again to the server, where the answer is judged to be correct or incorrect, and if the answer is incorrect, a specific explanation is automatically generated.

[0691] Furthermore, users can record their own coding videos with the smart glasses and upload them to the platform, where the server analyzes the videos to identify the user's coding style and areas where they are stuck.The emotion engine then analyzes the user's facial expressions and voice data to recognize their emotions in real time, and provides feedback such as lowering the difficulty level or displaying encouraging messages if the user is feeling frustrated.

[0692] An example of a prompt would be, "We have recorded a user solving a programming problem. Please use a machine learning model to analyze the user's coding style, stuck points, and error patterns from this video. Also, please analyze the user's facial expressions and voice, and use an emotion engine to recognize their emotions in real time and provide appropriate feedback."

[0693] Through this process, users receive optimal learning content suited to their skill level, as well as personalized feedback based on their weaknesses and emotional state.

[0694] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0695] Step 1:

[0696] A user registers an account on a smartphone or smart glasses. The user enters a username and password, which are then sent to the server, which hashes the password and stores it in a database.

[0697] Input: Username, Password

[0698] Output: Hashed password, saved user account information

[0699] Step 2:

[0700] A user attempts to log in. They enter their username and password and submit them to the server, which authenticates them by comparing the entered password with a hashed password stored in a database.

[0701] Input: Username, Password

[0702] Output: Authentication result (success or failure)

[0703] Step 3:

[0704] The user submits sample code for skill level assessment. The user puts on the smart glasses and inputs the sample code via voice or text input. The server executes the submitted code and determines the skill level.

[0705] Input: Sample code

[0706] Output: Skill level judgment result (beginner, intermediate)

[0707] Step 4:

[0708] The server automatically generates programming problems appropriate for each skill level, such as a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners.

[0709] Input: Skill Level

[0710] Output: Auto-generated programming questions

[0711] Step 5:

[0712] The user answers the provided programming questions and submits them to the server. The user enters the answer code using a smartphone or smart glasses and sends it to the server. The server compares the submitted code with the correct answer and determines whether it is correct or incorrect.

[0713] Input: Answer code

[0714] Output: Correct / incorrect result, feedback (including explanation)

[0715] Step 6:

[0716] The server analyzes the user's behavior log and submitted answers to identify the user's weaknesses. For example, if there are many incorrect answers regarding loops, it will extract those weaknesses.

[0717] Input: User behavior log, answer history

[0718] Output: Identified weaknesses

[0719] Step 7:

[0720] Users record their coding videos using smart glasses and upload them to a server, which then uses machine learning models to analyze the videos and identify the user's coding style and stumbling points.

[0721] Input: Coding video

[0722] Output: Analysis results (blockage points, error patterns)

[0723] Step 8:

[0724] The emotion engine recognizes the user's emotional state in real time from their facial expressions and voice. The user captures data using the camera and microphone in the smart glasses and sends it to the server. The server then performs emotion analysis and adjusts the learning program.

[0725] Input: facial expression data, voice data

[0726] Output: emotional state, adjusted learning program

[0727] Step 9:

[0728] The server provides customized feedback and programming problems based on the analyzed weaknesses and emotional state, and displays difficulty-adjusted problems and encouraging messages to reduce user frustration.

[0729] Input: Identified weaknesses, emotional state

[0730] Output: Customized feedback, programming problems

[0731] As a concrete example, the prompt sentence is as follows: "We have recorded a user solving a programming problem. Please use a machine learning model to analyze the user's coding style, stuck points, and error patterns from this video. Also, please analyze the user's facial expressions and voice, and use an emotion engine to recognize their emotions in real time and provide appropriate feedback."

[0732] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0733] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0734] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0735] [Second embodiment]

[0736] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0737] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0738] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0739] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0740] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0741] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0742] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0743] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0744] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0745] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0746] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0747] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0748] This invention is a system that automatically generates programming problems according to the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze the user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows.

[0749] User registration and login management

[0750] user:

[0751] Users create an account and log in to the system. When creating an account, users register by entering a username and password. When logging in, users enter their authentication information to access the system.

[0752] server:

[0753] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[0754] Determining a user's skill level

[0755] user:

[0756] Users submit sample code to the system for skill evaluation.

[0757] server:

[0758] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[0759] Automatic generation of programming problems

[0760] server:

[0761] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[0762] Automatic answer checking and explanation generation

[0763] user:

[0764] The user submits the answer code to the system.

[0765] server:

[0766] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[0767] User weakness analysis and customization

[0768] server:

[0769] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[0770] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[0771] Coding video analysis

[0772] user:

[0773] Users record themselves coding and upload videos to the platform.

[0774] server:

[0775] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[0776] Specific examples

[0777] Example 1: A beginner user registers and solves a problem

[0778] 1. User Registration

[0779] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[0780] 2. Log in

[0781] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[0782] 3. Skill Judgment

[0783] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[0784] 4. Providing questions

[0785] The server generates "beginner" questions based on skill level and presents them to the user.

[0786] 5. Submitting and checking your answers

[0787] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[0788] 6. Weakness analysis and customization problem provision

[0789] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[0790] This system provides an environment where beginners can effectively learn programming at their own pace. It dramatically improves learning efficiency by providing questions and explanations optimized based on the user's skill level and weaknesses.

[0791] The processing flow will be explained below.

[0792] User registration and login management

[0793] User Registration

[0794] Step 1:

[0795] The user enters a username and password.

[0796] Step 2:

[0797] The terminal sends the entered username and password to the server.

[0798] Step 3:

[0799] The server extracts the username and password from the request data.

[0800] Step 4:

[0801] The server hashes the password using an algorithm such as SHA-256.

[0802] Step 5:

[0803] The server stores the username and hashed password in a database.

[0804] Step 6:

[0805] The server returns a registration success message to the terminal.

[0806] Log in

[0807] Step 1:

[0808] The user enters the registered username and password.

[0809] Step 2:

[0810] The terminal sends the entered username and password to the server.

[0811] Step 3:

[0812] The server extracts the username and password from the request data.

[0813] Step 4:

[0814] The server retrieves the corresponding hashed password from its database.

[0815] Step 5:

[0816] The server hashes the entered password and verifies that it matches the stored hashed password.

[0817] Step 6:

[0818] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[0819] Determining a user's skill level

[0820] Step 1:

[0821] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[0822] Step 2:

[0823] The device sends the sample code to the server.

[0824] Step 3:

[0825] The server receives the submitted code.

[0826] Step 4:

[0827] The server runs the code in a secure environment.

[0828] Step 5:

[0829] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[0830] Step 6:

[0831] The server returns the skill level evaluation result to the terminal.

[0832] Automatic generation of programming problems

[0833] Step 1:

[0834] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[0835] Step 2:

[0836] The server generates programming problems corresponding to skill levels.

[0837] Step 3:

[0838] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[0839] Step 4:

[0840] The server transmits the generated programming questions to the terminal.

[0841] Automatic answer checking and explanation generation

[0842] Step 1:

[0843] The user enters the answer code into the terminal and clicks the submit button.

[0844] Step 2:

[0845] The terminal sends the answer code to the server.

[0846] Step 3:

[0847] The server receives the submitted answer code.

[0848] Step 4:

[0849] The server compares the answer to predefined correct answers.

[0850] Step 5:

[0851] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[0852] Step 6:

[0853] The server sends the generated message and explanation to the terminal.

[0854] User weakness analysis and customization

[0855] Step 1:

[0856] The server analyzes the user's past answer data and behavioral logs.

[0857] Step 2:

[0858] The server identifies user weaknesses (e.g., loops, conditional branches) based on data analysis.

[0859] Step 3:

[0860] The server invokes logic to generate customized questions according to the identified weaknesses.

[0861] Step 4:

[0862] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[0863] Step 5:

[0864] The server transmits the generated customization question to the terminal.

[0865] Coding video analysis

[0866] Step 1:

[0867] Users upload videos of themselves coding to their devices.

[0868] Step 2:

[0869] The device sends the uploaded video to the server.

[0870] Step 3:

[0871] The server analyzes the received video using a machine learning model.

[0872] Step 4:

[0873] The server identifies behaviors and choke points during coding.

[0874] Step 5:

[0875] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[0876] Step 6:

[0877] The server transmits the generated feedback to the terminal.

[0878] As a result, this fully automatic programming learning system helps users to learn effectively at their own pace.

[0879] Example 1

[0880] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0881] Conventional programming problem-solving systems have difficulty automatically generating appropriate problems based on the user's skill level and weaknesses, resulting in reduced learning efficiency. Additionally, there are generally delays in determining whether an answer is correct and in providing feedback, preventing users from checking their learning progress in real time. Furthermore, while analyzing coding videos, rather than relying solely on traditional behavioral log analysis, would enable more detailed and accurate feedback to be provided as a means of analyzing users' weaknesses, few such systems exist.

[0882] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0883] In this invention, the server includes a means for receiving user authentication information and hashing the password to ensure security, a means for automatically generating programming problems based on the user's skill level, and a means for determining whether the answers submitted by the user are correct, thereby enabling the provision of appropriate programming problems according to the user's skill level and fast and accurate feedback.

[0884] "User Credentials" means the username and password or other authentication credentials used to access a system.

[0885] "Password hashing" is the process of encrypting a user-entered password using a specific algorithm to ensure system security.

[0886] "Skill level" is an indicator of a user's programming skill and knowledge. For example, there is a distinction between "beginner" and "intermediate."

[0887] "Automatic programming problem generation" is the process by which the system automatically creates appropriate programming challenges based on the user's skill level.

[0888] The "means for determining whether the answer submitted by the user is correct" is a means for checking whether the program code submitted by the user is executed according to the specifications and determining whether the answer is appropriate.

[0889] An "action log" is a record of the operations and input a user performs on the system.

[0890] "Coding videos" are video data that record the process of a user programming.

[0891] "Weakness analysis" is the process of analyzing user behavior logs and coding videos to identify areas where users are particularly weak or frequently make errors.

[0892] "Customized programming problems" are personalized programming challenges generated based on a user's identified weaknesses.

[0893] This system automatically generates programming problems based on a user's skill level and judges the accuracy of the user's submitted solutions. Furthermore, it can analyze a user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows:

[0894] User registration and login management

[0895] A user creates an account and enters a username and password to log in to the system. The server receives the user's registration information and hashes and stores the password to ensure security. Also, when logging in, the entered password is compared with the hashed password for authentication. This maintains a high level of security.

[0896] Determining a user's skill level

[0897] Users submit sample code for skill assessment. The server runs the submitted code in a sandbox environment and determines the skill level based on whether an error occurs. For example, if no errors occur, the user is assessed as "intermediate," and if an error occurs, the user is assessed as "beginner."

[0898] Automatic generation of programming problems

[0899] The server automatically generates appropriate programming problems using a generative AI model based on the user's skill level. For example, it generates a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners. The generated problems are then presented to the user.

[0900] Automatic answer checking and explanation generation

[0901] The user submits the answer code. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct." If the answer is incorrect, it responds with a specific explanation. This allows the user to receive instant feedback.

[0902] User weakness analysis and customization

[0903] The server analyzes the user's past answer data and behavioral logs to identify the user's weaknesses. For example, if there are many incorrect answers regarding loops or conditional branching, it identifies these as "weaknesses." Based on the analysis results, the server provides customized programming problems to help the user overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates the corresponding programming problem.

[0904] Coding video analysis

[0905] Users upload videos of themselves coding to the platform. The server then uses machine learning models to analyze the uploaded videos and identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the platform identifies the user's weaknesses and provides appropriate feedback.

[0906] Specific examples

[0907] 1. Example of a beginner user registering and solving a problem

[0908] Create an account: The user creates an account by entering a username and password. The server hashes the password using the bcrypt library and stores it in the database along with the user information.

[0909] Login: The user enters a username and password. The server retrieves the hashed password for that username from the database and compares it with the entered password. If authentication is successful, the user is allowed to access the system.

[0910] Skill assessment: Users submit sample code. The server runs the submitted code in a Docker container and checks the error log to determine the skill level. For example, if sample code such as "print(2 + 2)" can be executed without error, the user is assessed as intermediate.

[0911] Problem provision: The server requests a problem for beginners from the AI ​​model and receives a program to add two numbers. The server then presents the generated problem to the user.

[0912] Answer submission and verification: The user submits an answer. The server compares the answer with predefined correct answers in a Docker container. If the answer is correct, the server returns a "correct" message. If the answer is incorrect, the server returns feedback including the exact error location.

[0913] Weakness analysis and provision of customized problems: The server analyzes the user's answer history and behavior log to identify loops as particularly difficult. It then asks the generative AI model to generate customized problems related to loops and provides them to the user. For example, it generates a problem such as "Create a program to calculate the sum of numbers from 1 to 10."

[0914] Prompt Sentence Examples

[0915] Prompt: Please generate a sample problem for a simple Python program for beginners. Also, please provide an explanation for this problem.

[0916] Example response of the generative AI model:

[0917] Question: Write a program that takes two integers as input and calculates and displays their sum.

[0918] Explanation: In this problem, we will use the input() function to read two integers from the user, add them, and display the result using the print() function. See the code below for an example.

[0919] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0920] Step 1: User Registration

[0921] User: A user creates an account by entering a username and password.

[0922] Input: Username and Password

[0923] Server: The server processes the received user information and hashes the password, specifically encrypting it using the bcrypt library.

[0924] Data manipulation: password hashing

[0925] Output: User information including hashed password

[0926] Server: The server stores this information in a database, specifically by inserting the hashed password and username into the database.

[0927] Step 2: Login authentication

[0928] User: The user enters their username and password into a login form.

[0929] Input: Username and Password

[0930] Server: The server retrieves the hashed password for the given username from its database.

[0931] Data manipulation: Obtaining a hashed password corresponding to a username

[0932] Output: Hashed password corresponding to username

[0933] Server: The server compares the entered password with the hashed password retrieved from the database.

[0934] Data calculation: Password verification

[0935] Output: Authentication result

[0936] Server: If authentication is successful, it generates session information and responds to the user. This includes generating a session ID and storing it with the corresponding user information.

[0937] Step 3: Submit your skill evaluation code

[0938] User: The user writes a sample code for skill evaluation and presses the submit button.

[0939] Input: Sample code

[0940] Server: The server receives the submitted code and executes it in a sandbox environment (Docker container).

[0941] Data processing: Running the sample code

[0942] Output: Execution results and error log

[0943] Server: Based on the execution results, if no errors occur, the user is judged as "Intermediate", and if an error occurs, the user is judged as "Beginner". This operation analyzes the error log and judges the skill level.

[0944] Step 4: Problem generation

[0945] Server: The server sends appropriate prompts to the generative AI model based on the user's skill level.

[0946] Input: Skill level and prompt

[0947] Generative AI model: A generative AI model generates programming problems based on prompts.

[0948] Data processing: Generating programming problems

[0949] Output: The generated programming problem

[0950] Server: Presents the generated questions to the user. This includes displaying the generated questions in a user interface.

[0951] Step 5: Submit your answer

[0952] User: The user writes the answer code for the displayed programming problem and presses the submit button.

[0953] Input: Answer code

[0954] Server: The server receives the submitted code and compares it with pre-stored correct answers.

[0955] Data processing: Analysis and comparison of submitted solution codes

[0956] Output: Correct / incorrect result and feedback

[0957] Server: If the answer is correct, it is deemed "correct" and a feedback message is generated and returned. If the answer is incorrect, feedback is provided, including specific error messages and suggestions for improvement. This action includes generating the feedback and responding to the user.

[0958] Step 6: Weakness analysis

[0959] Server: The server analyzes past answer data and behavioral logs to identify typical user error patterns and coding pitfalls.

[0960] Input: Past answer data and behavior log

[0961] Data processing: Error pattern analysis and statistical analysis

[0962] Output: Weakness identification results

[0963] Server: Based on the analysis results, the server requests the generative AI model to create customized questions that address the user's identified weaknesses. This action involves sending prompts to the generative AI model.

[0964] Step 7: Generate customization questions

[0965] Generative AI model: Based on the results of weakness analysis, the generative AI model generates personalized programming challenges for users to overcome.

[0966] Input: Weakness analysis results and prompt text

[0967] Data manipulation: custom problem generation

[0968] Output: Customized programming problems

[0969] Server: Presents the generated customization questions to the user. This includes displaying the customization questions in the user interface.

[0970] Step 8: Upload and analyze your video

[0971] Users: Users record themselves coding and upload videos to the platform.

[0972] Input: Coding video

[0973] Server: The server receives the uploaded video and requests the machine learning model to analyze it.

[0974] Data processing: Video data analysis

[0975] Output: Video analysis results

[0976] Machine learning model: The machine learning model analyzes the video and identifies points where users get stuck and where they frequently make errors.

[0977] Server: Generates specific feedback based on the video analysis results and provides it to the user. This operation includes generating feedback and responding to the user.

[0978] (Application example 1)

[0979] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0980] Modern factories require robot operators to learn robot control programs efficiently and effectively. However, it is difficult to provide customized learning content tailored to each operator's skill level and weaknesses. Furthermore, conventional programming education systems are unable to provide specific feedback or customized problems based on actual code writing or execution results, preventing improvements in learning efficiency. Furthermore, there is a lack of mechanisms for identifying problems operators encounter when operating robots and providing support based on those problems.

[0981] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0982] In this invention, the server includes means for automatically generating programming problems based on the skill level of a user, means for determining the correctness of answers submitted by the user, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, means for generating program problems for robot operators to improve the user's skills, and an application to be installed on robots used in factories. This makes it possible to provide customized program problems and specific feedback according to the skill level and weaknesses of robot operators.

[0983] "User" refers to a person who uses this system to solve programming problems.

[0984] "Skill level" is a standard for evaluating a user's programming ability and proficiency.

[0985] "Programming Problems" are tasks provided by the system to assess or improve programming skills.

[0986] "Automatic generation" refers to the system automatically creating programming problems according to the user's skill level.

[0987] "Solution" refers to the source code or program submitted by a user in response to a programming problem.

[0988] "Correctness determination" means that the system evaluates whether the submitted answer is correct or incorrect.

[0989] An "action log" refers to the operation history and activity record when a user uses the system.

[0990] A "coding video" is a video that records the process of a user creating a program.

[0991] "Weaknesses" are specific techniques or concepts that users have difficulty with or make repeated mistakes with.

[0992] "Analysis" refers to extracting specific information and patterns based on user behavior logs and coding videos.

[0993] "Customized programming problems" are programming problems that are tailored to each user to overcome their specific weaknesses.

[0994] A "robot operator" refers to a person whose job is to operate robots in a factory and control their movements.

[0995] A "factory" is a production facility for manufacturing products.

[0996] An "application" is software that realizes the functions provided by a system.

[0997] This invention is a system that automatically generates programming problems based on a user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze a user's behavioral log and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system is particularly applicable to programming problems for robot operators used in factories. The following describes specific embodiments of the invention.

[0998] System Configuration

[0999] The server manages a system that includes the following elements:

[1000] 1. User registration and login management:

[1001] User: A user creates an account and logs into the system. When creating an account, the user registers by entering a username and password. When logging in, the user enters their authentication information to access the system.

[1002] Server: The server receives the user's registration information and hashes the password to ensure security. When logging in, the server compares the entered password with the hashed password to authenticate.

[1003] 2. Skill Level Determination:

[1004] User: The user submits sample code to the system for skill evaluation.

[1005] Server: The server runs the submitted code and if the code runs correctly it will be judged as "Intermediate"; if an error occurs it will be judged as "Beginner".

[1006] 3. Automatic generation of programming problems:

[1007] Server: The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would generate a "program to add two numbers," and an intermediate would generate a "program to find factorials."

[1008] 4. Automatic answer checking and explanation generation:

[1009] User: The user submits the answer code to the system.

[1010] Server: The server examines the submitted code and compares it with a predefined answer. If it is correct, it marks it as "correct." If it is incorrect, it responds with a specific explanation.

[1011] 5. User weakness analysis and customization solutions:

[1012] Server: The server identifies the user's weak points based on the user's past answer data and the results of analyzing coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weak points."

[1013] Server: Next, it provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[1014] 6. Coding video analysis:

[1015] Users: Users record themselves coding and upload videos to the platform.

[1016] Server: The server analyzes the uploaded video using machine learning models to identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[1017] Computer Systems and Software

[1018] The server uses Flask to implement the web application and bcrypt to hash passwords. It also uses machine learning models to analyze user activity logs and coding videos. These pieces of software run on the robot's control unit and server.

[1019] Adding concrete examples and prompt sentence examples

[1020] As a specific example, there is the following usage.

[1021] 1. User Registration:

[1022] The user creates an account by entering a username and password.

[1023] 2. Login:

[1024] The user enters their login information and the server authenticates them.

[1025] 3. Skill Check:

[1026] Users submit sample code, and the server runs the code to determine skill level.

[1027] 4. Question submission:

[1028] The server generates "beginner" questions based on skill level and presents them to the user.

[1029] 5. Submitting and checking your answers:

[1030] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[1031] 6. Weakness analysis and customization problem provision:

[1032] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[1033] Prompt Sentence Examples

[1034] "Analyze the coding log for weakness analysis and generate optimal customization questions. If there are many errors related to loops or conditional branching, we will provide appropriate questions based on that."

[1035] This allows robot operators to receive program problems and specific feedback that are tailored to their skill level and weaknesses.

[1036] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1037] Step 1: User Registration

[1038] The user enters their username and password using a terminal and sends it to the server. This input data includes the user's identification and authentication information. The server hashes the received password using bcrypt and stores the user information along with the hashed password in the database. This registers a new user.

[1039] Input: Username, Password

[1040] Data processing: password hashing

[1041] Output: User information stored in the database

[1042] Step 2: User Login

[1043] The user enters the registered username and password using the terminal and sends them to the server. The server retrieves the corresponding user information from the database and verifies whether the sent password matches the hashed password. If the authentication is successful, the user can access the system.

[1044] Input: Username, Password

[1045] Data processing: Comparison with hashed passwords

[1046] Output: Login session started or error message

[1047] Step 3: Determine your skill level

[1048] Users submit sample code from their devices to the server for skill assessment. The server executes the submitted code, and if the code runs correctly, the user is judged as "intermediate," but if an error occurs, the user is judged as "beginner." The result of this assessment is saved in a database as the user's level.

[1049] Input: Sample code

[1050] Data processing: Executing code and determining results

[1051] Output: Updated skill level of the user

[1052] Step 4: Automatic generation of programming problems

[1053] The server automatically generates appropriate programming problems based on the user's skill level. For example, it generates a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners. The generated programming problems are provided to the user.

[1054] Input: User's skill level

[1055] Data manipulation: Generating programming problems

[1056] Output: Generated programming problem

[1057] Step 5: Check answers and automatically generate explanations

[1058] The user submits the answer code from their device to the server. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[1059] Input: Answer code

[1060] Data processing: Examining the code and comparing it to the correct answer

[1061] Output: Judgment result and explanation

[1062] Step 6: Analyze user weaknesses and provide customization solutions

[1063] The server identifies the user's weaknesses based on the analysis of the user's past answer data and coding videos. For example, if the user frequently makes mistakes regarding loops, it identifies that as a "weakness." It then generates customized programming problems to help the user overcome the identified weaknesses.

[1064] Input: Answer data, coding video

[1065] Data processing: Data analysis and weakness identification

[1066] Output: Customized Program Questions

[1067] Step 7: Analyzing the coding video

[1068] Users upload videos of themselves coding from their devices to the server. The server then analyzes the videos using machine learning models to identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[1069] Input: Coding video

[1070] Data processing: Video analysis using machine learning models

[1071] Output: Identified weaknesses and feedback

[1072] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1073] This invention is a system that automatically generates programming problems based on the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it analyzes the user's behavioral logs and coding videos to identify individual weaknesses and provides customized problems based on those weaknesses. By combining this with an emotion engine that recognizes the user's emotions, the user's learning experience becomes even more personalized and effective. Specific implementation methods for this system are described below.

[1074] User registration and login management

[1075] user:

[1076] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[1077] server:

[1078] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[1079] Determining a user's skill level

[1080] user:

[1081] Users submit sample code to the system for skill evaluation.

[1082] server:

[1083] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[1084] Automatic generation of programming problems

[1085] server:

[1086] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[1087] Automatic answer checking and explanation generation

[1088] user:

[1089] The user submits the answer code to the system.

[1090] server:

[1091] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[1092] User weakness analysis and customization

[1093] server:

[1094] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[1095] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[1096] Coding video analysis

[1097] user:

[1098] Users record themselves coding and upload videos to the platform.

[1099] server:

[1100] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[1101] Introducing the Emotion Engine

[1102] user:

[1103] As users code, they use a camera and microphone to record their facial expressions and voice.

[1104] server:

[1105] The server sends data acquired from the camera and microphone to the emotion engine, which analyzes the user's facial expressions and voice to recognize emotions in real time. This analysis can be done using, for example, facial recognition technology and voice analysis technology.

[1106] Emotion-Based Feedback

[1107] server:

[1108] The server adjusts the learning program based on the user's emotions, as detected by the emotion engine. For example, if the user is feeling stressed, it will provide easier problems or display encouraging messages.

[1109] Specific examples

[1110] Example 1: A beginner user registers and solves a problem

[1111] 1. User Registration

[1112] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[1113] 2. Log in

[1114] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[1115] 3. Skill Judgment

[1116] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[1117] 4. Providing questions

[1118] The server generates "beginner" questions based on skill level and presents them to the user.

[1119] 5. Submitting and checking your answers

[1120] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[1121] 6. Weakness analysis and customization problem provision

[1122] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[1123] 7. Analysis of coding videos

[1124] Users upload their own coding videos, which are then analyzed by the server using machine learning models. Based on the results, the server identifies the user's weaknesses and provides appropriate feedback.

[1125] 8. Leveraging Emotional Engines

[1126] The server recognizes the user's emotions from their facial expressions and voice and provides appropriate learning programs and feedback. For example, if the user shows signs of frustration, it provides them with questions of lower difficulty.

[1127] This process allows users to learn effectively at their own pace and overcome their weaknesses. The introduction of an emotional engine makes the learning experience more personalized and increases user motivation.

[1128] The processing flow will be explained below.

[1129] User registration and login management

[1130] User Registration

[1131] Step 1:

[1132] The user enters a username and password.

[1133] Step 2:

[1134] The terminal sends the entered username and password to the server.

[1135] Step 3:

[1136] The server extracts the username and password from the request data.

[1137] Step 4:

[1138] The server hashes the password using an algorithm such as SHA-256.

[1139] Step 5:

[1140] The server stores the username and hashed password in a database.

[1141] Step 6:

[1142] The server returns a registration success message to the terminal.

[1143] Log in

[1144] Step 1:

[1145] The user enters the registered username and password.

[1146] Step 2:

[1147] The terminal sends the entered username and password to the server.

[1148] Step 3:

[1149] The server extracts the username and password from the request data.

[1150] Step 4:

[1151] The server retrieves the corresponding hashed password from its database.

[1152] Step 5:

[1153] The server hashes the entered password and verifies that it matches the stored hashed password.

[1154] Step 6:

[1155] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[1156] Determining a user's skill level

[1157] Step 1:

[1158] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[1159] Step 2:

[1160] The device sends the sample code to the server.

[1161] Step 3:

[1162] The server receives the submitted code.

[1163] Step 4:

[1164] The server runs the code in a secure environment.

[1165] Step 5:

[1166] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[1167] Step 6:

[1168] The server returns the skill level evaluation result to the terminal.

[1169] Automatic generation of programming problems

[1170] Step 1:

[1171] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[1172] Step 2:

[1173] The server generates programming problems corresponding to skill levels.

[1174] Step 3:

[1175] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[1176] Step 4:

[1177] The server transmits the generated programming questions to the terminal.

[1178] Automatic answer checking and explanation generation

[1179] Step 1:

[1180] The user enters the answer code into the terminal and clicks the submit button.

[1181] Step 2:

[1182] The terminal sends the answer code to the server.

[1183] Step 3:

[1184] The server receives the submitted answer code.

[1185] Step 4:

[1186] The server compares the answer to predefined correct answers.

[1187] Step 5:

[1188] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[1189] Step 6:

[1190] The server sends the generated message and explanation to the terminal.

[1191] User weakness analysis and customization

[1192] Step 1:

[1193] The server analyzes the user's past answer data and behavioral logs.

[1194] Step 2:

[1195] The server identifies the user's weaknesses based on data analysis.

[1196] Step 3:

[1197] The server invokes logic to generate customized questions according to the identified weaknesses.

[1198] Step 4:

[1199] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[1200] Step 5:

[1201] The server transmits the generated customization question to the terminal.

[1202] Coding video analysis

[1203] Step 1:

[1204] Users upload videos of themselves coding to their devices.

[1205] Step 2:

[1206] The device sends the uploaded video to the server.

[1207] Step 3:

[1208] The server analyzes the received video using a machine learning model.

[1209] Step 4:

[1210] The server identifies behaviors and choke points during coding.

[1211] Step 5:

[1212] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[1213] Step 6:

[1214] The server transmits the generated feedback to the terminal.

[1215] Introducing the Emotion Engine

[1216] Step 1:

[1217] Set up your camera and microphone when you code.

[1218] Step 2:

[1219] The device collects data from the camera and microphone in real time and sends it to the emotion engine.

[1220] Step 3:

[1221] The server uses an emotion engine to analyze the user's facial expressions and voice in real time and recognize their emotions.

[1222] Emotion-Based Feedback

[1223] Step 1:

[1224] The server receives the analysis results from the emotion engine.

[1225] Step 2:

[1226] Based on the analysis results, the server generates customized feedback according to the user's emotions.

[1227] Step 3:

[1228] For example, if a user is feeling stressed, it offers a simple message of encouragement or an easier problem.

[1229] Step 4:

[1230] The server transmits feedback based on the generated emotion to the terminal.

[1231] This invention can improve the learning effect and reduce stress for users by providing real-time feedback and emotional support according to their learning progress.

[1232] Example 2

[1233] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1234] While conventional programmatic learning systems can generate questions and assess correct answers based on the user's skill level, they lack the ability to provide customized feedback based on a detailed analysis of each user's learning status and weaknesses. Furthermore, they are unable to adjust the learning program to take into account the user's emotional state, which means they are unable to adequately maintain learner motivation or reduce stress. Therefore, a new system that can provide a more effective and personalized learning experience is needed.

[1235] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for automatically generating programming problems based on the user's skill level, means for determining whether the answers submitted by the user are correct, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, and means for recognizing the user's emotions and adjusting the learning program. This makes it possible to provide problems and analyze weaknesses according to the user's skill level, and further to provide a personalized and effective learning experience by adjusting the learning based on the user's emotional state.

[1236] "User" refers to an individual or organization that uses this system to submit answers to programming problems and advance their learning.

[1237] "Skill level" is an indicator of a user's programming ability, and is classified as beginner, intermediate, etc. based on the results of executing the submitted sample code.

[1238] "Programming problems" refer to programming challenges that the system automatically generates for users to answer.

[1239] "Correctness determination" is the process of checking whether the program code submitted by the user matches a predefined answer and determining whether it is correct or incorrect.

[1240] "Behavior log" refers to a record of the operations and activities performed by a user on the system.

[1241] "Coding video" refers to video data that records the process of a user creating a program.

[1242] "Weaknesses" refer to areas or concepts where users frequently make mistakes when solving programming problems.

[1243] "Customized programming problems" refer to programming tasks that are individually designed based on the user's analysis results.

[1244] "Emotion" refers to the psychological or emotional state that a user experiences while coding.

[1245] "Adjusting the learning program" refers to changing the questions and feedback provided based on the user's learning progress and emotional state.

[1246] A "generative AI model" refers to an artificial intelligence method or system that automatically generates new questions, explanations, and feedback using natural language generation techniques, etc.

[1247] This invention is a system that automatically generates programming problems based on the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it analyzes the user's behavioral logs and coding videos to identify individual weaknesses and provides customized problems based on those weaknesses. By combining this with an emotion engine that recognizes the user's emotions, the user's learning experience becomes even more personalized and effective. Specific implementation methods for this system are described below.

[1248] User registration and login management

[1249] user:

[1250] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[1251] server:

[1252] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[1253] Determining a user's skill level

[1254] user:

[1255] Users submit sample code to the system for skill evaluation.

[1256] server:

[1257] The server runs the submitted code in an execution environment (e.g., Python execution environment), and if the code runs correctly, the user is judged as "intermediate," but if an error occurs, the user is judged as "beginner."

[1258] Automatic generation of programming problems

[1259] server:

[1260] The server automatically generates appropriate programming problems based on the user's skill level, using a generative AI model to create the problems. For example, a beginner might get a "program to add two numbers," while an intermediate learner might get a "program to find the factorial."

[1261] Automatic answer checking and explanation generation

[1262] user:

[1263] The user submits the answer code to the system.

[1264] server:

[1265] The server compares the submitted code with predefined answers. If the answer is correct, it is judged as "correct." If the answer is incorrect, a specific explanation is generated using a generative AI model and provided to the user.

[1266] User weakness analysis and customization

[1267] server:

[1268] The server analyzes the user's past answer data and behavioral logs to identify their weaknesses. Based on the identified weaknesses, the server then uses a generative AI model to generate appropriate, customized programming problems and provide them to the user. For example, if there are many incorrect answers about loops, the server generates programming problems with a "loop" theme.

[1269] Coding video analysis

[1270] user:

[1271] Users record themselves coding and upload videos to the platform.

[1272] server:

[1273] The server analyzes the uploaded video using machine learning models (e.g., YOLO or OpenCV) to identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[1274] Introducing the Emotion Engine

[1275] user:

[1276] As users code, they use a camera and microphone to record their facial expressions and voice.

[1277] server:

[1278] The server sends data acquired from the camera and microphone to the emotion engine, which analyzes the user's facial expressions and voice to recognize emotions in real time. This analysis uses facial recognition technology (e.g., OpenCV) and voice analysis technology (e.g., Microsoft Azure's Speech to Text).

[1279] Emotion-Based Feedback

[1280] server:

[1281] The server adjusts the learning program based on the user's emotions, as detected by the emotion engine. For example, if the user is feeling stressed, it will provide easier problems or display encouraging messages. This feedback is generated using a generative AI model.

[1282] Specific examples

[1283] Example 1: A beginner user registers and solves a problem:

[1284] 1. User Registration:

[1285] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[1286] 2. Login:

[1287] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[1288] 3. Skill Check:

[1289] Users submit sample code, and the server runs the code in an execution environment such as Python to determine their skill level. If the code runs without errors, they are considered intermediate.

[1290] 4. Question submission:

[1291] The server generates "beginner" questions based on skill level and presents them to the user.

[1292] 5. Submitting and checking your answers:

[1293] The user submits an answer, and the server compares it with predefined correct answers. If the answer is correct, a "correct" message is generated using a generative AI model, and if the answer is incorrect, a specific explanation is generated and provided to the user.

[1294] 6. Weakness analysis and customization problem provision:

[1295] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[1296] 7. Analysis of coding video:

[1297] Users upload their own coding videos, which are then analyzed by the server using machine learning models. Based on the results, the server identifies the user's weaknesses and provides appropriate feedback.

[1298] 8. Leveraging the Emotion Engine:

[1299] The server recognizes the user's emotions from their facial expressions and voice and provides appropriate learning programs and feedback. For example, if the user shows signs of frustration, it provides them with questions of lower difficulty.

[1300] Example prompts to input to the generative AI model

[1301] Example prompt sentence:

[1302] "Generate feedback and explanations for user-submitted code, assuming it's written in Python."

[1303] "If the emotion engine detects stress from the user's facial expression, generate an encouraging message for the user."

[1304] This process allows users to learn effectively at their own pace and overcome their weaknesses. The introduction of an emotional engine makes the learning experience more personalized and increases user motivation.

[1305] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1306] The flow of this system's program processing

[1307] Step 1: User Registration

[1308] Input: Username, Password

[1309] Output: Registration successful or error message

[1310] user:

[1311] Users access the system from their terminal and enter their username and password on the account creation screen.

[1312] server:

[1313] The server receives the username and password you entered.

[1314] The received password is hashed using a hashing algorithm (e.g., SHA-256).

[1315] The server checks the database to see if the entered username already exists, and if not, saves the new user information in the database.

[1316] Registration information such as username is stored in a database along with hashed passwords.

[1317] Step 2: Login authentication

[1318] Input: Username, Password

[1319] Output: Authentication successful or error message

[1320] user:

[1321] The user enters their username and password on the login screen.

[1322] server:

[1323] The server receives the username and password you entered.

[1324] The server retrieves the hashed password for that username from its database.

[1325] The entered password is hashed and compared with the information in the database.

[1326] If they match, the authentication is successful and the user is logged in to the system. If they do not match, an error message is returned.

[1327] Step 3: Skill Level Assessment

[1328] Input: Sample code

[1329] Output: Skill level judgment result (beginner, intermediate)

[1330] user:

[1331] Users submit sample code to the system for skill evaluation.

[1332] server:

[1333] The server receives sample code submitted by the user.

[1334] The server runs the sample code in an execution environment (e.g., Python execution environment).

[1335] Check the execution results and evaluate whether any errors occur.

[1336] If no error occurs, the player is judged as intermediate, and if an error occurs, the player is judged as beginner.

[1337] The result of the judgment is saved in the user's profile.

[1338] Step 4: Automatic generation of programming problems

[1339] Input: Skill Level

[1340] Output: Programming problem

[1341] server:

[1342] The server uses a generative AI model based on the user's skill level to automatically generate appropriate programming problems.

[1343] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[1344] The generated programming problem is presented to the user.

[1345] Step 5: Submit and check your answers

[1346] Input: Answer code

[1347] Output: Answer check result (correct answers, incorrect answers and explanations)

[1348] user:

[1349] The user submits the solution code to the system for the programming problem presented to them.

[1350] server:

[1351] The server receives the answer code submitted by the user.

[1352] The submitted answer code is compared with a predefined correct answer code.

[1353] If the answer is correct, it is judged as "correct," and if the answer is incorrect, a specific explanation is generated using a generative AI model.

[1354] Provide the results to the user.

[1355] Step 6: Analyze user weaknesses and provide customization solutions

[1356] Input: Past answer data, behavior log

[1357] Output: Custom programming questions

[1358] server:

[1359] The server analyzes the user's past answer data and behavioral logs.

[1360] Reinforcement learning algorithms and machine learning are used to identify weaknesses.

[1361] For example, if there are many loop-related errors, the generative AI model will generate customization problems related to the loop.

[1362] Providing users with customized programming problems.

[1363] Step 7: Analyzing the coding video

[1364] Input: Coding video

[1365] Output: Analysis results and feedback

[1366] user:

[1367] Users record themselves coding and upload videos to the platform.

[1368] server:

[1369] The server analyzes the coding videos uploaded by users.

[1370] Machine learning models (e.g., YOLO, OpenCV) are used to identify movements in the video and areas where errors frequently occur.

[1371] Based on the analysis results, we identify the user's weaknesses and provide appropriate feedback.

[1372] Step 8: Implementing the Emotion Engine and Feedback

[1373] Input: Video and audio data from the camera and microphone

[1374] Output: Emotion recognition results and feedback

[1375] user:

[1376] As users code, they use a camera and microphone to record their facial expressions and voice.

[1377] server:

[1378] The server analyzes the video and audio data acquired from the camera and microphone.

[1379] The emotion engine recognizes user emotions in real time using facial recognition technology (e.g., OpenCV) and voice analysis technology (e.g., Microsoft Azure's Speech to Text).

[1380] The learning program can be adjusted based on the perceived emotions, for example, providing easier problems or encouraging messages generated using a generative AI model if the user is feeling stressed.

[1381] (Application example 2)

[1382] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1383] While the demand for programming education is increasing, providing optimal learning programs tailored to each user's skill level is challenging. Furthermore, conventional systems are insufficient in assessing users' learning progress, identifying individual weaknesses, and providing customized feedback. Furthermore, to improve the user's learning experience, it is necessary to understand the user's emotional state in real time and provide appropriate support accordingly.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1385] In this invention, the server includes means for automatically generating programming problems based on the user's skill level, means for determining whether the answers submitted by the user are correct, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, and means for recognizing the user's emotions and adjusting the learning program. This makes it possible to automatically generate optimal programming problems according to the user's skill, identify the user's weaknesses, and provide customized feedback. Furthermore, by analyzing the user's emotional state in real time, the learning experience can be personalized and effective support can be provided.

[1386] "User skill level" refers to the user's current level of programming knowledge and ability.

[1387] "Automatic generation of programming problems" means automatically creating appropriate programming tasks based on the user's skill level.

[1388] "Means for determining whether the answer is correct" refers to the function of checking the correctness of the program submitted by the user and providing feedback on the results.

[1389] An "action log" is a record of a user's operation history and behavioral data when using the system.

[1390] A "coding video" is a video recording of a user writing a program.

[1391] "Means for analyzing weaknesses" refers to a function that analyzes user behavior logs and coding video data to identify areas in which the user is weak.

[1392] "Customized programming problems" refer to programming tasks that are specifically designed to overcome a user's analyzed weaknesses.

[1393] "Means of recognizing emotions" refers to technology for reading a user's psychological state from their facial expressions and voice.

[1394] "Means for adjusting the learning program" refers to the ability to change the learning content and difficulty level provided in real time depending on the user's emotional state and skill level.

[1395] In this invention, a system is constructed in which a server, a user terminal, and an emotion engine work together. The server automatically generates programming problems according to the user's skill level, judges whether the user's answers are correct, analyzes behavior logs and coding videos, provides customized problems, and adjusts the learning program based on the user's emotional state.

[1396] The server uses the following software to perform its processing:

[1397] Flask: A lightweight framework for providing server-side web services.

[1398] werkzeug.security: A security library used for password hashing and authentication.

[1399] emotion_recognition: A technology for identifying user emotions using an emotion analysis engine.

[1400] skill_evaluation: A module that determines skill levels and evaluates submitted program code.

[1401] video_analysis: A machine learning model that analyzes users' coding videos to identify behavioral trends and weaknesses.

[1402] The server analyzes data obtained from the user's submitted program code, behavioral logs, and coding videos to identify the user's weaknesses and learning progress. It also recognizes the user's emotional state in real time through an emotion engine, making it possible to adjust the difficulty of the learning program and provide appropriate feedback.

[1403] As a concrete example, the process of a user registering an account on a smartphone and solving a programming problem is shown below. The user uses a smartphone or smart glasses to submit programming code by voice input or keyboard input. The server then analyzes the submitted code, determines the user's skill level, and provides programming problems of an appropriate level of difficulty. The user answers the problem and submits it again to the server, where the answer is judged to be correct or incorrect, and if the answer is incorrect, a specific explanation is automatically generated.

[1404] Furthermore, users can record their own coding videos with the smart glasses and upload them to the platform, where the server analyzes the videos to identify the user's coding style and areas where they are stuck.The emotion engine then analyzes the user's facial expressions and voice data to recognize their emotions in real time, and provides feedback such as lowering the difficulty level or displaying encouraging messages if the user is feeling frustrated.

[1405] An example of a prompt would be, "We have recorded a user solving a programming problem. Please use a machine learning model to analyze the user's coding style, stuck points, and error patterns from this video. Also, please analyze the user's facial expressions and voice, and use an emotion engine to recognize their emotions in real time and provide appropriate feedback."

[1406] Through this process, users receive optimal learning content suited to their skill level, as well as personalized feedback based on their weaknesses and emotional state.

[1407] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1408] Step 1:

[1409] A user registers an account on a smartphone or smart glasses. The user enters a username and password, which are then sent to the server, which hashes the password and stores it in a database.

[1410] Input: Username, Password

[1411] Output: Hashed password, saved user account information

[1412] Step 2:

[1413] A user attempts to log in. They enter their username and password and submit them to the server, which authenticates them by comparing the entered password with a hashed password stored in a database.

[1414] Input: Username, Password

[1415] Output: Authentication result (success or failure)

[1416] Step 3:

[1417] The user submits sample code for skill level assessment. The user puts on the smart glasses and inputs the sample code via voice or text input. The server executes the submitted code and determines the skill level.

[1418] Input: Sample code

[1419] Output: Skill level judgment result (beginner, intermediate)

[1420] Step 4:

[1421] The server automatically generates programming problems appropriate for each skill level, such as a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners.

[1422] Input: Skill Level

[1423] Output: Auto-generated programming questions

[1424] Step 5:

[1425] The user answers the provided programming questions and submits them to the server. The user enters the answer code using a smartphone or smart glasses and sends it to the server. The server compares the submitted code with the correct answer and determines whether it is correct or incorrect.

[1426] Input: Answer code

[1427] Output: Correct / incorrect result, feedback (including explanation)

[1428] Step 6:

[1429] The server analyzes the user's behavior log and submitted answers to identify the user's weaknesses. For example, if there are many incorrect answers regarding loops, it will extract those weaknesses.

[1430] Input: User behavior log, answer history

[1431] Output: Identified weaknesses

[1432] Step 7:

[1433] Users record their coding videos using smart glasses and upload them to a server, which then uses machine learning models to analyze the videos and identify the user's coding style and stumbling points.

[1434] Input: Coding video

[1435] Output: Analysis results (blockage points, error patterns)

[1436] Step 8:

[1437] The emotion engine recognizes the user's emotional state in real time from their facial expressions and voice. The user captures data using the camera and microphone in the smart glasses and sends it to the server. The server then performs emotion analysis and adjusts the learning program.

[1438] Input: facial expression data, voice data

[1439] Output: emotional state, adjusted learning program

[1440] Step 9:

[1441] The server provides customized feedback and programming problems based on the analyzed weaknesses and emotional state, and displays difficulty-adjusted problems and encouraging messages to reduce user frustration.

[1442] Input: Identified weaknesses, emotional state

[1443] Output: Customized feedback, programming problems

[1444] As a concrete example, the prompt sentence is as follows: "We have recorded a user solving a programming problem. Please use a machine learning model to analyze the user's coding style, stuck points, and error patterns from this video. Also, please analyze the user's facial expressions and voice, and use an emotion engine to recognize their emotions in real time and provide appropriate feedback."

[1445] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1446] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1447] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1448] [Third embodiment]

[1449] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1450] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1451] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1452] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1453] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1454] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1455] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1456] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1457] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1458] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1459] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1460] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1461] This invention is a system that automatically generates programming problems according to the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze the user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows.

[1462] User registration and login management

[1463] user:

[1464] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[1465] server:

[1466] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[1467] Determining a user's skill level

[1468] user:

[1469] Users submit sample code to the system for skill evaluation.

[1470] server:

[1471] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[1472] Automatic generation of programming problems

[1473] server:

[1474] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[1475] Automatic answer checking and explanation generation

[1476] user:

[1477] The user submits the answer code to the system.

[1478] server:

[1479] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[1480] User weakness analysis and customization

[1481] server:

[1482] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[1483] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[1484] Coding video analysis

[1485] user:

[1486] Users record themselves coding and upload videos to the platform.

[1487] server:

[1488] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[1489] Specific examples

[1490] Example 1: A beginner user registers and solves a problem

[1491] 1. User Registration

[1492] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[1493] 2. Log in

[1494] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[1495] 3. Skill Judgment

[1496] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[1497] 4. Providing questions

[1498] The server generates "beginner" questions based on skill level and presents them to the user.

[1499] 5. Submitting and checking your answers

[1500] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[1501] 6. Weakness analysis and customization problem provision

[1502] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[1503] This system provides an environment where beginners can effectively learn programming at their own pace. It dramatically improves learning efficiency by providing questions and explanations optimized based on the user's skill level and weaknesses.

[1504] The processing flow will be explained below.

[1505] User registration and login management

[1506] User Registration

[1507] Step 1:

[1508] The user enters a username and password.

[1509] Step 2:

[1510] The terminal sends the entered username and password to the server.

[1511] Step 3:

[1512] The server extracts the username and password from the request data.

[1513] Step 4:

[1514] The server hashes the password using an algorithm such as SHA-256.

[1515] Step 5:

[1516] The server stores the username and hashed password in a database.

[1517] Step 6:

[1518] The server returns a registration success message to the terminal.

[1519] Log in

[1520] Step 1:

[1521] The user enters the registered username and password.

[1522] Step 2:

[1523] The terminal sends the entered username and password to the server.

[1524] Step 3:

[1525] The server extracts the username and password from the request data.

[1526] Step 4:

[1527] The server retrieves the corresponding hashed password from its database.

[1528] Step 5:

[1529] The server hashes the entered password and verifies that it matches the stored hashed password.

[1530] Step 6:

[1531] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[1532] Determining a user's skill level

[1533] Step 1:

[1534] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[1535] Step 2:

[1536] The device sends the sample code to the server.

[1537] Step 3:

[1538] The server receives the submitted code.

[1539] Step 4:

[1540] The server runs the code in a secure environment.

[1541] Step 5:

[1542] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[1543] Step 6:

[1544] The server returns the skill level evaluation result to the terminal.

[1545] Automatic generation of programming problems

[1546] Step 1:

[1547] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[1548] Step 2:

[1549] The server generates programming problems corresponding to skill levels.

[1550] Step 3:

[1551] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[1552] Step 4:

[1553] The server transmits the generated programming questions to the terminal.

[1554] Automatic answer checking and explanation generation

[1555] Step 1:

[1556] The user enters the answer code into the terminal and clicks the submit button.

[1557] Step 2:

[1558] The terminal sends the answer code to the server.

[1559] Step 3:

[1560] The server receives the submitted answer code.

[1561] Step 4:

[1562] The server compares the answer to predefined correct answers.

[1563] Step 5:

[1564] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[1565] Step 6:

[1566] The server sends the generated message and explanation to the terminal.

[1567] User weakness analysis and customization

[1568] Step 1:

[1569] The server analyzes the user's past answer data and behavioral logs.

[1570] Step 2:

[1571] The server identifies user weaknesses (e.g., loops, conditional branches) based on data analysis.

[1572] Step 3:

[1573] The server invokes logic to generate customized questions according to the identified weaknesses.

[1574] Step 4:

[1575] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[1576] Step 5:

[1577] The server transmits the generated customization question to the terminal.

[1578] Coding video analysis

[1579] Step 1:

[1580] Users upload videos of themselves coding to their devices.

[1581] Step 2:

[1582] The device sends the uploaded video to the server.

[1583] Step 3:

[1584] The server analyzes the received video using a machine learning model.

[1585] Step 4:

[1586] The server identifies behaviors and choke points during coding.

[1587] Step 5:

[1588] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[1589] Step 6:

[1590] The server transmits the generated feedback to the terminal.

[1591] As a result, this fully automatic programming learning system helps users to learn effectively at their own pace.

[1592] Example 1

[1593] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1594] Conventional programming problem-solving systems have difficulty automatically generating appropriate problems based on the user's skill level and weaknesses, resulting in reduced learning efficiency. Additionally, there are generally delays in determining whether an answer is correct and in providing feedback, preventing users from checking their learning progress in real time. Furthermore, while analyzing coding videos, rather than relying solely on traditional behavioral log analysis, would enable more detailed and accurate feedback to be provided, few such systems exist.

[1595] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1596] In this invention, the server includes a means for receiving user authentication information and hashing the password to ensure security, a means for automatically generating programming problems based on the user's skill level, and a means for determining whether the answers submitted by the user are correct, thereby enabling the provision of appropriate programming problems according to the user's skill level and fast and accurate feedback.

[1597] "User Credentials" means the username and password or other authentication credentials used to access a system.

[1598] "Password hashing" is the process of encrypting a user-entered password using a specific algorithm to ensure system security.

[1599] "Skill level" is an indicator of a user's programming skill and knowledge. For example, there is a distinction between "beginner" and "intermediate."

[1600] "Automatic programming problem generation" is the process by which the system automatically creates appropriate programming challenges based on the user's skill level.

[1601] The "means for determining whether the answer submitted by the user is correct" is a means for checking whether the program code submitted by the user is executed according to the specifications and determining whether the answer is appropriate.

[1602] An "action log" is a record of the operations and input a user performs on the system.

[1603] "Coding videos" are video data that record the process of a user programming.

[1604] "Weakness analysis" is the process of analyzing user behavior logs and coding videos to identify areas where users are particularly weak or frequently make errors.

[1605] "Customized programming problems" are personalized programming challenges generated based on a user's identified weaknesses.

[1606] This system automatically generates programming problems based on a user's skill level and judges the accuracy of the user's submitted solutions. Furthermore, it can analyze a user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows:

[1607] User registration and login management

[1608] A user creates an account and enters a username and password to log in to the system. The server receives the user's registration information and hashes and stores the password to ensure security. Also, when logging in, the entered password is compared with the hashed password for authentication. This maintains a high level of security.

[1609] Determining a user's skill level

[1610] Users submit sample code for skill assessment. The server runs the submitted code in a sandbox environment and determines the skill level based on whether an error occurs. For example, if no errors occur, the user is assessed as "intermediate," and if an error occurs, the user is assessed as "beginner."

[1611] Automatic generation of programming problems

[1612] The server automatically generates appropriate programming problems using a generative AI model based on the user's skill level. For example, it generates a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners. The generated problems are then presented to the user.

[1613] Automatic answer checking and explanation generation

[1614] The user submits the answer code. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct." If the answer is incorrect, it responds with a specific explanation. This allows the user to receive instant feedback.

[1615] User weakness analysis and customization

[1616] The server analyzes the user's past answer data and behavioral logs to identify the user's weaknesses. For example, if there are many incorrect answers regarding loops or conditional branching, it identifies these as "weaknesses." Based on the analysis results, the server provides customized programming problems to help the user overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates the corresponding programming problem.

[1617] Coding video analysis

[1618] Users upload videos of themselves coding to the platform. The server then uses machine learning models to analyze the uploaded videos and identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the platform identifies the user's weaknesses and provides appropriate feedback.

[1619] Specific examples

[1620] 1. Example of a beginner user registering and solving a problem

[1621] Create an account: The user creates an account by entering a username and password. The server hashes the password using the bcrypt library and stores it in the database along with the user information.

[1622] Login: The user enters a username and password. The server retrieves the hashed password for that username from the database and compares it with the entered password. If authentication is successful, the user is allowed to access the system.

[1623] Skill assessment: Users submit sample code. The server runs the submitted code in a Docker container and checks the error log to determine the skill level. For example, if sample code such as "print(2 + 2)" can be executed without error, the user is assessed as intermediate.

[1624] Problem provision: The server requests a problem for beginners from the AI ​​model and receives a program to add two numbers. The server then presents the generated problem to the user.

[1625] Answer submission and verification: The user submits an answer. The server compares the answer with predefined correct answers in a Docker container. If the answer is correct, the server returns a "correct" message. If the answer is incorrect, the server returns feedback including the specific error location.

[1626] Weakness analysis and provision of customized problems: The server analyzes the user's answer history and behavior log to identify loops as particularly difficult. It then asks the generative AI model to generate customized problems related to loops and provides them to the user. For example, it generates a problem such as "Create a program to calculate the sum of numbers from 1 to 10."

[1627] Prompt Sentence Examples

[1628] Prompt: Please generate a sample problem for a simple Python program for beginners. Also, please provide an explanation for this problem.

[1629] Example response of the generative AI model:

[1630] Question: Write a program that takes two integers as input and calculates and displays their sum.

[1631] Explanation: In this problem, we will use the input() function to read two integers from the user, add them, and display the result using the print() function. See the code below for an example.

[1632] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1633] Step 1: User Registration

[1634] User: A user creates an account by entering a username and password.

[1635] Input: Username and Password

[1636] Server: The server processes the received user information and hashes the password, specifically encrypting it using the bcrypt library.

[1637] Data manipulation: password hashing

[1638] Output: User information including hashed password

[1639] Server: The server stores this information in a database, specifically by inserting the hashed password and username into the database.

[1640] Step 2: Login authentication

[1641] User: The user enters their username and password into a login form.

[1642] Input: Username and Password

[1643] Server: The server retrieves the hashed password for the given username from its database.

[1644] Data manipulation: Obtaining a hashed password corresponding to a username

[1645] Output: Hashed password corresponding to username

[1646] Server: The server compares the entered password with the hashed password retrieved from the database.

[1647] Data calculation: Password verification

[1648] Output: Authentication result

[1649] Server: If authentication is successful, it generates session information and responds to the user. This includes generating a session ID and storing it with the corresponding user information.

[1650] Step 3: Submit your skill evaluation code

[1651] User: The user writes a sample code for skill evaluation and presses the submit button.

[1652] Input: Sample code

[1653] Server: The server receives the submitted code and executes it in a sandbox environment (Docker container).

[1654] Data processing: Running the sample code

[1655] Output: Execution results and error log

[1656] Server: Based on the execution results, if no errors occur, the user is judged as "Intermediate", and if an error occurs, the user is judged as "Beginner". This operation analyzes the error log and judges the skill level.

[1657] Step 4: Problem generation

[1658] Server: The server sends appropriate prompts to the generative AI model based on the user's skill level.

[1659] Input: Skill level and prompt

[1660] Generative AI model: A generative AI model generates programming problems based on prompts.

[1661] Data processing: Generating programming problems

[1662] Output: The generated programming problem

[1663] Server: Presents the generated questions to the user. This includes displaying the generated questions in a user interface.

[1664] Step 5: Submit your answer

[1665] User: The user writes the answer code for the displayed programming problem and presses the submit button.

[1666] Input: Answer code

[1667] Server: The server receives the submitted code and compares it with pre-stored correct answers.

[1668] Data processing: Analysis and comparison of submitted solution codes

[1669] Output: Correct / incorrect result and feedback

[1670] Server: If the answer is correct, it will be deemed "correct" and generate and return a feedback message. If the answer is incorrect, it will provide feedback with a specific error message and suggestions for improvement. This action includes generating the feedback and responding to the user.

[1671] Step 6: Weakness analysis

[1672] Server: The server analyzes past answer data and behavioral logs to identify typical user error patterns and coding pitfalls.

[1673] Input: Past answer data and behavior log

[1674] Data processing: Error pattern analysis and statistical analysis

[1675] Output: Weakness identification results

[1676] Server: Based on the analysis results, the server requests the generative AI model to create customized questions that address the user's identified weaknesses. This action involves sending prompts to the generative AI model.

[1677] Step 7: Generate customization questions

[1678] Generative AI model: Based on the results of weakness analysis, the generative AI model generates personalized programming challenges for users to overcome.

[1679] Input: Weakness analysis results and prompt text

[1680] Data manipulation: custom problem generation

[1681] Output: Customized programming problems

[1682] Server: Presents the generated customization questions to the user. This includes displaying the customization questions in the user interface.

[1683] Step 8: Upload and analyze your video

[1684] Users: Users record themselves coding and upload videos to the platform.

[1685] Input: Coding video

[1686] Server: The server receives the uploaded video and requests the machine learning model to analyze it.

[1687] Data processing: Video data analysis

[1688] Output: Video analysis results

[1689] Machine learning model: The machine learning model analyzes the video and identifies points where users get stuck and where they frequently make errors.

[1690] Server: Generates specific feedback based on the video analysis results and provides it to the user. This operation includes generating feedback and responding to the user.

[1691] (Application example 1)

[1692] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1693] Modern factories require robot operators to learn robot control programs efficiently and effectively. However, it is difficult to provide customized learning content tailored to each operator's skill level and weaknesses. Furthermore, conventional programming education systems are unable to provide specific feedback or customized problems based on actual code writing or execution results, preventing improvements in learning efficiency. Furthermore, there is a lack of mechanisms for identifying problems operators encounter when operating robots and providing support based on those problems.

[1694] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1695] In this invention, the server includes means for automatically generating programming problems based on the skill level of a user, means for determining the correctness of answers submitted by the user, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, means for generating program problems for robot operators to improve the user's skills, and an application to be installed on robots used in factories. This makes it possible to provide customized program problems and specific feedback according to the skill level and weaknesses of robot operators.

[1696] "User" refers to a person who uses this system to solve programming problems.

[1697] "Skill level" is a standard for evaluating a user's programming ability and proficiency.

[1698] "Programming Problems" are tasks provided by the system to assess or improve programming skills.

[1699] "Automatic generation" refers to the system automatically creating programming problems according to the user's skill level.

[1700] "Solution" refers to the source code or program submitted by a user in response to a programming problem.

[1701] "Correctness determination" means that the system evaluates whether the submitted answer is correct or incorrect.

[1702] An "action log" refers to the operation history and activity record when a user uses the system.

[1703] A "coding video" is a video that records the process of a user creating a program.

[1704] "Weaknesses" are specific techniques or concepts that users have difficulty with or make repeated mistakes with.

[1705] "Analysis" refers to extracting specific information and patterns based on user behavior logs and coding videos.

[1706] "Customized programming problems" are programming problems that are tailored to each user to overcome their specific weaknesses.

[1707] A "robot operator" refers to a person whose job is to operate robots in a factory and control their movements.

[1708] A "factory" is a production facility for manufacturing products.

[1709] An "application" is software that realizes the functions provided by a system.

[1710] This invention is a system that automatically generates programming problems based on a user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze a user's behavioral log and coding video to identify individual weaknesses and provide customized problems based on those weaknesses. This system is particularly applicable to programming problems for robot operators used in factories. The following describes specific embodiments of the invention.

[1711] System Configuration

[1712] The server manages a system that includes the following elements:

[1713] 1. User registration and login management:

[1714] User: A user creates an account and logs into the system. When creating an account, the user registers by entering a username and password. When logging in, the user enters their authentication information to access the system.

[1715] Server: The server receives the user's registration information and hashes the password to ensure security. When logging in, the server compares the entered password with the hashed password to authenticate.

[1716] 2. Skill Level Assessment:

[1717] User: The user submits sample code to the system for skill evaluation.

[1718] Server: The server runs the submitted code and if the code runs correctly it will be judged as "Intermediate"; if an error occurs it will be judged as "Beginner".

[1719] 3. Automatic generation of programming problems:

[1720] Server: The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would generate a "program to add two numbers," and an intermediate would generate a "program to find factorials."

[1721] 4. Automatic answer checking and explanation generation:

[1722] User: The user submits the answer code to the system.

[1723] Server: The server examines the submitted code and compares it with a predefined answer. If it is correct, it marks it as "correct." If it is incorrect, it responds with a specific explanation.

[1724] 5. User weakness analysis and customization solutions:

[1725] Server: The server identifies the user's weak points based on the user's past answer data and the results of analyzing coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weak points."

[1726] Server: Next, it provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[1727] 6. Coding video analysis:

[1728] Users: Users record themselves coding and upload videos to the platform.

[1729] Server: The server analyzes the uploaded video using machine learning models to identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[1730] Computer Systems and Software

[1731] The server uses Flask to implement web applications, bcrypt to hash passwords, and machine learning models to analyze user activity logs and coding videos. These pieces of software run on the robot's control unit and server.

[1732] Adding concrete examples and prompt sentence examples

[1733] As a specific example, there is the following usage.

[1734] 1. User Registration:

[1735] The user creates an account by entering a username and password.

[1736] 2. Login:

[1737] The user enters their login information and the server performs authentication.

[1738] 3. Skill Check:

[1739] Users submit sample code, and the server runs the code to determine skill level.

[1740] 4. Question submission:

[1741] The server generates "beginner" questions based on skill level and presents them to the user.

[1742] 5. Submitting and checking your answers:

[1743] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[1744] 6. Weakness analysis and customization problem provision:

[1745] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[1746] Prompt Sentence Examples

[1747] "Analyze the coding log for weakness analysis and generate optimal customization questions. If there are many errors related to loops or conditional branching, we will provide appropriate questions based on that."

[1748] This allows robot operators to receive program problems and specific feedback that are tailored to their skill level and weaknesses.

[1749] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1750] Step 1: User Registration

[1751] The user enters their username and password using a terminal and sends it to the server. This input data includes the user's identification and authentication information. The server hashes the received password using bcrypt and stores the user information along with the hashed password in the database. This registers a new user.

[1752] Input: Username, Password

[1753] Data processing: password hashing

[1754] Output: User information stored in the database

[1755] Step 2: User Login

[1756] The user enters the registered username and password using the terminal and sends them to the server. The server retrieves the corresponding user information from the database and verifies whether the sent password matches the hashed password. If the authentication is successful, the user can access the system.

[1757] Input: Username, Password

[1758] Data processing: Comparison with hashed passwords

[1759] Output: Login session started or error message

[1760] Step 3: Determine your skill level

[1761] Users submit sample code from their devices to the server for skill assessment. The server executes the submitted code, and if the code runs correctly, the user is judged as "intermediate," but if an error occurs, the user is judged as "beginner." The result of this assessment is saved in a database as the user's level.

[1762] Input: Sample code

[1763] Data processing: Executing code and determining results

[1764] Output: Updated skill level of the user

[1765] Step 4: Automatic generation of programming problems

[1766] The server automatically generates appropriate programming problems based on the user's skill level. For example, it generates a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners. The generated programming problems are provided to the user.

[1767] Input: User's skill level

[1768] Data manipulation: Generating programming problems

[1769] Output: Generated programming problem

[1770] Step 5: Check answers and automatically generate explanations

[1771] The user submits the answer code from their device to the server. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[1772] Input: Answer code

[1773] Data processing: Examining the code and comparing it to the correct answer

[1774] Output: Judgment result and explanation

[1775] Step 6: Analyze user weaknesses and provide customization solutions

[1776] The server identifies the user's weaknesses based on the analysis of the user's past answer data and coding videos. For example, if the user frequently makes mistakes regarding loops, it identifies that as a "weakness." It then generates customized programming problems to help the user overcome the identified weaknesses.

[1777] Input: Answer data, coding video

[1778] Data processing: Data analysis and weakness identification

[1779] Output: Customized Program Questions

[1780] Step 7: Analyzing the coding video

[1781] Users upload videos of themselves coding from their devices to the server. The server then analyzes the videos using machine learning models to identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[1782] Input: Coding video

[1783] Data processing: Video analysis using machine learning models

[1784] Output: Identified weaknesses and feedback

[1785] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1786] This invention is a system that automatically generates programming problems based on the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it analyzes the user's behavioral logs and coding videos to identify individual weaknesses and provides customized problems based on those weaknesses. By combining this with an emotion engine that recognizes the user's emotions, the user's learning experience becomes even more personalized and effective. Specific implementation methods for this system are described below.

[1787] User registration and login management

[1788] user:

[1789] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[1790] server:

[1791] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[1792] Determining a user's skill level

[1793] user:

[1794] Users submit sample code to the system for skill evaluation.

[1795] server:

[1796] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[1797] Automatic generation of programming problems

[1798] server:

[1799] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[1800] Automatic answer checking and explanation generation

[1801] user:

[1802] The user submits the answer code to the system.

[1803] server:

[1804] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[1805] User weakness analysis and customization

[1806] server:

[1807] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[1808] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[1809] Coding video analysis

[1810] user:

[1811] Users record themselves coding and upload videos to the platform.

[1812] server:

[1813] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[1814] Introducing the Emotion Engine

[1815] user:

[1816] As users code, they use a camera and microphone to record their facial expressions and voice.

[1817] server:

[1818] The server sends data acquired from the camera and microphone to the emotion engine, which analyzes the user's facial expressions and voice to recognize emotions in real time. This analysis can be done using, for example, facial recognition technology and voice analysis technology.

[1819] Emotion-Based Feedback

[1820] server:

[1821] The server adjusts the learning program based on the user's emotions, as detected by the emotion engine. For example, if the user is feeling stressed, it will provide easier problems or display encouraging messages.

[1822] Specific examples

[1823] Example 1: A beginner user registers and solves a problem

[1824] 1. User Registration

[1825] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[1826] 2. Log in

[1827] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[1828] 3. Skill Judgment

[1829] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[1830] 4. Providing questions

[1831] The server generates "beginner" questions based on skill level and presents them to the user.

[1832] 5. Submitting and checking your answers

[1833] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[1834] 6. Weakness analysis and customization problem provision

[1835] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[1836] 7. Analysis of coding videos

[1837] Users upload their own coding videos, which are then analyzed by the server using machine learning models. Based on the results, the server identifies the user's weaknesses and provides appropriate feedback.

[1838] 8. Leveraging Emotional Engines

[1839] The server recognizes the user's emotions from their facial expressions and voice and provides appropriate learning programs and feedback. For example, if the user shows signs of frustration, it provides them with questions of lower difficulty.

[1840] This process allows users to learn effectively at their own pace and overcome their weaknesses. The introduction of an emotional engine makes the learning experience more personalized and increases user motivation.

[1841] The processing flow will be explained below.

[1842] User registration and login management

[1843] User Registration

[1844] Step 1:

[1845] The user enters a username and password.

[1846] Step 2:

[1847] The terminal sends the entered username and password to the server.

[1848] Step 3:

[1849] The server extracts the username and password from the request data.

[1850] Step 4:

[1851] The server hashes the password using an algorithm such as SHA-256.

[1852] Step 5:

[1853] The server stores the username and hashed password in a database.

[1854] Step 6:

[1855] The server returns a registration success message to the terminal.

[1856] Log in

[1857] Step 1:

[1858] The user enters the registered username and password.

[1859] Step 2:

[1860] The terminal sends the entered username and password to the server.

[1861] Step 3:

[1862] The server extracts the username and password from the request data.

[1863] Step 4:

[1864] The server retrieves the corresponding hashed password from its database.

[1865] Step 5:

[1866] The server hashes the entered password and verifies that it matches the stored hashed password.

[1867] Step 6:

[1868] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[1869] Determining a user's skill level

[1870] Step 1:

[1871] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[1872] Step 2:

[1873] The device sends the sample code to the server.

[1874] Step 3:

[1875] The server receives the submitted code.

[1876] Step 4:

[1877] The server runs the code in a secure environment.

[1878] Step 5:

[1879] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[1880] Step 6:

[1881] The server returns the skill level evaluation result to the terminal.

[1882] Automatic generation of programming problems

[1883] Step 1:

[1884] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[1885] Step 2:

[1886] The server generates programming problems corresponding to skill levels.

[1887] Step 3:

[1888] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[1889] Step 4:

[1890] The server transmits the generated programming questions to the terminal.

[1891] Automatic answer checking and explanation generation

[1892] Step 1:

[1893] The user enters the answer code into the terminal and clicks the submit button.

[1894] Step 2:

[1895] The terminal sends the answer code to the server.

[1896] Step 3:

[1897] The server receives the submitted answer code.

[1898] Step 4:

[1899] The server compares the answer to predefined correct answers.

[1900] Step 5:

[1901] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[1902] Step 6:

[1903] The server sends the generated message and explanation to the terminal.

[1904] User weakness analysis and customization

[1905] Step 1:

[1906] The server analyzes the user's past answer data and behavioral logs.

[1907] Step 2:

[1908] The server identifies the user's weaknesses based on data analysis.

[1909] Step 3:

[1910] The server invokes logic to generate customized questions according to the identified weaknesses.

[1911] Step 4:

[1912] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[1913] Step 5:

[1914] The server transmits the generated customization question to the terminal.

[1915] Coding video analysis

[1916] Step 1:

[1917] Users upload videos of themselves coding to their devices.

[1918] Step 2:

[1919] The device sends the uploaded video to the server.

[1920] Step 3:

[1921] The server analyzes the received video using a machine learning model.

[1922] Step 4:

[1923] The server identifies behaviors and choke points during coding.

[1924] Step 5:

[1925] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[1926] Step 6:

[1927] The server transmits the generated feedback to the terminal.

[1928] Introducing the Emotion Engine

[1929] Step 1:

[1930] Set up your camera and microphone when you code.

[1931] Step 2:

[1932] The device collects data from the camera and microphone in real time and sends it to the emotion engine.

[1933] Step 3:

[1934] The server uses an emotion engine to analyze the user's facial expressions and voice in real time and recognize their emotions.

[1935] Emotion-Based Feedback

[1936] Step 1:

[1937] The server receives the analysis results from the emotion engine.

[1938] Step 2:

[1939] Based on the analysis results, the server generates customized feedback according to the user's emotions.

[1940] Step 3:

[1941] For example, if a user is feeling stressed, it offers a simple message of encouragement or an easier problem.

[1942] Step 4:

[1943] The server transmits feedback based on the generated emotion to the terminal.

[1944] This invention can improve the learning effect and reduce stress for users by providing real-time feedback and emotional support according to their learning progress.

[1945] Example 2

[1946] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1947] While conventional programmatic learning systems can generate questions and assess correct answers based on the user's skill level, they lack the ability to provide customized feedback based on a detailed analysis of each user's learning status and weaknesses. Furthermore, they are unable to adjust the learning program to take into account the user's emotional state, which means they are unable to adequately maintain learner motivation or reduce stress. Therefore, a new system that can provide a more effective and personalized learning experience is needed.

[1948] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for automatically generating programming problems based on the user's skill level, means for determining whether the answers submitted by the user are correct, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, and means for recognizing the user's emotions and adjusting the learning program. This makes it possible to provide problems and analyze weaknesses according to the user's skill level, and further to provide a personalized and effective learning experience by adjusting the learning based on the user's emotional state.

[1949] "User" refers to an individual or organization that uses this system to submit answers to programming problems and advance their learning.

[1950] "Skill level" is an indicator of a user's programming ability, and is classified as beginner, intermediate, etc. based on the results of executing the submitted sample code.

[1951] "Programming problems" refer to programming challenges that the system automatically generates for users to answer.

[1952] "Correctness determination" is the process of checking whether the program code submitted by the user matches a predefined answer and determining whether it is correct or incorrect.

[1953] "Behavior log" refers to a record of the operations and activities performed by a user on the system.

[1954] "Coding video" refers to video data that records the process of a user creating a program.

[1955] "Weaknesses" refer to areas or concepts where users frequently make mistakes when solving programming problems.

[1956] "Customized programming problems" refer to programming tasks that are individually designed based on the user's analysis results.

[1957] "Emotion" refers to the psychological or emotional state that a user experiences while coding.

[1958] "Adjusting the learning program" refers to changing the questions and feedback provided based on the user's learning progress and emotional state.

[1959] A "generative AI model" refers to an artificial intelligence method or system that automatically generates new questions, explanations, and feedback using natural language generation techniques, etc.

[1960] This invention is a system that automatically generates programming problems based on the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it analyzes the user's behavioral logs and coding videos to identify individual weaknesses and provides customized problems based on those weaknesses. By combining this with an emotion engine that recognizes the user's emotions, the user's learning experience becomes even more personalized and effective. Specific implementation methods for this system are described below.

[1961] User registration and login management

[1962] user:

[1963] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[1964] server:

[1965] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[1966] Determining a user's skill level

[1967] user:

[1968] Users submit sample code to the system for skill evaluation.

[1969] server:

[1970] The server runs the submitted code in an execution environment (e.g., Python execution environment), and if the code runs correctly, the user is judged as "intermediate," but if an error occurs, the user is judged as "beginner."

[1971] Automatic generation of programming problems

[1972] server:

[1973] The server automatically generates appropriate programming problems based on the user's skill level, using a generative AI model to create the problems. For example, a beginner might get a "program to add two numbers," while an intermediate learner might get a "program to find the factorial."

[1974] Automatic answer checking and explanation generation

[1975] user:

[1976] The user submits the answer code to the system.

[1977] server:

[1978] The server compares the submitted code with predefined answers. If the answer is correct, it is judged as "correct." If the answer is incorrect, a specific explanation is generated using a generative AI model and provided to the user.

[1979] User weakness analysis and customization

[1980] server:

[1981] The server analyzes the user's past answer data and behavioral logs to identify their weaknesses. Based on the identified weaknesses, the server then uses a generative AI model to generate appropriate, customized programming problems and provide them to the user. For example, if there are many incorrect answers about loops, the server generates programming problems with a "loop" theme.

[1982] Coding video analysis

[1983] user:

[1984] Users record themselves coding and upload videos to the platform.

[1985] server:

[1986] The server analyzes the uploaded video using machine learning models (e.g., YOLO or OpenCV) to identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[1987] Introducing the Emotion Engine

[1988] user:

[1989] As users code, they use a camera and microphone to record their facial expressions and voice.

[1990] server:

[1991] The server sends data acquired from the camera and microphone to the emotion engine, which analyzes the user's facial expressions and voice to recognize emotions in real time. This analysis uses facial recognition technology (e.g., OpenCV) and voice analysis technology (e.g., Microsoft Azure's Speech to Text).

[1992] Emotion-Based Feedback

[1993] server:

[1994] The server adjusts the learning program based on the user's emotions, as detected by the emotion engine. For example, if the user is feeling stressed, it will provide easier problems or display encouraging messages. This feedback is generated using a generative AI model.

[1995] Specific examples

[1996] Example 1: A beginner user registers and solves a problem:

[1997] 1. User Registration:

[1998] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[1999] 2. Login:

[2000] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[2001] 3. Skill Check:

[2002] Users submit sample code, and the server runs the code in an execution environment such as Python to determine their skill level. If the code runs without errors, they are considered intermediate.

[2003] 4. Question submission:

[2004] The server generates "beginner" questions based on skill level and presents them to the user.

[2005] 5. Submitting and checking your answers:

[2006] The user submits an answer, and the server compares it with predefined correct answers. If the answer is correct, a "correct" message is generated using a generative AI model, and if the answer is incorrect, a specific explanation is generated and provided to the user.

[2007] 6. Weakness analysis and customization problem provision:

[2008] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[2009] 7. Analysis of coding video:

[2010] Users upload their own coding videos, which are then analyzed by the server using machine learning models. Based on the results, the server identifies the user's weaknesses and provides appropriate feedback.

[2011] 8. Leveraging the Emotion Engine:

[2012] The server recognizes the user's emotions from their facial expressions and voice and provides appropriate learning programs and feedback. For example, if the user shows signs of frustration, it provides them with questions of lower difficulty.

[2013] Example prompts to input to the generative AI model

[2014] Example prompt sentence:

[2015] "Generate feedback and explanations for user-submitted code, assuming it's written in Python."

[2016] "If the emotion engine detects stress from the user's facial expression, generate an encouraging message for the user."

[2017] This process allows users to learn effectively at their own pace and overcome their weaknesses. The introduction of an emotional engine makes the learning experience more personalized and increases user motivation.

[2018] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2019] The flow of this system's program processing

[2020] Step 1: User Registration

[2021] Input: Username, Password

[2022] Output: Registration successful or error message

[2023] user:

[2024] Users access the system from their terminal and enter their username and password on the account creation screen.

[2025] server:

[2026] The server receives the username and password you entered.

[2027] The received password is hashed using a hashing algorithm (e.g., SHA-256).

[2028] The server checks the database to see if the entered username already exists, and if not, saves the new user information in the database.

[2029] Registration information such as username is stored in a database along with hashed passwords.

[2030] Step 2: Login authentication

[2031] Input: Username, Password

[2032] Output: Authentication successful or error message

[2033] user:

[2034] The user enters their username and password on the login screen.

[2035] server:

[2036] The server receives the username and password you entered.

[2037] The server retrieves the hashed password for that username from its database.

[2038] The entered password is hashed and compared with the information in the database.

[2039] If they match, the authentication is successful and the user is logged in to the system. If they do not match, an error message is returned.

[2040] Step 3: Skill Level Assessment

[2041] Input: Sample code

[2042] Output: Skill level judgment result (beginner, intermediate)

[2043] user:

[2044] Users submit sample code to the system for skill evaluation.

[2045] server:

[2046] The server receives sample code submitted by the user.

[2047] The server runs the sample code in an execution environment (e.g., Python execution environment).

[2048] Check the execution results and evaluate whether any errors occur.

[2049] If no error occurs, the player is judged as intermediate, and if an error occurs, the player is judged as beginner.

[2050] The result of the judgment is saved in the user's profile.

[2051] Step 4: Automatic generation of programming problems

[2052] Input: Skill Level

[2053] Output: Programming problem

[2054] server:

[2055] The server uses a generative AI model based on the user's skill level to automatically generate appropriate programming problems.

[2056] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[2057] The generated programming problem is presented to the user.

[2058] Step 5: Submit and check your answers

[2059] Input: Answer code

[2060] Output: Answer check result (correct answers, incorrect answers and explanations)

[2061] user:

[2062] The user submits the solution code to the system for the programming problem presented to them.

[2063] server:

[2064] The server receives the answer code submitted by the user.

[2065] The submitted answer code is compared with a predefined correct answer code.

[2066] If the answer is correct, it is judged as "correct," and if the answer is incorrect, a specific explanation is generated using a generative AI model.

[2067] Provide the results to the user.

[2068] Step 6: Analyze user weaknesses and provide customization solutions

[2069] Input: Past answer data, behavior log

[2070] Output: Custom programming questions

[2071] server:

[2072] The server analyzes the user's past answer data and behavioral logs.

[2073] Reinforcement learning algorithms and machine learning are used to identify weaknesses.

[2074] For example, if there are many loop-related errors, the generative AI model will generate customization problems related to the loop.

[2075] Providing users with customized programming problems.

[2076] Step 7: Analyzing the coding video

[2077] Input: Coding video

[2078] Output: Analysis results and feedback

[2079] user:

[2080] Users record themselves coding and upload videos to the platform.

[2081] server:

[2082] The server analyzes the coding videos uploaded by users.

[2083] Machine learning models (e.g., YOLO, OpenCV) are used to identify movements in the video and areas where errors frequently occur.

[2084] Based on the analysis results, we identify the user's weaknesses and provide appropriate feedback.

[2085] Step 8: Implementing the Emotion Engine and Feedback

[2086] Input: Video and audio data from the camera and microphone

[2087] Output: Emotion recognition results and feedback

[2088] user:

[2089] As users code, they use a camera and microphone to record their facial expressions and voice.

[2090] server:

[2091] The server analyzes the video and audio data acquired from the camera and microphone.

[2092] The emotion engine recognizes user emotions in real time using facial recognition technology (e.g., OpenCV) and voice analysis technology (e.g., Microsoft Azure's Speech to Text).

[2093] The learning program can be adjusted based on the perceived emotions, for example, providing easier problems or encouraging messages generated using a generative AI model if the user is feeling stressed.

[2094] (Application example 2)

[2095] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2096] While the demand for programming education is increasing, providing optimal learning programs tailored to each user's skill level is challenging. Furthermore, conventional systems are insufficient in assessing users' learning progress, identifying individual weaknesses, and providing customized feedback. Furthermore, to improve the user's learning experience, it is necessary to understand the user's emotional state in real time and provide appropriate support accordingly.

[2097] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2098] In this invention, the server includes means for automatically generating programming problems based on the user's skill level, means for determining whether the answers submitted by the user are correct, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, and means for recognizing the user's emotions and adjusting the learning program. This makes it possible to automatically generate optimal programming problems according to the user's skill, identify the user's weaknesses, and provide customized feedback. Furthermore, by analyzing the user's emotional state in real time, the learning experience can be personalized and effective support can be provided.

[2099] "User skill level" refers to the user's current level of programming knowledge and ability.

[2100] "Automatic generation of programming problems" means automatically creating appropriate programming tasks based on the user's skill level.

[2101] "Means for determining whether the answer is correct" refers to the function of checking the correctness of the program submitted by the user and providing feedback on the results.

[2102] An "action log" is a record of a user's operation history and behavioral data when using the system.

[2103] A "coding video" is a video recording of a user writing a program.

[2104] "Means for analyzing weaknesses" refers to a function that analyzes user behavior logs and coding video data to identify areas in which the user is weak.

[2105] "Customized programming problems" refer to programming tasks that are specifically designed to overcome a user's analyzed weaknesses.

[2106] "Means of recognizing emotions" refers to technology for reading a user's psychological state from their facial expressions and voice.

[2107] "Means for adjusting the learning program" refers to the ability to change the learning content and difficulty level provided in real time depending on the user's emotional state and skill level.

[2108] In this invention, a system is constructed in which a server, a user terminal, and an emotion engine work together. The server automatically generates programming problems according to the user's skill level, judges whether the user's answers are correct, analyzes behavior logs and coding videos, provides customized problems, and adjusts the learning program based on the user's emotional state.

[2109] The server uses the following software to perform its processing:

[2110] Flask: A lightweight framework for providing server-side web services.

[2111] werkzeug.security: A security library used for password hashing and authentication.

[2112] emotion_recognition: A technology for identifying user emotions using an emotion analysis engine.

[2113] skill_evaluation: A module that determines skill levels and evaluates submitted program code.

[2114] video_analysis: A machine learning model that analyzes users' coding videos to identify behavioral trends and weaknesses.

[2115] The server analyzes data obtained from the user's submitted program code, behavioral logs, and coding videos to identify the user's weaknesses and learning progress. It also recognizes the user's emotional state in real time through an emotion engine, making it possible to adjust the difficulty of the learning program and provide appropriate feedback.

[2116] As a concrete example, the process of a user registering an account on a smartphone and solving a programming problem is shown below. The user uses a smartphone or smart glasses to submit programming code by voice input or keyboard input. The server then analyzes the submitted code, determines the user's skill level, and provides programming problems of an appropriate level of difficulty. The user answers the problem and submits it again to the server, where the answer is judged to be correct or incorrect, and if the answer is incorrect, a specific explanation is automatically generated.

[2117] Furthermore, users can record their own coding videos with the smart glasses and upload them to the platform, where the server analyzes the videos to identify the user's coding style and areas where they are stuck.The emotion engine then analyzes the user's facial expressions and voice data to recognize their emotions in real time, and provides feedback such as lowering the difficulty level or displaying encouraging messages if the user is feeling frustrated.

[2118] An example of a prompt would be, "We have recorded a user solving a programming problem. Please use a machine learning model to analyze the user's coding style, stuck points, and error patterns from this video. Also, please analyze the user's facial expressions and voice, and use an emotion engine to recognize their emotions in real time and provide appropriate feedback."

[2119] Through this process, users receive optimal learning content suited to their skill level, as well as personalized feedback based on their weaknesses and emotional state.

[2120] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2121] Step 1:

[2122] A user registers an account on a smartphone or smart glasses. The user enters a username and password, which are then sent to the server, which hashes the password and stores it in a database.

[2123] Input: Username, Password

[2124] Output: Hashed password, saved user account information

[2125] Step 2:

[2126] A user attempts to log in. They enter their username and password and submit them to the server, which authenticates them by comparing the entered password with a hashed password stored in a database.

[2127] Input: Username, Password

[2128] Output: Authentication result (success or failure)

[2129] Step 3:

[2130] The user submits sample code for skill level assessment. The user puts on the smart glasses and inputs the sample code via voice or text input. The server executes the submitted code and determines the skill level.

[2131] Input: Sample code

[2132] Output: Skill level judgment result (beginner, intermediate)

[2133] Step 4:

[2134] The server automatically generates programming problems appropriate for each skill level, such as a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners.

[2135] Input: Skill Level

[2136] Output: Auto-generated programming questions

[2137] Step 5:

[2138] The user answers the provided programming questions and submits them to the server. The user enters the answer code using a smartphone or smart glasses and sends it to the server. The server compares the submitted code with the correct answer and determines whether it is correct or incorrect.

[2139] Input: Answer code

[2140] Output: Correct / incorrect result, feedback (including explanation)

[2141] Step 6:

[2142] The server analyzes the user's behavior log and submitted answers to identify the user's weaknesses. For example, if there are many incorrect answers regarding loops, it will extract those weaknesses.

[2143] Input: User behavior log, answer history

[2144] Output: Identified weaknesses

[2145] Step 7:

[2146] Users record their coding videos using smart glasses and upload them to a server, which then uses machine learning models to analyze the videos and identify the user's coding style and stumbling points.

[2147] Input: Coding video

[2148] Output: Analysis results (blockage points, error patterns)

[2149] Step 8:

[2150] The emotion engine recognizes the user's emotional state in real time from their facial expressions and voice. The user captures data using the camera and microphone in the smart glasses and sends it to the server. The server then performs emotion analysis and adjusts the learning program.

[2151] Input: facial expression data, voice data

[2152] Output: emotional state, adjusted learning program

[2153] Step 9:

[2154] The server provides customized feedback and programming problems based on the analyzed weaknesses and emotional state, and displays difficulty-adjusted problems and encouraging messages to reduce user frustration.

[2155] Input: Identified weaknesses, emotional state

[2156] Output: Customized feedback, programming problems

[2157] As a concrete example, the prompt sentence is as follows: "We have recorded a user solving a programming problem. Please use a machine learning model to analyze the user's coding style, stuck points, and error patterns from this video. Also, please analyze the user's facial expressions and voice, and use an emotion engine to recognize their emotions in real time and provide appropriate feedback."

[2158] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2159] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2160] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2161] [Fourth embodiment]

[2162] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2163] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2164] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2165] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2166] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2167] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2168] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2169] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2170] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2171] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2172] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2173] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2174] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2175] This invention is a system that automatically generates programming problems according to the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze the user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows.

[2176] User registration and login management

[2177] user:

[2178] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[2179] server:

[2180] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[2181] Determining a user's skill level

[2182] user:

[2183] Users submit sample code to the system for skill evaluation.

[2184] server:

[2185] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[2186] Automatic generation of programming problems

[2187] server:

[2188] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[2189] Automatic answer checking and explanation generation

[2190] user:

[2191] The user submits the answer code to the system.

[2192] server:

[2193] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[2194] User weakness analysis and customization

[2195] server:

[2196] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[2197] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[2198] Coding video analysis

[2199] user:

[2200] Users record themselves coding and upload videos to the platform.

[2201] server:

[2202] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[2203] Specific examples

[2204] Example 1: A beginner user registers and solves a problem

[2205] 1. User Registration

[2206] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[2207] 2. Log in

[2208] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[2209] 3. Skill Judgment

[2210] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[2211] 4. Providing questions

[2212] The server generates "beginner" questions based on skill level and presents them to the user.

[2213] 5. Submitting and checking your answers

[2214] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[2215] 6. Weakness analysis and customization problem provision

[2216] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[2217] This system provides an environment where beginners can effectively learn programming at their own pace. It dramatically improves learning efficiency by providing questions and explanations optimized based on the user's skill level and weaknesses.

[2218] The processing flow will be explained below.

[2219] User registration and login management

[2220] User Registration

[2221] Step 1:

[2222] The user enters a username and password.

[2223] Step 2:

[2224] The terminal sends the entered username and password to the server.

[2225] Step 3:

[2226] The server extracts the username and password from the request data.

[2227] Step 4:

[2228] The server hashes the password using an algorithm such as SHA-256.

[2229] Step 5:

[2230] The server stores the username and hashed password in a database.

[2231] Step 6:

[2232] The server returns a registration success message to the terminal.

[2233] Log in

[2234] Step 1:

[2235] The user enters the registered username and password.

[2236] Step 2:

[2237] The terminal sends the entered username and password to the server.

[2238] Step 3:

[2239] The server extracts the username and password from the request data.

[2240] Step 4:

[2241] The server retrieves the corresponding hashed password from its database.

[2242] Step 5:

[2243] The server hashes the entered password and verifies that it matches the stored hashed password.

[2244] Step 6:

[2245] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[2246] Determining a user's skill level

[2247] Step 1:

[2248] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[2249] Step 2:

[2250] The device sends the sample code to the server.

[2251] Step 3:

[2252] The server receives the submitted code.

[2253] Step 4:

[2254] The server runs the code in a secure environment.

[2255] Step 5:

[2256] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[2257] Step 6:

[2258] The server returns the skill level evaluation result to the terminal.

[2259] Automatic generation of programming problems

[2260] Step 1:

[2261] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[2262] Step 2:

[2263] The server generates programming problems corresponding to skill levels.

[2264] Step 3:

[2265] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[2266] Step 4:

[2267] The server transmits the generated programming questions to the terminal.

[2268] Automatic answer checking and explanation generation

[2269] Step 1:

[2270] The user enters the answer code into the terminal and clicks the submit button.

[2271] Step 2:

[2272] The terminal sends the answer code to the server.

[2273] Step 3:

[2274] The server receives the submitted answer code.

[2275] Step 4:

[2276] The server compares the answer to predefined correct answers.

[2277] Step 5:

[2278] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[2279] Step 6:

[2280] The server sends the generated message and explanation to the terminal.

[2281] User weakness analysis and customization

[2282] Step 1:

[2283] The server analyzes the user's past answer data and behavioral logs.

[2284] Step 2:

[2285] The server identifies user weaknesses (e.g., loops, conditional branches) based on data analysis.

[2286] Step 3:

[2287] The server invokes logic to generate customized questions according to the identified weaknesses.

[2288] Step 4:

[2289] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[2290] Step 5:

[2291] The server transmits the generated customization question to the terminal.

[2292] Coding video analysis

[2293] Step 1:

[2294] Users upload videos of themselves coding to their devices.

[2295] Step 2:

[2296] The device sends the uploaded video to the server.

[2297] Step 3:

[2298] The server analyzes the received video using a machine learning model.

[2299] Step 4:

[2300] The server identifies behaviors and choke points during coding.

[2301] Step 5:

[2302] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[2303] Step 6:

[2304] The server transmits the generated feedback to the terminal.

[2305] As a result, this fully automatic programming learning system helps users to learn effectively at their own pace.

[2306] Example 1

[2307] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2308] Conventional programming problem-solving systems have difficulty automatically generating appropriate problems based on the user's skill level and weaknesses, resulting in reduced learning efficiency. Additionally, there are generally delays in determining whether an answer is correct and in providing feedback, preventing users from checking their learning progress in real time. Furthermore, while analyzing coding videos, rather than relying solely on traditional behavioral log analysis, would enable more detailed and accurate feedback to be provided, few such systems exist.

[2309] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2310] In this invention, the server includes a means for receiving user authentication information and hashing the password to ensure security, a means for automatically generating programming problems based on the user's skill level, and a means for determining whether the answers submitted by the user are correct, thereby enabling the provision of appropriate programming problems according to the user's skill level and fast and accurate feedback.

[2311] "User Credentials" means the username and password or other authentication credentials used to access a system.

[2312] "Password hashing" is the process of encrypting a user-entered password using a specific algorithm to ensure system security.

[2313] "Skill level" is an indicator of a user's programming skill and knowledge. For example, there is a distinction between "beginner" and "intermediate."

[2314] "Automatic programming problem generation" is the process by which the system automatically creates appropriate programming challenges based on the user's skill level.

[2315] The "means for determining whether the answer submitted by the user is correct" is a means for checking whether the program code submitted by the user is executed according to the specifications and determining whether the answer is appropriate.

[2316] An "action log" is a record of the operations and input a user performs on the system.

[2317] "Coding videos" are video data that record the process of a user programming.

[2318] "Weakness analysis" is the process of analyzing user behavior logs and coding videos to identify areas where users are particularly weak or frequently make errors.

[2319] "Customized programming problems" are personalized programming challenges generated based on a user's identified weaknesses.

[2320] This system automatically generates programming problems based on a user's skill level and judges the accuracy of the user's submitted solutions. Furthermore, it can analyze a user's behavioral logs and coding videos to identify individual weaknesses and provide customized problems based on those weaknesses. This system operates as follows:

[2321] User registration and login management

[2322] A user creates an account and enters a username and password to log in to the system. The server receives the user's registration information and hashes and stores the password to ensure security. Also, when logging in, the entered password is compared with the hashed password for authentication. This maintains a high level of security.

[2323] Determining a user's skill level

[2324] Users submit sample code for skill assessment. The server runs the submitted code in a sandbox environment and determines the skill level based on whether an error occurs. For example, if no errors occur, the user is assessed as "intermediate," and if an error occurs, the user is assessed as "beginner."

[2325] Automatic generation of programming problems

[2326] The server automatically generates appropriate programming problems using a generative AI model based on the user's skill level. For example, it generates a "program to add two numbers" for beginners and a "program to find factorials" for intermediate learners. The generated problems are then presented to the user.

[2327] Automatic answer checking and explanation generation

[2328] The user submits the answer code. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct." If the answer is incorrect, it responds with a specific explanation. This allows the user to receive instant feedback.

[2329] User weakness analysis and customization

[2330] The server analyzes the user's past answer data and behavioral logs to identify the user's weaknesses. For example, if there are many incorrect answers regarding loops or conditional branching, it identifies these as "weaknesses." Based on the analysis results, the server provides customized programming problems to help the user overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates the corresponding programming problem.

[2331] Coding video analysis

[2332] Users upload videos of themselves coding to the platform. The server then uses machine learning models to analyze the uploaded videos and identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the platform identifies the user's weaknesses and provides appropriate feedback.

[2333] Specific examples

[2334] 1. Example of a beginner user registering and solving a problem

[2335] Create an account: The user creates an account by entering a username and password. The server hashes the password using the bcrypt library and stores it in the database along with the user information.

[2336] Login: The user enters a username and password. The server retrieves the hashed password for that username from the database and compares it with the entered password. If authentication is successful, the user is allowed to access the system.

[2337] Skill assessment: Users submit sample code. The server runs the submitted code in a Docker container and checks the error log to determine the skill level. For example, if sample code such as "print(2 + 2)" can be executed without error, the user is assessed as intermediate.

[2338] Problem provision: The server requests a problem for beginners from the AI ​​model and receives a program to add two numbers. The server then presents the generated problem to the user.

[2339] Answer submission and verification: The user submits an answer. The server compares the answer with predefined correct answers in a Docker container. If the answer is correct, the server returns a "correct" message. If the answer is incorrect, the server returns feedback including the specific error location.

[2340] Weakness analysis and provision of customized problems: The server analyzes the user's answer history and behavior log to identify loops as particularly difficult. It then asks the generative AI model to generate customized problems related to loops and provides them to the user. For example, it generates a problem such as "Create a program to calculate the sum of numbers from 1 to 10."

[2341] Prompt Sentence Examples

[2342] Prompt: Please generate a sample problem for a simple Python program for beginners. Also, please provide an explanation for this problem.

[2343] Example response of the generative AI model:

[2344] Question: Write a program that takes two integers as input and calculates and displays their sum.

[2345] Explanation: In this problem, we will use the input() function to read two integers from the user, add them, and display the result using the print() function. See the code below for an example.

[2346] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2347] Step 1: User Registration

[2348] User: A user creates an account by entering a username and password.

[2349] Input: Username and Password

[2350] Server: The server processes the received user information and hashes the password, specifically encrypting it using the bcrypt library.

[2351] Data manipulation: password hashing

[2352] Output: User information including hashed password

[2353] Server: The server stores this information in a database, specifically by inserting the hashed password and username into the database.

[2354] Step 2: Login authentication

[2355] User: The user enters their username and password into a login form.

[2356] Input: Username and Password

[2357] Server: The server retrieves the hashed password for the given username from its database.

[2358] Data manipulation: Obtaining a hashed password corresponding to a username

[2359] Output: Hashed password corresponding to username

[2360] Server: The server compares the entered password with the hashed password retrieved from the database.

[2361] Data calculation: Password verification

[2362] Output: Authentication result

[2363] Server: If authentication is successful, it generates session information and responds to the user. This includes generating a session ID and storing it with the corresponding user information.

[2364] Step 3: Submit your skill evaluation code

[2365] User: The user writes a sample code for skill evaluation and presses the submit button.

[2366] Input: Sample code

[2367] Server: The server receives the submitted code and executes it in a sandbox environment (Docker container).

[2368] Data processing: Running the sample code

[2369] Output: Execution results and error log

[2370] Server: Based on the execution results, if no errors occur, the user is judged as "Intermediate", and if an error occurs, the user is judged as "Beginner". This operation analyzes the error log and judges the skill level.

[2371] Step 4: Problem generation

[2372] Server: The server sends appropriate prompts to the generative AI model based on the user's skill level.

[2373] Input: Skill level and prompt

[2374] Generative AI model: A generative AI model generates programming problems based on prompts.

[2375] Data processing: Generating programming problems

[2376] Output: The generated programming problem

[2377] Server: Presents the generated questions to the user. This includes displaying the generated questions in a user interface.

[2378] Step 5: Submit your answer

[2379] User: The user writes the answer code for the displayed programming problem and presses the submit button.

[2380] Input: Answer code

[2381] Server: The server receives the submitted code and compares it with pre-stored correct answers.

[2382] Data processing: Analysis and comparison of submitted solution codes

[2383] Output: Correct / incorrect result and feedback

[2384] Server: If the answer is correct, it will be deemed "correct" and generate and return a feedback message. If the answer is incorrect, it will provide feedback with a specific error message and suggestions for improvement. This action includes generating the feedback and responding to the user.

[2385] Step 6: Weakness analysis

[2386] Server: The server analyzes past answer data and behavioral logs to identify typical user error patterns and coding pitfalls.

[2387] Input: Past answer data and behavior log

[2388] Data processing: Error pattern analysis and statistical analysis

[2389] Output: Weakness identification results

[2390] Server: Based on the analysis results, the server requests the generative AI model to create customized questions that address the user's identified weaknesses. This action involves sending prompts to the generative AI model.

[2391] Step 7: Generate customization questions

[2392] Generative AI model: Based on the results of weakness analysis, the generative AI model generates personalized programming challenges for users to overcome.

[2393] Input: Weakness analysis results and prompt text

[2394] Data manipulation: custom problem generation

[2395] Output: Customized programming problems

[2396] Server: Presents the generated customization questions to the user. This includes displaying the customization questions in the user interface.

[2397] Step 8: Upload and analyze your video

[2398] Users: Users record themselves coding and upload videos to the platform.

[2399] Input: Coding video

[2400] Server: The server receives the uploaded video and requests the machine learning model to analyze it.

[2401] Data processing: Video data analysis

[2402] Output: Video analysis results

[2403] Machine learning model: The machine learning model analyzes the video and identifies points where users get stuck and where they frequently make errors.

[2404] Server: Generates specific feedback based on the video analysis results and provides it to the user. This operation includes generating feedback and responding to the user.

[2405] (Application example 1)

[2406] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2407] Modern factories require robot operators to learn robot control programs efficiently and effectively. However, it is difficult to provide customized learning content tailored to each operator's skill level and weaknesses. Furthermore, conventional programming education systems are unable to provide specific feedback or customized problems based on actual code writing or execution results, preventing improvements in learning efficiency. Furthermore, there is a lack of mechanisms for identifying problems operators encounter when operating robots and providing support based on those problems.

[2408] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2409] In this invention, the server includes means for automatically generating programming problems based on the skill level of a user, means for determining the correctness of answers submitted by the user, means for analyzing the user's weaknesses from the user's behavior log or coding video, means for providing customized programming problems based on the analysis results, means for generating program problems for robot operators to improve the user's skills, and an application to be installed on robots used in factories. This makes it possible to provide customized program problems and specific feedback according to the skill level and weaknesses of robot operators.

[2410] "User" refers to a person who uses this system to solve programming problems.

[2411] "Skill level" is a standard for evaluating a user's programming ability and proficiency.

[2412] "Programming Problems" are tasks provided by the system to assess or improve programming skills.

[2413] "Automatic generation" refers to the system automatically creating programming problems according to the user's skill level.

[2414] "Solution" refers to the source code or program submitted by a user in response to a programming problem.

[2415] "Correctness determination" means that the system evaluates whether the submitted answer is correct or incorrect.

[2416] An "action log" refers to the operation history and activity record when a user uses the system.

[2417] A "coding video" is a video that records the process of a user creating a program.

[2418] "Weaknesses" are specific techniques or concepts that users have difficulty with or make repeated mistakes with.

[2419] "Analysis" refers to extracting specific information and patterns based on user behavior logs and coding videos.

[2420] "Customized programming problems" are programming problems that are tailored to each user to overcome their specific weaknesses.

[2421] A "robot operator" refers to a person whose job is to operate robots in a factory and control their movements.

[2422] A "factory" is a production facility for manufacturing products.

[2423] An "application" is software that realizes the functions provided by a system.

[2424] This invention is a system that automatically generates programming problems based on a user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it can analyze a user's behavioral log and coding video to identify individual weaknesses and provide customized problems based on those weaknesses. This system is particularly applicable to programming problems for robot operators used in factories. The following describes specific embodiments of the invention.

[2425] System Configuration

[2426] The server manages a system that includes the following elements:

[2427] 1. User registration and login management:

[2428] User: A user creates an account and logs into the system. When creating an account, the user registers by entering a username and password. When logging in, the user enters their authentication information to access the system.

[2429] Server: The server receives the user's registration information and hashes the password to ensure security. When logging in, the server compares the entered password with the hashed password to authenticate.

[2430] 2. Skill Level Assessment:

[2431] User: The user submits sample code to the system for skill evaluation.

[2432] Server: The server runs the submitted code and if the code runs correctly it will be judged as "Intermediate"; if an error occurs it will be judged as "Beginner".

[2433] 3. Automatic generation of programming problems:

[2434] Server: The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would generate a "program to add two numbers," and an intermediate would generate a "program to find factorials."

[2435] 4. Automatic answer checking and explanation generation:

[2436] User: The user submits the answer code to the system.

[2437] Server: The server examines the submitted code and compares it with a predefined answer. If it is correct, it marks it as "correct." If it is incorrect, it responds with a specific explanation.

[2438] 5. User weakness analysis and customization solutions:

[2439] Server: The server identifies the user's weak points based on the user's past answer data and the results of analyzing coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weak points."

[2440] Server: Next, it provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[2441] 6. Coding video analysis:

[2442] Users: Users record themselves coding and upload videos to the platform.

[2443] Server: The server analyzes the uploaded video using machine learning models to identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[2444] Computer Systems and Software

[2445] The server uses Flask to implement web applications, bcrypt to hash passwords, and machine learning models to analyze user activity logs and coding videos. These pieces of software run on the robot's control unit and server.

[2446] Adding concrete examples and prompt sentence examples

[2447] As a specific example, there is the following usage.

[2448] 1. User Registration:

[2449] The user creates an account by entering a username and password.

[2450] 2. Login:

[2451] The user enters their login information and the server performs authentication.

[2452] 3. Skill Check:

[2453] Users submit sample code, and the server runs the code to determine skill level.

[2454] 4. Question submission:

[2455] The server generates "beginner" questions based on skill level and presents them to the user.

[2456] 5. Submitting and checking your answers:

[2457] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[2458] 6. Weakness analysis and customization problem provision:

[2459] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[2460] Prompt Sentence Examples

[2461] "Analyze the coding log for weakness analysis and generate optimal customization questions. If there are many errors related to loops or conditional branching, we will provide appropriate questions based on that."

[2462] This allows robot operators to receive program problems and specific feedback that are tailored to their skill level and weaknesses.

[2463] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2464] Step 1: User Registration

[2465] The user enters their username and password using a terminal and sends it to the server. This input data includes the user's identification and authentication information. The server hashes the received password using bcrypt and stores the user information along with the hashed password in the database. This registers a new user.

[2466] Input: Username, Password

[2467] Data processing: password hashing

[2468] Output: User information stored in the database

[2469] Step 2: User Login

[2470] The user enters the registered username and password using the terminal and sends them to the server. The server retrieves the corresponding user information from the database and verifies whether the sent password matches the hashed password. If the authentication is successful, the user can access the system.

[2471] Input: Username, Password

[2472] Data processing: Comparison with hashed passwords

[2473] Output: Login session started or error message

[2474] Step 3: Determine your skill level

[2475] Users submit sample code from their devices to the server for skill assessment. The server executes the submitted code, and if the code runs correctly, the user is judged as "intermediate," and if an error occurs, the user is judged as "beginner." The result of this assessment is saved in a database as the user's level.

[2476] Input: Sample code

[2477] Data processing: Executing code and determining results

[2478] Output: Updated skill level of the user

[2479] Step 4: Automatic generation of programming problems

[2480] The server automatically generates appropriate programming problems based on the user's skill level. For example, it generates a "program to add two numbers" for a beginner, and a "program to find factorials" for an intermediate learner. The generated programming problems are provided to the user.

[2481] Input: User's skill level

[2482] Data manipulation: Generating programming problems

[2483] Output: Generated programming problem

[2484] Step 5: Check answers and automatically generate explanations

[2485] The user submits the answer code from their device to the server. The server examines the submitted code and compares it with the predefined correct answer. If the answer is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[2486] Input: Answer code

[2487] Data processing: Examining the code and comparing it to the correct answer

[2488] Output: Judgment result and explanation

[2489] Step 6: Analyze user weaknesses and provide customization solutions

[2490] The server identifies the user's weaknesses based on the analysis of the user's past answer data and coding videos. For example, if the user frequently makes mistakes regarding loops, it identifies that as a "weakness." It then generates customized programming problems to help the user overcome the identified weaknesses.

[2491] Input: Answer data, coding video

[2492] Data processing: Data analysis and weakness identification

[2493] Output: Customized Program Questions

[2494] Step 7: Analyzing the coding video

[2495] Users upload videos of themselves coding from their devices to the server. The server then analyzes the videos using machine learning models to identify coding stumbling points and areas where errors frequently occur. Based on the analysis results, the server identifies the user's weaknesses and provides appropriate feedback.

[2496] Input: Coding video

[2497] Data processing: Video analysis using machine learning models

[2498] Output: Identified weaknesses and feedback

[2499] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2500] This invention is a system that automatically generates programming problems based on the user's skill level and judges the accuracy of the answers submitted by the user. Furthermore, it analyzes the user's behavioral logs and coding videos to identify individual weaknesses and provides customized problems based on those weaknesses. By combining this with an emotion engine that recognizes the user's emotions, the user's learning experience becomes even more personalized and effective. Specific implementation methods for this system are described below.

[2501] User registration and login management

[2502] user:

[2503] Users create an account and log in to the system. When creating an account, they register by entering a username and password. When logging in, they enter their authentication information to access the system.

[2504] server:

[2505] The server receives the user's registration information and hashes the password to ensure security. When logging in, the entered password is compared with the hashed password for authentication.

[2506] Determining a user's skill level

[2507] user:

[2508] Users submit sample code to the system for skill evaluation.

[2509] server:

[2510] The server executes the submitted code, and if the code executes correctly, the user is judged as "intermediate", but if an error occurs, the user is judged as "beginner".

[2511] Automatic generation of programming problems

[2512] server:

[2513] The server automatically generates appropriate programming problems based on the user's skill level. For example, a beginner would get a "program to add two numbers," while an intermediate learner would get a "program to find the factorial."

[2514] Automatic answer checking and explanation generation

[2515] user:

[2516] The user submits the answer code to the system.

[2517] server:

[2518] The server examines the submitted code and compares it with a predefined answer. If it is correct, it is judged as "correct," and if it is incorrect, it responds with a specific explanation.

[2519] User weakness analysis and customization

[2520] server:

[2521] The server identifies the user's weaknesses based on the analysis of the user's past answers and coding videos. For example, if there are many incorrect answers regarding loops or conditional branching, it will identify them as "weaknesses."

[2522] It then provides customized programming problems to help users overcome the identified weaknesses. For example, if a "loop" is a weakness, it generates a corresponding programming problem.

[2523] Coding video analysis

[2524] user:

[2525] Users record themselves coding and upload videos to the platform.

[2526] server:

[2527] The server uses machine learning models to analyze uploaded videos and identify coding stumbling blocks and areas where errors frequently occur. Based on the analysis results, it identifies the user's weaknesses and provides appropriate feedback.

[2528] Introducing the Emotion Engine

[2529] user:

[2530] As users code, they use a camera and microphone to record their facial expressions and voice.

[2531] server:

[2532] The server sends data acquired from the camera and microphone to the emotion engine, which analyzes the user's facial expressions and voice to recognize emotions in real time. This analysis can be done using, for example, facial recognition technology and voice analysis technology.

[2533] Emotion-based feedback

[2534] server:

[2535] The server adjusts the learning program based on the user's emotions, as detected by the emotion engine. For example, if the user is feeling stressed, it will provide easier problems or display encouraging messages.

[2536] Specific examples

[2537] Example 1: A beginner user registers and solves a problem

[2538] 1. User Registration

[2539] A user creates an account by entering a username and password, and the server hashes the password and stores the registration information.

[2540] 2. Log in

[2541] The user enters their login information, the server authenticates them, and if authentication is successful, the user is allowed access to the system.

[2542] 3. Skill Judgment

[2543] Users submit sample code, and the server runs it to determine their skill level. If the code runs without errors, they are considered intermediate.

[2544] 4. Providing questions

[2545] The server generates "beginner" questions based on skill level and presents them to the user.

[2546] 5. Submitting and checking your answers

[2547] The user submits an answer, and the server compares it with predefined correct answers, returning a "correct" message if correct, or a specific explanation if incorrect.

[2548] 6. Weakness analysis and customization problem provision

[2549] The server analyzes the user's answer history and behavior log to identify weak points. For example, if there are many loop-related questions, it will provide related customized questions.

[2550] 7. Analysis of coding videos

[2551] Users upload their own coding videos, which are then analyzed by the server using machine learning models. Based on the results, the server identifies the user's weaknesses and provides appropriate feedback.

[2552] 8. Leveraging Emotional Engines

[2553] The server recognizes the user's emotions from their facial expressions and voice and provides appropriate learning programs and feedback. For example, if the user shows signs of frustration, it provides them with questions that are less difficult.

[2554] This process allows users to learn effectively at their own pace and overcome their weaknesses. The introduction of an emotional engine makes the learning experience more personalized and increases user motivation.

[2555] The processing flow will be explained below.

[2556] User registration and login management

[2557] User Registration

[2558] Step 1:

[2559] The user enters a username and password.

[2560] Step 2:

[2561] The terminal sends the entered username and password to the server.

[2562] Step 3:

[2563] The server extracts the username and password from the request data.

[2564] Step 4:

[2565] The server hashes the password using an algorithm such as SHA-256.

[2566] Step 5:

[2567] The server stores the username and hashed password in a database.

[2568] Step 6:

[2569] The server returns a registration success message to the terminal.

[2570] Log in

[2571] Step 1:

[2572] The user enters the registered username and password.

[2573] Step 2:

[2574] The terminal sends the entered username and password to the server.

[2575] Step 3:

[2576] The server extracts the username and password from the request data.

[2577] Step 4:

[2578] The server retrieves the corresponding hashed password from its database.

[2579] Step 5:

[2580] The server hashes the entered password and verifies that it matches the stored hashed password.

[2581] Step 6:

[2582] If they match, the server generates a JWT token containing the username and returns it to the device. If they don't match, it returns an error message.

[2583] Determining a user's skill level

[2584] Step 1:

[2585] The user enters the sample code for skill evaluation into the terminal and clicks the submit button.

[2586] Step 2:

[2587] The device sends the sample code to the server.

[2588] Step 3:

[2589] The server receives the submitted code.

[2590] Step 4:

[2591] The server runs the code in a secure environment.

[2592] Step 5:

[2593] If the code executes correctly, the server will classify the user as "intermediate"; if an error occurs, the user will be classified as "beginner."

[2594] Step 6:

[2595] The server returns the skill level evaluation result to the terminal.

[2596] Automatic generation of programming problems

[2597] Step 1:

[2598] The server invokes logic to automatically generate questions based on the user's skill level (e.g., beginner, intermediate, advanced).

[2599] Step 2:

[2600] The server generates programming problems corresponding to skill levels.

[2601] Step 3:

[2602] For example, for beginners, it generates a "program to add two numbers," and for intermediate learners, it generates a "program to find factorials."

[2603] Step 4:

[2604] The server transmits the generated programming questions to the terminal.

[2605] Automatic answer checking and explanation generation

[2606] Step 1:

[2607] The user enters the answer code into the terminal and clicks the submit button.

[2608] Step 2:

[2609] The terminal sends the answer code to the server.

[2610] Step 3:

[2611] The server receives the submitted answer code.

[2612] Step 4:

[2613] The server compares the answer to predefined correct answers.

[2614] Step 5:

[2615] If the answer is correct, the server generates a "correct" message and a detailed explanation, and if the answer is incorrect, it generates an "incorrect" message and an explanation including points for improvement.

[2616] Step 6:

[2617] The server sends the generated message and explanation to the terminal.

[2618] User weakness analysis and customization

[2619] Step 1:

[2620] The server analyzes the user's past answer data and behavioral logs.

[2621] Step 2:

[2622] The server identifies the user's weaknesses based on data analysis.

[2623] Step 3:

[2624] The server invokes logic to generate customized questions according to the identified weaknesses.

[2625] Step 4:

[2626] For example, if a "loop" is a weakness, a problem containing a loop is generated.

[2627] Step 5:

[2628] The server transmits the generated customization question to the terminal.

[2629] Coding video analysis

[2630] Step 1:

[2631] Users upload videos of themselves coding to their devices.

[2632] Step 2:

[2633] The device sends the uploaded video to the server.

[2634] Step 3:

[2635] The server analyzes the received video using a machine learning model.

[2636] Step 4:

[2637] The server identifies behaviors and choke points during coding.

[2638] Step 5:

[2639] The server identifies the user's weaknesses based on the analysis results and generates appropriate feedback.

[2640] Step 6:

[2641] The server transmits the generated feedback to the terminal.

[2642] Introducing the Emotion Engine

[2643] Step 1:

[2644] Set up your camera and microphone when you code.

[2645] Step 2:

[2646] The device collects data from the camera and microphone in real time and sends it to the emotion engine.

[2647] Step 3:

[2648] The server uses an emotion engine to analyze the user's facial expressions and voice in real time and recognize their emotions.

[2649] Emotion-based feedback

[2650] Step 1:

[2651] The server receives the analysis results from the emotion engine.

[2652] Step 2:

[2653] Based on the analysis results, the server generates customized feedback according to the user's emotions.

[2654] Step 3:

[2655] For example, if a user is feeling stressed, it offers a simple message of encouragement or an easier problem.

[2656] Step 4:

[2657] The server transmits feedback based on the generated emotion to the terminal.

[2658] This invention can improve the learning effect and reduce stress for users by providing real-time feedback and emotional support according to their learning progress.

[2659] Example 2

[2660] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2661] While conventional programmatic learning systems can generate questions and assess correct answers based on the user's skill level, they lack the ability to provide customized feedback based on a detailed analysis of each user's learning status and weaknesses. Furthermore, they are unable to adjust the learning program to take into account the user's emotional state, which means they are unable to adequately maintain ...

Claims

1. means for automatically generating programming problems based on a user's skill level; a means for determining whether a user's submitted answer is correct; and A means of analyzing user weaknesses from user behavior logs or coding videos; a means for providing customized programming problems based on the analysis results; A system including:

2. 10. The system of claim 1, further comprising means for automatically generating explanations for user-submitted answers.

3. The system according to claim 1, wherein the analysis of the user's weaknesses is based on the analysis of the coding video.

Images