Method for generating feedback information for math problem solving process

Abstract

Disclosed, according to one embodiment of the present disclosure, is a method for generating feedback information for a math problem solving process, performed by a computing device. The method may comprise the steps of: acquiring user input information related to math problem solving; analyzing the math problem solving process on the basis of the acquired user input information by utilizing a pre-trained artificial intelligence model; and generating feedback information for the math problem solving on the basis of the analysis.

Description

How to generate feedback information about the math problem-solving process

[0001] The present invention relates to a method for generating feedback information for a mathematical problem solving process.

[0002] Compared to other subjects, math problems are logically complex and have a high rate of curriculum linkage across grades, making accurate assessments extremely challenging. Since grades are merely a superficial phenomenon, math requires a method to accurately assess whether students have understood the logical flow from the problem to the correct answer.

[0003] Republic of Korea Patent Publication No. 10-2022-0070888 (May 31, 2022) discloses an effective mathematical concept learning and problem-solving textbook.

[0004] This invention was filed with support from the Gyeonggi Province and the Gyeonggi Province Economic and Science Promotion Agency's '2024 Global Startup Commercialization Support Project.'

[0005] This disclosure aims to provide a method for analyzing a student's problem-solving method in real time during the process of solving a mathematical problem, thereby providing diagnosis and feedback on the problem-solving process itself, rather than simply confirming whether the problem is correct.

[0006] Furthermore, the present disclosure aims to provide a method for providing customized feedback tailored to the learning level of each student, as each student has different mathematical problem-solving abilities and error types, and for simplifying and improving the efficiency of teachers' grading and feedback work by automatically analyzing the solution process and generating feedback using an artificial intelligence model.

[0007] Meanwhile, the technical task to be achieved by the present disclosure is not limited to the technical task mentioned above, and may include various technical tasks within a scope obvious to a person skilled in the art from the contents described below.

[0008] According to one embodiment of the present disclosure for achieving the aforementioned task, a method for generating feedback information regarding a mathematical problem-solving process performed by a computing device is disclosed. The method may include the steps of: acquiring user input information related to mathematical problem-solving; analyzing the mathematical problem-solving process based on the acquired user input information using a pre-trained artificial intelligence model; and generating feedback information regarding the mathematical problem-solving process based on the analysis.

[0009] In one embodiment, the step of obtaining user input information related to solving the math problem may include at least one of the steps of obtaining user input information entered through an app or a web; or the step of obtaining user input information scanned from a piece of paper containing information related to solving the math problem.

[0010] In one embodiment, the step of obtaining user input information related to solving the mathematical problem may include the steps of distinguishing letters, numbers, and mathematical symbols from the obtained user input information; performing recognition on the distinguished letters using a character recognition model; performing recognition on the distinguished numbers using the character recognition model; and performing recognition on the distinguished mathematical symbols using the character recognition model.

[0011] In one embodiment, the step of analyzing the solution process for the acquired user input information using the pre-trained artificial intelligence model may include the step of sequentially dividing the solution process for the acquired user input information; and the step of analyzing the sequentially divided solution process in response to rubric items using the pre-trained artificial intelligence model.

[0012] In one embodiment, the rubric items may include at least one of problem understanding, solution method, calculation accuracy, or derivation of a final correct answer.

[0013] In one embodiment, the pre-trained artificial intelligence model may be trained based on the following operations: acquiring learning solution process data; sequentially dividing the learning solution process data; acquiring the rubric items, which are evaluation criteria for each of the sequentially divided learning solution process data; and predicting whether the sequentially divided learning solution process data satisfies the rubric items.

[0014] In one embodiment, the pre-trained artificial intelligence model may include at least one of a first model for measuring the ability to understand the problem's objectives and conditions; a second model for evaluating a solution approach or logical development; a third model for evaluating the computational accuracy of each of the sequentially divided learning solution process data; or a fourth model for determining whether the final derived answer satisfies the problem's requirements.

[0015] In one embodiment, the step of generating feedback information based on the analyzed solution process may include the step of generating feedback information for the sequentially divided solution process based on the analyzed results corresponding to the rubric items.

[0016] In one embodiment, the step of generating feedback information for the sequentially divided solving process based on the analyzed results corresponding to the rubric items may include either the step of generating feedback information for the solving process in need of improvement corresponding to the rubric items; or the step of generating feedback information for the sequentially divided solving process by assigning partial scores according to the degree of satisfaction of each rubric item.

[0017] In one embodiment, the step of analyzing the mathematical problem solving process based on the acquired user input information using the pre-learned artificial intelligence model may include a step of analyzing whether characters included in the user input information are characters directly related to solving the mathematical problem or are additional characters not directly related to solving the mathematical problem.

[0018] In one embodiment, the step of analyzing the mathematical problem-solving process based on the acquired user input information by utilizing the pre-learned artificial intelligence model may include the step of identifying the language of the characters included in the user input information, if the characters are directly related to the mathematical problem-solving process; the step of identifying a formula associated with the characters among formulas included in the solving process, considering the system of the identified language; and the step of generating analysis information based on the semantic information of the characters and the identified formula.

[0019] In one embodiment, the step of analyzing the mathematical problem solving process based on the acquired user input information by utilizing the pre-learned artificial intelligence model may include a step of generating analysis information based on the remaining portion excluding the additional characters, if the additional characters are not directly related to the mathematical problem solving.

[0020] In one embodiment, the method may further include a step of generating a problem related to the solution in a user-customized manner, taking into account the analyzed solution process.

[0021] According to one embodiment of the present disclosure for achieving the above-described task, a computer program stored in a computer-readable storage medium is disclosed. When the computer program is executed on one or more processors, the computer program causes the one or more processors to perform the following operations for generating feedback information regarding a mathematical problem-solving process, wherein the operations may include: obtaining user input information related to mathematical problem-solving; analyzing the mathematical problem-solving process based on the obtained user input information by utilizing a pre-learned artificial intelligence model; and generating feedback information regarding the mathematical problem-solving based on the analysis.

[0022] In one embodiment, the operation of obtaining user input information related to solving the mathematical problem may include an operation of distinguishing letters, numbers, and mathematical symbols from the obtained user input information; an operation of performing recognition on the distinguished letters using a character recognition model; an operation of performing recognition on the distinguished numbers using the character recognition model; and an operation of performing recognition on the distinguished mathematical symbols using the character recognition model.

[0023] In one embodiment, the operation of analyzing the solution process for the acquired user input information using the pre-trained artificial intelligence model may include the operation of sequentially dividing the solution process for the acquired user input information; and the operation of analyzing the sequentially divided solution process in response to rubric items using the pre-trained artificial intelligence model.

[0024] In one embodiment, the operation of generating feedback information based on the analyzed solution process may include an operation of generating feedback information for the sequentially divided solution process based on the analyzed results corresponding to the rubric items.

[0025] In one embodiment, the operation of analyzing the process of solving the math problem based on the acquired user input information by utilizing the pre-learned artificial intelligence model may include an operation of analyzing whether characters included in the user input are characters directly related to solving the math problem or are additional characters not directly related to solving the math problem.

[0026] In one embodiment, the operation may further include providing a problem related to the solution in a user-customized manner, taking into account the analyzed solution process.

[0027] A computing device according to one embodiment of the present disclosure for achieving the aforementioned task is disclosed. The device comprises at least one processor; and a memory, wherein the at least one processor is configured to: acquire user input information related to solving a mathematical problem; analyze the mathematical problem-solving process based on the acquired user input information using a pre-trained artificial intelligence model; and generate feedback information regarding the mathematical problem-solving based on the analysis.

[0028] In one embodiment, the at least one processor may be configured to distinguish letters, numbers, and mathematical symbols from the acquired user input information; perform recognition of the distinguished letters using a character recognition model; perform recognition of the distinguished numbers using the character recognition model; and perform recognition of the distinguished mathematical symbols using the character recognition model.

[0029] In one embodiment, the at least one processor may be configured to sequentially divide the solution process for the acquired user input information; and to analyze the sequentially divided solution process in response to a rubric item using the pre-learned artificial intelligence model.

[0030] In one embodiment, the at least one processor may be configured to generate feedback information for the sequentially segmented solving process based on the analyzed results corresponding to the rubric items.

[0031] In one embodiment, the at least one processor may be configured to analyze whether characters included in the user input are characters directly related to the solution of the math problem or are additional characters not directly related to the solution of the math problem.

[0032] In one embodiment, the at least one processor may be further configured to provide a problem related to the solution in a user-customizable manner, taking into account the analyzed solution process.

[0033] This disclosure provides sophisticated learning feedback information by enabling a pre-trained artificial intelligence model to learn various error patterns that may occur during the process of solving mathematical problems, thereby clearly indicating specific problems and directions for improvement in students' solutions.

[0034] Furthermore, the present disclosure allows students to correct and improve their incorrect problem-solving processes through immediate and personalized feedback, thereby improving their understanding of the learning process by allowing them to address any gaps in their understanding in real time while solving problems.

[0035] Furthermore, since the present disclosure can analyze and score each step of the solution process through an artificial intelligence model, it goes beyond simply awarding points based on correctness, and enables step-by-step scoring and partial scoring. This can help students not only improve their overall understanding but also specifically identify which parts of the solution process need further improvement.

[0036] Furthermore, this disclosure allows teachers to focus on guidance by understanding each student's learning status, freeing them from the repetitive and time-consuming task of grading, thereby enabling them to design learning directions based on statistics on the types of errors students frequently make or areas where they lack understanding.

[0037] Meanwhile, the effects of the present disclosure are not limited to the effects mentioned above, and various effects may be included within a range apparent to those skilled in the art from the contents described below.

[0038] FIG. 1 is a block diagram of a computing device for generating feedback information for a mathematical problem solving process according to one embodiment of the present disclosure.

[0039] FIG. 2 illustrates an exemplary structure of an artificial intelligence-based model according to one embodiment of the present disclosure.

[0040] FIG. 3 is a flowchart illustrating a method for generating feedback information for a mathematical problem solving process according to one embodiment of the present disclosure.

[0041] FIG. 4 is a diagram for explaining an operation of obtaining user input information related to solving a mathematical problem according to one embodiment of the present disclosure.

[0042] FIG. 5 is a diagram for explaining an operation of analyzing a solution process for user input information obtained by utilizing a pre-learned artificial intelligence model according to one embodiment of the present disclosure.

[0043] FIG. 6 is a diagram for explaining an operation of generating feedback information according to one embodiment of the present disclosure.

[0044] FIG. 7 is a schematic diagram illustrating the result of awarding partial points for solving a math problem using a user interface according to one embodiment of the present disclosure.

[0045] FIG. 8 is a schematic diagram illustrating a user interface that provides comprehensive results for a mathematical problem solving process according to one embodiment of the present disclosure.

[0046] FIG. 9 is a schematic diagram illustrating a user interface related to a dashboard that allows checking learning history for each student according to one embodiment of the present disclosure.

[0047] FIG. 10 is a simplified, general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0048] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to facilitate understanding of the present disclosure. However, it will be apparent that these embodiments may be practiced without these specific details.

[0049] As used herein, the terms "component," "module," "system," and the like refer to computer-related entities, hardware, firmware, software, a combination of software and hardware, or an execution of software. For example, a component may be, but is not limited to, a procedure running on a processor, a processor, an object, a thread of execution, a program, and / or a computer. For example, both an application running on a computing device and the computing device may be a component. One or more components may reside within a processor and / or a thread of execution. A component may be localized within a single computer. A component may be distributed between two or more computers. Furthermore, these components may execute from various computer-readable media having various data structures stored therein. Components may communicate via local and / or remote processes, for example, by signals comprising one or more data packets (e.g., data from one component interacting with another component in a local system, a distributed system, and / or data transmitted to another system via a network such as the Internet via signals).

[0050] Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from context, "X employs A or B" is intended to mean either of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, "X employs A or B" can apply to any of these cases. Furthermore, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the associated items listed.

[0051] Additionally, the terms "comprises" and / or "comprising" should be understood to imply the presence of the features and / or components in question. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, components, and / or groups thereof. Furthermore, unless otherwise specified or clear from the context to refer to the singular form, the singular in the specification and claims should generally be construed to mean "one or more."

[0052] And, the term "at least one of A or B" should be interpreted to mean "if it includes only A", "if it includes only B", or "if it is combined in the composition of A and B".

[0053] Those skilled in the art should further appreciate that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0054] The description of the disclosed embodiments is provided to enable a person skilled in the art to make or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Therefore, the present invention is not limited to the embodiments disclosed herein. The present invention is to be construed in the widest scope consistent with the principles and novel features disclosed herein.

[0055] In the present disclosure, network function, artificial neural network and neural network can be used interchangeably.

[0056]

[0057] FIG. 1 is a block diagram of a computing device for generating feedback information for a mathematical problem solving process according to one embodiment of the present disclosure.

[0058] The configuration of the computing device (100) illustrated in FIG. 1 is merely a simplified example. In one embodiment of the present disclosure, the computing device (100) may include other configurations for performing the computing environment of the computing device (100), and only some of the disclosed configurations may constitute the computing device (100).

[0059] A computing device (100) may include a processor (110), memory (130), and network unit (150).

[0060] The processor (110) may be configured with one or more cores, and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. The processor (110) may read a computer program stored in the memory (130) to perform data processing for machine learning according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor (110) may perform operations for learning a neural network. The processor (110) may perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating weights of a neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) may process learning of a network function. For example, a CPU and a GPGPU can jointly process network function learning and data classification using network functions. Furthermore, in one embodiment of the present disclosure, processors of multiple computing devices can be jointly used to process network function learning and data classification using network functions. Furthermore, a computer program executed on a computing device according to one embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

[0061] According to one embodiment of the present disclosure, the memory (130) can store any form of information generated or determined by the processor (110) and any form of information received by the network unit (150).

[0062] According to one embodiment of the present disclosure, the memory (130) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may also operate in relation to web storage that performs the storage function of the memory (130) on the internet. The description of the above-described memory is merely an example, and the present disclosure is not limited thereto.

[0063] The network unit (150) according to one embodiment of the present disclosure can use various wired communication systems such as a public switched telephone network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed ​​DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and a local area network (LAN).

[0064] In addition, the network unit (150) presented in this specification can use various wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA) and other systems.

[0065] In the present disclosure, the network unit (150) may be configured regardless of the communication mode, such as wired or wireless, and may be configured as various communication networks, such as a local area network (LAN), a personal area network (PAN), and a wide area network (WAN). In addition, the network may be the well-known World Wide Web (WWW), and may also utilize a wireless transmission technology used for short-distance communication, such as infrared (IrDA: Infrared Data Association) or Bluetooth.

[0066] The techniques described in this specification can be used in other networks as well as the networks mentioned above.

[0067]

[0068] FIG. 2 illustrates an exemplary structure of an artificial intelligence-based model according to one embodiment of the present disclosure.

[0069] Throughout this specification, the terms artificial intelligence model, artificial intelligence-based model, computational model, neural network, network function, and neural network may be used interchangeably.

[0070] A neural network can be composed of a set of interconnected computational units, generally referred to as nodes. These nodes can also be referred to as neurons. A neural network consists of at least one node. The nodes (or neurons) that make up a neural network can be interconnected by one or more links.

[0071] Within a neural network, one or more nodes connected via links can form a relationship between input nodes and output nodes. The concept of input nodes and output nodes is relative, meaning that any node that is in an output node relationship with one node can also be in an input node relationship with another node, and vice versa. As described above, the relationship between input nodes and output nodes can be created based on links. One input node can be connected to one or more output nodes via links, and vice versa.

[0072] In a relationship between input nodes and output nodes connected through a single link, the data of the output node can have its value determined based on the data input to the input node. Here, the link interconnecting the input nodes and output nodes can have a weight. The weight can be variable and can be varied by the user or an algorithm so that the neural network can perform a desired function. For example, when one or more input nodes are interconnected to one output node through each link, the output node can determine the output node value based on the values ​​input to the input nodes connected to the output node and the weight set on the link corresponding to each input node.

[0073] As described above, a neural network is a network in which one or more nodes are interconnected through one or more links, forming input and output node relationships within the network. The characteristics of a neural network can be determined based on the number of nodes and links within the network, the relationships between the nodes and links, and the weights assigned to each link. For example, if two neural networks have the same number of nodes and links but different weight values ​​for the links, the two neural networks can be perceived as different from each other.

[0074] A neural network can be composed of a set of one or more nodes. A subset of the nodes comprising the neural network can form a layer. Some of the nodes comprising the neural network can form a layer based on their distances from the initial input node. For example, a set of nodes that are n distances from the initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links required to reach the node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within a neural network can be defined in a different way than described above. For example, a layer of nodes can be defined by its distance from the final output node.

[0075] In one embodiment of the present disclosure, a set of neurons or nodes may be defined as a layer.

[0076] An initial input node may refer to one or more nodes within a neural network into which data is directly input without going through links with other nodes. Alternatively, within a neural network, it may refer to nodes that do not have other input nodes connected by links in the relationship between nodes based on links. Similarly, a final output node may refer to one or more nodes within a neural network that do not have output nodes in their relationship with other nodes. Furthermore, a hidden node may refer to nodes that constitute a neural network other than the initial input node and the final output node.

[0077] A neural network according to one embodiment of the present disclosure may be a neural network in which the number of nodes in an input layer may be the same as the number of nodes in an output layer, and the number of nodes decreases and then increases as it progresses from the input layer to the hidden layer. In addition, a neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in an input layer may be less than the number of nodes in an output layer, and the number of nodes increases as it progresses from the input layer to the hidden layer. In addition, a neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in an input layer may be greater than the number of nodes in an output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer. A neural network according to another embodiment of the present disclosure may be a neural network in the form of a combination of the above-described neural networks.

[0078] An AI-based model according to one embodiment of the present disclosure may include a deep neural network (DNN). A DNN may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. Using a DNN, it is possible to identify latent structures in data. That is, the latent structures of a photo, text, video, voice, protein sequence structure, gene sequence structure, peptide sequence structure, music (e.g., what objects are in a photo, what the content and emotion of a text are, what the content and emotion of a voice are, etc.), and / or the binding affinity between a peptide and MHC can be identified. Deep neural networks may include convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, restricted Boltzmann machines (RBMs), deep belief networks (DBNs), Q-networks, U-networks, Siamese networks, generative adversarial networks (GANs), transformers, and the like. The description of the above-described deep neural networks is merely an example and the present disclosure is not limited thereto.

[0079] The artificial intelligence-based model of the present disclosure can be represented by a network structure of any structure described above, including an input layer, a hidden layer, and an output layer.

[0080] The neural network that can be used in the artificial intelligence-based model of the present disclosure may be trained using at least one of supervised learning, unsupervised learning, semi-supervised learning, transfer learning, active learning, or reinforcement learning. Training of the neural network may be a process of applying knowledge to the neural network to perform a specific action.

[0081] Neural networks can be trained to minimize output errors. This process involves repeatedly inputting training data into the neural network, calculating the neural network output and target error for the training data, and backpropagating the neural network error from the output layer to the input layer to update the weights of each node in the neural network to reduce the error. In supervised learning, training data with the correct answer for each training data is used (i.e., labeled training data). In unsupervised learning, the correct answer may not be labeled for each training data. For example, in supervised learning for data classification, the training data may be data with each category labeled. Labeled training data is input to the neural network, and the error can be calculated by comparing the output (category) of the neural network with the training data labels. Alternatively, in unsupervised learning for data classification, the error can be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated in the neural network in the backward direction (i.e., from the output layer to the input layer), and the connection weights of each node in each layer of the neural network can be updated according to the backpropagation. The amount of change in the connection weights of each node to be updated can be determined by the learning rate. The neural network's calculation of the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network's learning cycle. For example, a high learning rate can be used in the early stages of neural network training to quickly achieve a certain level of performance, thereby increasing efficiency. A lower learning rate can be used in the later stages of training to increase accuracy.

[0082] In neural network training, training data can typically be a subset of real-world data (i.e., the data to be processed using the trained neural network). Therefore, there can be a learning cycle where errors on the training data decrease but errors on the real-world data increase. Overfitting is a phenomenon where excessive training on the training data leads to increased errors on the real-world data. For example, a neural network trained on yellow cats may fail to recognize cats when shown non-yellow colors, a type of overfitting. Overfitting can increase errors in machine learning algorithms. Various optimization methods can be used to prevent overfitting. These methods include increasing the training data, regularization, dropout, which disables some nodes in the network during the learning process, and the use of batch normalization layers.

[0083]

[0084] Figure 3 is a flowchart illustrating a method for generating feedback information regarding a mathematical problem-solving process according to one embodiment of the present disclosure. Meanwhile, the method for generating feedback information regarding a mathematical problem-solving process, described below, can be performed by a computing device (100). For reference, the "user" referred to below may include students, course participants, etc. who solve mathematical problems provided by the computing device (100).

[0085] According to one embodiment of the present disclosure, with reference to FIG. 3, a method for generating feedback information for a math problem solving process may include a step of obtaining user input information related to math problem solving (S110), a step of analyzing the math problem solving process based on the obtained user input information by utilizing a pre-learned artificial intelligence model (S120), and a step of generating feedback information for the math problem solving based on the analysis (S130). Meanwhile, the computing device (100) may obtain user input information related to math problem solving obtained through various methods, analyze the math problem solving process by utilizing an artificial intelligence model, and provide feedback in a stepwise manner.

[0086]

[0087] FIG. 4 is a diagram for explaining an operation of obtaining user input information related to solving a mathematical problem according to one embodiment of the present disclosure.

[0088] According to one embodiment of the present disclosure, the computing device (100) can obtain user input information related to solving a math problem (S110). For example, the user input information related to solving a math problem may include handwritten letters, numbers, and mathematical symbols. For convenience of explanation, the user input information related to solving a math problem will be described hereinafter as being in Korean, but is not limited thereto and may include various languages ​​(e.g., English, Japanese, Chinese, etc.). For example, the user input information related to solving a math problem may include information in which the user writes an explanation for the problem or inputs a calculation process in the form of a formula. For example, the computing device (100) can obtain user input information entered through an app or the web. For example, the computing device (100) can obtain user input information in which the user solves a math problem through a web application or mobile app installed on the user's device. A user can directly input the process of solving a mathematical problem in digital format using a user device (e.g., a smartphone, a tablet PC, etc.), and the computing device (100) can obtain the user input information input through the user device. The computing device (100) can obtain the user input information through a mobile app, a tablet app, a tablet web, a PC web, etc. According to an embodiment, referring to FIG. 4, the computing device (100) can provide a mathematical problem (Q) to the user device before step S110. The user can solve the mathematical problem output on the user device (e.g., a mobile web or application, etc.) and input the solving process (S1 to S5) through a touch display of the user device, etc. For example, the user can sequentially input information related to solving a mathematical problem through the user device as shown in S1 to S5 in FIG. 4.Additionally, the computing device (100) may sequentially divide the process of interpreting the acquired user input information into S1 to S5.

[0089] As another example, the computing device (100) may obtain user input information obtained by scanning a piece of paper containing information related to solving a math problem. For example, a user may write a solution to a math problem on paper, and the computing device (100) may obtain the information obtained by scanning the paper as user input information. For example, a user may write a solution to a math problem step by step on paper, and the computing device (100) may obtain an image of the paper taken with a mobile camera or scanner as the user input information.

[0090] According to one embodiment, the computing device (100) can distinguish letters, numbers, and mathematical symbols by utilizing a character recognition model in order to accurately recognize and process information related to solving a mathematical problem in the acquired user input information. Here, letters may include variables, function symbols, text descriptions, etc. In addition, mathematical symbols may include +, -, =, ^, fractions, parentheses, subscripts, exponents, etc. For example, the character recognition model may include an optical character recognition (OCR) model, an intelligent character recognition (ICR) model, etc. First, the computing device (100) can distinguish elements included in the user input information by type. Since the user input information related to solving a mathematical problem includes general text (letters), numbers, mathematical symbols (operators and symbols), etc., the computing device (100) can distinguish elements included in the user input information by type in order to individually recognize them. For example, if user input information related to solving a math problem includes a formula such as f(x)= x^2+ 2x+1, the computing device (100) can distinguish “f” and “x” as letters, “^” and “+” as math symbols, and “2” and “1” as numbers. By distinguishing letters, numbers, and math symbols, the computing device (100) can enable a character recognition model to more accurately recognize elements included in the user input information. For example, the computing device (100) can distinguish letters, numbers, and math symbols from user input information obtained by a network such as CNN or R-CNN.

[0091] For example, the computing device (100) may utilize a character recognition model to perform recognition of each of the above-described letters, numbers, and mathematical symbols. For example, the character recognition model for recognizing letters, numbers, and mathematical symbols of the computing device (100) may be trained using a specialized data set. For example, the specialized data set may include various styles such as handwriting and printing, and may be data that reflects the characteristics of symbols or numbers frequently used in solving mathematical problems. In addition, the computing device (100) may secure diversity in the training data of the character recognition model through various forms of transformation such as rotation, distortion, and addition of noise. In addition, the character recognition model may include layers specialized for recognizing letters, numbers, and symbols, respectively, or specific layers of the model may be designed to optimize recognition performance for each category. In addition, the character recognition model may also be trained using a contextual flow (e.g., a pattern in which symbols or numbers are located at specific locations). For example, a character recognition model acquires specialized recognition capabilities for each element by learning letters, numbers, and mathematical symbols, enabling it to better understand mathematical meanings such as variables, constants, and operators, rather than simply general characters. Furthermore, the character recognition model can clearly distinguish between similar characters (e.g., "1" and "I," "0" and "O") without confusion, and can reduce recognition errors caused by misinterpretation of symbols (e.g., "-" and "="). Furthermore, by learning to accurately recognize various handwriting styles, sizes, and inclinations, the character recognition model can reduce errors in input information during actual problem solving. In other words, the character recognition model can acquire user input information related to mathematical problem solving, categorize it into letters, numbers, and mathematical symbols, and recognize them individually. For example, the character recognition model can accurately identify "x" as a variable and "+" as an operation symbol.Additionally, character recognition models can interpret consecutive occurrences of numbers and the operator "+" as an addition symbol. Furthermore, character recognition models can interpret symbols such as exponentiation (e.g., "^") as exponentiation when used with numbers, reducing potential misunderstandings during problem-solving.

[0092] According to one embodiment, the computing device (100) can perform recognition of separated letters, numbers, and mathematical symbols using a single character recognition model. Furthermore, the computing device (100) can perform recognition of each separated letter, number, and mathematical symbol using multiple character recognition models. Alternatively, the computing device (100) can perform recognition of the separated letters using a first optical recognition model. For example, the first optical recognition model may be an optical recognition model that is based on a language model to recognize various languages ​​and can supplement the context of the text through morphological analysis. An optical recognition model method that reflects the language structure and context can exhibit high performance, especially in long sentences or syntax analysis. Furthermore, the computing device (100) can perform recognition of the separated numbers using a second optical recognition model. For example, the second optical recognition model may be an optical recognition model for recognizing numbers. Since simple pattern recognition is important for numbers, a second optical recognition model specialized for recognizing numbers can be utilized to reduce misrecognition (e.g., confusion between “1” and “l”). The second optical recognition model may include a simple pattern matching model specialized for numbers and a network structure reflecting number characteristics. In addition, the computing device (100) may perform recognition of the distinguished mathematical symbols by utilizing a third optical recognition model. The third optical recognition model may be an optical recognition model for recognizing mathematical symbols. Since symbol and position information are important for mathematical expression recognition, the third optical recognition model may include a model that learns the relative positions of symbols. For example, a character recognition model that recognizes mathematical symbols may be a model that has learned to understand fractions, parentheses, subscripts, exponents, etc.

[0093] Alternatively, the computing device (100) may utilize an Intelligent Character Recognition (ICR) model to perform text recognition on the acquired user input information. For example, the ICR model may perform preprocessing on the acquired user input information for accurate recognition. Here, the acquired user input information may be collected in the form of an image. The ICR model may perform preprocessing such as increasing the resolution of the image, removing background noise, and adjusting the contrast. In addition, the ICR model may distinguish letters, numbers, and mathematical symbols from the acquired user input information. For example, the ICR model may distinguish between sentences and formulas by referring to positional rules and contexts commonly found in Korean sentences. In addition, the ICR model may distinguish between sentences and formulas by referring to positional rules and contexts commonly found in various languages. The ICR model may be trained to recognize various languages, numbers, mathematical formulas, special characters, etc. For example, if user input related to solving a math problem is in Korean, an intelligent character recognition model can recognize it and convert it into digital text. Furthermore, since numbers are a crucial element in solving math problems, an intelligent character recognition model can recognize numbers separately to avoid confusion with other characters and accurately recognize consecutive numbers. Furthermore, mathematical symbols such as '+', '-', '=', and '√' have different forms than regular text, so an intelligent character recognition model can be trained to distinguish them. In particular, because equations often consist of a combination of symbols and numbers, an intelligent character recognition model can recognize them as a whole, accurately distinguishing and analyzing them.For example, after recognizing letters, numbers, and mathematical symbols, an intelligent character recognition model can review the context of the recognized text to confirm whether the sentence was recognized correctly. For example, an intelligent character recognition model can check whether the formula "3x + 2 = 11" was recognized according to mathematical rules, and if numbers and symbols are listed in the wrong order, it can detect and correct errors. Furthermore, an intelligent character recognition model can use a language model in conjunction with the text to correct parts that do not match the context. For example, if the 'x' in the sentence "The area of ​​the rectangle is 3x2 = 11" is mistakenly recognized as the mathematical symbol 'x' instead of the letter, the language model can analyze the context to detect the error and correct it to "3x + 2 = 11."

[0094]

[0095] According to one embodiment of the present disclosure, the computing device (100) can analyze the solution process for acquired user input information by utilizing a pre-trained artificial intelligence model (S120). First, the computing device (100) can sequentially divide the solution process for acquired user input information. For example, the computing device (100) can set the division criteria based on the steps or logical flow that occur in the process of solving a mathematical problem. For example, the solution process for an equation problem can be sequentially divided by following each operation step. For example, the computing device (100) can sequentially divide the solution process for user input information by dividing it into problem understanding, solution method setting, calculation, and deriving the correct answer. For example, the computing device (100) can understand the conditions of the problem and identify necessary information. The computing device (100) can analyze the text of the problem, identify necessary formulas, or reorganize the conditions to sequentially divide the solution process for the acquired user input information. For example, the computing device (100) may sequentially divide the solution process for the acquired user input information based on a calculation step, which is a process of expanding an equation or performing a calculation and obtaining an intermediate result. In addition, the computing device (100) may sequentially divide the solution process for the acquired user input information based on a correct answer derivation step, which includes a process of reviewing the calculated value, confirming whether the unit or condition is correct, and deriving the final answer. For example, a math problem may be [Find the area of ​​a rectangle. Width: 3x + 2, Height: x - 1, x = 3].The computing device (100) can obtain, as user input information related to solving a mathematical problem, width = 3(3) + 2 = 11, height = 3 - 1 = 2, area of ​​the rectangle = width * height = 11 * 2 = 22 by substituting x = 3. Here, the computing device (100) can sequentially divide the solution process for the obtained user input information by utilizing a pre-learned artificial intelligence model into Step 1: Substituting x = 3, width = 3(3) + 2 = 11, height = 3 - 1 = 2, Step 2: area of ​​the rectangle = width * height = 11 * 2 = 22. Alternatively, the computing device (100) can also sequentially divide the solution process for the obtained user input information based on the problem type (e.g., differentiation, integration, equation, etc.). The computing device (100) can analyze the operations and access methods performed at each stage by sequentially dividing the interpretation process for the acquired user input information as described above. Referring to FIG. 4, for example, the computing device (100) can sequentially divide the interpretation process for the acquired user input information into stages S1 through S5.

[0096] According to one embodiment, the computing device (100) can analyze the sequentially segmented solving process in response to a rubric item by utilizing a pre-trained artificial intelligence model. For example, the computing device (100) can utilize the pre-trained artificial intelligence model to check whether the sequentially segmented solving process satisfies the rubric item. For example, the rubric item may include at least one of problem understanding, solution method, calculation accuracy, or derivation of a final correct answer. The above is merely an example and the present disclosure is not limited thereto. For example, the following description will be made based on an embodiment in which the math problem is [Find the area of ​​a rectangle. Width: 3x + 2, Height: x - 1, x = 3], and the segmented solving process is Step 1: Substituting x = 3, width = 3(3) + 2 = 11, height = 3 - 1 = 2, Step 2: Area of ​​the rectangle = width * height = 11 * 2 = 22. For example, in the Problem Understanding item, it is possible to evaluate whether the user accurately interpreted and reflected the conditions given in the problem. The computing device (100) can use a pre-trained artificial intelligence model to determine whether the value x = 3 given in Step 1 was correctly substituted in the Problem Understanding item among the rubric items. In addition, the computing device (100) can use a pre-trained artificial intelligence model to compare the input problem content with the initial stage of the solving process to analyze whether the user correctly understood the conditions of the problem (the process of substituting the x value for the width and height). In addition, in the Solution Approach item, it is possible to evaluate whether a solution method appropriate for the problem was applied. The computing device (100) can use a pre-trained artificial intelligence model to evaluate whether the user appropriately used the formula for multiplying the width and height to find the area of ​​a rectangle (geometric area calculation method).In addition, the Calculation Accuracy item can evaluate whether the formula calculation of each step was performed accurately. The computing device (100) can evaluate whether the substitution and calculation in Step 1 and Step 2 are accurate by utilizing a pre-trained artificial intelligence model. For example, the computing device (100) can evaluate whether the calculations 3(3)+2=11 and 3-1=2 are correct, and whether the final calculation 11*2=22 is correct by utilizing a pre-trained artificial intelligence model. In addition, the Final Answer Derivation item can evaluate whether the correct answer was derived in the final step of problem solving. The computing device (100) can evaluate whether the finally derived answer is correct as the area of ​​a rectangle, that is, whether the final correct answer was accurately calculated as 22 by utilizing a pre-trained artificial intelligence model. For example, the computing device (100) inputs a sequentially divided solution process into a pre-trained artificial intelligence model, and the pre-trained artificial intelligence model can predict how much each of the sequentially divided solution processes satisfies a specific rubric item.

[0097] According to one embodiment, the above-described pre-trained artificial intelligence model may be trained based on sequentially segmented learning solution process data and operations such as evaluating each of the sequentially segmented learning solution process data based on rubric items. First, the computing device (100) may acquire learning solution process data for training the pre-trained artificial intelligence model. For example, the learning solution process data may include data for model training by collecting various types of problems and solution cases. In addition, the learning solution process data may include all steps required in the actual solution process, and each step may be linked to a specific rubric criterion. In addition, the computing device (100) may sequentially segment the acquired learning solution process data. For example, the computing device (100) may divide the solution process into several steps according to the problem type, and may distinguish them based on the purpose and solution method performed by each step. For example, a differentiation problem may be divided into steps such as "formula organization → application of differentiation rule → result organization." In addition, the computing device (100) can obtain rubric items, which are evaluation criteria for each sequentially segmented learning solution process data. For example, the computing device (100) can set a rubric criterion that includes at least one of problem understanding, solution method, calculation accuracy, or derivation of a final answer that aligns with the learning objective for each step, so as to evaluate how well the solution process satisfies the criteria. For example, in the "problem understanding" step, rubric items for various evaluation criteria can be obtained to evaluate whether the key information of the problem has been accurately grasped, and in the "calculation accuracy" step, whether the calculation has been performed accurately.Additionally, the computing device (100) may train a pre-trained artificial intelligence model to evaluate each of the sequentially segmented learning solution process data based on the previously acquired sequentially segmented learning solution process data and the rubric items. The pre-trained artificial intelligence model may be trained to predict the extent to which the sequentially segmented learning solution process data satisfies a specific rubric item.

[0098] In another embodiment, the pre-trained AI model may be a model trained through supervised learning. For example, the computing device (100) may acquire learning solution process data. Furthermore, the computing device (100) may sequentially segment the learning solution process data. The sequentially segmented learning solution process data may be assigned a label (e.g., met or not met) indicating whether the criteria corresponding to the rubric item are met for each step. Furthermore, the computing device (100) may extract features for at least one of problem understanding, solution method, calculation accuracy, or final answer derivation, and may obtain information indicating whether the criteria are met (e.g., keywords, operation structure, suitability of the development method, etc.). Furthermore, the computing device (100) may set an optimized model for each specific rubric item to predict feedback for each step of the problem. For example, the computing device (100) may utilize an RNN or Transformer-based model suitable for text analysis for a rubric item that evaluates problem understanding. In addition, the computing device (100) can utilize a classification model for formula verification, since computational accuracy is important for computational verification. In addition, the computing device (100) can perform learning using a loss function (e.g., cross-entropy loss) to reduce the difference between the value predicted by a pre-trained artificial intelligence model for each rubric item and the actual label. The loss function can determine the learning direction so that the pre-trained artificial intelligence model reduces the error when predicting whether the criterion is met. When new solution process data is input, the pre-trained artificial intelligence model can output a predicted score or probability value to determine whether the criterion for each rubric item is met. For example, the pre-trained artificial intelligence model can provide a result such as “This step satisfies 85% of the problem understanding item”, and can quantitatively provide whether the criterion is met.Furthermore, pre-trained AI models can provide learners with feedback on areas for improvement based on their predictions regarding whether each rubric criterion has been met. For example, they can provide feedback such as, "The calculation accuracy is 60% met, so please review the calculation in that step." Furthermore, pre-trained AI models can award partial points based on the degree to which rubric criteria have been met.

[0099] In another embodiment, a pre-trained AI model may include sub-models specialized for each rubric item. For example, the sub-models may include a first model for measuring the ability to understand the problem's objectives and conditions. Furthermore, the sub-models may include a second model for evaluating the solution approach or logical development. Furthermore, the sub-models may include a third model for evaluating the computational accuracy of each sequentially segmented training solution process data. Furthermore, the sub-models may include a fourth model for determining whether the final derived answer satisfies the problem's requirements. These sub-models may be configured to share a common intermediate layer and exchange information with each other. For example, the intermediate layer may employ a multi-task learning method to learn the characteristics of multiple rubric items and reflect the relationships between them. For example, since problem understanding and solution approaches are highly correlated, sharing information between these two items in the intermediate layer can improve prediction accuracy. This structure allows for independent evaluation of each rubric item while efficiently utilizing information between highly related items. For example, a pre-trained AI model can take new data and predict how well it meets the criteria for each rubric item.

[0100]

[0101] FIG. 5 is a diagram for explaining an operation of analyzing a solution process for user input information obtained by utilizing a pre-learned artificial intelligence model according to one embodiment of the present disclosure.

[0102] According to one embodiment, the computing device (100) can analyze whether the characters included in the user input are directly related to the solution of the math problem or are additional characters that are not directly related to the solution of the math problem. For example, referring to FIG. 5, the acquired user input information may include first input information (1) and second input information (2). The computing device (100) can analyze whether the first input information (1) and the second input information (2) are characters that are directly related to the solution of the math problem or are additional characters that are not directly related to the solution of the math problem. For reference, the third input information (3) is feedback information. For example, referring to FIG. 5, the computing device (100) can analyze the second input information (2) that is additional characters that are not directly related to the solution of the math problem by utilizing a pre-trained artificial intelligence model. For example, the computing device (100) can identify additional characters in the acquired user input information by utilizing natural language processing (NLP) and context analysis techniques. For example, the computing device (100) can classify sentences based on context. The computing device (100) can first analyze sentences included in the user input to separate out words that are not related to key words related to problem solving (e.g., differentiation, maximum, minimum, function, extremum, etc.). For example, a sentence such as the second input information (2) corresponding to “I don’t know how to find the minimum” includes key terms directly related to solving a math problem, but the computing device (100) can recognize that it is an expression indicating a user explanation or difficulty in the context. In addition, the computing device (100) can determine the intent of each sentence. For example, the computing device (100) can determine whether the intent is problem solving, a question or request, or a general opinion unrelated to the solution.For example, the computing device (100) may interpret the second input information (2) corresponding to “I don’t know how to find the minimum” as a request for help or a statement of opinion rather than a calculation or logic directly related to problem solving. As another example, the computing device (100) may analyze whether the characters included in the user input are characters directly related to the solution of the mathematical problem or are additional characters not directly related to the solution of the mathematical problem, considering keyword filtering, pattern recognition, location-based analysis, semantic similarity evaluation, etc. For another example, if the characters included in the user input information are “What if you take the log of ~?”, the input information may be determined to be characters directly related to the solution of the mathematical problem. Meanwhile, the computing device (100) may determine that input information including formulas and mathematical symbols, such as the first input information (1), is characters directly related to the solution of the mathematical problem.

[0103] For example, the computing device (100) can identify the language of characters included in the user input if the characters are directly related to the solution of the mathematical problem. For example, the computing device (100) can identify whether the characters included in the user input are Korean or another language (e.g., English, Japanese, Chinese, etc.) by utilizing natural language processing (NLP) technology. Through this, the grammar and structure (language system) of each language can be reflected in the subsequent analysis process to more accurately associate text and formula. In addition, the computing device (100) can identify formulas associated with characters among formulas included in the solution process by considering the identified language system. For example, in Korean, when an expression such as “What if you take the log of ~?” appears, this can be interpreted to mean applying a logarithmic operation to the formula. On the other hand, in English, an expression such as “Apply log to ~” is used with the same meaning. In this way, by reflecting the system of each language, sentence patterns that signify specific operations (e.g., logarithms, differentiation, integration) can be recognized, and the correlation with the corresponding formula can be analyzed based on this. In addition, the computing device (100) can generate analysis information based on the semantic information of the characters and the identified formula. Through semantic analysis, the computing device (100) can determine which formula an operation, function, or variable name mentioned in the text is associated with. For example, if there is a text such as “take log,” the computing device (100) can search for formulas or variables related to the logarithmic function to identify the associated formula, and generate analysis information based on the identified formula. In addition, in the case of English, the computing device (100) can identify that a text such as “Apply log to x” is associated with the variable x and the logarithmic function, and generate analysis information based on the identified formula. For reference, the analysis information may be input data input to a pre-trained artificial intelligence model.

[0104] For example, if the computing device (100) contains additional characters that are not directly related to solving a mathematical problem, the computing device (100) may generate analysis information based on the remaining portion excluding the additional characters. Referring again to FIG. 5, the computing device (100) may input the first input information (1) into a pre-trained artificial intelligence model, excluding the second input information (2), which is an additional character that is not directly related to solving a mathematical problem, to analyze the mathematical problem-solving process.

[0105]

[0106] FIG. 6 is a diagram for explaining an operation of generating feedback information according to one embodiment of the present disclosure, and FIG. 7 is a diagram schematically showing a result of granting partial scores for solving a math problem using a user interface according to one embodiment of the present disclosure.

[0107] According to one embodiment of the present disclosure, the computing device (100) may generate feedback information regarding the solution of a math problem based on the analysis (S130). For example, the computing device (100) may generate feedback information regarding the solution of a math problem based on whether a rubric item, which is an analysis result of a pre-trained artificial intelligence model, is satisfied. For example, if the satisfaction of a rubric item, which is an analysis result of a pre-trained artificial intelligence model, is not 100% satisfied, the computing device (100) may generate feedback information regarding the solution of a math problem in response to the unsatisfied item. For example, the feedback information may include feedback related to the type of error corresponding to the rubric item, quantitative feedback, qualitative feedback, etc. In addition, the computing device (100) may generate feedback information regarding the sequentially divided solution process based on the analyzed results corresponding to the rubric item. For example, the computing device (100) may generate feedback information regarding the solution process that requires improvement in response to the rubric item. Alternatively, the computing device (100) may utilize a language model to generate feedback information for solving a mathematical problem based on analysis. For example, the computing device (100) may generate prompts based on analysis results from a pre-trained artificial intelligence model, rubric items, and sequentially segmented problem-solving processes to generate feedback information using a language model.

[0108] For example, if a specific rubric item is not met, the computing device (100) can generate specific feedback information such as “The distributive law was not applied correctly in this step” or “The sign was used incorrectly in addition.” In addition, the computing device (100) can provide an explanation for each rubric item in a language that is easy for the user to understand. For example, the computing device (100) can generate feedback information that encourages learning by adding feedback such as “Your understanding of the problem is correct and your method of solving it is also correct. However, there was an error in the final calculation, so the answer is incorrect. Please check the calculation again.” In addition, the computing device (100) can also generate feedback information such as “Since the calculation accuracy is 60% met, please review the calculation in the corresponding step again” based on the prediction result of whether each rubric criterion is met. Referring to FIG. 6 as an example, the computing device (100) can acquire 1-1 user input information (S-1) and 1-2 user input information (S-2) related to solving a math problem in response to a 1-1 math problem (Q-1). In addition, the computing device (100) can analyze the math problem solving process based on the acquired user input information by utilizing a pre-learned artificial intelligence model. At this time, the computing device (100) can sequentially divide the solving process for the acquired user input information and analyze the solving process in response to the rubric items by utilizing the pre-learned artificial intelligence model. The computing device (100) can generate feedback information on the solving process that requires improvement in response to the rubric items. For example, the computing device (100) can generate feedback information on the solution process that requires improvement in response to the rubric item, such as 1-1 feedback information (F-1), based on the analysis results of the pre-learned artificial intelligence model for the 1-2 user input information (S-2).

[0109] In addition, the computing device (100) can generate feedback information for the sequentially divided solution process by assigning partial scores according to the degree of satisfaction of each rubric item. The computing device (100) can quantitatively provide partial scores for each rubric item to generate scores received by the learner at each stage. For example, if the problem understanding and solution method are satisfied but an error occurs in the calculation, the computing device (100) can reflect this as a partial score. For example, referring to FIG. 7, the computing device (100) can assign partial scores for the analyzed solution process and add the rubric item to which the partial score was assigned to generate feedback information.

[0110] According to one embodiment, the computing device (100) may generate feedback information based on additional characters that are not directly related to solving a math problem, if the additional characters satisfy a predetermined condition. The predetermined condition may include a case where the additional characters include keywords requesting feedback, such as “I don’t know,” “Please help me,” or “It’s difficult.” Referring again to FIG. 5 as an example, the computing device (100) may input the second input information (2), which is an additional character that is not directly related to solving a math problem, but satisfies a predetermined condition, into a pre-trained artificial intelligence model to generate feedback information for the additional characters. In this case, the pre-trained artificial intelligence model may generate feedback information in a way that provides only partial assistance to the user, rather than providing the entire solution process for the second input information (2). A pre-trained AI model may not provide a complete solution to the second input (2), but rather provide some necessary information (e.g., the next formula to use, hints, etc.) to help the user consider the next solution. This provides the user with additional clues for problem-solving and allows them to continue solving the problem on their own.

[0111]

[0112] According to one embodiment of the present disclosure, the computing device (100) may provide user-tailored problems related to the solution, taking into account the analyzed solution process. For example, the computing device (100) may analyze user input information related to solving a mathematical problem to identify errors, weaknesses, and insufficient conceptual understanding that occurred during the process of solving a specific problem. For example, if it is confirmed that the user repeatedly makes calculation errors during the problem-solving process or does not understand a specific mathematical principle (e.g., the rules for addition and multiplication of fractions), the computing device (100) may identify areas in which the user needs to improve based on this and generate user-tailored problems related to the solution. For example, the computing device (100) may also generate customized problems, taking into account the analyzed solution process according to rubric items. If the analysis results show that there are many incorrect answers to the problem understanding item among the rubric items, the computing device (100) may generate problems that help users who have difficulty understanding the problem clearly understand the key conditions of the problem. For example, the computing device (100) can help the user grasp the essence of the problem by providing problems that are reconstructed with simpler conditions or problems that emphasize important conditions, and can then generate customized problems that can increase the user's level of understanding by gradually increasing the complexity of the conditions. In addition, if the analysis results show that there are many incorrect answers regarding the solution method among the rubric items, the computing device (100) can generate problems that allow the user to apply specific strategies or concepts necessary for problem solving. For example, the computing device (100) can generate problems that can teach the user the understanding of the solution process and the logical development method by providing problems that induce the solution method through hints or by informing the user in advance of frequently used formulas in a specific type of problem.In addition, if the analysis results show that there are many incorrect answers regarding calculation accuracy among the rubric items, the computing device (100) can generate problems that start with relatively simple calculations and then require complex calculations to improve calculation accuracy. For example, if mistakes frequently occur in addition or multiplication, the computing device (200) can generate problems that focus on the relevant calculations so that calculation accuracy can be improved through repeated learning. In addition, if the analysis results show that there are many incorrect answers regarding deriving the final answer among the rubric items, the computing device (100) can provide problems that require verification of answers in the middle of the solution process or generate problems that enhance attention in the final stage to improve the accuracy of deriving the final answer. This can help the user reduce mistakes in the final stage of the solution and learn while checking the flow of the entire solution process. This method of generating customized problems for each rubric item can help the user clearly identify their shortcomings and supplement their weaknesses through repeated learning. Since feedback is provided in real time at each stage, it can maximize learning effectiveness and ultimately contribute greatly to comprehensively improving mathematical thinking and problem-solving skills.

[0113]

[0114] FIG. 8 is a schematic diagram illustrating a user interface that provides comprehensive results for a mathematical problem solving process according to one embodiment of the present disclosure.

[0115] According to one embodiment of the present disclosure, the computing device (100) can provide learning outcomes, customized learning, personalized learning status, etc., as shown in (a) of FIG. 8. The computing device (100) can display learning achievements, such as "Reached the top 5%," which can visually show the user's achievements and increase motivation. In addition, the computing device (100) can provide a notification, such as "New learning has arrived!", so that the user can check the learning progress of a specific period. In addition, the computing device (100) can display progress and understanding information related to recently studied subjects (e.g., advanced mathematics) as a percentage, so that the user can understand his or her learning progress and understanding. For example, the computing device (100) can also provide detailed information on individual lectures to be studied today, as shown in (b) of FIG. 8. The section marked "Concept Lecture" introduces the title and outline of the lecture to be learned today, and the lecture title can help users clearly understand the content to be learned by including details related to the topic (e.g., "Least Common Multiple and Greatest Common Divisor"). In addition, the lecture details include detailed information such as the lecture length (42 minutes), so that users can plan the time required to proceed with the lecture in advance. In addition, the name of the instructor who provided the lecture (e.g., Tutor Kim Seol-myeong) can be displayed. In addition, the play button in the center allows the user to immediately start the lecture, which is an interface element that helps facilitate the learning process. For example, the computing device (100) can display a summary of the learning record for the past week, as shown in (c) of FIG. 8, and can provide the user with the opportunity to periodically review the user's learning performance. The computing device (100) can provide numerical values ​​for items such as the units studied, the predicted achievement level for each unit, and the amount of learning.

[0116]

[0117] Figure 9 is a schematic diagram of a user interface related to a dashboard that allows teachers to check individual student learning histories according to one embodiment of the present disclosure. For reference, Figure 9 is a user interface screen of the dashboard, which allows teachers to efficiently manage and check individual student learning histories. It is designed to allow teachers to view the progress of each assignment and the status of individual student assignment submissions at a glance, allowing teachers to quickly assess students' learning participation and achievement.

[0118] The computing device (100) can display a list of collaborative assignments managed by the teacher by date and visually provide the submission status of each assignment. For example, if the "09 / 29 Joint Assignment" item designated on a specific date is marked with 0% submission, it means that the assignment has not yet been submitted. If the "09 / 28 Joint Assignment" and "09 / 27 Joint Assignment" items are marked with 100% submission, it means that all students have submitted the assignment. In addition, each assignment item can be configured to display the grading progress status, such as "grading in progress" or "grading complete," in addition to the submission status, so that the teacher can easily check the grading status. For example, the computing device (100) can include assignment details in the assignment list, providing information such as the number of questions, target student group, submission start and deadline times, and whether late submissions are allowed. This allows the teacher to clearly understand the assignment conditions and deadlines and provide appropriate guidance to students based on this information. In addition, each assignment includes information about the learning unit associated with it, allowing teachers and students to easily identify the unit they are learning through the assignment (e.g., remainder theorem, exponential and logarithmic functions). The computing device (100) can provide learning history for each student, allowing teachers to comprehensively manage each student's learning status. The individual submission status of each student is displayed, and the frequency with which each student has submitted the joint assignment and the submission status can be intuitively confirmed. For example, if a specific student has no submission history or has a low assignment submission rate, the computing device (100) can help the teacher easily recognize the need to encourage the student's participation in the assignment. In addition, the computing device (100) also displays the personalized assignment status, indicating the progress of each student's personalized assignment as "awaiting submission" or "awaiting grading," allowing the teacher to systematically manage the student's learning progress.Additionally, information on the most recent learning units currently being studied by each student is provided, allowing teachers to identify learning units that require supplementation based on the student's learning history and use this as a basis for providing additional learning assignments.

[0119] Meanwhile, a user interface related to a dashboard that allows teachers to view individual student learning histories can help teachers comprehensively analyze each student's learning history and provide personalized instruction tailored to their individual learning needs. Teachers can easily check the submission and grading status of each assignment to understand the learning status of the entire class and provide supplementary instruction or feedback based on individual student learning histories and understanding.

[0120] The steps described in the above description may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present disclosure. Furthermore, some steps may be omitted as needed, and the order of the steps may be changed.

[0121]

[0122] Meanwhile, a computer-readable medium storing a data structure according to an embodiment of the present disclosure is disclosed.

[0123] A computer-readable medium storing a data structure according to one embodiment of the present disclosure is disclosed. The aforementioned data structure can be stored in a memory within the present disclosure, executed by a processor, and transmitted and received by a network unit.

[0124] A data structure can refer to the organization, management, and storage of data that enables efficient access and modification. A data structure can refer to the organization of data to solve specific problems (e.g., data analysis, data retrieval, data storage, data modification). A data structure can also be defined as the physical or logical relationships between data elements designed to support specific data processing functions. Logical relationships between data elements can include connections between user-defined data elements. Physical relationships between data elements can include actual relationships between data elements physically stored on a computer-readable storage medium (e.g., persistent storage). Specifically, a data structure can include a collection of data, relationships between data, and functions or commands applicable to the data. An effectively designed data structure allows a computing device to perform operations while minimizing the use of its resources. Specifically, a computing device can improve the efficiency of operations, reading, inserting, deleting, comparing, exchanging, and searching through an effectively designed data structure.

[0125] Data structures can be categorized as linear or nonlinear, depending on their form. A linear data structure can be a structure in which only one piece of data is linked to the next. Linear data structures can include lists, stacks, queues, and deques. A list can refer to a series of data sets with an internal order. Lists can also include linked lists. A linked list is a data structure in which data is linked in a single line, each piece having a pointer. In a linked list, a pointer can contain information about the next or previous piece of data. Linked lists can be expressed as singly linked lists, doubly linked lists, or circular linked lists, depending on their form. A stack can be a data listing structure with limited data access. A stack can be a linear data structure in which data operations (e.g., insertion or deletion) can only be performed at one end of the data structure. Data stored in a stack can be a Last-in-First-out (LIFO) data structure. A queue is a data structure with limited access to data. Unlike a stack, it can be a first-in, first-out (FIFO) data structure, with later data being retrieved later. A deck can be a data structure that can process data at both ends.

[0126] A nonlinear data structure can be a structure in which multiple pieces of data are connected behind a single piece of data. Nonlinear data structures can include graph data structures. A graph data structure can be defined by vertices and edges, and an edge can include a line connecting two different vertices. Graph data structures can include tree data structures. A tree data structure can be a data structure in which there is only one path connecting two different vertices among multiple vertices included in the tree. In other words, it can be a data structure that does not form a loop in a graph data structure.

[0127] Throughout this specification, the terms artificial intelligence-based model, computational model, neural network, network function, and neural network may be used interchangeably. Hereinafter, they are collectively referred to as neural networks. A data structure may include a neural network. And, a data structure including a neural network may be stored on a computer-readable medium. A data structure including a neural network may also include preprocessed data for processing by a neural network, data input to a neural network, neural network weights, neural network hyperparameters, data obtained from a neural network, activation functions associated with each node or layer of a neural network, loss functions for neural network learning, etc. A data structure including a neural network may include any of the components disclosed above. That is, a data structure including a neural network may be configured to include all or any combination of preprocessed data for processing by a neural network, data input to a neural network, neural network weights, neural network hyperparameters, data obtained from a neural network, activation functions associated with each node or layer of a neural network, loss functions for neural network learning, etc. In addition to the aforementioned configurations, a data structure including a neural network may include any other information that determines the characteristics of the neural network. Furthermore, the data structure may include any form of data used or generated in the computational process of the neural network, and is not limited to the aforementioned. The computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. A neural network may be composed of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node.

[0128] The data structure may include data input to a neural network. The data structure including the data input to the neural network may be stored on a computer-readable medium. The data input to the neural network may include training data input during the neural network training process and / or input data input to the neural network after training has been completed. The data input to the neural network may include data that has undergone preprocessing and / or data that is the target of preprocessing. Preprocessing may include a data processing process for inputting data to the neural network. Accordingly, the data structure may include data that is the target of preprocessing and data generated by the preprocessing. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

[0129] The data structure may include weights of the neural network. (In this specification, the terms "weight" and "parameter" may be used interchangeably.) And the data structure including the weights of the neural network may be stored in a computer-readable medium. The neural network may include a plurality of weights. The weights may be variable and may be varied by a user or an algorithm so that the neural network can perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node may determine a data value output from the output node based on the values ​​input to the input nodes connected to the output node and the weights set for the links corresponding to each input node. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

[0130] By way of example and not limitation, the weights may include weights that vary during the neural network training process and / or weights that have completed neural network training. The weights that vary during the neural network training process may include weights at the start of the training cycle and / or weights that vary during the training cycle. The weights that have completed neural network training may include weights that have completed the training cycle. Accordingly, a data structure including the weights of a neural network may include a data structure including weights that vary during the neural network training process and / or weights that have completed neural network training. Therefore, the above-described weights and / or combinations of each weight are included in the data structure including the weights of a neural network. The above-described data structures are merely examples and the present disclosure is not limited thereto.

[0131] A data structure including neural network weights can be stored in a computer-readable storage medium (e.g., memory, hard disk) after going through a serialization process. Serialization can be a process of converting a data structure into a form that can be stored on the same or different computing devices and later reconstructed and used. A computing device can serialize the data structure to transmit and receive data over a network. The serialized data structure including neural network weights can be reconstructed on the same computing device or another computing device through deserialization. The data structure including neural network weights is not limited to serialization. Furthermore, the data structure including neural network weights can include a data structure that increases computational efficiency while minimizing the use of computing device resources (e.g., a B-Tree, an R-Tree, a Trie, an m-way search tree, an AVL tree, a Red-Black Tree in nonlinear data structures). The foregoing is merely an example, and the present disclosure is not limited thereto.

[0132] The data structure may include hyperparameters of a neural network. Furthermore, the data structure including the hyperparameters of the neural network may be stored on a computer-readable medium. The hyperparameters may be variables that can be varied by the user. The hyperparameters may include, for example, a learning rate, a cost function, the number of learning cycle repetitions, weight initialization (e.g., setting a range of weight values ​​to be subject to weight initialization), and the number of hidden units (e.g., the number of hidden layers, the number of nodes in the hidden layer). The above-described data structure is merely an example, and the present disclosure is not limited thereto.

[0133]

[0134] FIG. 10 is a simplified, general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0135] Although the present disclosure has been described above as being generally implemented by a computing device, those skilled in the art will appreciate that the present disclosure may also be implemented in combination with computer-executable instructions and / or other program modules that may be executed on one or more computers and / or as a combination of hardware and software.

[0136] Generally, program modules include routines, programs, components, data structures, and the like that perform specific tasks or implement specific abstract data types. Furthermore, those skilled in the art will appreciate that the methods of the present disclosure can be implemented with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which may be operatively connected to one or more associated devices.

[0137] The described embodiments of the present disclosure can also be practiced in distributed computing environments, where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0138] Computers typically include a variety of computer-readable media. Computer-readable media can be any media that can be accessed by a computer, and includes both volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes both volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be accessed by a computer and used to store the desired information.

[0139] Computer-readable transmission media typically includes any information delivery media that embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal that has one or more of its characteristics set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, or other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

[0140] An exemplary environment for implementing various aspects of the present disclosure is illustrated, including a computer (1102), which includes a processing unit (1104), system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including but not limited to the system memory (1106), to the processing unit (1104). The processing unit (1104) may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be utilized as the processing unit (1104).

[0141] The system bus (1108) may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. The system memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). A basic input / output system (BIOS) is stored in non-volatile memory (1110), such as ROM, EPROM, or EEPROM, and includes basic routines that help transfer information between components within the computer (1102), such as during start-up. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.

[0142] The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA) - which may also be configured for external use within a suitable chassis (not shown), a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from or writing to a CD-ROM disk (1122) or other high-capacity optical media such as a DVD). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128), respectively. The interface (1124) for implementing an external drive includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

[0143] These drives and their associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of the computer (1102), the drives and media correspond to storing any data in a suitable digital format. While the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those of ordinary skill in the art will appreciate that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, may also be used in the exemplary operating environment, and that any such media may contain computer-executable instructions for performing the methods of the present disclosure.

[0144] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or portions of the operating system, applications, modules, and / or data may also be cached in RAM (1112). It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0145] A user may enter commands and information into the computer (1102) via one or more wired / wireless input devices, such as a keyboard (1138) and a pointing device such as a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, and the like. These and other input devices are often connected to the processing unit (1104) via an input device interface (1142) that is connected to the system bus (1108), but may be connected by other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, and the like.

[0146] A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface, such as a video adapter (1146). In addition to the monitor (1144), the computer typically includes other peripheral output devices (not shown), such as speakers, a printer, and so on.

[0147] The computer (1102) may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) (1148), via wired and / or wireless communications. The remote computer(s) (1148) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and generally include many or all of the components described for the computer (1102), although for simplicity, only the memory storage device (1150) is shown. The logical connections shown include wired / wireless connections to a local area network (LAN) (1152) and / or a larger network, such as a wide area network (WAN) (1154). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which may be connected to a worldwide computer network, such as the Internet.

[0148] When used in a LAN networking environment, the computer (1102) is connected to a local network (1152) via a wired and / or wireless communication network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communications to the LAN (1152), which may also include a wireless access point installed therein for communicating with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communications computing device on the WAN (1154), or have other means of establishing communications over the WAN (1154), such as via the Internet. The modem (1158), which may be internal or external and wired or wireless, is connected to the system bus (1108) via a serial port interface (1142). In a networked environment, program modules or portions thereof described for the computer (1102) may be stored in a remote memory / storage device (1150). It will be appreciated that the network connections depicted are exemplary and other means of establishing a communications link between the computers may be used.

[0149] The computer (1102) operates to communicate with any wireless device or object that is arranged and operates via wireless communication, such as a printer, a scanner, a desktop and / or portable computer, a portable data assistant (PDA), a communication satellite, any equipment or location associated with a radio-detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network, or may simply be an ad hoc communication between at least two devices.

[0150] Wi-Fi (Wireless Fidelity) enables connections to the Internet and other devices without wires. Wi-Fi is a wireless technology that allows devices, such as computers, to send and receive data anywhere within the coverage area of ​​a base station, both indoors and outdoors, similar to cell phones. Wi-Fi networks use wireless technologies called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, at data rates of, for example, 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual-band).

[0151] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0152] Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein, for convenience, as software), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0153] The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Furthermore, various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0154] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented.

[0155] The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments disclosed herein, but is to be construed in the broadest scope consistent with the principles and novel features disclosed herein.

[0156] As described above, the relevant contents have been described in the best form for carrying out the invention.

Claims

1. A method for generating feedback information on a mathematical problem solving process performed by a computing device, A step of obtaining user input information related to solving a mathematical problem; A step of analyzing the process of solving the mathematical problem based on the acquired user input information by utilizing a pre-learned artificial intelligence model; and Based on the above analysis, a step of generating feedback information on the solution to the math problem Including, method.

2. In paragraph 1, The step of obtaining user input information related to solving the above mathematical problem is: A step for obtaining user input information entered through an app or web; or A step of obtaining user input information by scanning a paper containing information related to solving the above mathematical problem; Containing at least one of: method.

3. In paragraph 1, The step of obtaining user input information related to solving the above mathematical problem is: A step of distinguishing letters, numbers and mathematical symbols from the acquired user input information; A step of performing recognition on the above-described separated characters by utilizing a character recognition model; A step of performing recognition of the separated numbers by utilizing the above character recognition model; and A step of performing recognition of the above-described mathematical symbols by utilizing the above-described character recognition model. Including, method.

4. In paragraph 1, The step of analyzing the interpretation process for the acquired user input information by utilizing the above pre-learned artificial intelligence model is as follows. A step of sequentially dividing the interpretation process for the acquired user input information; and A step of analyzing the sequentially divided solution process corresponding to the rubric items by utilizing the above pre-learned artificial intelligence model; Including, method.

5. In paragraph 4, The above rubric items are: Including at least one of problem understanding, solution method, computational accuracy, or deriving a final answer. method.

6. In paragraph 4, The above pre-trained artificial intelligence model is, The action of acquiring learning solution process data; An action of sequentially dividing the above learning solution process data; An operation of obtaining the rubric items, which are evaluation criteria for each of the sequentially divided learning solution process data; and An operation of predicting whether the sequentially divided learning solution process data satisfies the rubric items; Based on learning, method.

7. In paragraph 6, The above pre-trained artificial intelligence model is, Model 1, which measures the ability to identify the goals and conditions of a problem; A second model that evaluates the solution approach or logical development; A third model for evaluating the computational accuracy for each of the sequentially divided learning solution process data; or A fourth model that determines whether the final solution meets the requirements of the problem. Containing at least one of: method.

8. In paragraph 4, Based on the above analyzed solution process, the step of generating feedback information is: A step of generating feedback information for the sequentially divided solution process based on the analyzed results corresponding to the above rubric items; Including, method.

9. In paragraph 8, Based on the results analyzed in response to the above rubric items, the step of generating feedback information for the sequentially divided solution process is: A step of generating feedback information on the solution process that requires improvement in response to the above rubric items; or A step of generating feedback information for the sequentially divided solution process by granting partial scores according to the degree of satisfaction of each rubric item; Including any one of the following, method.

10. In paragraph 1, The step of analyzing the mathematical problem solving process based on the acquired user input information by utilizing the above pre-learned artificial intelligence model is as follows. A step of analyzing whether the characters included in the user input information are directly related to the solution to the math problem or are additional characters not directly related to the solution to the math problem. Including, method.

11. In Article 10, The step of analyzing the mathematical problem solving process based on the acquired user input information by utilizing the above pre-learned artificial intelligence model is as follows. A step of identifying the language of a character included in the user input information, if the character is directly related to solving the mathematical problem; A step of identifying a formula associated with the character among the formulas included in the solution process, considering the system of the identified language; and A step of generating analysis information based on the semantic information of the above characters and the identified formula. Including, method.

12. In paragraph 10, The step of analyzing the mathematical problem solving process based on the acquired user input information by utilizing the above pre-learned artificial intelligence model is as follows. In the case of additional characters that are not directly related to the solution of the above mathematical problem, a step of generating analysis information based on the remaining part excluding the additional characters Including, method.

13. In paragraph 1, The above method, Considering the above analyzed solution process, a step of creating a problem related to the above solution in a user-customized manner Including more, method.

14. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, causes the one or more processors to perform the following operations for generating feedback information on a mathematical problem solving process, the operations being: An action to obtain user input information related to solving a mathematical problem; An operation of analyzing the process of solving the mathematical problem based on the acquired user input information by utilizing a pre-learned artificial intelligence model; and An operation for generating feedback information on the solution of the mathematical problem based on the above analysis. Including, A computer program stored on a computer-readable storage medium.

15. In paragraph 14, The action of obtaining user input information related to solving the above mathematical problem is: An action of distinguishing letters, numbers and mathematical symbols from the user input information obtained above; An action of performing recognition on the above-described separated characters by utilizing a character recognition model; An operation of performing recognition of the separated numbers by utilizing the above character recognition model; and An operation of performing recognition of the above-described mathematical symbols by utilizing the above-described character recognition model. Including, A computer program stored on a computer-readable storage medium.

16. In paragraph 14, The operation of analyzing the interpretation process for the acquired user input information by utilizing the above pre-learned artificial intelligence model is as follows. An operation of sequentially dividing the interpretation process for the user input information obtained above; and An operation of analyzing the sequentially divided solution process corresponding to the rubric items by utilizing the above pre-learned artificial intelligence model; Including, A computer program stored on a computer-readable storage medium.

17. As a computing device, at least one processor; and memory; Including, At least one processor of the above, Obtain user input information related to solving a mathematical problem; Analyzing the process of solving the mathematical problem based on the acquired user input information by utilizing a pre-learned artificial intelligence model; and Based on the above analysis, configured to generate feedback information for solving the math problem, device.

18. In paragraph 17, At least one processor of the above, Distinguish letters, numbers and mathematical symbols from the user input information obtained above; Recognition of the above-described characters is performed using a character recognition model; Recognition of the separated numbers is performed using the above character recognition model; and configured to perform recognition of the above-described mathematical symbols by utilizing the above-described character recognition model; device.

19. In paragraph 17, At least one processor of the above, Sequentially dividing the interpretation process for the user input information obtained above; and It is configured to analyze the sequentially divided solution process corresponding to the rubric items by utilizing the above pre-learned artificial intelligence model. device

Images