Learning support device, program, method and system

Abstract

This learning support device appropriately predicts and presents questions that the target user finds difficult. [Solution] The learning support device obtains the target user's answers to a set of preliminary test questions, calculates a first index indicating the target user's difficulty with each question based on the target user's answers to the preliminary test questions and the answers of multiple other users to the question set, calculates a second index indicating the target user's difficulty with each question based on the target user's answers to the preliminary test questions and the label information assigned to each question, and predicts which questions the target user will have difficulty with from the question set based on the first and second indices. The first index is calculated to be larger for questions that have been answered incorrectly by multiple other users whose answer tendencies are similar to those of the target user. The second index is calculated to be larger for questions in the preliminary test question set that are similar to questions that the target user has answered incorrectly.

Description

Technical Field

[0001] The present invention relates to a learning support apparatus, a program, a method and a system.

Background Art

[0002] Various learning support systems have been developed to enhance learning effects by selecting and presenting questions that learners are not good at. For example, Patent Document 1 discloses that artificial intelligence technology is used to identify another user whose answer correctness and incorrectness are similar to that of a target user, and a question to be recommended to the target user is selected from the questions incorrectly answered by the identified other user. Patent Document 2 discloses that questions related to the questions that the learner has answered incorrectly are presented.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Patent Document 2

Summary of Invention

Problem to be Solved by the Invention

[0004] As in Patent Document 1, when a question is selected from the questions incorrectly answered by another user similar to the target user, a problem may occur that questions that the target user is inherently not good at are not appropriately selected. Since it is rare that the tendency of answer correctness and incorrectness of the target user completely matches that of another user, the above problem may occur if the characteristics of another user are excessively referred to. In addition, as in Patent Document 2, when a question related to a question that a learner answered incorrectly in the past is selected, questions that the target user is truly not good at tend not to be appropriately selected until a large amount of the learner's answer history data is accumulated.

[0005] The objective of the present invention is to provide a learning support device, program, method, and system that can appropriately predict problems that a target user finds difficult and present those problems to the target user. [Means for solving the problem]

[0006] Item 1. A learning support device that presents a user with a question selected from a set of questions, It is connected to a database that stores the answers of multiple other users to the aforementioned set of problems, The answers of the target user to the preliminary test questions included in the aforementioned set of questions are obtained. Based on the target user's answers to the preliminary test question set and the answers of the multiple other users to the question set, a first index indicating the target user's level of difficulty with each question included in the question set is calculated. Based on the target user's answers to the preliminary test question set and the label information assigned to each question in the question set, a second indicator is calculated that shows the target user's level of difficulty with each question in the question set. Based on the first and second indicators, the system predicts which problems the target user will find difficult from the set of problems and presents those problems to the target user. It includes a control unit that performs the following: The first indicator is calculated such that it becomes larger for questions answered incorrectly by users among the multiple other users whose answering tendencies are similar to those of the target user. The second indicator is calculated such that it is larger for questions in the preliminary test question set that are similar to the questions that the target user answered incorrectly. Learning support device.

[0007] Section 2. The control unit shall An integrated index is calculated by combining the first and second indicators, and problems with a larger integrated index are predicted to be problems that the target user will find difficult. A learning support device as described in item 1.

[0008] Section 3. The integrated index is the harmonic mean of the first index and the second index. A learning support device as described in item 2.

[0009] Section 4. Each question included in the aforementioned set of questions is a multiple-choice question with multiple options, and each of the multiple options is assigned a weighted score. The control unit calculates an evaluation value indicating each user's level of difficulty with each problem, according to the score assigned to the option selected by each user for each problem, and calculates the first index based on the evaluation value. A learning support device as described in any of items 1 to 3.

[0010] Item 5. The options include one correct answer and multiple incorrect answers, and the multiple incorrect answers are assigned different scores. A learning support device as described in item 4.

[0011] Item 6. The control unit calculates an evaluation value indicating each user's level of difficulty with each problem, according to the time each user takes to answer each problem, and calculates the first index based on the evaluation value. A learning support device as described in any of items 1 to 5.

[0012] Item 7. A learning support program that presents a selected set of questions to the target user, A computer capable of accessing a database that stores the answers of multiple other users to the aforementioned set of problems, The answers of the target user to the preliminary test questions included in the aforementioned set of questions are obtained. Based on the target user's answers to the preliminary test question set and the answers of the multiple other users to the question set, a first index indicating the target user's level of difficulty with each question included in the question set is calculated. Based on the target user's answers to the preliminary test question set and the label information assigned to each question in the question set, a second indicator is calculated that shows the target user's level of difficulty with each question in the question set. predicting questions that the target user is not good at from the question group based on the first index and the second index, and presenting the questions to the target user to cause a computer to execute the steps of: the first index is calculated such that the greater the value of the first index, the more incorrectly the question was answered by users whose answer tendencies are similar to those of the target user among the plurality of other users, the second index is calculated such that the greater the value of the second index, the more similar the question is to a question incorrectly answered by the target user in the preliminary test question group, a learning support program.

[0013] Item 8. A learning support method for presenting questions selected from a question group to a target user, comprising: using a computer capable of referencing a database that stores answers from a plurality of other users to the question group, acquiring the target user's answers to a preliminary test question group included in the question group, calculating a first index indicating the target user's degree of difficulty with each question included in the question group based on the target user's answers to the preliminary test question group and the plurality of other users' answers to the question group, calculating a second index indicating the target user's degree of difficulty with each question included in the question group based on the target user's answers to the preliminary test question group and label information assigned to each question included in the question group, predicting questions that the target user is not good at from the question group based on the first index and the second index, and presenting the questions to the target user the steps comprising: the first index is calculated such that the greater the value of the first index, the more incorrectly the question was answered by users whose answer tendencies are similar to those of the target user among the plurality of other users, the second index is calculated such that the greater the value of the second index, the more similar the question is to a question incorrectly answered by the target user in the preliminary test question group, a learning support method.

[0014] Item 9. A terminal operated by a target user, A learning support device that presents a selected problem from a set of problems to the target user via the terminal. A learning support system comprising, The aforementioned learning support device is It is connected to a database that stores the answers of multiple other users to the aforementioned set of problems, The answers of the target user to the preliminary test questions included in the aforementioned set of questions are obtained. Based on the target user's answers to the preliminary test question set and the answers of the multiple other users to the question set, a first index indicating the target user's level of difficulty with each question included in the question set is calculated. Based on the target user's answers to the preliminary test question set and the label information assigned to each question in the question set, a second indicator is calculated that shows the target user's level of difficulty with each question in the question set. Based on the first and second indicators, the system predicts which problems the target user will find difficult from the set of problems and presents those problems to the target user. It includes a control unit that performs the following: The first indicator is calculated such that it becomes larger for questions answered incorrectly by users among the multiple other users whose answering tendencies are similar to those of the target user. The second indicator is calculated such that it is larger for questions in the preliminary test question set that are similar to the questions that the target user answered incorrectly. Learning support system. [Effects of the Invention]

[0015] According to the present invention, a learning support device, program, method, and system are provided that can appropriately predict problems that a target user finds difficult and present those problems to the target user. [Brief explanation of the drawing]

[0016] [Figure 1] A diagram showing the overall configuration of a learning support system including a learning support device according to one embodiment. [Figure 2]A schematic diagram of a learning support device according to one embodiment. [Figure 3] A sequence diagram showing the flow of the preliminary test process included in the learning process according to one embodiment. [Figure 4] A sequence diagram showing the flow of individual test processing included in the learning process according to one embodiment. [Figure 5] A diagram showing the data structure of a problem database according to one embodiment. [Figure 6] A diagram showing the data structure of an answer database according to one embodiment. [Modes for carrying out the invention]

[0017] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and their descriptions will not be repeated. Furthermore, each drawing is schematic, with parts omitted or exaggerated as appropriate, for ease of understanding.

[0018] [1. Overall structure of the learning support system] Figure 1 shows the overall configuration of the learning support system 100, including the learning support device 1 according to this embodiment. The learning support system 100 is a system that supports learners' learning by providing them with a learning environment through various problems. As shown in Figure 1, the learning support system 100 includes the learning support device 1 and terminals 3 connected to the learning support device 1 via a network 2. Network 2 is, for example, the internet or a local area network (LAN). The learning support device 1 selects a problem from a pre-prepared set of problems and provides it to terminals 3 via the network 2. Terminals 3 are devices operated by learners. That is, the learning support device 1 presents learners with problems selected from a pre-prepared set of problems via terminals 3. Many learners can connect to the learning support device 1 via their respective terminals 3. Although not limited to this, for example, the learning support device 1 may be implemented from a server computer, and terminals 3 may be implemented from personal computers, tablets, smartphones, etc. that function as client computers. The learning support device 1 may be a device on the cloud or a device in an on-premise environment.

[0019] The learner operates terminal 3 to access learning support device 1 via network 2, thereby receiving and answering questions from learning support device 1. The learner's answers are transmitted from terminal 3 to learning support device 1 via network 2, where they are graded. Learning support device 1 provides the grading results and explanations to terminal 3 via network 2 for the learner to view. Learning support device 1 supports the learner's learning by providing questions to the learner, receiving answers from the learner, grading, and providing feedback to the learner (provision of grading results and explanations) via terminal 3.

[0020] [2. Configuration of the learning support device] Figure 2 shows a schematic configuration of the learning support device 1 according to this embodiment. As shown in Figure 2, the learning support device 1 is a computer in terms of hardware and comprises a control unit 10, a storage unit 20, a communication module 30, and an input / output interface 40.

[0021] The control unit 10 includes a CPU (Central Processing Unit) 11, RAM (Random Access Memory) 12, and ROM (Read Only Memory) 13. The storage unit 20 is a non-volatile auxiliary storage device such as a hard disk drive or a solid-state drive. The storage unit 20 stores the learning support program PG1. The learning support program PG1 is a program that causes the learning support device 1 to execute various processes, including the learning process described later. The control unit 10 loads the learning support program PG1 into RAM, and the CPU interprets and executes it to perform the various processes.

[0022] The communication module 30 is a device that provides communication functionality to the learning support device 1. The learning support device 1 is connected to the network 2 via the communication module 30 and communicates with the terminal 3. The input / output interface 40 is an input / output device and includes a keyboard, mouse, and display, etc.

[0023] The control unit 10 is connected to the problem database 21 and the answer database 22 via the communication module 30, and can access these databases 21 and 22. As will be described in detail later, the problem database 21 stores information on each of the problems that make up the set of problems that the learning support device 1 can present (see Figure 5). Similarly, as will be described in detail later, the answer database 22 stores information on the answers of many learners to the set of problems stored in the problem database 21 (see Figure 6). Within the answer database 22, the answer information is stored in association with the learner's identifier (hereinafter referred to as the user ID). In the example in Figure 2, the databases 21 and 22 are located outside the learning support device 1 and are connected to the learning support device 1 via the communication module 30. However, all or part of the information stored in the databases 21 and 22 may be stored inside the learning support device 1 (for example, in the storage unit 20), as long as the control unit 10 can access it.

[0024] [3. Learning Process] The learning process by the learning support system 100 will be described below with reference to Figures 3 and 4. The learning process includes a preliminary test process in which a preliminary test is administered to the target user (learner) when they begin using the learning support system 100, and an individual test process in which a personalized individual test is administered to the target user based on the results of the preliminary test. Figure 3 is a sequence diagram showing the flow of the preliminary test process, and Figure 4 is a sequence diagram showing the flow of the individual test process.

[0025] In the preliminary test process shown in Figure 3, a pre-prepared set of preliminary test questions (hereinafter referred to as the preliminary test question set) is presented. The preliminary test questions are questions that are presented to learners in advance. The preliminary test question set constitutes a part of the set of questions stored in the question database 21. For example, if the question database 21 stores 1000 questions, the preliminary test question set can consist of 50 specific questions from that set. The preliminary test question set is a comprehensive collection of questions from various genres (themes), making it suitable for measuring the learner's proficiency in each genre (theme). In this embodiment, the preliminary test question set is common to all learners.

[0026] In the individual test processing shown in Figure 4, individual test questions are presented that are selected individually to suit the target user. The individual test questions are selected from the set of questions stored in the question database 21 that have not yet been presented to the target user (hereinafter referred to as "unpresented questions"). More specifically, based on the results of the target user's preliminary test, questions that the target user may have difficulty with are predicted from the unpresented questions and selected as individual test questions for the target user.

[0027] In this embodiment, each question is a multiple-choice question with multiple options. The number of options is not particularly limited as long as there are at least two, but it is preferably three or more, and more preferably four or more. The multiple options include correct and incorrect answers, and in this embodiment, it includes one correct answer and multiple incorrect answers.

[0028] Figure 5 shows the data structure of the problem database 21 according to this embodiment. As shown in the figure, the problem database 21 has fields for storing information such as a problem identifier (hereinafter referred to as problem ID), the problem, the correct answer, the score of the correct answer, the incorrect answer, the score of the incorrect answer, and the explanation. Therefore, the problem database 21 stores the problem ID, the problem identified by the problem ID, the correct answer and its score for the problem, the incorrect answer and its score for the problem, and the explanation for the problem, all of which are associated with each other for each problem. In the example in Figure 5, there are three fields, "Incorrect Answer 1," "Incorrect Answer 2," and "Incorrect Answer 3," for storing three incorrect answers, and three fields for storing the scores of the three incorrect answers.

[0029] The score for each option is an evaluation point given to the learner who selected that option. In this embodiment, each of the multiple options for each question is assigned a weighted score, as shown in Figure 5. It is preferable that the range of scores assigned to each of the multiple options for each question is constant (the minimum and maximum scores are the same for all questions). The score assigned to a learner can represent the learner's level of difficulty with that question; a higher value means greater difficulty, and a lower value means greater proficiency. Therefore, among the multiple options, the correct answer is assigned the smallest value, and incorrect answers are assigned a value larger than the correct answer. If there are multiple incorrect answers, the same score may be assigned to the multiple incorrect answers, or different scores may be assigned, but in this embodiment, different scores are assigned. This allows for a more detailed evaluation of the learner's level of difficulty with the question, depending on the degree of incorrectness. For example, among the multiple incorrect answers, the most significant incorrect answer is assigned the largest value, while incorrect answers that were close to being correct are assigned a value larger than the correct answer but smaller than the significant incorrect answer. For example, if there are three incorrect answers—"almost," "almost," and "completely wrong"—the correct answer can be assigned 0 points, an almost incorrect answer 2 points, an almost incorrect answer 2.5 points, and a completely wrong answer 3 points. In the example in Figure 5, the scores assigned to multiple choices are set for each question, but they could also be common to all questions. For example, all questions could be uniformly assigned 0 points for the correct answer, 2 points for "incorrect answer 1" (almost incorrect), 2.5 points for "incorrect answer 2" (almost incorrect), and 3 points for "incorrect answer 3" (completely wrong). This is preferable because it prevents the weighting from becoming subjective, even when multiple people are setting the score weights.

[0030] Figure 6 shows the data structure of the answer database 22 according to this embodiment. As shown in the figure, the answer database 22 has fields for storing the user ID and information on the learner's answers to each problem, respectively. Therefore, the answer database 22 stores the user ID and the learner's answers to each problem identified by the user ID, associated with each other for each learner. Thus, the answer database 22 stores the learner's answers for each learner and for each problem. The fields for storing answers are defined for each problem included in the set of problems stored in the problem database 21.

[0031] As shown in Figure 6, the answer database 22 according to this embodiment stores the learner's answer itself, the time it took the learner to answer (hereinafter referred to as "answer time"), and the scoring result of the learner's answer as the learner's answer. Here, when we refer to the learner's (or user's) answer, it may mean not only the learner's (or user's) answer itself, but also the information associated with it, such as the answer time and / or the scoring result. The answer database 22 may store, for example, only the learner's answer itself and the answer time, or only the scoring result as the learner's answer. As will be described in detail later, if the scoring result is known, subsequent processing can be carried out, and if the learner's answer and answer time are known, the scoring result can be obtained. In the example in Figure 6, the scoring result includes information indicating the correctness of the learner's answer obtained by scoring the answer (hereinafter referred to as "correctness information") and an evaluation value. The evaluation value for each problem indicates each user's level of difficulty with that problem. In other words, the difficulty level is an evaluation value given to a user's answer to a problem; a higher value means the user finds the problem more difficult, and a lower value means the user finds the problem more difficult. The difficulty level, or the evaluation value that indicates it, is a numerical value that quantifies the degree to which a user finds a problem difficult. Therefore, calculating the difficulty level, or the evaluation value that indicates it, makes it possible to numerically compare and evaluate which problems a user finds difficult and to what extent. It is preferable that the range of evaluation values ​​for each problem is constant (the minimum and maximum evaluation values ​​are the same for all problems). The answer itself, the time taken to answer, the correctness information, and the evaluation value are stored in different fields.

[0032] Referring to Figure 3, the flow of the preliminary test process will be explained in detail. The target user takes a preliminary test when they start using the learning support system 100. First, the target user operates terminal 3 to access the learning support device 1 and requests the preliminary test question set from the learning support device 1 (step S1). In response, the control unit 10 of the learning support device 1 sends the preliminary test question set to terminal 3 and displays it on terminal 3, thereby presenting the preliminary test question set to the target user (step S2). At this time, the control unit 10 extracts the preliminary test question set from the question set stored in the question database 21. For example, a list of question IDs of the questions that make up the preliminary test question set is stored in the storage unit 20 in advance, and the preliminary test question set can be extracted from the question database 21 using the question IDs included in this list as keys. Alternatively, a field can be defined in the question database 21 to store a flag indicating whether or not a question is a preliminary test question, and the preliminary test question set can be extracted from the question database 21 using this flag as a key. For example, in the former case, the administrator can arbitrarily select a set of preliminary test questions by appropriately changing the list of question IDs stored in the memory unit 20. Alternatively, in the latter case, the administrator can arbitrarily select a set of preliminary test questions by appropriately changing the flag values ​​in the question database 21.

[0033] Next, the target user views the set of preliminary test questions displayed on terminal 3, creates answers to the preliminary test questions, and sends them from terminal 3 to the learning support device 1 (step S3). At this time, terminal 3 measures the time it takes the target user to answer each question and sends the answer time to the learning support device 1 in association with the answer to each question. For example, if terminal 3 displays each question one by one, the answer time is measured as the time from when each question is displayed on terminal 3 until the target user finishes entering the answer. In other words, in step S3, the control unit 10 of the learning support device 1 obtains the target user's answers and answer times for the set of preliminary test questions. Although Figure 3 shows an example where the set of preliminary test questions, their answers, and answer times are sent all at once, it is also possible to alternate between sending each question in the set of preliminary test questions one by one and sending its answer and answer time one by one.

[0034] The control unit 10 of the learning support device 1 stores the target user's answers and answer times for the preliminary test question set acquired in step S3 in the answer database 22 (step S4). More specifically, the control unit 10 stores the target user's answers and answer times acquired in step S3 in the appropriate fields of the record for the target user in the answer database 22.

[0035] Next, the control unit 10 of the learning support device 1 scores the target user's answers to the preliminary test question set acquired in step S3 and obtains the scoring results (step S5). First, the control unit 10 refers to the question database 21, determines whether the target user's answer to each question in the preliminary test question set is correct or incorrect, and generates correct / incorrect information. The control unit 10 also calculates an evaluation value indicating the target user's level of difficulty with each question based on the target user's answers to the preliminary test question set. In this embodiment, the calculation of the evaluation value takes into account not only the answers themselves acquired in step S3, but also the answer time acquired in step S3. In this embodiment, the evaluation value is calculated according to the score assigned to the option selected by the target user for each question and the target user's answer time for each question.

[0036] More specifically, in step S5, the following calculations are performed for each problem. First, the control unit 10 identifies the option that matches the target user's answer (i.e., the option selected by the target user) from among the multiple options for that problem stored in the problem database 21, and identifies the score assigned to that option as the first evaluation score. Next, the control unit 10 calculates the second evaluation score according to the target user's answer time based on predetermined rules. For example, the second evaluation score is calculated according to rules such as: if the answer time is 0 to 20 seconds, the second evaluation score is 0 points; if it is 20 to 40 seconds, it is 1 point; if it is 40 to 60 seconds, it is 1.5 points; and if it is more than 1 minute, it is 2 points. The predetermined rules may be set in common for all problems, or they may be determined for each problem. In the latter case, it is advantageous in that the second evaluation score can be determined according to the characteristics of each problem, such as the amount of text in the problem, whether it is a problem that requires thinking, or whether it is a problem that only tests knowledge. In the latter case, it is preferable that the range of the second evaluation score for each problem is constant (the minimum and maximum values ​​of the second evaluation score are common to all problems). Next, the control unit 10 calculates an evaluation value indicating the target user's level of difficulty with the problem based on the first evaluation score and the second evaluation score. The evaluation value is calculated such that it increases as the first evaluation score increases and as the second evaluation score increases. The evaluation value can be, for example, the sum of the first evaluation score and the second evaluation score, but it may also be the harmonic mean, simple mean, or weighted mean of the first evaluation score and the second evaluation score.

[0037] Next, the control unit 10 stores the scoring results of the target user's answers to the preliminary test question set calculated in step S5 in the answer database 22 (step S6). That is, the control unit 10 stores the correct / incorrect information and evaluation value calculated in step S5 in the appropriate fields of the record for the target user in the answer database 22.

[0038] Next, the control unit 10 transmits the scoring results of the target user's answers to the preliminary test question set calculated in step S5 to terminal 3 and displays them on terminal 3 (step S7). Note that only correct / incorrect information is transmitted as the scoring result, and the evaluation value does not need to be transmitted. The control unit 10 also extracts explanations for each question included in the preliminary test question set from the question database 21 and transmits them to terminal 3 along with the scoring results and displays them on terminal 3. The target user can deepen their understanding of the knowledge related to the questions they have answered by looking at the scoring results and explanations displayed on terminal 3.

[0039] This concludes the preliminary testing. After this, the target users can conduct individual tests at any time. For example, they may conduct individual tests immediately after completing the preliminary testing, or they may conduct individual tests at a later date after completing the preliminary testing.

[0040] Referring to Figure 4, the flow of the individual test processing will be explained in detail. After taking a preliminary test, if the target user continues learning using the learning support system 100, they can take the individual test. The individual test can be taken any number of times, and the individual test processing shown in Figure 4 is executed each time. First, the target user operates terminal 3 to access the learning support device 1 and requests individual test questions from the learning support device 1 (step S21). In response, the control unit 10 of the learning support device 1 extracts individual test questions from the group of questions stored in the question database 21 that have not yet been asked to the target user. Whether each question has not yet been asked can be determined, for example, by whether or not the answer to that question is stored in the answer database 22.

[0041] The control unit 10 calculates a first indicator (step S22) and a second indicator (step S23) in order to extract individual test questions. Subsequently, based on the first and second indicators, the control unit 10 predicts which questions the target user will find difficult from among the unasked questions and determines them as individual test questions (step S24). The first indicator is an indicator that shows the target user's level of difficulty with each question included in the question set. The second indicator is another indicator that shows the target user's level of difficulty with each question included in the question set. In step S24, the unasked questions are ranked according to the target user's level of difficulty by comprehensively evaluating the first and second indicators.

[0042] Step S22 is executed as follows: The control unit 10 calculates a first index for each problem in the problem set based on the target user's answer to the problem set (more precisely, the scoring result of the answer itself) and the answers of multiple other users to the problem set (more precisely, the scoring result of the answer itself). That is, the first index is a value calculated for each problem in the problem set based on the target user's answer to the problem set and the answers of multiple other users to the problem set. Note that at this stage, not all problems stored in the problem database 21 have been solved by each user (the target user and other users). Therefore, the answers of users (the target user and other users) to the problem set that form the basis for calculating the first index refer to the answers of users to problems stored in the problem database 21 that have been presented to the user and answered by the user (hereinafter referred to as "presented problems"). Whether each problem is a presented problem can be determined, for example, by whether or not the answer to that problem is stored in the answer database 22. Furthermore, for target users, the questions already presented will include only the preliminary test questions if it is their first time taking the individual test, but for subsequent individual test attempts, they will include both the preliminary test questions and previously answered individual test questions. In this embodiment, the first indicator is calculated for questions that have not yet been presented.

[0043] The first indicator is calculated such that it is higher for questions that were answered incorrectly by users with similar answer tendencies to the target user among multiple other users (i.e., questions that similar users find difficult). Therefore, by using the first indicator, it is possible to extract questions that users similar to the target user are likely to answer incorrectly. Although not limited to this, in this embodiment, a recommendation system using collaborative filtering is used to calculate the first indicator as described above. As a result, even for unassigned questions that the target user has not yet answered, the target user's potential weakness is evaluated based on the scoring results of other users for those unassigned questions.

[0044] More specifically, in step S22, the control unit 10 creates the evaluation value matrix A shown below.

number

[0045] When the evaluation value matrix A is an M×N matrix, N corresponds to the number of problems included in the problem set stored in the problem database 21, and M corresponds to the number of users (including target users) included in the evaluation. Each column of the evaluation value matrix A corresponds to each problem, and each row corresponds to each user included in the evaluation. Although not limited to this, for example, the first row can be the row corresponding to the target user, and the rows from the second row onward can be the rows corresponding to other users. However, the row corresponding to the target user can be set arbitrarily as it does not affect the calculation. Note that the users included in the evaluation may be all users registered in the answer database 22, but if the following calculation is performed using the answer information of other users who have answered few problems, the error may become large. For this reason, it is preferable to include only users who have answered a certain number of problems or more in the evaluation, excluding target users.

[0046] The value V in the m row and n column of the evaluation matrix. m,nrepresents an evaluation value (scoring result) indicating the user's weakness level corresponding to the m-th row for the question corresponding to the n-th column (n=1,2,···,N, m=1,2,···,M). Accordingly, the control unit 10 extracts the evaluation values of the users (the target user and other users) included in the evaluation from the answer database 22, and appropriately arranges the extracted evaluation values to create an evaluation value matrix A. At this time, V in the answer database 22 m,n does not include a corresponding evaluation value, such V m,n can be complemented with a value of 0. For example, when a user takes an individual test for the first time, in the row corresponding to the target user, all values in columns corresponding to questions other than the preliminary test questions are complemented with 0.

[0047] Next, the control unit 10 performs singular value decomposition on the evaluation value matrix A. As a result, the evaluation value matrix A is decomposed into a product form of a user factor matrix P and a question factor matrix Q (A=P×Q) as shown below. Note that when the singular value decomposition of A is expressed as A=UΣV T , P=UΣ and Q=V T holds.

Formula

[0048] P is an M×N matrix, and Q is an N×N matrix. The arrangement of N values in each row of the user factor matrix P indicates the characteristics of the user corresponding to the row. The arrangement of N values in each column of the question factor matrix Q indicates the characteristics of the question corresponding to the column.

[0049] Next, the control unit 10 multiplies the row corresponding to the target user included in the user factor matrix P by each column corresponding to an unasked question included in the question factor matrix Q, thereby calculating a 1×K matrix. K is the number of unasked questions (K<N). Each of the K values included in this 1×K matrix is a first index for each question (each unasked question). That is, the value at the 1st row and k-th column of this 1×K matrix is the first index indicating the target user's weakness level for the question (unasked question) corresponding to the k-th column. The first index calculated here is a prediction result of the target user's weakness level based on answers from other users.

[0050] Step S23 is executed as follows: The control unit 10 calculates a second index for each problem in the problem set based on the target user's answers to the problem set (more precisely, the scoring results of the answers themselves) and the label information assigned to each problem in the problem set. That is, the second index is a value calculated for each problem in the problem set based on the target user's answers to the problem set and the label information assigned to each problem in the problem set. At this stage, the target user has not yet solved all the problems stored in the problem database 21. Therefore, the target user's answers to the problem set, which form the basis for calculating the second index, refer to the target user's answers to the problems that have already been presented. If it is the first time the target user is taking an individual test, the problems that have already been presented to the target user include only the preliminary test problems. However, if it is the second or subsequent time the target user is taking an individual test, the problems that have already been presented to the target user include the preliminary test problems as well as individual test problems that have been solved in the past. In this embodiment, the second index is calculated for problems that have not yet been presented.

[0051] The second indicator is calculated so that it is higher for questions that are similar to those the target user answered incorrectly (i.e., questions that are similar to those the target user finds difficult) among the questions that have already been asked. Therefore, by using the second indicator, it is possible to extract questions that are similar to those that the target user finds difficult. Although not limited to this, in this embodiment, a content-based recommendation system is used to calculate the second indicator as described above. As a result, even for unasked questions that the target user has not yet answered, the level of difficulty of the target user is evaluated based on the scoring results of the questions that the target user has already answered.

[0052] More specifically, each problem in the problem database 21 is assigned label information. Label information is a tag that indicates the genre of each problem. There is no particular limit to the number of tags (number of genres) assigned to each problem; it may be one, but it is preferable to have multiple tags. The number of tags may be the same for all problems, or it may differ from problem to problem.

[0053] Label information is pre-assigned to each problem and stored in the memory unit 20. Alternatively, label information may be stored in the problem database 21. Label information may be assigned manually, but it can also be assigned using AI (Artificial Intelligence), particularly generative AI. For example, a prompt can be created to instruct the generative AI model to generate label information for each problem, and this prompt can be provided to the generative AI model to obtain the label information assigned to each problem as its response. The generative AI model may be stored within the learning support device 1, but an external server that provides a web service that accepts prompts and returns answers, such as ChatGPT (registered trademark), may be used. A prompt can be created, for example, as follows: "From the following list of 100 tags, please tell us four appropriate tags for the following problem Q1. Tag list: valve, piping, ...combustion, tools, operation; Problem Q1: You will be working with the tools shown in the figure below. What kind of work will you be doing?..." Label information assigned using generative AI may be set directly for each problem, or the administrator may evaluate the validity of the label information assigned by the generative AI for each problem. For example, in the latter case, the administrator can use the label information assigned by the generating AI as is if there are no problems with it, or manually correct the label information and set it to the problem if there are problems.

[0054] The control unit 10 extracts from the answer database 22 the questions that the target user answered incorrectly (hereinafter referred to as "incorrect questions") from the questions that have already been asked to the target user. At this time, the correct / incorrect information can be used as a key to extract the incorrect questions. Note that the number of incorrect questions is not limited to multiple questions, and may be just one. The control unit 10 also extracts questions that have not yet been asked to the target user from the answer database 22.

[0055] Next, the control unit 10 compares the tags assigned to each incorrectly answered question with the tags assigned to each unanswered question and calculates the similarity between each incorrectly answered question and each unanswered question. While not limited to this, in this embodiment, cosine similarity is calculated. More specifically, for each incorrectly answered question and each unanswered question, the control unit 10 creates a vector T representing the genre of the question, in other words, a vector representing the tags assigned to that question (hereinafter referred to as a tag vector). The tag vector T is a vector having L values, where L corresponds to the number of types of tags assigned to the question set stored in the question database 21. Each value in the tag vector T corresponds to each tag. The l-th value from the left of the tag vector T is v l The value v in that problem l If the corresponding tag is assigned, the value will be 1, and that problem will have that value v l If no corresponding tag is assigned, the result will be 0. For example, the tag vector T is constructed as follows. In the following equation, "valve", "piping", "combustion", "tools", and "operation" are examples of genres.

[0056]

number

[0057] The control unit 10 calculates the cosine similarity between the tag vector T of each unasked question and the tag vector T of each incorrectly answered question by brute force. For example, if there are 950 unasked questions and 6 incorrectly answered questions, the cosine similarity is calculated for each unasked question as there are incorrectly answered questions, resulting in a total of 950 × 6 = 5700 cosine similarities being calculated. The control unit 10 then calculates a representative similarity that represents the cosine similarity for each unasked question as there are incorrectly answered questions. The representative similarity is calculated so that it increases as the cosine similarity increases. The representative similarity for each unasked question can be, for example, the average or sum of the cosine similarities for each incorrectly answered question. The average here may be a weighted average, and the weights can be values ​​corresponding to the evaluation value of each incorrectly answered question or the score assigned to the answer of the target user. In this case, based on the evaluation value or score, the weights may be set so that the cosine similarity between problems judged to be difficult for the target user (problems with high evaluation values, problems with high scores) contributes more to the representative similarity.

[0058] In this embodiment, the above representative similarity is a second indicator that shows the target user's level of difficulty with each problem (each unasked problem). The second indicator calculated here is a prediction of the target user's level of difficulty with unasked problems, based on the target user's answers to the problems that have already been asked.

[0059] Next, the control unit 10 comprehensively evaluates the first indicator calculated in step S22 and the second indicator calculated in step S23 to predict which questions the target user will find difficult from among the unasked questions, and determines them as individual test questions (step S24). In this embodiment, the control unit 10 calculates an integrated indicator for each unasked question by combining the first indicator and the second indicator, and predicts that questions with a larger integrated indicator will be difficult for the target user.

[0060] The integrated metric can be, for example, the average of the first metric and the second metric, or the sum of them. The average can be various types, such as a simple average or a weighted average, but a preferred example is the harmonic mean. The harmonic mean tends to mitigate the impact of large outliers and weight the impact of small outliers, and tends to approach the minimum value in the data set. Therefore, if the first and second metrics are unbalanced (their values ​​are far apart), the integrated metric, which is the harmonic mean, will be swayed by the smaller value. As a result, the problem corresponding to that integrated metric is less likely to be predicted as a problem that the target user will struggle with, and is less likely to be selected as an individual test question. Conversely, if the first and second metrics are well-balanced (their values ​​are close), the problem is more likely to be predicted as a problem that the target user will struggle with, and is more likely to be selected as an individual test question. Therefore, the validity and reliability of the judgment as to whether a problem is difficult for the target user increases. Furthermore, the harmonic mean is 0 when either of the data sets (i.e., either the first or second metric) is 0. Therefore, if either the first or second indicator is 0 (correct answer), the harmonic mean will be 0, making it less likely that the problem will be predicted as one that the target user will struggle with, and thus less likely to be selected as an individual test question. Consequently, in this sense as well, the validity and reliability of the judgment as to whether a problem is one that the target user will struggle with is increased.

[0061] The integrated index is preferably calculated after normalizing the first and second indicators. For example, it is preferable to scale the values ​​of the first and second indicators from 0 to 1 and then calculate the integrated index using the scaled values.

[0062] When the untested questions are sorted in descending order of the integrated index, they correspond to the questions that the target users find most difficult. The control unit 10 ranks the target users' difficulty with the untested questions according to the integrated index. The control unit 10 then selects the top-ranked questions from the untested questions that the target users find difficult, and determines them as individual test questions. The number of individual test questions selected here is not particularly limited; it may be predetermined or specified by the target users.

[0063] Next, the control unit 10 sends individual test questions to the terminal 3 and displays them on the terminal 3, thereby presenting the target user with individual test questions that are predicted to be difficult for the target user (step S25). The target user looks at the individual test questions displayed on the terminal 3, creates answers to the individual test questions, and sends the answers and answer times from the terminal 3 to the learning support device 1 (step S26). As a result, in step S26, the control unit 10 obtains the target user's answers and answer times for each individual test question. If there are multiple individual test questions, the individual test questions, answers, and answer times may be sent all at once, or the individual test questions may be sent one by one, and the answers and answer times for each question may be sent one by one, alternating between the two.

[0064] The control unit 10 saves the target user's answers and response times to the individual test questions obtained in step S26 to the answer database 22 (step S27). Next, the control unit 10 scores the target user's answers to the individual test questions obtained in step S26 and obtains the scoring results (step S28). Next, the control unit 10 saves the scoring results of the target user's answers to the individual test questions calculated in step S28 to the answer database 22 (step S29). Next, the control unit 10 transmits the scoring results and explanations of the target user's answers to the individual test questions calculated in step S28 to the terminal 3 and displays them on the terminal 3 (step S30). Steps S27 to S30 are executed in the same way as steps S4 to S7 by changing the preliminary test question set to individual test questions. With this, the individual test is completed.

[0065] Furthermore, the preliminary and individual tests described above are administered to various learners, and the individual tests may be administered repeatedly. Therefore, as steps S4-S6 and S27-S29 are repeatedly executed for various learners, the answer database 22 accumulates answers from various learners to the set of questions. As a result, when an individual test is administered to a specific user, the answer database 22 contains answers from multiple other users to the set of questions referenced in step S22, as well as the answers of that specific user to the set of questions referenced in steps S22 and S23.

[0066] The learning support system 100 described above can be used in various fields of learning. For example, it can be used to give quizzes to employees of a waste incineration facility (e.g., operators) to help them acquire the knowledge necessary for the operation of the facility. In general, to enhance learning effectiveness, it is desirable to prioritize questions on areas where learners are weak. Therefore, if there is variation in each person's strengths and weaknesses, giving everyone the same questions will not yield sufficient learning results. In this respect, the learning support system 100 can select questions on each person's weak areas and give them personalized individual test questions, thus enabling high learning effectiveness.

[0067] [4. Features] In the above embodiment, the types of problems a target user struggles with are predicted based on a first indicator calculated based on the answers of other users and a second indicator calculated based on the target user's answers to other problems. The first indicator is calculated to be larger the more problems that other users with similar answer tendencies to the target user have answered incorrectly (i.e., the more problems that similar users find difficult). The second indicator is calculated to be larger the more similar the problems are to those that the target user has answered incorrectly among the previously given questions (at least including the preliminary test questions) (i.e., the more similar the problems are to those that the target user finds difficult). This makes it possible to appropriately predict the types of problems a target user struggles with and present them to the target user.

[0068] In the above embodiment, individual test questions are determined based on a first metric calculated using collaborative filtering and a second metric calculated using a content-based recommendation system. If individual test questions were determined based only on the second metric and not the first metric, only questions from genres the user answered incorrectly or similar genres would be recommended, resulting in a lack of diverse questions. In this case, the user would find it uninteresting. In this respect, in the above embodiment, individual test questions are selected based on the first metric as well as the second metric, so questions from unexpected genres are also recommended, keeping the user engaged and enabling learning across a wide range of fields.

[0069] In the above embodiment, a content-based recommendation system is used to calculate the second indicator, so each problem is labeled with a genre, and the level of difficulty is predicted based on that label. Therefore, compared to the first indicator, it is possible to predict problems that the user will struggle with more directly, thus increasing the reliability of the prediction of problems that the user will struggle with.

[0070] In the above embodiment, the first metric is calculated using collaborative filtering. Therefore, problems that the target user struggles with are predicted from the answer history of users similar to the target user. In this case, if there are no users similar to the target user, the recommended problems may not be appropriate. This problem is resolved as the number of users increases, making it easier to find compatible users, but it takes some time to secure a sufficient number of users. Furthermore, in the above embodiment, the second metric is calculated using a content-based recommendation system. Therefore, if the genre labeling of a huge number of problems is done manually, the criteria for labeling will differ from person to person, potentially leading to inaccurate labeling. This problem can be resolved, for example, by pre-defining detailed labeling rules, but defining such rules is not always easy. In this respect, in the above embodiment, problems that users struggle with are predicted based on both the first and second metrics, thus mitigating problems caused by inaccurate labeling in the content-based recommendation system and ensuring the appropriateness of the problems recommended by collaborative filtering.

[0071] [5. Other Embodiments] Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the invention. For example, the following modifications are possible. Furthermore, the gist of the following modifications can be combined as appropriate.

[0072] [5-1] In step S24, the problems that the target user struggles with were predicted based on the integrated metric, which is a combination of the first and second metrics, and individual test questions were determined. However, it is also possible to predict problems that the user struggles with based on the first metric, and separately, predict problems that the user struggles with based on the second metric, and then merge the problems predicted based on each metric to create individual test questions. When predicting problems that the user struggles with based on only the second metric, it is not necessary to calculate the representative similarity as described above. For example, the cosine similarity values ​​described above could be sorted in descending order, and the unasked problems corresponding to the cosine similarity values ​​could be ranked according to this order, under the constraint that no problems should be duplicated. The problems with the highest rankings could then be given priority for selection as individual test questions.

[0073] [5-2] In the above embodiment, step S22, which calculates the first indicator, step S23, which calculates the second indicator, and step S24, which predicts the problems that the target user has difficulty with based on the first and second indicators and determines them as individual test problems, were executed when an individual test problem was requested by a specific user. However, the timing of the execution of steps S22 to S24 is not limited to this. For example, steps S22 to S24 may be executed at predetermined time intervals (for example, every day), the results may be saved, and then, when an individual test problem is requested by a user, the individual test problem may be determined based on the saved results. In this case, it is advantageous because computation time can be secured even if the computation load of steps S22 to S24 increases due to an increase in the number of problems, etc.

[0074] [5-3] In steps S22 and S23 of the above embodiment, the questions that serve as the basis for calculating the first and / or second indicators include, in addition to the preliminary test questions, the individual test questions that have been answered in the past, if the individual test is being taken for the second time or later. However, in the case of heavy users, it is expected that as the number of unasked questions decreases, questions in their areas of expertise will be selected more frequently as individual test questions. In this regard, in order to enable the selection of questions in areas of weakness, if the number of individual tests taken or the number of questions that have been asked exceeds a certain value, the questions that serve as the basis for calculating the first and / or second indicators may be the individual test questions that have been asked in the most recent predetermined number of individual tests, or the most recent predetermined number of individual test questions that have been answered. For example, if a particular user has taken 10 individual tests in the past, the first and / or second indicators can be calculated by referring to the answers to the questions that have been asked in the 6th to 10th individual tests. In this case, the questions that have been asked before that (in this example, the preliminary tests and the questions that have been asked in the 1st to 5th individual tests) may be treated as unasked questions.

[0075] [5-4] The method for calculating the first indicator is not limited to the method described above. For example, in the above embodiment, the score used to determine the first evaluation score was set to a larger value the more difficult the user was (closer to "big mistake" among "correct," "almost," "almost," and "big mistake"), and the value allocated to the answer time used to determine the second evaluation score was also set to a larger value the more difficult the user was (longer the answer time). However, it is also possible to set a smaller value for the score the more difficult the user was, and to set a smaller value for the answer time the more difficult the user was. In this case, the first indicator is calculated so that numerically it decreases as similar users have answered the same questions incorrectly, but as an indicator of difficulty, it is calculated so that it increases as similar users have answered the same questions incorrectly (in the direction of increasing difficulty).

[0076] [5-5] The method for calculating the second indicator is not limited to the method described above. For example, in the above embodiment, the cosine similarity between the tag vector T of each unasked question and the tag vector T of each incorrectly answered question was calculated, and the representative similarity representing the cosine similarity for each unasked question and the number of incorrectly answered questions was used as the second indicator. However, instead of cosine similarity, distances such as the Euclidean distance or Manhattan distance may be calculated between the tag vector T of each unasked question and the tag vector T of each incorrectly answered question, and the representative similarity may be calculated from these distances in a similar manner. Cosine similarity indicates a higher degree of similarity as the value increases, while the distances exemplified here indicate a higher degree of similarity as the value decreases. Therefore, in this case, although the second indicator is calculated numerically so that it decreases for questions similar to those answered incorrectly by the target user, as an indicator of difficulty, it is calculated so that it increases for questions similar to those answered incorrectly by the target user (in the direction of increasing difficulty). [Explanation of Symbols]

[0077] 100 Learning Support Systems 1. Learning support device 2 Network 3 terminals 10 Control Unit 20 Memory section 21 Problem Database 22 Answer Database 30 Communication Modules 40 Input / Output 1 / F PG1 Learning Support Program A: Evaluation matrix P User Factor Matrix Q problem factor matrix T tag vector

Claims

1. A learning support device that presents a user with a problem selected from a set of problems, It is connected to a database that stores the answers of multiple other users to the aforementioned set of problems, The answers of the target user to the preliminary test questions included in the aforementioned set of questions are obtained. Based on the target user's answers to the preliminary test question set and the answers of the multiple other users to the question set, a first index indicating the target user's level of difficulty with each question included in the question set is calculated. Based on the target user's answers to the preliminary test question set and the label information assigned to each question in the question set, a second index is calculated that indicates the target user's level of difficulty with each question in the question set. Based on the first and second indicators, the system predicts which problems the target user will find difficult from the set of problems and presents those problems to the target user. It includes a control unit that performs the following: The first indicator is calculated such that it becomes larger for questions answered incorrectly by users among the multiple other users whose answering tendencies are similar to those of the target user. The second indicator is calculated such that it is larger for questions in the preliminary test question set that are similar to the questions that the target user answered incorrectly. Learning support device.

2. The control unit, An integrated index is calculated by combining the first index and the second index, and problems with a larger integrated index among the set of problems are predicted to be problems that the target user finds difficult. The learning support device according to claim 1.

3. The aforementioned integrated index is the harmonic mean of the first index and the second index. The learning support device according to claim 2.

4. Each question in the aforementioned set of questions is a multiple-choice question with multiple options, and each of these options is assigned a weighted score. The control unit calculates an evaluation value indicating each user's level of difficulty with each problem, according to the score assigned to the option selected by each user for each problem, and calculates the first index based on the evaluation value. The learning support device according to claim 1.

5. The aforementioned options include one correct answer and multiple incorrect answers, and each of the multiple incorrect answers is assigned a different score. The learning support device according to claim 4.

6. The control unit calculates an evaluation value indicating each user's level of difficulty with each problem, based on the time each user takes to answer each problem, and calculates the first index based on the evaluation value. A learning support device according to any one of claims 1 to 5.

7. A learning support program that presents a selected set of problems to the target user, A computer capable of accessing a database that stores the answers of multiple other users to the aforementioned set of problems, The answers of the target user to the preliminary test questions included in the aforementioned set of questions are obtained. Based on the target user's answers to the preliminary test question set and the answers of the multiple other users to the question set, a first index indicating the target user's level of difficulty with each question included in the question set is calculated. Based on the target user's answers to the preliminary test question set and the label information assigned to each question in the question set, a second index is calculated that indicates the target user's level of difficulty with each question in the question set. Based on the first and second indicators, the system predicts which problems the target user will find difficult from the set of problems and presents those problems to the target user. Let them do it, The first indicator is calculated such that it becomes larger for questions answered incorrectly by users among the multiple other users whose answering tendencies are similar to those of the target user. The second indicator is calculated such that it is larger for questions in the preliminary test question set that are similar to the questions that the target user answered incorrectly. Learning support program.

8. A learning support method that presents a target user with a problem selected from a set of problems, Using a computer capable of accessing a database that stores the answers of multiple other users to the aforementioned set of problems, The answers of the target user to the preliminary test questions included in the aforementioned set of questions are obtained. Based on the target user's answers to the preliminary test question set and the answers of the multiple other users to the question set, a first index indicating the target user's level of difficulty with each question included in the question set is calculated. Based on the target user's answers to the preliminary test question set and the label information assigned to each question in the question set, a second index is calculated that indicates the target user's level of difficulty with each question in the question set. Based on the first and second indicators, the system predicts which problems the target user will find difficult from the set of problems and presents those problems to the target user. To carry out the task, The first indicator is calculated such that it becomes larger for questions answered incorrectly by users among the multiple other users whose answering tendencies are similar to those of the target user. The second indicator is calculated such that it is larger for questions in the preliminary test question set that are similar to the questions that the target user answered incorrectly. Learning support methods.

9. The device operated by the target user, A learning support device that presents a selected problem from a set of problems to the target user via the terminal. A learning support system comprising, The aforementioned learning support device is It is connected to a database that stores the answers of multiple other users to the aforementioned set of problems, The answers of the target user to the preliminary test questions included in the aforementioned set of questions are obtained. Based on the target user's answers to the preliminary test question set and the answers of the multiple other users to the question set, a first index indicating the target user's level of difficulty with each question included in the question set is calculated. Based on the target user's answers to the preliminary test question set and the label information assigned to each question in the question set, a second index is calculated that indicates the target user's level of difficulty with each question in the question set. Based on the first and second indicators, the system predicts which problems the target user will find difficult from the set of problems and presents those problems to the target user. It includes a control unit that performs the following: The first indicator is calculated such that it becomes larger for questions answered incorrectly by users among the multiple other users whose answering tendencies are similar to those of the target user. The second indicator is calculated such that it is larger for questions in the preliminary test question set that are similar to the questions that the target user answered incorrectly. Learning support system.

Images