Lever determination device, lever determination method, and program
The level determination device in CAT systems uses probability distributions and superimposed academic ability distributions to address the inefficiencies in conventional methods, reducing test time and user burden by accurately determining user levels.
Patent Information
- Application Number
- JP2023070809
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-24
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2043-04-24
AI Technical Summary
Conventional Computerized Adaptive Testing (CAT) methods fail to utilize user's previous answers to determine the first question, leading to increased test time and user burden.
A level determination device that includes a question section, correctness determination, and judgment level generation, using probability distributions based on initial user academic ability to ask questions and correct/incorrect answers, superimposing academic ability distributions to generate a judgment level.
The device shortens test time and reduces user burden by accurately determining the user's level through iterative processing and superimposing academic ability distributions, improving precision and accuracy of the judgment level.
Smart Images

Figure 2024156391000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a level determination device, a level determination method, and a program for determining a level representing a user's academic ability. [Background technology]
[0002] In recent tests, computerized adaptive testing (CAT) is being used (for example, Non-Patent Document 1). CAT measures the level of understanding based on the user's answers and determines the next question to be asked based on the measured level of understanding. As a result, it is possible to measure the user's level of understanding with a small number of questions, shortening the test time and reducing the burden on the user. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] Educational Measurement Research Institute, Inc., “Changing Test Formats”, [online], Educational Measurement Research Institute, Inc., [Retrieved March 31, 2023], Internet〈 URL: https: / / www.jiem.co.jp / concept / pop_keitai.html 〉 Summary of the Invention [Problem to be solved by the invention]
[0004] However, with conventional CAT methods, there is no prior answer data from the user at the start of the test, so the first question cannot be set based on the user's level of understanding, which means that shortening the test time and reducing the burden on the user are insufficient.
[0005] Therefore, an object of the present disclosure is to provide a level judgment device that can shorten the test time and reduce the burden on the user. [Means for solving the problem]
[0006] The level judgment device of the present disclosure includes a question setting section, a correct / incorrect judgment section, and a judgment level generating section.
[0007] The question setting unit sets questions to the user corresponding to a probability distribution based on an initial level representing the user's academic ability in a specified unit. The correctness determining unit obtains answers to the questions from the user and determines whether the answers are correct. The judgment level generating unit corrects the probability distribution by superimposing the academic ability distribution of a group of people who answered correctly in a question corresponding to a correct answer on the probability distribution, and corrects the probability distribution by superimposing the academic ability distribution of a group of people who answered incorrectly in a question corresponding to an incorrect answer on the probability distribution to generate a judgment level. Effect of the Invention
[0008] According to the level judgment device of the present disclosure, it is possible to shorten the test time and reduce the burden on the user. [Brief description of the drawings]
[0009] [Figure 1] FIG. 2 is a block diagram showing the functional configuration of the level determination device according to the first embodiment. [Diagram 2] 4 is a flowchart showing the operation of the level determining device according to the first embodiment. [Diagram 3] A diagram illustrating the relationship between units and clauses. [Figure 4] FIG. 13 is a diagram illustrating a procedure for generating an initial level based on comprehension for each passage. [Diagram 5] FIG. 13 is a diagram illustrating a case where the initial level is obtained or generated as one value. [Figure 6] FIG. 13 illustrates a case where the initial level is obtained or generated as a probability distribution. [Figure 7] FIG. 11 is a diagram showing Example 1 in which an initial level, which is one value, is corrected over time. [Figure 8] FIG. 11 is a diagram showing Example 2 in which an initial level, which is one value, is corrected over time. [Figure 9] FIG. 11 is a diagram showing Example 3 in which the initial level, which is a probability distribution, is corrected according to the passage of time. [Figure 10]FIG. 13 is a diagram showing an example of generating a judgment level by superimposing the academic ability distribution of students who answered correctly on a correction level. [Figure 11] FIG. 13 is a diagram showing an example of generating a judgment level by superimposing the academic ability distribution of incorrect answerers on a correction level. [Figure 12] FIG. 2 is a diagram showing an example of the functional configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, an embodiment of the present invention will be described in detail. Note that components having the same functions are given the same reference numbers and duplicated explanations will be omitted. EXAMPLES
[0011] The functional configuration of the level judgment device of the first embodiment will be described below with reference to Fig. 1. As shown in the figure, the level judgment device 1 of the present embodiment includes an initial level storage unit 1A, a question database 1B, an initial level generation unit 11, a correction level generation unit 12, a question setting unit 13, a correct / incorrect judgment unit 14, and a judgment level generation unit 15.
[0012] The contents of the data stored in each storage unit will be described below.
[0013] <Initial Level Memory Unit 1A> The initial level storage unit 1A stores an initial level that indicates the academic ability of the user in a predetermined unit. In this embodiment, the initial level is expressed in five stages from 1 to 5. Since the level determination device 1 of this embodiment only needs to use the initial level in some way, the initial level may be acquired from the initial level storage unit 1A, or the initial level may be generated by the initial level generation unit 11 described later. Therefore, the level determination device 1 of this embodiment only needs to include the component of either the initial level storage unit 1A or the initial level generation unit 11, and the other component may be omitted.
[0014] <Problem Database 1B> The problem database 1B stores the problem ID of the problem to be posed to the user, the problem statement, the problem level set for that problem, the answer (or the model answer in the case of an essay question, etc.), the academic ability distribution of those who answered that problem correctly, and the academic ability distribution of those who answered that problem incorrectly, all linked together. The data stored in the problem database 1B is shown in the table below. [Table 1] The operation of each component will be described below with reference to FIG.
[0015] <<Initial level generation unit 11>> The initial level generating unit 11 generates an initial level in some way (S11). For example, as shown in FIG. 3, a case will be described where a given unit is broken down into a plurality of sections. In the example shown in the figure, the learning contents of the junior high school mathematics unit "Calculation of positive and negative numbers" are divided into sections 1-10. Typically, the smaller the section number, the more elementary the contents recorded. Also, the larger the section number, the more applied the contents recorded, and the contents that are difficult to understand unless the level of understanding of the learning contents of the smaller section numbers of the same unit is high. Although the example of mathematics is shown in the figure, it is possible to define the learning contents by dividing them into units and sections even in the case of other subjects such as Japanese, English, science, and history.
[0016] Each user will receive a level of understanding of 0-100% for each section based on the correctness of the exercise questions for each section, the status of submission of work for each section, the status of taking correspondence courses for each section, etc. In the example of Figure 4, the user will receive a level of understanding of 100% for sections 1 and 2, 90% for section 3, 80% for section 4, 60% for section 5, and so on.
[0017] Each of the multiple sections corresponds to one of the initial levels. In the example of the figure, one or more sections are assigned to each of the initial levels 1-5 without overlapping. More specifically, section 1 is assigned to initial level 1, sections 2-5 are assigned to initial level 2, sections 6-7 are assigned to initial level 3, sections 8-9 are assigned to initial level 4, and section 10 is assigned to initial level 5.
[0018] In this case, the initial level generating unit 11 generates a corresponding initial level based on the user's understanding of each of the multiple sections (S11). More specifically, the initial level generating unit 11 generates a corresponding initial level according to the proportion of the multiple sections whose understanding has reached a predetermined rate (S11). In the example of the figure, the predetermined rate is 70%. In the example of the figure, the understanding level is 100% in section 1 corresponding to initial level 1, so the proportion of sections that have reached the predetermined rate (70%) in initial level 1 is 1 / 1=100%. In sections 2-5 corresponding to initial level 2, the understanding levels are 70% or more except for section 5, so the proportion of sections that have reached the predetermined rate (70%) in initial level 2 is 3 / 4=75%. In sections 6-7 corresponding to initial level 3, the understanding level of only section 6 is 70% or more, so the proportion of sections that have reached the predetermined rate (70%) in initial level 3 is 1 / 2=50%. Among sections 8-10, which correspond to initial levels 4 and 5, there are no sections for which the comprehension level reaches 70% or more, so the percentage of sections that reach the specified rate (70%) in initial levels 4 and 5 is 0%.
[0019] The ratio of the clauses that have reached a certain level of understanding or higher for each initial level thus generated is regarded as a cumulative probability distribution, and a probability distribution can be generated by converting this cumulative probability distribution. In the example of the figure, a probability distribution of initial level 1=25%, initial level 2=25%, and initial level 3=50% can be obtained. This probability distribution indicates that for the corresponding user, the initial level in this unit is 25% likely to be 1, 25% likely to be 2, and 50% likely to be 3. Note that the initial level does not necessarily have to be given as a probability distribution as shown in FIG. 4, and may be given as a single value. For example, if only the maximum value of the probability distribution in FIG. 4 is adopted, the initial level will be 3 (100%) as shown in FIG. 5. If the entire probability distribution in FIG. 4 is used as is, the initial level will be the result shown in FIG. 6.
[0020] <Correction level generating unit 12> The correction level generating unit 12 acquires the initial level, corrects the initial level in accordance with the time that has elapsed since the time the initial level was determined until the present, and generates a correction level that is a probability distribution of academic ability (S12).
[0021] For example, when the initial level is 3 as shown in FIG. 5, the correction level generating unit 12 corrects the initial level according to the time lapse from the time when the initial level was determined to the present, and generates a probability distribution as a normal distribution as shown in FIG. 7 and FIG. 8. The variance value of the normal distribution can take into account both the possibility that the user's academic ability has improved due to growth in the relevant unit over time and the possibility that the academic ability has decreased due to forgetting. If the time elapsed since the generation of the initial level is short, the effects of growth and forgetting are small, and the correction level is considered to have a probability distribution close to the state of the initial level, so the variance should be small (for example, FIG. 7). If the time elapsed since the generation of the initial level is long, it is estimated that the effects of growth and forgetting are large, so the variance should be large (for example, FIG. 8).
[0022] In addition, when the initial level is acquired and generated not as a single value but as a probability distribution as shown in FIG. 6, as shown in FIG. 9, a normal distribution (solid line graph in the same figure) to which the effects of growth and forgetting are added may be generated corresponding to each initial level value of the probability distribution, and the correction level may be generated by superposing these normal distributions (long dashed line graph in the same figure).
[0023] Figure 7-9 shows an example of generating a correction level using normal distribution, but normal distribution is only one example and does not need to be limited to normal distribution. For example, t distribution, F distribution, binomial distribution, chi-square distribution, Poisson distribution, gamma distribution, etc. can be used. Normal distribution is symmetric, but there may be cases where an asymmetric distribution such as F distribution is preferable.
[0024] <<Question Section 13>> The question setting unit 13 sets a question corresponding to the correction level to the user (S13). For example, the question setting unit 13 searches the question database 1B for questions of a problem level corresponding to the level at which the correction level (probability distribution) peaks. The question setting unit 13 transmits the searched question to the user's terminal. Alternatively, the question setting unit 13 displays the searched question on a display connected to the level determination device 1. The user answers the question received by the user's terminal by inputting the answer into the user's terminal. Alternatively, the user answers the question displayed on a display connected to the level determination device 1 by inputting the answer into the level determination device 1.
[0025] The question setting unit 13 may set a plurality of questions corresponding to the correction level to the user at once without going through the convergence determination described later.
[0026] <Correctness Judgment Section 14> The correctness judgment unit 14 acquires the answer to the question from the user (user's terminal) and judges whether the answer is correct or incorrect (S14). For example, the correctness judgment unit 14 may compare the answer to the question from the user with the answer stored in the question database 1B to judge whether the answer is correct or incorrect.
[0027] When the question setting section 13 sets a plurality of questions at once, the correctness determining section 14 also performs correctness determination for the number of answers.
[0028] <Decision level generating unit 15> The judgment level generating unit 15 generates a judgment level by superimposing the academic ability distribution of either correct answerers or incorrect answerers on the correction level (S15).
[0029] More specifically, the judgment level generating unit 15 superimposes the academic ability distribution of the group of correct answerers for the question corresponding to the case where the answer is correct on the probability distribution, and superimposes the academic ability distribution of the group of incorrect answerers for the question corresponding to the case where the answer is incorrect on the probability distribution, thereby correcting the probability distribution and generating the judgment level (S15). As described above, the academic ability distribution of the group of correct answerers / incorrect answerers is stored in advance in the question database 1B.
[0030] For example, as shown in Fig. 10, if question a, which corresponds to question level 3, is asked based on the correction level (solid line graph) that peaks at level 3, and the user's answer to question a is correct, the judgment level (dash-dotted line graph) can be generated by superimposing the academic ability distribution of those who answered question a correctly (dashed line graph) on the correction level. Comparing the correction level (solid line graph) in the same figure with the judgment level (dash-dotted line graph) in the same figure, it can be seen that the peak position and the entire distribution have shifted to the right.
[0031] Also, for example, as shown in Fig. 11, if question b, which corresponds to question level 3, is posed based on the same correction level (solid line graph) as above, and the user's answer to question b is incorrect, the judgment level (dash-dotted line graph) can be generated by superimposing the academic ability distribution (dashed line graph) of those who answered question b incorrectly on the correction level. Comparing the correction level (solid line graph) in the same figure with the judgment level (dash-dotted line graph) in the same figure, it can be seen that the peak position and the entire distribution are shifted to the left.
[0032] If the convergence judgment described later is not performed, the judgment level generation unit 15 can output the judgment level at any timing. If the question setting unit 13 sets multiple questions at once, the judgment level generation unit 15 also repeatedly performs superimposition of academic ability distributions for the number of correct answers and incorrect answers to generate a judgment level. In this case, the judgment level generation unit 15 may determine and output the judgment level without performing the convergence judgment described later. As described above, the judgment level that is output may be the probability distribution itself, or may be the peak value, average value, etc. of the probability distribution.
[0033] <Iterative processing, convergence judgment> The process for executing the convergence judgment will be described below. It is assumed that the above-mentioned steps of setting a question based on the correction level, judging whether the answer is correct, and generating a judgment level (S13-S15) have been executed at least once before this process.
[0034] The question setting unit 13 executes a process of setting a question corresponding to the generated judgment level to the user. The correctness judgment unit 14 executes a process of acquiring an answer to the question from the user and judging whether the answer is correct or not. The judgment level generation unit 15 executes a process of correcting the judgment level by superimposing the academic ability distribution in a group of people who answered correctly on the judgment level when the answer is correct, and by superimposing the academic ability distribution in a group of people who answered incorrectly on the judgment level when the answer is incorrect, thereby generating a new judgment level.
[0035] The question setting unit 13, the correctness judgment unit 14, and the judgment level generation unit 15 repeatedly execute the above-mentioned process until a predetermined condition is satisfied. The predetermined condition may be, for example, when the probability value of the peak of the judgment level is equal to or greater than a predetermined threshold, or when the variance of the judgment level is equal to or less than a predetermined threshold. In addition, the predetermined condition may be when the number of questions set by the question setting unit 13 reaches a predetermined threshold, or when the time elapsed since the first question set by the question setting unit 13 exceeds a predetermined threshold.
[0036] According to the level judgment device 1 of this embodiment, the first questions for CAT are set based on the probability distribution with the initial level corrected, so that the test time can be shortened and the burden on the user can be reduced. In addition, since the academic ability distributions of correct and incorrect answerers are superimposed, the judgment level can be generated with high accuracy. When the repetitive process is executed, the judgment level is continuously updated until a predetermined convergence condition is reached, so that the accuracy of the finally generated judgment level can be improved.
[0037] <Additional Notes> The device of the present invention has, as a single hardware entity, an input section to which a keyboard or the like can be connected, an output section to which a liquid crystal display or the like can be connected, a communication section to which a communication device (e.g., a communication cable) capable of communicating with the outside of the hardware entity can be connected, a CPU (which may also have a central processing unit, cache memory, registers, etc.), memories such as RAM and ROM, an external storage device such as a hard disk, and buses connecting the input section, output section, communication section, CPU, RAM, ROM, and external storage device so that data can be exchanged between them. If necessary, the hardware entity may also be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity equipped with such hardware resources is, for example, a general-purpose computer.
[0038] The external storage device of the hardware entity stores the programs required to realize the above-mentioned functions and the data required in the processing of these programs (not limited to the external storage device, but for example the programs may be stored in a ROM, which is a read-only storage device). Data obtained by the processing of these programs is stored appropriately in the RAM, the external storage device, etc.
[0039] In the hardware entity, each program stored in an external storage device (or ROM, etc.) and the data required to process each program are loaded into memory as necessary, and interpreted, executed, and processed by the CPU as appropriate, so that the CPU realizes a predetermined function (each of the components represented as the above, "units," "means," etc.).
[0040] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention. In addition, the processes described in the above-described embodiment are not limited to being executed in chronological order according to the order described, but may be executed in parallel or individually depending on the processing capacity of the device that executes the processes or as necessary.
[0041] As described above, when the processing functions of the hardware entities (the devices of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entities should have are described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.
[0042] The various processes described above can be implemented by loading a program for executing each step of the above method into the recording unit 10020 of the computer shown in FIG. 12 and operating the control unit 10010, input unit 10030, output unit 10040, etc.
[0043] The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable medium. Specifically, for example, a hard disk drive, a flexible disk, a magnetic tape, or the like can be used as the magnetic recording device, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (ReWritable), or the like can be used as the optical disk, an MO (Magneto-Optical disc), or the like can be used as the magneto-optical recording medium, and an EEP-ROM (Electrically Erasable and Programmable-Read Only Memory) or the like can be used as the semiconductor memory.
[0044] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.
[0045] A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or the program transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. The above-mentioned process may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result, without transferring a program from the server computer to the computer. Note that the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (data that is not a direct command to the computer but has a nature that specifies the processing of the computer, etc.).
[0046] In addition, in this embodiment, a hardware entity is configured by executing a specific program on a computer, but at least a part of the processing contents may be realized by hardware.
Claims
1. a question setting unit that sets to the user questions corresponding to a probability distribution based on an initial level that represents the user's academic ability in a predetermined unit; a correctness determining unit that obtains an answer to the question from the user and determines whether the answer is correct; and a judgment level generating unit that corrects the probability distribution by superimposing an academic ability distribution in a group of people who answered correctly in a question corresponding to a case where the answer is correct, on the probability distribution, and by superimposing an academic ability distribution in a group of people who answered incorrectly in a question corresponding to a case where the answer is incorrect, on the probability distribution, thereby generating a judgment level. Level determination device.
2. 2. The level determination device according to claim 1, The question setting unit includes: execute a process of presenting a question corresponding to the assessment level to the user; The correctness determining unit is execute a process of acquiring an answer to the question from the user and determining whether the answer is correct; The judgment level generating unit is execute a process of correcting the judgment level by superimposing the academic ability distribution in the group of correct answerers on the judgment level when the answer is correct, and by superimposing the academic ability distribution in the group of incorrect answerers on the judgment level when the answer is incorrect, thereby generating a new judgment level; The question setting unit, the correctness determining unit, and the evaluation level generating unit repeatedly execute the process until a predetermined condition is satisfied. Level determination device.
3. 2. The level determination device according to claim 1, a correction level generating unit that acquires the initial level, corrects the initial level in accordance with the elapsed time from the time when the initial level was determined to the present, and generates a correction level that is a probability distribution of the academic ability, The question setting unit includes: Presenting a question corresponding to the correction level to the user; The judgment level generating unit is A judgment level is generated by superimposing any of the academic achievement distributions on the correction level. Level determination device.
4. 2. The level determination device according to claim 1, The predetermined unit is broken down into a plurality of sections, each of the plurality of sections corresponds to one of the initial levels, and an initial level generating unit is provided for generating the corresponding initial level based on the user's understanding of each of the plurality of sections. Level determination device.
5. 5. The level determination device according to claim 4, each of said initial levels being assigned one or more nodes without overlapping; The initial level generating unit includes: A corresponding initial level is generated according to a proportion of the plurality of passages whose comprehension has reached a predetermined proportion. Level determination device.
6. Presenting a question to the user corresponding to a probability distribution based on an initial level representing the user's academic ability in a predetermined unit; obtaining an answer to the question from the user and determining whether the answer is correct; a step of correcting the probability distribution by superimposing an academic ability distribution in a group of people who answered correctly in a question corresponding to a case where the answer was correct, on the probability distribution, and correcting the probability distribution by superimposing an academic ability distribution in a group of people who answered incorrectly in a question corresponding to a case where the answer was incorrect, to generate a judgment level. How to determine the level.
7. A program for causing a computer to function as the level determination device according to any one of claims 1 to 5.
Citation Information
Patent Citations
Work skill supporting device
JP2001166680A
Learning system
JP2004117947A
Learning drill delivery system
JP2005070465A
User-adaptive test program, apparatus, and method for selecting model of problem group according to comprehension degree probability
JP2017097139A
Learning support system having continuous evaluation function of learner and teaching material
JP2017134184A
