Academic ability estimation system, academic ability estimation method, and academic ability estimation program

Abstract

By minimizing the influence of factors such as the difficulty level of the questions and the quality of the answers in the test, a more accurate and quantifiable representation of the user's academic ability can be obtained. [Solution] The academic ability estimation system 1 comprises at least one processor, which acquires answer information including answer data indicating whether the user's answer to each of a plurality of problems is correct or incorrect, determines answer data inappropriate for calculating the user's academic ability based on predetermined conditions, estimates the difficulty level of at least each problem based on the answer information from which the answer data determined to be inappropriate has been excluded, estimates the academic ability of each user based on the estimated difficulty level of each problem and the answer information before the exclusion of answer data, and outputs at least each user's academic ability.

Description

[Technical Field]

[0001] One aspect of this disclosure relates to an academic ability estimation system, an academic ability estimation method, and an academic ability estimation program. [Background technology]

[0002] To assess the effectiveness of online language learning, systems for determining the language proficiency of learners are known (see, for example, Patent Document 1). In the system described in Patent Document 1, the language proficiency level is calculated by performing statistical processing on evaluation data stored in a database. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2018-87920 [Overview of the project] [Problems that the invention aims to solve]

[0004] In examinations administered to users, there is a need to quantify users' academic ability without being affected by differences in the difficulty of questions from one examination to another. Furthermore, especially in online examinations conducted without proctoring, there is a risk of answers being given due to cheating by test-takers. In addition, in academic ability measurement tests that are not intended for qualification acquisition or entrance examinations, some test-takers may give answers that are not based on careful consideration or answers given randomly. Answer data obtained due to factors other than genuine academic ability has been a contributing factor to a decrease in statistical reliability.

[0005] This disclosure aims to obtain a more accurate, quantified representation of user academic ability by minimizing the influence of factors such as the difficulty level of the questions and the quality of the answers in the test. [Means for solving the problem]

[0006] A system for estimating academic ability relating to one aspect of this disclosure comprises at least one processor, which acquires answer information including answer data indicating whether a user's answer to each of a plurality of problems is correct or incorrect, determines answer data inappropriate for calculating the user's academic ability based on predetermined conditions, estimates the difficulty level of at least each problem based on the answer information from which the answer data deemed inappropriate has been excluded, estimates the academic ability of each user based on the estimated difficulty level of each problem and the answer information before the exclusion of answer data, and outputs at least each user's academic ability.

[0007] From this perspective, the appropriate difficulty level of each question can be appropriately estimated based on answer data that excludes answer data inappropriate for calculating the user's academic ability. Furthermore, since the user's academic ability is estimated based on the appropriate difficulty level of each question, it is possible to provide the user with more accurate and quantified information about their academic ability. [Effects of the Invention]

[0008] According to one aspect of this disclosure, it becomes possible to obtain a more accurate, quantified representation of the user's academic ability by mitigating the influence of factors such as the difficulty level of the questions and the quality of the answers in the test. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the device configuration of an academic ability estimation system. [Figure 2] This figure shows an example of the hardware configuration of the academic ability estimation system and user terminal. [Figure 3] This is a block diagram showing an example of the functional configuration of an academic ability estimation server and user terminal. [Figure 4] This figure shows an example of the structure and content of test management information. [Figure 5] This figure shows an example of the structure and content of the answer information. [Figure 6] This is a flowchart illustrating an example of how the academic ability estimation system works. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described in detail below with reference to the attached drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] Figure 1 shows an example of the application of an academic ability estimation system 1. In this example, the academic ability estimation system 1 includes an academic ability estimation server 10. The academic ability estimation server 10 is a computer that estimates the quantified academic ability of users. The academic ability estimation system 1 is configured to communicate with one or more user terminals 20 and a group of databases 30 via a communication network N. The communication network N may include the internet or may include an intranet.

[0012] User terminal 20 is a computer used by the user. User terminal 20 has the function of accessing the academic ability estimation system 1 and conducting tests for academic ability estimation, and the function of receiving and presenting information on estimated academic ability. User terminal 20 may be a mobile device such as a high-function mobile phone (smartphone), tablet device, wearable device (e.g., head-mounted display (HMD), smart glasses, etc.), laptop personal computer, or mobile phone. Alternatively, user terminal 20 may be a stationary device such as a desktop personal computer.

[0013] The database group 30 is a collection of one or more databases that store data used for conducting tests and estimating users' academic abilities. In one example, the database group 30 includes a test management database 31 and an answer information database 32.

[0014] The test management database 31 is a non-temporary storage device that manages and stores various information related to the test questions and information about the test, which consists of multiple questions. Details of the test management database 31 will be described later with reference to Figure 4.

[0015] The answer information database 32 is a non-temporary storage device that stores answer information, including answer data for tests administered to users. Details of the answer information database 32 will be described later with reference to Figure 5.

[0016] Each database in the database group 30 may be a component of the academic ability estimation system 1, or it may be located in a computer system separate from the academic ability estimation system 1.

[0017] Figure 2 shows an example of the hardware configuration of the academic ability estimation system 1 and the user terminal 20. Figure 2 shows a server computer 100 that functions as the academic ability estimation server 10 and a terminal computer 200 that functions as the user terminal 20.

[0018] As an example, the server computer 100 comprises a processor 101, a main memory unit 102, an auxiliary memory unit 103, and a communication unit 104 as hardware components. The processor 101 is an arithmetic unit that executes the operating system and application programs, and is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The main memory unit 102 is a device that stores programs to be executed, calculation results, etc., and is, for example, composed of ROM (Read Only Memory) or RAM (Random Access Memory). The auxiliary memory unit 103 is a device that can generally store a larger amount of data than the main memory unit 102, and is, for example, composed of a non-volatile storage medium such as a hard disk or flash memory. The auxiliary memory unit 103 stores the server program P1 and various data for making the server computer 100 function as an academic ability estimation server 10. The communication unit 104 is a device that performs data communication with other computers via a communication network N, and is, for example, composed of a network card or a wireless communication module.

[0019] The academic ability estimation program described herein is implemented as server program P1. Each functional element of the academic ability estimation server 10 is realized by loading the corresponding server program P1 onto the processor 101 or main memory unit 102 and having the processor 101 execute the program. Server program P1 includes code for realizing each functional element of the academic ability estimation server 10. The processor 101 operates the communication unit 104 according to the server program P1 and performs data reading and writing in the main memory unit 102 or auxiliary memory unit 103.

[0020] The academic ability estimation server 10 may consist of one or more computers. When multiple computers are used, these computers are connected to each other via a communication network N, thereby logically forming a single academic ability estimation server 10.

[0021] As an example, the terminal computer 200 comprises a processor 201, a main memory unit 202, an auxiliary memory unit 203, a communication unit 204, an input interface 205, and an output interface 206 as hardware components. The processor 201 is an arithmetic unit that executes the operating system and application programs, and is, for example, a CPU or GPU. The main memory unit 202 is a device that stores programs to be executed, calculation results, etc., and is, for example, composed of ROM or RAM. The auxiliary memory unit 203 is a device that can generally store a larger amount of data than the main memory unit 202, and is, for example, composed of a non-volatile storage medium such as a hard disk or flash memory. The auxiliary memory unit 203 stores the client program P2 and various data for making the terminal computer 200 function as a user terminal 20. The communication unit 204 is a device that performs data communication with other computers via a communication network N, and is, for example, composed of a network card or a wireless communication module. The input interface 205 is a device that receives data based on user operations or actions, and is comprised of at least one of the following: a keyboard, operation buttons, a pointing device, a touch panel, a microphone, a sensor, and a camera. The output interface 206 is a device that outputs data processed by the terminal computer 200, and is comprised of a display device such as a monitor, a touch panel, or an HMD.

[0022] The academic ability estimation program relating to this disclosure may be implemented as a client program P2. Each functional element of the user terminal 20 is realized by loading the client program P2 into the processor 201 or main memory 202 and having the processor 201 execute the program. The client program P2 includes code for realizing each functional element of the user terminal 20. The processor 201 operates the communication unit 204, input interface 205, or output interface 206 according to the client program P2 and performs data reading and writing in the main memory 202 or auxiliary memory 203.

[0023] At least one of the server program P1 and the client program P2 may be provided on a non-temporary recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, at least one of these programs may be provided via a communication network N as a data signal superimposed on a carrier wave.

[0024] Figure 3 shows the functional configuration of the academic ability estimation system 1 and the user terminal 20. As shown in Figure 3, the academic ability estimation server 10 comprises an examination management unit 11, an answer data acquisition unit 12, a judgment unit 13, an estimation unit 14, and an output unit 15. The user terminal 20 comprises an examination execution unit 21 and a presentation unit 22 as functional components.

[0025] The test management unit 11 has the function of managing and conducting tests for estimating the user's academic ability. Specifically, the test management unit 11 conducts tests for users using tests and questions managed in the test management database 31.

[0026] Figure 4 schematically shows the structure of the test management database 31 and an example of the test information stored therein. As shown in Figure 4, the test information stores information on multiple questions associated with each test identifier Ti. The question information is associated with a question identifier q that identifies each question. i This includes information such as the question, correct answer, question characteristics (difficulty level), and question characteristics (difficulty level series) associated with the question. The question is the content of the question, including, for example, the question text and answer choices. The correct answer indicates the correct answer to the question. The question characteristics (difficulty level) is the difficulty level b of the question. i As will be detailed later, it is estimated based on the answer information. The problem characteristics (difficulty series) are the difficulty levels of each problem estimated over multiple instances (sample size t: 1 to T). it This is a series that shows the time-series progression of the difficulty level of the problem in question. For example, if the estimated difficulty level decreases at a certain point in time, it is suspected that the problem has been leaked to users, and based on the characteristics of the problem, it is possible to detect the problem leak.

[0027] The test management unit 11 transmits a plurality of questions q in a group that constitute, for example, a single test T to the user U. The test execution unit 21 of the user terminal 20 receives the plurality of questions q in a group that constitute the test T transmitted from the academic ability estimation server 10. The test execution unit 21 presents each question q to the user and acquires the answers to each question q input by the user. Then, the test execution unit 21 transmits the acquired answers, the time stamp indicating the time when the answers were input, the time stamps at the start and end of the test, etc. to the academic ability estimation server 10. The test management unit 11 receives the information transmitted from the user terminal 20 as answer information, and performs scoring of correct and incorrect answers by comparing the answers to each question with the correct answers to each question.

[0028] FIG. 5 is a diagram schematically showing an example of the configuration of the answer information database 32 and the answer information stored therein. As shown in FIG. 5, the answer information includes, for each examinee identifier U j and test identifier T, the question identifier q of each question i , answer data X ij , information such as time, time D, etc. The answer data X ij is data indicating the correctness of the answers to each question. The time of each question is the time stamp of the time when the answer was input. The time D represents the length of time required for the answer to each question. Further, the answer information may have the time stamps at the start and end of the test that was conducted online at the user terminal 20, and may further include the time stamps at the time of interruption and restart of the test.

[0029] The test management unit 11 stores various information received from the user terminal 20 in the answer information database 32 as answer information. Also, the test management unit 11 stores the answer data X ij indicating the scoring result of the answers to each question transmitted from the user terminal 20 in the answer information database 32 as answer information.

[0030] The answer data acquisition unit 12 acquires answer information including the answer data X ij indicating the correctness of the answers by the user for each of the plurality of questions. Specifically, the answer data acquisition unit 12 acquires answer information from the answer information database 32.

[0031] The determination unit 13 determines, based on predetermined conditions, that the answer data X is inappropriate for calculating the user's academic ability. ij This determines whether or not it is true. The determination process will be explained in detail below.

[0032] The answer information is the time D required to answer each question. i If (i=1,2,3,··) is present, the determination unit 13 determines the answer time D i Answer data X for problems where the score is below a predetermined level. ij It may be determined that the answer data is inappropriate. Specifically, for example, the answer time D i Setting the threshold for this to 5 seconds, the determination unit 13 determines the answer time D i Answer data X where the time is less than 5 seconds. ij The answer data may be judged as inappropriate. That is, answers that were not completed within a certain time are likely to be inappropriate for estimating academic ability, for example, because they were answered randomly. Therefore, answer data for questions that were completed within a certain time will be excluded from the estimation of academic ability. This will improve the accuracy of the estimation of academic ability.

[0033] If the answer information includes the total answer time, which is the time taken by the user to answer a group of multiple questions, the determination unit 13 may determine that the answer data for multiple questions whose total answer time is less than a predetermined amount is inappropriate answer data. For example, the determination unit 13 may consider all the questions in one exam as a group of multiple questions and the answer time D for each of all the questions in one exam i The total answer time may be calculated as the sum of the individual times, or as the time from the start timestamp to the end timestamp of a single test.

[0034] Then, the threshold for the total response time is set to the minimum time normally required to answer the test, and the determination unit 13 determines the answer data X of the test for which the total response time is less than the threshold. ij The determination unit 13 may determine that the answer data is inappropriate. That is, the determination unit 13 may decide to exclude the test information of the user for that one test in the estimation of academic ability.

[0035] Thus, given that answers that were not completed within a specified time are likely to be unsuitable for estimating academic ability due to reasons such as being answered randomly, the answer data for a group of multiple questions with a total completion time of less than the specified amount will be excluded from the estimation of academic ability. This will improve the accuracy of the academic ability estimation.

[0036] If the answer information includes answer execution time information indicating the time spent by the user during and after the execution of answers to a series of multiple questions, the determination unit 13 may determine that the answer data for the user's answers to a series of multiple questions is inappropriate answer data if the pattern of the time spent during and after the execution of answers indicated in the answer execution time information matches a predetermined pattern.

[0037] Specifically, the determination unit 13 may acquire answer execution time information based, for example, on timestamps included in the answer information for the start and end of the exam, as well as for interruption and resumption. When the exam is conducted online via the user terminal 20, there is a possibility of cheating, such as interrupting the exam to refer to information that would assist in answering. In such cases, the determination unit 13 may determine to exclude the exam information for a single exam taken by a user in the estimation of academic ability if the answer pattern shown in the user's answer execution time information corresponds to a predetermined pattern that suggests cheating. This makes it possible to improve the accuracy of the academic ability estimation.

[0038] If the test questions are multiple-choice and the answer is provided by selecting one option from multiple choices, including an unknown option for when the answer is unknown, the determination unit 13 may determine that the answer data provided by selecting the unknown option is inappropriate. Excluding the answer data provided by selecting the unknown option in the estimation of academic ability improves the accuracy of the academic ability estimation.

[0039] The estimation unit 14 estimates the difficulty level of at least each question based on the answer information from which answer data deemed inappropriate has been excluded. Then, the estimation unit 14 estimates each user's academic ability based on the estimated difficulty level of each question and the answer information before the exclusion of answer data. The estimation process will be described in detail below.

[0040] Here, the parameter corresponding to question i in the exam is the discriminative power a. i , difficulty level b i Set the parameters for user j, and academic ability θ as the parameter for user j. j Set the discriminant value a. i This is an indicator of whether the problem accurately assesses the user's abilities. In this case, the probability of answering problem i correctly is P. i (θ j ) can be expressed as follows according to item response theory.

number

[0041] Then, the estimation unit 14 calculates the estimated academic ability θ of each user. j The value of and the observed answer data X ij Using this, the difficulty level b of each problem ii Update the following for each difficulty level b. i Regarding this, give the likelihood function for all users who answered the problem, and find the value of b that maximizes that likelihood. i To estimate academic ability θ, the Newton-Raphson method or similar methods may be used. j and difficulty level b i When the likelihood function value has almost stopped changing after repeated updates, and the amount of each parameter has been updated has become very small, the estimation unit 14 will determine academic ability θ j and difficulty level b i Confirm. In the above explanation, the above probability P in item response theory is used. i In the formula for calculating the discriminant power a, assuming a one-parameter logistic model, i Although we treated it as a fixed value, as a logistic model with two or more parameters, the discriminant power a i It may be treated as a variable. In that case, the estimation unit 14 determines the discriminant value a i , difficulty level b i and academic ability θ j We estimate this. Furthermore, for compatibility with models using the cumulative normal distribution function in item response theory, the above probability P i Regarding the formula for calculating the discriminant power a, i Alternatively, an expression multiplied by a constant D (where D is, for example, approximately 1.7) may be used.

[0042] Further specific examples of parameter estimation are described below. The above correct answer probability P i (θ j ) is preferably non-negative and has a maximum value of 1. i If we consider this to be positive, then the probability of getting the correct answer is P. i (θ j ) is a monotonically increasing function, and θ i When P is set to infinity i (θ j ) approaches 1, θ j When P is set to -∞ i (θ j ) approaches 0, so the probability of getting the correct answer P i (θ jThe range of values ​​for ) is (0,1).

[0043] User J's solution data X for problem i ij However, if the answer is 1 for a correct answer and 0 for an incorrect answer, then the answer data X ij The probability of obtaining this is expressed as follows:

number

number

[0044] For the sake of ease of numerical calculation, the estimation unit 14 may perform maximum likelihood estimation by maximizing the log-likelihood, which is obtained by taking the logarithm of the likelihood function described above, as follows.

number

[0045] Note that the functions shown in the maximum likelihood estimation described above are examples and are not limited to these examples. For example, a penalty term for the parameter may be included in the logarithmic likelihood function. Also, while the above description is point estimation for obtaining one point of the parameter to be maximized, the estimation unit 14, when the answer data X ij is given, may estimate the academic ability θ j and the difficulty level b i by distribution estimation (Bayesian estimation) for obtaining the posterior probability of each parameter.

[0046] Based on the maximum likelihood estimation described above, the estimation unit 14 estimates the academic ability θ ij and the difficulty level b j from the answer information excluding the answer data X i that has been determined to be inappropriate. When the discrimination power a i is treated as a variable, the estimation unit 14 estimates the academic ability θ j , the discrimination power a i and the difficulty level b i .

[0047] Furthermore, the estimation unit 14 may estimate the academic ability of all users including the academic ability of the user whose answer data has been excluded in the estimation of the academic ability, based on the estimated difficulty level (and discrimination power) of each question and the answer information before the exclusion of the inappropriate answer data. That is, based on the answer information excluding the answer data X ij that has been determined to be inappropriate, since the difficulty level b i (and the discrimination power a i ) is estimated with high accuracy, the estimation unit 14 uses the estimated difficulty level b i (and the discrimination power a i ) to estimate the academic ability θ ij of all users including the user who is the answerer of the answer data X i that has been determined to be inappropriate.

[0048] By using the answer information from which the answer data determined to be inappropriate has been excluded according to such an estimation flow, it is possible to accurately estimate the difficulty level of the question without being affected by the quality of the answer data. Then, by using the estimated difficulty level of the question, it is possible to estimate the academic ability of all users including the users whose answer data has been excluded once.

[0049] In the test assuming the application of the academic ability estimation system 1 of the present embodiment, one question may be repeatedly presented in a plurality of tests. In such a case, so-called leakage of the question may occur, where one question (and its answer) presented in a certain test becomes known to the examinees (users) of a later test. The answer data of the leaked question is not suitable for estimating the academic ability of the users.

[0050] In order to improve the accuracy of academic ability estimation in such a case, the determination unit 13 obtains a series of difficulty levels b of each question estimated along with multiple presentations. Then, when the series of difficulty levels of one question corresponds to a given tendency where leakage of the question is suspected, the determination unit 13 may determine the answer data of the one question as inappropriate answer data. i Specifically, the determination unit 13 obtains the question characteristics from the test information in the test management database 31. The question characteristics are the series of difficulty levels b along with multiple presentations of the difficulty level of the question. Here, the subscript t (1 to T) is the number of samples of the difficulty level of the question. The determination unit 13 calculates the correlation between the series of difficulty levels b including the latest sample T and the series of difficulty levels b before the latest sample is added. When the correlation is greater than or equal to a predetermined threshold, it is determined that no leakage of the question has occurred and the question is effective for academic ability estimation. When the correlation is less than the predetermined threshold, it is determined that there may have been leakage of the question and the question is not effective for academic ability estimation.

[0051] Specifically, the determination unit 13 obtains the question characteristics from the test information in the test management database 31. The question characteristics are the series of difficulty levels b along with multiple presentations of the difficulty level of the question. Here, the subscript t (1 to T) is the number of samples of the difficulty level of the question. The determination unit 13 calculates the correlation between the series of difficulty levels b including the latest sample T and the series of difficulty levels b before the latest sample is added. When the correlation is greater than or equal to a predetermined threshold, it is determined that no leakage of the question has occurred and the question is effective for academic ability estimation. When the correlation is less than the predetermined threshold, it is determined that there may have been leakage of the question and the question is not effective for academic ability estimation. it Here, the subscript t (1 to T) is the number of samples of the difficulty level of the question. The determination unit 13 calculates the correlation between the series of difficulty levels b including the latest sample T and the series of difficulty levels b before the latest sample is added. When the correlation is greater than or equal to a predetermined threshold, it is determined that no leakage of the question has occurred and the question is effective for academic ability estimation. When the correlation is less than the predetermined threshold, it is determined that there may have been leakage of the question and the question is not effective for academic ability estimation. it and the series of difficulty levels b before the latest sample is added, it and when the correlation is greater than or equal to a predetermined threshold, it is determined that no leakage of the question has occurred and the question is effective for academic ability estimation, and when the correlation is less than the predetermined threshold, it is determined that there may have been leakage of the question and the question is not effective for academic ability estimation. Also, the determination unit 13 compares the newly estimated difficulty level b i with the previously estimated difficulty level b iBy comparing these two factors, if the degree of discrepancy is less than a predetermined threshold, it may be determined that the problem is effective for estimating academic ability. If the degree of discrepancy is greater than or equal to the predetermined threshold, it may be determined that the problem is not effective for estimating academic ability.

[0052] Then, the determination unit 13 determines that the answer data X of the problems is not effective for estimating academic ability. ij This will be judged as inappropriate answer data.

[0053] Thus, if the difficulty level of a problem corresponds to a given trend, such as when problem leaks are suspected, the answer data for that problem is excluded from the estimation of academic ability. This makes it possible to improve the accuracy of the academic ability estimation.

[0054] The output unit 15 outputs at least the user's academic ability. Specifically, it outputs the estimated user's academic ability θ. j This is transmitted to the user terminal 20 of the user concerned. The output unit 15 outputs the estimated academic ability θ over multiple tests. j The sequence may be sent to the user terminal 20.

[0055] Furthermore, the output unit 15 outputs the estimated difficulty level b of each problem. i The output unit 15 may output the estimated difficulty level b of each problem to a database that manages problems, such as the test management database 31. i This may be stored in association with the identifier of the problem in question.

[0056] The display unit 22 displays the academic ability θ transmitted from the academic ability estimation server 10. j The system receives and presents the received academic ability θ to the user. The presentation unit 22 displays the received academic ability θ on a display means such as a display, for example. j You may display it.

[0057] Next, with reference to Figure 6, an example of the academic ability estimation method in academic ability estimation system 1 will be explained. Figure 6 is a flowchart showing an example of the processing flow of the academic ability estimation method.

[0058] In step S1, the answer data acquisition unit 12 acquires answer data X indicating whether the user's answer to each of the multiple questions is correct or incorrect. ij The system obtains answer information that includes the answer. Specifically, the answer data acquisition unit 12 obtains answer information from the answer information database 32.

[0059] In step S2, the determination unit 13 determines, based on predetermined conditions, that the answer data X is inappropriate for calculating the user's academic ability. ij The estimation unit determines that the answer data X is inappropriate in step S3. ij This information will be excluded from the answer data used to estimate academic ability.

[0060] In step S4, the estimation unit 14 determines the difficulty level b of at least each question based on the answer information from which the answer data deemed inappropriate has been excluded. i The estimation unit 14 estimates the academic ability θ of the users whose answer data was not excluded. j It may also be estimated.

[0061] In step S5, the estimation unit 14 calculates the estimated difficulty level b of each problem. i Using this, the answer data X that was excluded in step S3. ij The academic ability θ of all users, including the user involved. j This estimates the academic ability θ that is unaffected by inappropriate answer data for users whose answer data was not excluded. j The estimation result is obtained. This estimation result is the academic ability θ obtained in step S4. j This closely matches the estimated result. Furthermore, for users whose answer data was excluded, the academic ability θ based on their inappropriate answer data was calculated. j This will yield the following estimation result.

[0062] In step S6, the output unit 15 outputs at least the user's academic ability θ j It outputs the estimated user's academic ability θ. j This is sent to the user's terminal 20 and displayed.

[0063] According to the academic ability estimation system 1, academic ability estimation method, and academic ability estimation program (server program P1, terminal program P2) of this embodiment described above, the user's academic ability is estimated based on answer information from which inappropriate answer data for calculating the user's academic ability has been excluded, thus enabling the user to be provided with highly accurate and quantified academic ability information.

[0064] The present invention has been described in detail above based on its embodiments. However, the present invention is not limited to the above embodiments. The present invention can be modified in various ways without departing from its spirit.

[0065] The internal configuration of the academic ability estimation system is not limited to the above embodiment and may be designed according to any policy. For example, the devices (server and terminal) that comprise the functional units 11-15 and 21-22 are not limited to the examples of this embodiment. In this embodiment, the functional units configured in the academic ability estimation server 10 may be configured in the user terminal 20, or the functional units configured in the user terminal 20 may be configured in the academic ability estimation server 10. Furthermore, the entities (i.e., processors) that realize each of the functional units 11-15 and 21-22 are not limited to the examples of this disclosure and may differ for each functional unit.

[0066] The processing steps of a method executed by at least one processor are not limited to the examples in the above embodiments. For example, some of the steps (processes) described above may be omitted, or each step may be executed in a different order. Also, any two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to each of the above steps.

[0067] Any part or all of the functional components described herein may be implemented by program. The programs referred to herein may be distributed by non-temporarily recording them on a computer-readable recording medium, by distributing them via communication lines such as the Internet (including wireless communication), or by distributing them installed on any terminal.

[0068] Based on the above description, those skilled in the art may be able to conceive of additional effects and various modifications of the present invention, but the embodiments of the present invention are not limited to the individual embodiments described above. Various additions, modifications, and partial deletions are possible without departing from the conceptual idea and spirit of the present invention derived from the claims and their equivalents.

[0069] For example, what is described herein as a single device (or component, hereinafter the same) (including what is depicted as a single device in the drawings) may be implemented by multiple devices. Conversely, what is described herein as multiple devices (including what is depicted as multiple devices in the drawings) may be implemented by a single device. Alternatively, some or all of the means or functions included in one device (e.g., a server) may be included in another device (e.g., a user terminal). Furthermore, "system" may consist of a single device or of two or more devices (e.g., a server and a user terminal, or multiple user terminals).

[0070] Furthermore, not all matters described herein are mandatory requirements. In particular, matters described herein but not included in the claims can be considered optional additional matters.

[0071] It should be noted that the applicant is only aware of the prior art inventions described in the "Prior Art Documents" section of this specification, and that the present invention is not necessarily intended to solve the problems described in those prior art inventions. The problems that the present invention aims to solve should be determined by considering this specification as a whole. For example, if this specification describes that a certain effect is achieved by a particular configuration, it may also mean that the problem that is the inverse of that predetermined effect is solved. However, this does not necessarily mean that such a particular configuration is an essential requirement.

[0072] [Note] As can be seen from the various examples above, this disclosure includes the following aspects:

[0073] [Note 1] Equipped with at least one processor, The at least one processor, We obtain answer information that includes answer data indicating whether the user's answer to each of multiple problems is correct or incorrect. Based on predetermined conditions, the system determines that the answer data is inappropriate for calculating the user's academic ability. Based on the answer information from which the answer data deemed inappropriate has been excluded, the difficulty level of at least each question is estimated. Based on the estimated difficulty level of each problem and the answer information before the exclusion of the answer data, the academic ability of each user is estimated. Outputting at least the academic ability of each user, Academic ability estimation system.

[0074] [Note 2] The aforementioned answer information includes the time taken to answer each question. The at least one processor, For questions where the answer time is less than a predetermined amount, the answer data is determined to be the inappropriate answer data. The academic ability estimation system described in Appendix 1.

[0075] [Note 3] The aforementioned answer information includes the total answer time, which is the time taken by the user to answer a group of multiple questions. The at least one processor, The answer data for the aforementioned multiple questions, whose total answer time is less than a predetermined amount, is determined to be inappropriate answer data. The academic ability estimation system described in Appendix 1 or 2.

[0076] [Note 4] The aforementioned answer information includes answer execution time information indicating the time during and interrupted in the user's answering of a series of multiple questions. The at least one processor, If the pattern of time during and after answering, as indicated in the answering time information, matches a predetermined pattern, the answer data of the user's answers to the series of questions is determined to be the inappropriate answer data. The academic ability estimation system described in any one of the appendices 1 to 3.

[0077] [Note 5] The at least one processor, Based on the answer information from which the answer data deemed inappropriate has been excluded, the difficulty level and discriminatory power of each question are estimated. The academic ability of each user is estimated using the aforementioned difficulty level and discriminative ability of each estimated problem. The academic ability estimation system described in any one of the appendices 1 to 4.

[0078] [Note 6] The answer to the aforementioned problem is provided by selecting one option from a set of multiple choices, including an unknown option to select when the answer is unknown. The at least one processor, The answer data obtained by selecting the unknown option is determined to be the inappropriate answer data. The academic ability estimation system described in any one of the appendices 1 to 5.

[0079] [Note 7] The at least one processor, Based on the aforementioned answer information, the academic ability of each user and the difficulty level of each problem are estimated. We obtain a series of difficulty levels for each question estimated based on multiple instances of the question being asked. If the series of difficulty levels for a given problem matches a given trend, the answer data for that problem is determined to be inappropriate answer data. The academic ability estimation system described in any one of the appendices 1 to 6.

[0080] [Note 8] A method for estimating academic ability, which is performed by an academic ability estimation system having at least one processor, The steps include obtaining answer information, which includes answer data indicating whether the user's answer to each of multiple problems is correct or incorrect, and A step of determining the answer data that is inappropriate for calculating the user's academic ability based on predetermined conditions, Based on the answer information from which the answer data deemed inappropriate has been excluded, the step of estimating at least the difficulty level of each question, A step of estimating each user's academic ability based on the estimated difficulty level of each problem and the answer information before the exclusion of the answer data, The process includes at least the step of outputting the academic ability of each user, Methods for estimating academic ability, including those mentioned above.

[0081] [Note 9] The steps include obtaining answer information, which includes answer data indicating whether the user's answer to each of multiple problems is correct or incorrect, and A step of determining the answer data that is inappropriate for calculating the user's academic ability based on predetermined conditions, Based on the answer information from which the answer data deemed inappropriate has been excluded, the step of estimating at least the difficulty level of each question, A step of estimating each user's academic ability based on the estimated difficulty level of each problem and the answer information before the exclusion of the answer data, The process includes at least the step of outputting the academic ability of each user, A program that uses a computer to estimate academic ability.

[0082] According to the embodiments described in Appendices 1, 8, and 9, the appropriate difficulty level of each question is appropriately estimated based on answer information from which inappropriate answer data for calculating the user's academic ability has been excluded. Since the user's academic ability is then estimated based on the appropriate difficulty level of each question, the user can be provided with highly accurate, quantified information about their academic ability.

[0083] According to the method described in Appendix 2, given that answers that were not completed within a predetermined time are likely to be unsuitable for estimating academic ability due to reasons such as being answered randomly, answer data for questions that were completed within a predetermined time are excluded from the estimation of academic ability. This makes it possible to improve the accuracy of the estimation of academic ability.

[0084] According to the method described in Appendix 3, given that answers that were not completed within a predetermined time are likely to be unsuitable for estimating academic ability due to reasons such as being answered randomly, the answer data for a group of multiple questions with a total answer time of less than a predetermined amount is excluded from the estimation of academic ability. This makes it possible to improve the accuracy of the estimation of academic ability.

[0085] According to the embodiment described in Appendix 4, if the answer pattern shown in the user's answer time information falls under a predetermined pattern that suggests cheating, the user's answer data for a series of questions is excluded from the estimation of academic ability. This improves the accuracy of the academic ability estimation.

[0086] According to the method described in Appendix 5, by using answer information from which answer data deemed inappropriate has been excluded, the difficulty level of the questions can be estimated with high accuracy without being affected by the quality of the answer data. Furthermore, by using the estimated difficulty level of the questions, it becomes possible to estimate the academic ability of users whose answer data has been initially excluded.

[0087] According to the configuration described in Appendix 6, answer data from the unknown option is excluded from the estimation of academic ability. This makes it possible to improve the accuracy of the academic ability estimation.

[0088] According to the method described in Appendix 7, if the series of difficulty levels of a problem falls under a given trend, such as when the leakage of the problem is suspected, the answer data for that problem is excluded from the estimation of academic ability. This makes it possible to improve the accuracy of the estimation of academic ability. [Explanation of symbols]

[0089] 1...Academic ability estimation system, 10...Academic ability estimation server, 11...Exam management unit, 12...Answer data acquisition unit, 13...Judgment unit, 14...Estimation unit, 15...Output unit, 20...User terminal, 21...Exam execution unit, 22...Presentation unit, 30...Database group, 31...Test management database, 32...Answer information database, P1...Server program, P2...Client program.

Claims

1. Equipped with at least one processor, The at least one processor, We obtain answer information that includes answer data indicating whether the user's answer to each of multiple problems is correct or incorrect. Based on predetermined conditions, the system determines that the answer data is inappropriate for calculating the user's academic ability. As a first estimation step, the difficulty level of at least each question is estimated based on the answer information from which the answer data deemed inappropriate has been excluded. As a second estimation step, based on the difficulty level of each problem estimated in the first estimation step, and the answer information before the exclusion of the answer data, the academic ability of each user, including the user whose answer information includes answer data that was determined to be inappropriate, is estimated. Outputting at least the academic ability of each user, Academic ability estimation system.

2. The aforementioned answer information includes the time taken to answer each question. The at least one processor, For questions where the answer time is less than a predetermined amount, the answer data is determined to be the inappropriate answer data. The academic ability estimation system according to claim 1.

3. The aforementioned answer information includes the total answer time, which is the time taken by the user to answer a group of multiple questions. The at least one processor, The answer data for the aforementioned multiple questions, whose total answer time is less than a predetermined amount, is determined to be inappropriate answer data. The academic ability estimation system according to claim 1 or 2.

4. The aforementioned answer information includes answer execution time information indicating the time during and interrupted in the user's answering of a series of multiple questions. The at least one processor, If the pattern of time during and after answering, as indicated in the answering time information, matches a predetermined pattern, the answer data of the user's answers to the series of questions is determined to be the inappropriate answer data. The academic ability estimation system according to claim 1 or 2.

5. The at least one processor, Based on the answer information from which the answer data deemed inappropriate has been excluded, the difficulty level of each question and the discriminative power, which is an indicator of whether the question can accurately assess the user's ability, are estimated. The academic ability of each user is estimated using the aforementioned difficulty level and discriminative ability of each estimated problem. The academic ability estimation system according to claim 1 or 2.

6. The answer to the aforementioned problem is provided by selecting one option from a set of multiple choices, including an unknown option to select when the answer is unknown. The at least one processor, The answer data obtained by selecting the unknown option is determined to be the inappropriate answer data. The academic ability estimation system according to claim 1 or 2.

7. The at least one processor, Based on the aforementioned answer information, the academic ability of each user and the difficulty level of each problem are estimated. We obtained a series showing the estimated progression of difficulty levels for each question based on multiple instances of the question being asked. If the series of difficulty levels for a given problem corresponds to a given trend regarding a decrease in correlation or an increase in the degree of deviation within that series, the answer data for that problem is determined to be inappropriate answer data. The academic ability estimation system according to claim 1 or 2.

8. A method for estimating academic ability, which is performed by an academic ability estimation system having at least one processor, The steps include obtaining answer information, which includes answer data indicating whether the user's answer to each of multiple problems is correct or incorrect, and A step of determining the answer data that is inappropriate for calculating the user's academic ability based on predetermined conditions, A first estimation step of estimating the difficulty level of at least each question based on the answer information from which the answer data deemed inappropriate has been excluded, A second estimation step in which, based on the difficulty level of each problem estimated in the first estimation step and the answer information before the exclusion of the answer data, estimates the academic ability of each user, including the user whose answer information includes answer data that has been determined to be inappropriate. The process includes at least the step of outputting the academic ability of each user, Methods for estimating academic ability, including those mentioned above.

9. The steps include obtaining answer information, which includes answer data indicating whether the user's answer to each of multiple problems is correct or incorrect, and A step of determining the answer data that is inappropriate for calculating the user's academic ability based on predetermined conditions, A first estimation step of estimating the difficulty level of at least each question based on the answer information from which the answer data deemed inappropriate has been excluded, A second estimation step in which, based on the difficulty level of each problem estimated in the first estimation step and the answer information before the exclusion of the answer data, estimates the academic ability of each user, including the user whose answer information includes answer data that has been determined to be inappropriate. The process includes at least the step of outputting the academic ability of each user, A program that uses a computer to estimate academic ability.

Images