Marker Evaluation System Consistency and Time Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In description type writing tests, it is challenging to mark answers mechanically due to differences in sentence expression and character count among examinees, making it difficult for a single marker to accurately evaluate responses, often requiring multiple markers to ensure fairness and accuracy.
Innovation Solution
A marker evaluation system that compares marking results and times across multiple markers for the same answer, using a server device to derive comparison results and evaluate each marker based on consistency and efficiency, ensuring fair and accurate scoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple markers mark the same answer, then marking accuracy is improved, but marking time increases
Solution Approach 1:
The system performs preliminary actions by automatically deriving comparison results and marking times before full evaluation occurs. The server device pre-processes marking data, compares results across multiple markers, and identifies potential issues before final scoring decisions are made, reducing the overall time required for accurate multi-marker evaluation
2Reliability
If multiple markers are used for description type tests, then evaluation fairness is improved, but system complexity increases
Solution Approach 1:
The server device performs multiple functions within a single system: it stores marking results, derives comparison results, calculates marking times, evaluates marker performance, and manages the overall marking process. This multi-functional approach maintains evaluation fairness through multiple markers while avoiding the need for separate complex systems for each function
Solution Approach 2:
The server device acts as an intermediary between multiple markers and the evaluation system. It receives marking results from various markers, standardizes the data format, performs automated comparisons, and facilitates fair evaluation by eliminating direct marker-to-marker conflicts, thereby managing complexity through centralized mediation
Data Source
AI summary
A marker evaluation system includes: a memory configured to store a program; and at least one processor configured to execute a program stored in the memory, in which the processor is configured to derive at least one of a comparison result obtained by comparing a decided marking result decided based on marking results by a plurality of markers for an identical answer of an identical answerer with a marking result by each of the plurality of markers for the identical answer, and a marking time required for each of the plurality of markers to mark the identical answer, and evaluate each of the plurality of markers based on the at least one of the comparison result and the marking time that has been derived.