Online Test Server for Evaluating Idea Creation Skills
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for evaluating idea creation skills in online tests are inadequate, as they rely on comparing answers with predefined model answers, which fails to assess innovative talent effectively, and peer evaluation without clear assessment skills is unreliable.
Innovation Solution
An online test method where examinees select situation settings from 5W1H elements and describe ideas within a time limit, with mathematical analysis and peer evaluation, allowing for objective scoring and evaluator assessment through repeated evaluations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional model answer comparison is used for scoring, then scoring procedure is simple and standardized, but it cannot evaluate idea creation skills and innovative talent
Solution Approach 1:
The patent introduces an intermediary evaluation system where AI models and peer evaluators act as mediators between examinee answers and final scores. The AI model analyzes answer characteristics (originality, feasibility, completeness) and peer evaluators provide additional assessments, combining multiple intermediary judgments to achieve accurate evaluation of idea creation skills without requiring complex manual grading of each response.
Solution Approach 2:
The patent replaces traditional mechanical comparison with model answers using AI-based automated evaluation systems. The AI model automatically analyzes examinee responses against multiple evaluation dimensions (originality, feasibility, completeness) and generates scores, substituting manual mechanical grading with intelligent automated systems that can handle complex creative assessment tasks.
2Reliability
If peer evaluation is used without clear assessment skills, then evaluation process is simple, but assessment reliability is poor
Solution Approach 1:
The patent implements feedback mechanisms where peer evaluators receive guidance on evaluation criteria (originality, feasibility, completeness) and their assessments are compared against AI-generated evaluations. This feedback loop helps improve evaluator reliability over time while maintaining system simplicity through standardized feedback protocols and iterative learning processes.
Solution Approach 2:
The patent creates a universal evaluation framework that serves multiple functions: AI models provide baseline assessments, peer evaluators offer diverse perspectives, and the system adapts to different idea creation contexts. This multi-functional approach enhances reliability by combining multiple evaluation sources while maintaining operational simplicity through integrated processing.
3Measurement precision
If comprehensive idea evaluation is implemented, then evaluation accuracy improves, but processing time and system complexity increase
Solution Approach 1:
The patent applies preliminary action by pre-training AI models with extensive evaluation criteria and guidelines before actual assessment. Evaluation dimensions (originality, feasibility, completeness) are pre-defined and configured in advance, allowing the system to quickly process examinee responses without time-consuming analysis during the actual evaluation phase, thus maintaining high accuracy while reducing processing time.
Solution Approach 2:
The patent segments the comprehensive evaluation process into distinct analytical components: originality assessment, feasibility analysis, and completeness checking. Each segment is handled by specialized AI functions or evaluation modules that process specific aspects independently, then combine results to form the overall evaluation. This segmentation enables parallel processing and reduces total evaluation time while maintaining comprehensive accuracy.
Data Source
AI summary
Provided are a method of on online test and an online test server for simply evaluating idea creation skills. A test is conducted online, in which a number of examinees are prompted to repeat a task of choosing a situation setting involving 5W1H from a plurality of options describing ideas corresponding to the situation setting as many times as possible within a time limit, and total scores are calculated for answers of the examinees by assigning weight according to predetermined criteria, thereby evaluating an innovation ability, such as an ability to create a lot of valuable ideas, an ability to create a variety of valuable ideas, or an ability to create rare valuable ideas.