Automated Examination Item Similarity Detection via Vector N-grams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies lack an automated and objective mechanism for identifying and removing similar examination items, known as 'enemy items,' from digitized examination forms, making the process labor-intensive and subjective, especially as assessment evolves to computer-adaptive administrations.
Innovation Solution
A system and method using natural language processing (NLP) and machine learning to evaluate linguistic, lexical, syntactic, and semantic similarity between examination items by converting n-grams into vectors and calculating congruence values, with a predictive model to determine and remove potential enemy item pairs based on threshold values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual comparison of examination items is performed to identify enemy items, then identification accuracy can be maintained through expert judgment, but the process becomes labor-intensive and time-consuming
Solution Approach 1:
The patent replaces manual mechanical comparison of examination items by human experts with an automated computational system that uses natural language processing, vector representation, and cosine similarity calculations to objectively measure item similarity, thereby eliminating labor-intensive manual work while maintaining identification accuracy
Solution Approach 2:
The system enables examination items to evaluate their own similarity through automated computational processes, where the computer algorithm independently performs the comparison and identification of enemy items without requiring continuous human intervention, making the process self-sufficient and efficient
2Reliability
If comprehensive pairwise comparison of all examination items is performed to ensure complete identification of enemy items, then identification completeness is improved, but the complexity and computational burden increase significantly
Solution Approach 1:
The patent replaces complex manual pairwise comparison processes with an automated computational system that efficiently handles large numbers of item comparisons through computer algorithms, making the comprehensive comparison of all item pairs feasible and manageable without overwhelming complexity
Solution Approach 2:
The system transforms the complexity of item comparison into a standardized mathematical parameter calculation (cosine similarity between vectors), converting the subjective and complex task of determining item similarity into an objective numerical computation that is easier to manage and scale
3Productivity
If automated methods are used to identify enemy items, then productivity and efficiency are improved, but the subjectivity and nuance of human expert judgment may be lost
Solution Approach 1:
The patent substitutes human expert judgment with an automated system based on natural language processing and vector representation, where the computational method objectively measures linguistic, lexical, syntactic, and semantic similarity, providing consistent and reproducible results without human subjectivity
Solution Approach 2:
The system introduces vector representations and cosine similarity calculations as intermediary mathematical constructs that bridge the gap between automated computation and the nuanced concept of item similarity, allowing the computer to capture subtle semantic relationships that might be missed in purely keyword-based approaches
4Stability of the object's composition
If fixed-length examination forms are used, then the structure and organization of assessments are maintained, but the ability to present a broad range of non-cumulative items is limited
Solution Approach 1:
The patent extracts and removes enemy items (redundant or highly similar items) from the examination pool through automated identification and elimination, thereby enabling the construction of examination forms with a broader range of non-cumulative items while maintaining the desired form length and structure
Data Source
AI summary
A computer implemented method for identifying and removing similar examination items in a digital examination includes obtaining, with an examination evaluation server, a digital examination with a plurality of examination items, tokenizing a first examination item of the plurality of examination items into a first set of n-grams, tokenizing a second examination item of the plurality of examination items into a second set of n-grams, generating a first item vector from the first set of n-grams, generating a second item vector from the second set of n-grams, generating multiple item congruence values for corresponding similarity dimensions by comparing the first item vector with the second item vector, and identifying that the first and second examination items are a potential enemy item pair if one or more of the item congruence values exceed corresponding threshold values.