Abnormal Peer Review Scoring Detection and Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting abnormal scoring in peer reviews are inadequate, particularly in multidimensional scoring data with unknown distributions, and fail to effectively correct anomalies, leading to fairness and objectivity issues.
Innovation Solution
A method and device that acquire and clean scoring data, perform one-way and two-way anomaly detection, and use information entropy to correct abnormal scores, incorporating data normalization and index structures like Kd-trees for efficient detection and correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical methods (three times standard deviation detection, Grubbs-based detection, t-based detection) are used to detect abnormal scoring, then abnormal data can be identified, but the method requires pre-assumed data distribution forms and parameters which are not available in reality
Solution Approach 1:
The patent replaces statistical hypothesis testing methods (which rely on assumed data distributions) with distance-based anomaly detection methods. Specifically, it calculates the distance between data points and their k-nearest neighbors using distance functions, and identifies anomalies based on whether this distance exceeds a threshold. This substitution eliminates the need for pre-assumed distribution forms while maintaining detection capability.
Solution Approach 2:
The patent changes the detection parameter from statistical parameters (mean, standard deviation, p-values) to geometric parameters (distance, neighborhood density). By transforming the anomaly detection problem into a spatial relationship problem, it achieves distribution-free detection while adapting to the actual data structure through k-nearest neighbor calculations.
2Adaptability or versatility
If distance-based anomaly detection is used, then abnormal data can be detected without assuming data distribution, but the method requires calculating distances between all data objects which increases computational complexity
Solution Approach 1:
The patent performs preliminary action by pre-calculating and storing the k-nearest neighbor relationships for each data point before anomaly detection. This preprocessing step creates a ready-to-use spatial index that avoids the need to compute all pairwise distances during the actual anomaly detection phase, significantly reducing real-time computational complexity.
Solution Approach 2:
The patent uses the k-nearest neighbor approach to create a simplified representation of the data space structure. Instead of working with all data points simultaneously, it copies and utilizes only the local neighborhood information (k nearest neighbors) for each point, reducing the computational burden while preserving the essential spatial relationships needed for anomaly detection.
3Measurement precision
If density anomaly detection using clustering algorithms is used, then objects with sparse density can be found, but the method requires complex clustering algorithms and parameter tuning
Solution Approach 1:
The patent extracts the essential idea of density-based anomaly detection (finding sparse regions) while removing the complex clustering algorithm component. It uses a simplified k-nearest neighbor distance threshold approach that inherently identifies sparse density regions without requiring clustering algorithms, parameter tuning, or complex computational geometry operations.
Solution Approach 2:
Instead of using complex algorithms to cluster data and then identify anomalies within clusters, the patent inverts the approach by directly calculating local density through k-nearest neighbor distances and immediately identifying anomalies based on this local density measure. This inversion simplifies the process by eliminating the intermediate clustering step.
4Reliability
If peer review scoring is used to assess employee performance, then managers can understand employee ability and status, but reviewers may engage in fraudulent practices such as deliberately depressing scores or huddling together for common profits
Solution Approach 1:
The patent implements feedback mechanisms that provide real-time monitoring and detection of abnormal scoring patterns. By continuously analyzing scoring data for anomalies (such as unusually low scores, consistent patterns suggesting collusion, or deviations from expected distributions), the system generates feedback signals that can alert managers to potential fraudulent behavior and trigger investigative procedures.
Solution Approach 2:
The patent introduces an intermediary anomaly detection system that acts as a mediator between the peer review process and the final scoring outcomes. This intermediary layer analyzes scoring data for abnormal patterns and provides corrected or adjusted scores, thereby filtering out fraudulent practices while preserving the overall peer review mechanism's functionality.
Data Source
AI summary
The present application disclose a method and a device for detecting and correcting abnormal scoring of peer reviews, which includes: converting collected scoring data into a two-dimensional matrix and preprocessing the data; determining the anomaly of the processed structured data with a one-way anomaly detection method, a consistency check method and a two-way anomaly detection method, and classifying the detected abnormal data into an abnormal data set; repairing the abnormal data for the abnormal data set with an information entropy correction method; generating an ability evaluation report according to the abnormal data set, performing weighed averaging on the corrected scoring data according to the scoring weights of reviewers in the ability evaluation report to obtain a final scoring result, and generating an abnormal scoring correction report. The present application can effectively detect the abnormal phenomenon of peer reviews in the performance appraisal of enterprise personnel.