Automated Fact Checking System with Hybrid Verification Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The proliferation of user-generated content on social media platforms without editorial oversight leads to a significant challenge in verifying the accuracy of information, as existing methods struggle with complex situations and are either unscalable or labor-intensive, resulting in a high likelihood of false or exaggerated content being shared.
Innovation Solution
A method combining automated and assisted fact-checking techniques, using natural language processing and machine learning classifiers to determine the accuracy of information, which includes both automated scoring for easier verifications and manual scoring by human experts for complex claims, generating a combined verification score that assesses the confidence in the accuracy of the input data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If expert moderators manually verify content, then verification accuracy is improved, but scalability and cost deteriorate
Solution Approach 1:
The verification process is segmented into two distinct pathways: automated verification for straightforward claims and manual expert verification for complex or sensitive claims. This segmentation allows the system to maintain high accuracy for manual reviews while achieving scalability through automated processing of the majority of content.
Solution Approach 2:
An automated verification system acts as an intermediary between content publishers and human expert moderators. This intermediary layer filters and pre-verifies content using AI and machine learning techniques, reducing the volume of content requiring manual review and enabling scalable operation while maintaining verification quality.
2Productivity
If automated verification systems are used, then scalability is improved, but handling of complex situations deteriorates
Solution Approach 1:
The verification system dynamically adapts its approach based on the complexity and nature of each claim. Simple, verifiable claims are processed automatically at scale, while the system identifies and routes complex, nuanced, or high-stakes claims to human experts, optimizing both scalability and reliability according to situational requirements.
Solution Approach 2:
Different verification quality levels are applied locally to different types of content. High-volume, low-complexity content receives automated verification, while low-volume, high-complexity content receives manual expert verification. This localized quality approach ensures appropriate handling of complex situations while maintaining overall system scalability.
3Reliability
If manual expert verification is used, then handling of complex situations is improved, but cost and labor intensity deteriorate
Solution Approach 1:
Instead of applying manual expert verification to all content (excessive action), the system applies it only partially to the specific subset of claims that require human judgment. This partial application of manual verification reduces cost and labor intensity while still ensuring complex situations are handled appropriately by experts.
Data Source
AI summary
The present invention relates to a method and system for verification scoring and automated fact checking. More particularly, the present invention relates to a combination of automated and assisted fact checking techniques to provide a verification score. According to a first aspect, there is a method of verifying input data, comprising the steps of: receiving one or more items of input data; determining one or more pieces of information to be verified from the or each item of input data; determining which of the one or more pieces of information are to be verified automatically and which of the one or more pieces of information require manual verification; determining an automated score indicative of the accuracy of the at least one piece of information which is to be verified automatically; and generating a combined verification score which gives a measure of confidence of the accuracy of the information which forms the or each item of input data.