Offensive Content Analysis System for Dynamic Communication Monitoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing internet and messaging security features are inadequate in protecting individuals, particularly children, from accessing inappropriate or offensive material due to the dynamic nature of online content, as filtering websites does not cover all potential sources of offensive content.

Innovation Solution

A method that detects offensive behavior in communications by determining a current offensive behavior metric and historical pattern, comparing it to a threshold level set by an enforcement entity, and notifying the entity if the behavior exceeds this threshold, using an offensive content analysis system that monitors communications and takes appropriate actions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If website filtering is used to block offensive content, then protection against known offensive material is improved, but the system cannot protect against ever-changing content on the internet

Engineering Contradiction:

Improveprotection effectivenessVSAvoidability to handle dynamic content

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary analysis of communications by evaluating current offensive behavior metrics and comparing them against historical patterns before allowing content to reach users. This proactive approach enables the system to block offensive content based on learned patterns rather than relying solely on pre-defined filters, thus addressing both reliability and adaptability requirements

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms by continuously monitoring communication content, comparing current offensive behavior metrics against historical patterns, and adjusting its protection strategies accordingly. This closed-loop approach allows the system to adapt to ever-changing content while maintaining reliable protection through continuous learning and pattern recognition

Inventive Principle:

Principle #23Feedback

2Measurement precision

If comprehensive monitoring of all communications is implemented to detect offensive behavior, then detection accuracy is improved, but system complexity increases

Engineering Contradiction:

Improveoffensive behavior detection accuracyVSAvoidmonitoring system complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system applies partial monitoring by focusing computational resources on evaluating specific offensive behavior metrics that are most relevant to user safety. Rather than analyzing every aspect of each communication in equal detail, the system selectively measures key indicators of offensive behavior, achieving high detection accuracy while managing system complexity through prioritized analysis

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS8904547B2Notification upon exposure to offensive behavioral patterns in collaboration

Publication Date: 2014.12.02 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A system and method for protecting a user from offensive behavior in communications and notifying the user and/or an enforcement entity of the offensive behavior. The offensive content analysis system monitors communications between users for offensive behavior. The offensive content analysis system may measure the level of current offense in the communication and determine a historical offensive behavior pattern for the user. The offensive content analysis system may then determine if the offensive behavior, both current and historical, rises to a threshold behavior level. The offensive content analysis system may take notification action if the offensive behavior meets the threshold level.