Embedding Space Mapping for Unsupervised Neologism Normalization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing neologism detection and normalization methods rely on expensive supervised machine learning models and exclusion lists, which are inefficient for dynamically evolving social media content.

Innovation Solution

An unsupervised machine learning model using embedding space mapping for detecting and normalizing neologisms by analyzing contextual similarities and implementing lattice decoding to identify and score canonical candidates.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If supervised machine learning models are used for neologism detection, then detection accuracy is improved, but development and update costs increase

Engineering Contradiction:

Improveneologism detection accuracyVSAvoidmodel development and update cost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The system performs self-training by automatically identifying candidate neologisms from social media content, generating synthetic training data, and updating its own language model without requiring external supervised resources. This self-service mechanism resolves the contradiction by eliminating the need for expensive manual annotation while maintaining detection capability through autonomous learning from unlabeled data.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system dynamically adjusts language model parameters and thresholds based on observed neologism patterns and frequency metrics from social media streams. By changing operational parameters adaptively rather than requiring complete model retraining, the system maintains accuracy while reducing update costs and complexity.

Inventive Principle:

Principle #35Parameter changes

2Quantity of substance

If traditional dictionary extension processes are used, then lexical coverage is improved, but adaptability to rapid social media evolution deteriorates

Engineering Contradiction:

Improvelexical coverageVSAvoidadaptability to social media evolution

Core Design Contradiction:

Quantity of substanceVSAdaptability or versatility

Solution Approach 1:

The system implements dynamic neologism detection that continuously adapts to emerging language patterns in social media by monitoring content streams in real-time. Unlike static dictionary updates, the system automatically identifies and incorporates new terms as they emerge, maintaining both lexical coverage and adaptability through continuous dynamic adjustment of its vocabulary.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system performs preliminary identification and validation of candidate neologisms from social media content before incorporating them into the normalized output. This preliminary action allows the system to proactively capture emerging terms before they become widespread, improving both lexical coverage and timeliness of adaptation.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If comprehensive neologism tracking is implemented, then detection capability is improved, but computational resources increase

Engineering Contradiction:

Improveneologism detection capabilityVSAvoidcomputational resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system focuses computational resources on identifying only the most promising candidate neologisms that meet specific frequency and novelty thresholds, rather than comprehensively analyzing every token in social media streams. This partial action approach maintains detection capability for significant neologisms while reducing overall computational burden by filtering out common or clearly non-neologism content.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system segments the neologism detection process into distinct stages: candidate identification, frequency analysis, similarity checking against existing lexica, and validation. By segmenting the workflow, computational resources are distributed across specialized sub-tasks rather than requiring exhaustive simultaneous processing, improving efficiency while maintaining detection precision.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12505293B2Systems and methods for unsupervised neologism normalization of electronic content using embedding space mapping

Publication Date: 2025.12.23 YAHOO ASSETS LLC

AI summary

Systems and methods are disclosed for utilizing a comment moderation bot for detecting and normalizing neologisms in social media. One method comprises transmitting, by a neologism normalization system, a comment moderation bot for detecting neologisms on an online platform maintained by one or more publisher systems. The comment moderation bot may aggregate data related to user comments and transmit the aggregated data to the neologism normalization system for further processing. The neologism normalization system implements unsupervised machine learning models for detecting neologisms in the aggregated data through tokenization and filtering; and normalizing the neologisms through similarity analysis and lattice decoding.