AR Sign Language Translation via Classifier Handshape Recognition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Sign languages, such as American Sign Language (ASL), incorporate rich gesture-based descriptions of verbs of motion through classifier constructions, which are difficult to accurately translate into spoken languages as some information about path, position, and manner of movement is lost.

Innovation Solution

A method and system that utilize an augmented reality device to observe classifier handshapes, analyze them using object recognition algorithms, convert the contextual meaning into a graphical representation, and display it alongside the observed handshape, effectively translating the detailed information in classifier constructions into a graphical format.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If classifier constructions are translated into spoken languages, then communication with non-signers is enabled, but detailed information about path, position, and manner of movement is lost

Engineering Contradiction:
Improvecommunication accessibilityVSAvoidpath and position information
Core Design Contradiction:
Adaptability or versatilityVSLoss of information

Solution Approach 1:

The patent introduces graphical representations as an intermediary medium between sign language classifier constructions and spoken language. These graphical representations serve as a visual bridge that preserves the spatial and motion information from sign language while making it accessible to non-signers, thereby resolving the contradiction between communication accessibility and information preservation.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent transitions the translation from a one-dimensional spoken language output to a two-dimensional or three-dimensional graphical representation. This dimensional change allows the preservation of spatial path, position, and movement information that would otherwise be lost in traditional spoken language translation, enabling non-signers to perceive the full semantic content.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Ease of operation

If traditional translation methods are used, then spoken language output is produced, but the rich gesture-based description of verbs of motion cannot be accurately conveyed

Engineering Contradiction:
Improvetranslation simplicityVSAvoidtranslation accuracy
Core Design Contradiction:
Ease of operationVSMeasurement precision

Solution Approach 1:

The patent creates visual copies or representations of the classifier constructions directly from the sign language gestures. By copying the essential visual and spatial characteristics of the handshapes and movements into graphical forms, the system maintains high translation accuracy while keeping the process relatively simple, as the graphical output directly mirrors the input gesture semantics.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent replaces traditional mechanical or algorithmic translation approaches with a visual representation system. Instead of converting sign language into spoken words through complex linguistic processing, the system directly generates graphical representations that preserve the motion and spatial information, simplifying the translation mechanism while improving accuracy.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12307764B2Augmented reality translation of sign language classifier constructions
Publication Date: 2025.05.20 INTERNATIONAL BUSINESS MACHINE CORPORATION
  • US12307764B2 patent drawing
  • US12307764B2 patent drawing
  • US12307764B2 patent drawing

AI summary

A method, computer system, and a computer program product for translating a classifier construction into a graphical representation is provided. The present invention may include observing a classifier handshape by an augmented reality device. The present invention may include analyzing the observed classifier handshape according to an object recognition algorithm to determine a contextual meaning of the classifier handshape. The present invention may include converting the contextual meaning of the observed classifier handshape into a graphical representation. The present invention may include displaying the graphical representation alongside the observed classifier handshape on the augmented reality device.