Sign Language Morpheme Segmentation With Frame Position Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sign language translation models face limitations due to insufficient data resources, leading to inaccurate morpheme prediction and translation, as they struggle to determine the frames where morphemes are generated in sign language videos.
Innovation Solution
A method involving a sign language analysis system that recognizes key points from a sign language video, segments morphemes, predicts their positions, and augments training data by combining morphemes, using a skeleton model and AI model to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a learning-based model is used to translate sign languages into language, then translation capability is improved, but data requirements increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-segmenting sign language videos into morphemes and pre-predicting their positions before translation. This preprocessing creates structured training data that enables the translation model to learn more efficiently, reducing the overall data requirements while maintaining translation capability.
Solution Approach 2:
The patent segments sign language videos into discrete morphemes with predicted position information. This segmentation transforms continuous video data into structured units that can be more efficiently learned by the translation model, reducing the amount of raw data needed while improving adaptability.
2Ease of operation
If morpheme prediction is performed without position information, then processing simplicity is maintained, but translation accuracy deteriorates
Solution Approach 1:
The patent adds a temporal dimension to morpheme prediction by predicting position information (frame indices) alongside morpheme identification. This transforms the prediction task from a single-dimensional classification problem into a multi-dimensional problem that includes both what morpheme appears and when it appears, thereby improving translation accuracy while maintaining reasonable processing complexity.
3Reliability
If more sign language data is collected for training, then model performance is improved, but resource requirements and time cost increase
Solution Approach 1:
The patent implements self-service through automated morpheme segmentation and position prediction that generates training data without requiring extensive manual annotation. The system uses AI models to automatically process and structure sign language videos, reducing both the time and resources needed for data preparation while improving model performance.
4Measurement precision
If manual data construction is performed, then data quality is improved, but construction cost and time increase
Solution Approach 1:
The patent replaces manual mechanical annotation processes with automated AI-based morpheme segmentation and position prediction systems. This substitution maintains high data quality through algorithmic precision while dramatically reducing construction costs and time requirements by eliminating manual labor.
Data Source
AI summary
Provided are a method for segmenting sign language into morphemes, a method for predicting morpheme positions, and a method for augmenting data. A system for analyzing sign language according to an embodiment of the present invention comprises: a recognition unit which recognizes key points of a speaker from a sign language video; and a prediction unit which inputs the recognized key points into an artificial intelligence model, segments the sign language into morphemes, and predicts position information of the segmented morphemes. Accordingly, by recognizing the morphemes of the sign language video frame by frame on the basis of a skeletal model and thereby segmenting the sign language into morphemes and predicting morpheme positions, it is possible to lay the foundations for accurate sign language translation.