The present invention facilitates communication between sign language users and machines by translating sign language and text using AI models,
deep learning computer vision, and word embeddings. Users interact via sign language, captured and processed through
deep learning and NLP modules. The
system converts sign language videos into text, constructs coherent sentences, and generates contextually appropriate responses using a Retrieve and Generate (RAG) model. Responses are translated back into sign language videos, spelling out words not found in the dictionary. If requested, a
human agent can respond. Key features include high-accuracy recognition, context-aware
response generation, dynamic vocabulary updates, and optional
human interaction. The method ensures efficient
processing with LLM, embedding techniques, and
deep learning, optimizing translation accuracy and user experience. The
system adapts to multiple languages and dialects by training on specific sign languages, making it applicable globally.