Game Text Translation Pipeline With OCR and Model Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing real-time translation methods for games using large language models (LLMs) face challenges such as high costs, limited input data length, and difficulty in maintaining context and memory, especially when using retrieval-augmented generation (RAG) or fine-tuning, which are impractical for the vast amount of game text and numerous games.
Innovation Solution
A method integrating computer vision to extract features from video frames, using an association process to match machine learning models for translation, incorporating optical character recognition (OCR) and retrieval-augmented generation (RAG) to preprocess and translate game text in real-time, and rendering translated text on a display device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If LLM and RAG service are frequently called for real-time translation, then translation responsiveness is improved, but calling cost rapidly increases
Solution Approach 1:
The system performs preliminary actions by extracting features from video frames and pre-processing text before translation. The association process pre-matches features with appropriate translation models, so when translation is needed, the system can quickly proceed without performing expensive LLM calls for every translation task, thus reducing calling costs while maintaining responsiveness
Solution Approach 2:
The patent introduces an intermediary association process that sits between feature extraction and translation model selection. This intermediary layer efficiently matches extracted features with appropriate translation models, acting as a mediator that prevents direct, expensive LLM calls for every translation task, thereby reducing overall calling costs while maintaining fast translation response
2Reliability
If input data length is increased to maintain context memory, then translation accuracy is improved, but LLM input data length limit is exceeded
Solution Approach 1:
The system segments the translation process into multiple components: feature extraction, association process, text pre-processing, and translation model selection. By dividing the overall task into these segments, the system can manage context information in smaller, more manageable portions that fit within LLM input limits while maintaining overall translation accuracy through the coordinated action of all segments
Solution Approach 2:
The patent extracts essential information from the game video frames using computer vision and feature extraction. By taking out only the relevant visual features and extracting key textual information through OCR and pre-processing, the system reduces the input data length to within LLM limits while preserving the necessary context for accurate translation
3Reliability
If RAG architecture is used to improve translation accuracy, then translation quality is improved, but calling cost increases
Solution Approach 1:
The system performs preliminary feature extraction and text pre-processing before translation. By preparing the input data in advance and using the association process to select appropriate models, the system reduces the need for expensive RAG and LLM calls, thereby maintaining translation quality while reducing overall calling costs
Solution Approach 2:
The patent employs lighter, more efficient translation models that can be quickly selected and executed based on feature association. These cheaper, shorter-lived model instances replace the need for expensive, persistent RAG architecture, providing sufficient translation quality at lower computational cost
4Reliability
If fine-tuning is used to improve translation accuracy, then translation quality is improved, but it becomes impractical for vast game text and numerous games
Solution Approach 1:
The system segments the translation approach into a general association process that works across all games, combined with game-specific feature extraction and model selection. This segmentation allows the system to handle vast game text and numerous games without requiring fine-tuning of the entire system for each game, thereby reducing overall system complexity while maintaining translation quality
Solution Approach 2:
The patent creates a universal association process that can handle multiple games and translation tasks through a single unified system. This universal approach eliminates the need for separate fine-tuning processes for each game, making the system scalable across vast game text and numerous titles while maintaining consistent translation quality
Data Source
AI summary
A real time translation method for a game includes extracting features from video frames using computer vision and a database, performing an association process to find a machine learning model best matching the features for translation, obtaining texts in the game through optical character recognition (OCR), preprocessing the texts, translating the texts using the machine learning model to generate translated texts, and rendering the translated texts to images of the video frames for displaying the images with the translated texts on a display device.