Image Translation Using Non-Textual Context Disambiguation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional OCR and machine language translation systems fail to accurately translate text from images due to lack of consideration for non-textual context information, leading to incorrect translations as words or phrases can have multiple meanings based on context.
Innovation Solution
A computer-implemented technique that receives an image from a mobile device, performs OCR to extract text, identifies non-textual context information such as objects, colors, and shapes within the image, and adjusts the baseline translation based on this information to provide a more accurate translation tailored to the image's context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional OCR and machine language translation systems are used to translate text from images, then the translation process is simple and fast, but the translation accuracy is low due to lack of context information
Solution Approach 1:
The system segments the image into multiple regions of interest using object detection, identifying different contextual elements (objects, scenes, entities) separately. This segmentation allows the translation system to process and weigh different context types independently, improving translation accuracy without overwhelming system complexity
Solution Approach 2:
The system introduces an intermediary context analysis layer between OCR text extraction and machine translation. This intermediary layer extracts non-textual context information (objects, scenes, entities) and uses it to disambiguate translations, acting as a mediator that bridges the gap between raw text and accurate translation
2Measurement precision
If non-textual context information is extracted and used to adjust translations, then translation accuracy improves, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary object detection and context extraction before the translation process. By pre-identifying regions of interest and their contextual meanings, the system prepares disambiguation information in advance, allowing the translation step to be faster and more accurate without redundant processing
Solution Approach 2:
The system applies different levels of context analysis to different regions of the image based on their importance. Not all objects receive equal processing depth - the system prioritizes context extraction for objects most likely to influence translation accuracy, optimizing the balance between processing time and translation quality
Data Source
AI summary
A computer-implemented technique can include receiving, at a server from a mobile computing device, the server having one or more processors, an image including a text. The technique can include obtaining, at the server, optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image. The technique can include identifying, at the server, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself and (ii) being indicative of a context of the image. The technique can include based on the non-textual context information, obtaining, at the server, a translation of the OCR text to a target language to obtain a translated OCR text. The technique can include outputting, from the server to the mobile computing device, the translated OCR text.