Subtitle Generation Using Background Text Correction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing subtitle generation technologies, particularly those using speech recognition, often mistranslate domain-specific vocabulary, mis-split, or mis-merge words, leading to inaccuracies in subtitle translation, especially when dealing with specific terminology.

Innovation Solution

The method involves determining similar words from the video background and using them to update subtitles, calculating a sentence generation probability, and combining semantic information and context to generate new subtitles, which are then used to refine speech recognition models.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If speech recognition technology is used to generate subtitles automatically, then productivity is improved by removing human interaction, but manufacturing precision deteriorates due to mistranslation of domain-specific vocabulary and word errors

Engineering Contradiction:

Improvesubtitle generation efficiencyVSAvoidsubtitle translation accuracy

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The system extracts background text from video frames, compares it with generated subtitles, and uses the comparison results to refine and correct subtitle translations. This feedback loop continuously improves accuracy by leveraging contextual information from the visual background.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

Background text extracted from video frames serves as an intermediary element that mediates between the audio input and subtitle generation process. This intermediary provides contextual clues and domain-specific vocabulary that help correct speech recognition errors.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If speech recognition technology processes audio directly, then device complexity is reduced, but measurement precision deteriorates due to mis-splitting or mis-merging of words

Engineering Contradiction:

Improvesystem structure simplicityVSAvoidword boundary detection accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The system performs additional processing beyond standard speech recognition by extracting and analyzing background text. This partial additional action focuses specifically on contextual verification and correction, improving word boundary detection without completely redesigning the speech recognition system.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11646030B2Subtitle generation using background information

Publication Date: 2023.05.09 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A video is received. One or more subtitles are determined for the video. Whether a word found in a background of the video is similar to a word found in the one or more subtitles is determined. Responsive to determining the word found in the background of the video is similar to the word found in the one or more subtitles, one or more updated subtitles are generated. The one or more updated subtitles include the word found in the background of the video and remove the word found in the one or more subtitles that is similar. A metric for the one or more updated subtitles is calculated. Whether the metric is larger than a threshold is determined. Responsive to determining the metric is larger than the threshold, the video is updated to include the one or more updated subtitles.