Text Refinement Network for Image Segmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional image segmentation tools are ineffective in accurately identifying and segmenting text in images due to their unawareness of unique visual properties of text, often mistaking curves and decorations as text or missing portions with similar color and texture.

Innovation Solution

A neural network-based text segmentation system that performs initial segmentation and refines it by identifying key features of text using a query-key attention mechanism, combining these features with image features to improve classification accuracy at the pixel level.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional image segmentation tools are used, then general image processing capability is maintained, but text segmentation accuracy deteriorates due to unawareness of unique visual properties of text

Engineering Contradiction:
Improvetext segmentation accuracyVSAvoidawareness of unique visual properties
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent applies local quality by introducing a text-specific refinement module that processes only the text portions identified in the initial segmentation. This module uses text-aware features (such as character structure, font properties, and text-specific patterns) to refine the segmentation results specifically for text regions, while leaving non-text regions processed by the general segmentation network unchanged. This resolves the contradiction by enhancing text segmentation accuracy through specialized processing without requiring the entire system to be reconfigured for text-specific properties.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent employs a two-stage segmentation approach: first, a general image segmentation network performs initial segmentation to identify potential text regions; second, a text refinement network processes these regions with text-specific features to improve accuracy. This multi-level segmentation strategy allows the system to maintain general image processing capability while adding specialized text awareness where needed, thus resolving the contradiction between general versatility and text-specific precision.

Inventive Principle:
Principle #1Segmentation

2Manufacturing precision

If conventional segmentation is applied, then processing speed is maintained, but segmentation precision deteriorates by mistaking curves and decorations as text or missing text portions

Engineering Contradiction:
Improvesegmentation precisionVSAvoidsegmentation system complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by first performing a general image segmentation to identify candidate text regions, then using this initial segmentation as input to the text refinement network. The refinement network uses text-specific features to pre-process and highlight likely text portions before final classification. This two-stage approach improves precision by preparing the data in advance with text-aware processing, while keeping the overall system complexity manageable through modular architecture.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary text refinement network that acts as a bridge between the general image segmentation and final text classification. This intermediary module processes the initial segmentation results through text-specific feature extraction and refinement, improving precision by filtering out false positives (curves and decorations) and enhancing true text regions. The intermediary approach adds only moderate complexity while significantly improving segmentation precision.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Measurement precision

If initial segmentation is performed without refinement, then computational efficiency is maintained, but classification accuracy deteriorates in identifying foreground text portions

Engineering Contradiction:
Improveclassification accuracyVSAvoidcomputational energy consumption
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies local quality by directing computational resources specifically to text regions identified in the initial segmentation. The refinement network processes only these candidate text regions using text-specific features, rather than applying heavy processing to the entire image. This localized refinement approach improves classification accuracy for text portions while minimizing additional computational energy consumption by avoiding unnecessary processing of non-text regions.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent implements partial action by applying the computationally intensive text refinement process only to regions identified as potential text in the initial segmentation, rather than processing the entire image. This selective application of refinement reduces overall computational energy consumption while still achieving high classification accuracy for text portions. The system performs excessive processing only where needed (in text regions) and minimal processing elsewhere.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11688190B2Text refinement network
Publication Date: 2023.06.27 ADOBE INC
  • US11688190B2 patent drawing
  • US11688190B2 patent drawing
  • US11688190B2 patent drawing

AI summary

Systems and methods for text segmentation are described. Embodiments of the inventive concept are configured to receive an image including a foreground text portion and a background portion, classify each pixel of the image as foreground text or background using a neural network that refines a segmentation prediction using a key vector representing features of the foreground text portion, wherein the key vector is based on the segmentation prediction, and identify the foreground text portion based on the classification.