Character String Recognition Using Attention Mechanisms

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing character string recognition techniques face challenges in achieving high accuracy, particularly in languages with a large number of characters, such as Japanese, due to the bias in label frequency and ambiguity in character borders, leading to lower recognition accuracy for labels with lower frequency and those not present in learning data.

Innovation Solution

A character string recognition apparatus that computes an attention single character certainty factor using a recurrent neural network, sharing parameters among labels and employing an attention technique to selectively extract relevant features, thereby improving recognition accuracy and addressing the ambiguity in character borders.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional character string recognition is used with label frequency bias, then recognition speed is maintained, but recognition accuracy deteriorates for low-frequency labels

Engineering Contradiction:
Improverecognition accuracyVSAvoidlabel frequency coverage
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent transforms the certainty factor computation by introducing attention mechanisms that dynamically adjust the weighting of input features based on their relevance to each character position. This parameter change allows the system to adapt to varying label frequencies without being constrained by traditional frequency-based biases, thereby improving recognition accuracy for low-frequency labels while maintaining overall system performance

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent introduces an attention mechanism as an intermediary component between the input character sequence and the prediction unit. This attention mechanism computes attention scores that mediate the interaction between different character positions and labels, allowing the system to selectively focus on relevant features regardless of label frequency, thus resolving the contradiction between accuracy and frequency coverage

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If recurrent neural network with attention mechanism is used, then recognition accuracy is improved, but computational complexity increases

Engineering Contradiction:
Improverecognition accuracyVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the computational process into distinct modules: an encoding module that processes the input sequence, an attention mechanism that computes relevance scores, and a prediction unit that generates outputs. This segmentation allows each component to be optimized independently and enables parallel computation of attention scores for different character positions, reducing overall computational complexity while maintaining high recognition accuracy

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements dynamic computation where the attention mechanism adaptively adjusts the weight of different input features based on their relevance to the current prediction task. This dynamic approach allows the system to focus computational resources on the most relevant features rather than processing all features uniformly, thereby improving accuracy without proportionally increasing computational complexity

Inventive Principle:
Principle #15Dynamics

3Productivity

If shared parameters among labels are used, then model efficiency is improved, but ability to capture label-specific features is reduced

Engineering Contradiction:
Improvemodel efficiencyVSAvoidlabel-specific recognition
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The patent segments the parameter structure into shared components (encoding module parameters) and label-specific components (prediction unit parameters). The shared encoding parameters capture general features across all labels, improving model efficiency, while the label-specific prediction parameters capture distinctive characteristics of each label, maintaining high recognition precision. This segmentation resolves the contradiction between efficiency and label-specific accuracy

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS11126883B2Character string recognition apparatus, and non-transitory computer readable medium
Publication Date: 2021.09.21 FUJIFILM BUSINESS INNOVATION CORP
  • US11126883B2 patent drawing
  • US11126883B2 patent drawing
  • US11126883B2 patent drawing

AI summary

A character string recognition apparatus includes a prediction unit that computes an attention single character certainty factor from a single character certainty factor, computed for a sequence of a partial pattern of an input pattern that is a character string serving as a recognition target, and from attention information computed by a recurrent neural network. The prediction unit shares a parameter among labels serving as recognition results, and uses the attention single character certainty factor as a feature value.