Dual-Submodel Image Inference for Accuracy-Speed Trade-off

Solution Overview

Problem

Machine learning models for semantic segmentation face a trade-off between inference speed and accuracy, where high-resolution training data improves accuracy but increases calculation load, and using low-resolution data reduces accuracy while speeding up inference.

Innovation Solution

An image processing apparatus that employs a dual-sub-model architecture, where a first sub-model performs feature extraction and a second sub-model performs resolution enhancement, allowing for the generation of higher-resolution output images and evaluation results to update the learning model, enabling faster and more accurate inference.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If high-resolution image is used as correct answer data for learning, then inference accuracy is improved, but calculation amount increases and output speed decreases

Engineering Contradiction:

Improveinference accuracyVSAvoidoutput speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The learning model is divided into two separate sub-models: a first sub-model for feature extraction and a second sub-model for resolution enhancement. This segmentation allows each sub-model to specialize in specific tasks, enabling the system to process high-resolution data during learning while maintaining fast inference by using only the necessary sub-models during deployment.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically adjusts the learning process by selectively using different sub-models based on the task requirements. During learning, both sub-models are activated to process high-resolution data and improve accuracy. During inference, the system can dynamically choose to use only the first sub-model for fast processing or activate the second sub-model when higher resolution is needed, thus adapting to different performance requirements.

Inventive Principle:

Principle #15Dynamics

2Productivity

If low-resolution image is used as correct answer data for learning, then calculation amount is suppressed and output speed is improved, but information amount of data decreases and inference accuracy decreases

Engineering Contradiction:

Improveoutput speedVSAvoidinference accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The learning model is divided into two separate sub-models: a first sub-model for feature extraction and a second sub-model for resolution enhancement. This segmentation allows each sub-model to specialize in specific tasks, enabling the system to process high-resolution data during learning while maintaining fast inference by using only the necessary sub-models during deployment.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The first sub-model performs preliminary feature extraction on the input image, creating a compressed feature representation. This preliminary processing enables subsequent resolution enhancement by the second sub-model to work with reduced computational complexity, effectively preparing the data for high-resolution output without requiring the entire learning process to operate at high resolution.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If decoder network gradually increases image size during learning, then semantic segmentation accuracy is improved, but calculation load increases

Engineering Contradiction:

Improvesegmentation accuracyVSAvoidcalculation load

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The learning model is divided into two separate sub-models: a first sub-model for feature extraction and a second sub-model for resolution enhancement. This segmentation allows each sub-model to specialize in specific tasks, enabling the system to process high-resolution data during learning while maintaining fast inference by using only the necessary sub-models during deployment.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different parts of the processing pipeline have different quality requirements. The first sub-model operates with lower computational intensity to extract essential features, while the second sub-model applies higher computational quality only where needed for resolution enhancement. This local differentiation of processing quality reduces overall calculation load while maintaining segmentation accuracy in critical regions.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20240404251A1Image processing apparatus, operation method therefor, inference apparatus, and learning apparatus

Publication Date: 2024.12.05 FUJIFILM CORP

AI summary

A learning input image is input to a first sub-model to extract a first feature map, and a first output image is output based on the first feature map. The first feature map is input to a second sub-model to extract a second feature map, and a second output image having a higher resolution than the first output image is output. In response to an inference input image being input to a trained learned model, the first output image as an inference result image is output.