Convolutional Neural Network Segmentation for Image Morphing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional methods for morphing images using convolutional neural networks are inefficient due to complex network structures, leading to low morphing speed, poor quality, and high processing time, often requiring excessive convolutional kernels and interconnected layers that result in low operation efficiency.

Innovation Solution

A computer-implemented method utilizing a convolutional neural network with a densely connected segment and cross-scale concatenations, comprising a down-sampling, densely connected, and up-sampling segment, which reduces the number of convolutional kernels and prevents overlapping transactions between layers, improving operation efficiency and morphing quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional convolutional neural networks with complex interconnected layers are used for image morphing, then the network can capture detailed features, but the processing speed decreases and operation efficiency becomes low

Engineering Contradiction:
Improvefeature extraction accuracyVSAvoidprocessing speed
Core Design Contradiction:
Measurement precisionVSSpeed

Solution Approach 1:

The network is divided into three independent segments (down-sampling segment, densely connected segment, up-sampling segment) that process images in sequence. Each segment has specialized functions and operates independently, reducing computational overhead while maintaining feature extraction accuracy. The segmentation allows parallel processing within each segment without the need for complex inter-layer connections.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent extracts and removes redundant convolutional kernels and interconnected layers from the conventional CNN structure. By taking out only the essential processing components needed for style transfer and eliminating unnecessary complexity, the network achieves faster processing speed while preserving the core functionality of feature extraction and transformation.

Inventive Principle:
Principle #2Taking out (Extraction)

2Adaptability or versatility

If conventional convolutional neural networks with excessive convolutional kernels are used, then comprehensive feature coverage is achieved, but the number of operations increases and operation efficiency decreases

Engineering Contradiction:
Improvefeature coverageVSAvoidoperation efficiency
Core Design Contradiction:
Adaptability or versatilityVSProductivity

Solution Approach 1:

Different segments of the network are assigned different levels of processing complexity appropriate to their specific functions. The down-sampling segment uses fewer kernels focused on structural features, the densely connected segment uses moderate kernels for style features, and the up-sampling segment uses fewer kernels for reconstruction. This local optimization of kernel density maintains comprehensive feature coverage while reducing total operation count.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent applies partial action by using sufficient but not excessive convolutional kernels in each segment. Rather than deploying excessive kernels throughout the entire network, the design uses just enough kernels in each specialized segment to achieve the required feature coverage, eliminating redundant operations and improving overall productivity.

Inventive Principle:
Principle #16Partial or excessive action

3Measurement precision

If densely connected layers with many interconnected layers are used, then deep feature extraction is achieved, but the network complexity increases and processing time extends

Engineering Contradiction:
Improvedeep feature extractionVSAvoidnetwork complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The network architecture is segmented into three distinct functional blocks with clear boundaries. Each segment performs specific processing tasks (down-sampling, dense connection, up-sampling) and passes results to the next segment. This segmentation reduces network complexity by eliminating the need for dense interconnections between all layers while maintaining the capability for deep feature extraction through the sequential processing pipeline.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent transitions from the traditional horizontal dimension of many interconnected layers to a vertical dimension of three stacked segments. Each segment processes features at different scales and representations, effectively adding a dimensional aspect to feature extraction. This dimensional change allows deep feature extraction without requiring a large number of interconnected layers, thereby reducing network complexity.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

4Reliability

If conventional network structures with overlapping layer transactions are used, then comprehensive processing is achieved, but operation efficiency becomes low due to redundant computations

Engineering Contradiction:
Improveprocessing completenessVSAvoidoperation efficiency
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

By segmenting the network into distinct processing stages with clear input-output interfaces, the patent eliminates overlapping transactions between layers. Each segment processes its assigned data independently and passes results forward, ensuring that no computation is performed more than once. This segmentation maintains processing completeness while eliminating redundant operations, thereby improving operation efficiency.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The segmented architecture ensures continuous useful action by passing processed data sequentially from one segment to the next without backtracking or reprocessing. Each segment performs its designated function completely before passing control to the next segment, eliminating gaps or overlaps in the processing flow. This continuous action ensures comprehensive processing while maximizing operation efficiency by avoiding redundant computations.

Inventive Principle:
Principle #20Continuity of useful action

Data Source

PatentUS11842267B2Computer-implemented method using a neural network comprising down-sampling segment, densely connected segment, and up-sampling segment, apparatus, and computer-program product
Publication Date: 2023.12.12 BOE TECHNOLOGY GROUP CO LTD
  • US11842267B2 patent drawing
  • US11842267B2 patent drawing
  • US11842267B2 patent drawing

AI summary

A computer-implemented method using a convolutional neural network is provided. The computer-implemented method using a convolutional neural network includes processing an input image through at least one channel of the convolutional neural network to generate an output image including content features of the input image morphed with style features of a reference style image. The at least one channel includes a down-sampling segment, a densely connected segment, and an up-sampling segment sequentially connected together. Processing the input image through the at least one channel of the convolutional neural network includes processing an input signal through the down-sampling segment to generate a down-sampling segment output; processing the down-sampling segment output through the densely connected segment to generate a densely connected segment output; and processing the densely connected segment output through the up-sampling segment to generate an up-sampling segment output. The input signal includes a component of the input image.