DNN Training Recognition Device Using GAN Simulation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Deep learning for image recognition in logistics and production fields faces challenges in reducing manual annotation efforts, power consumption, and hardware requirements, particularly due to differences between simulated and real images used for training, and the inefficiencies of backpropagation-based training methods.

Innovation Solution

A training recognition device that employs a generative adversarial network to convert simulation images into artificial site images, allowing for re-training of deep neural networks without manual annotation, and updates coefficients based on feature point differences to adapt to real image conditions, thereby reducing power consumption and hardware needs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If backpropagation is used to train the entire DNN from scratch, then training accuracy can be achieved, but power consumption, time, and hardware amount increase significantly

Engineering Contradiction:
Improvetraining accuracyVSAvoidpower consumption
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent segments the DNN training process into two distinct stages: pre-training using simulation images (where computational resources can be optimized) and re-training using actual site images (where only necessary portions are updated). This segmentation allows the system to achieve high accuracy while reducing overall power consumption by avoiding full re-training from scratch.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary pre-training using simulation images before actual deployment. This preliminary action establishes a foundational model that can be efficiently refined later through re-training with actual site images, reducing the computational burden during the final training phase and thereby lowering power consumption.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If the entire DNN is re-trained using actual site images, then recognition accuracy for real images improves, but power consumption and training time increase

Engineering Contradiction:
Improverecognition accuracy for real imagesVSAvoidtraining time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent divides the training process into pre-training with simulation images and selective re-training with actual site images. By segmenting this way, the system only updates necessary portions of the DNN for real image conditions rather than re-training the entire network, thus improving recognition accuracy for real images while reducing training time.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by updating only the necessary portions of the DNN that are relevant to actual site image conditions during re-training, while maintaining the pre-trained parameters for other parts of the network. This localized update approach improves recognition accuracy for real images without the time cost of re-training the entire DNN.

Inventive Principle:
Principle #3Local quality

3Extent of automation

If simulation images are used for training, then manual annotation efforts are reduced, but recognition accuracy deteriorates due to differences between simulation and real images

Engineering Contradiction:
Improvemanual annotation effortsVSAvoidrecognition accuracy
Core Design Contradiction:
Extent of automationVSMeasurement precision

Solution Approach 1:

The patent introduces actual site images as an intermediary between simulation images and the final DNN model. The process uses simulation images for initial pre-training (reducing manual annotation), then introduces actual site images through re-training to bridge the gap between simulation and real-world conditions, thereby improving recognition accuracy while maintaining the benefits of automated simulation-based training.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent performs preliminary pre-training using simulation images with automated annotation, then conducts subsequent re-training using actual site images to refine the model. This preliminary action with simulation images reduces manual annotation efforts, while the follow-up re-training with actual images ensures high recognition accuracy by adapting to real-world conditions.

Inventive Principle:
Principle #10Preliminary action

4Adaptability or versatility

If full DNN training is performed, then comprehensive learning is achieved, but hardware amount and computational resources increase

Engineering Contradiction:
Improvecomprehensive learning capabilityVSAvoidhardware amount
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the training process into pre-training and re-training phases, allowing comprehensive learning to be achieved through staged computation rather than requiring full computational resources simultaneously. This segmentation enables the system to reach comprehensive learning capability while reducing peak hardware requirements.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary pre-training with simulation images to establish comprehensive learning foundations, then conducts re-training with actual site images to refine and adapt the model. This preliminary action allows comprehensive learning to be built incrementally, reducing the need for extensive hardware resources during the final training phase.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS20220391698A1Training Recognition Device
Publication Date: 2022.12.08 HITACHI LTD
  • US20220391698A1 patent drawing
  • US20220391698A1 patent drawing
  • US20220391698A1 patent drawing

AI summary

Provided is a training recognition device that implements training of a DNN for article recognition that does not require manual annotation for an image for training and can reduce power consumption, time, and hardware amount required for training. The training recognition device includes: an image conversion unit that inputs a simulation image and an actual site image into a generative adversarial network and converts the simulation image into an artificial site image; a pre-trained feature extraction unit that inputs the simulation image to a trained deep neural network trained using the simulation image and annotation data for the simulation image and outputs a feature point of the simulation image at time of re-training; a re-training feature extraction unit that inputs the artificial site image to a deep neural network for re-training, re-trains a difference between the simulation image and the artificial site image, and outputs a feature point of the artificial site image; an error calculation unit for feature extraction unit that calculates a difference between the feature point output by the re-training feature extraction unit and the feature point output by the pre-trained feature extraction unit; a coefficient update unit for feature extraction unit that updates a coefficient of the re-training feature extraction unit used for re-training based on the difference; and a re-training identification unit that re-trains a method for identifying an article based on a feature point output from the deep neural network for re-training of the coefficient updated by the coefficient update unit for feature extraction unit.