Object State Identification Using Predicted Image Regions

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods for predicting the motion of objects near a vehicle using neural networks require significant computational resources, leading to excessive hardware demands and potential delays in processing time.

Innovation Solution

An apparatus and method that utilize a processor to input images into a first classifier to detect object regions and determine predictable positions, with characteristics from predicted object regions being processed by a second classifier with a recursive structure or convolution operation in the temporal direction, allowing for parallel processing and reduced computational load.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If neural network is used to predict the motion of objects near a vehicle, then the prediction accuracy is improved, but the computational load increases excessively

Engineering Contradiction:
Improveprediction accuracyVSAvoidcomputational load
Core Design Contradiction:
Measurement precisionVSPower

Solution Approach 1:

The patent divides the processing into two separate classifiers: a first classifier that detects object regions in images, and a second classifier that predicts motion trajectories. This segmentation allows each classifier to specialize in a specific task, reducing the overall computational load compared to using a single comprehensive neural network, while maintaining prediction accuracy through coordinated operation of both classifiers.

Inventive Principle:
Principle #1Segmentation

2Measurement precision

If neural network processing is performed for object motion prediction, then the prediction capability is improved, but the processing time exceeds the desired timeframe

Engineering Contradiction:
Improveprediction capabilityVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The first classifier performs preliminary detection of object regions and extracts relevant features before the second classifier performs motion prediction. This preliminary action prepares the data in advance, allowing the second classifier to focus solely on trajectory prediction with reduced computational requirements, thereby decreasing overall processing time while maintaining prediction capability.

Inventive Principle:
Principle #10Preliminary action

3Measurement precision

If comprehensive object analysis is performed using neural network, then the identification accuracy is improved, but the hardware requirements become excessive

Engineering Contradiction:
Improveidentification accuracyVSAvoidhardware requirements
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the comprehensive object analysis into two distinct processing stages handled by separate classifiers. The first classifier handles object detection and region identification, while the second classifier handles motion trajectory prediction. This segmentation reduces the computational complexity required in each individual component, allowing the system to achieve high identification accuracy without requiring excessively complex hardware infrastructure.

Inventive Principle:
Principle #1Segmentation

4Device complexity

If sequential processing is used for object detection and state identification, then the processing logic is simplified, but the throughput is reduced

Engineering Contradiction:
Improveprocessing logicVSAvoidthroughput
Core Design Contradiction:
Device complexityVSProductivity

Solution Approach 1:

The first classifier performs preliminary object detection and extracts features in advance, preparing the data structure and identifying regions of interest before the second classifier performs state identification. This preliminary action enables the second classifier to process only relevant extracted features rather than raw images, significantly improving throughput while keeping the processing logic relatively simple through the structured workflow.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS11829153B2Apparatus, method, and computer program for identifying state of object, and controller
Publication Date: 2023.11.28 TOYOTA JIDOSHA KK
  • US11829153B2 patent drawing
  • US11829153B2 patent drawing
  • US11829153B2 patent drawing

AI summary

An apparatus for identifying the state of an object includes a processor configured to input, every time obtaining an image from a camera, the image into a first classifier to detect, for each of one or more predetermined objects represented in the image, an object region including the object; determine a predicted object region in a subsequent image to be obtained from the camera for an object whose position in the subsequent image is predictable; and input characteristics into a second classifier to identify the state of an object involving time-varying changes in outward appearance. When the object has a predicted object region, the characteristics are obtained from pixel values of the predicted object region in the subsequent image. On the other hand, when the object does not have a predicted object region, the characteristics are obtained from pixel values of the object region detected from the subsequent image.