Real-Time Roadway Object Detection Using Superpixel Segmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current object detection systems for vehicles require computationally expensive three-dimensional vision systems, making them commercially infeasible for real-time operation, especially in environments without geo-reference information.

Innovation Solution

A method using a camera sensor and onboard computer to process real-time images by dividing them into superpixels, merging similar ones, generating prior maps, drawing bounding boxes, and performing feature extraction and categorization to identify objects, reducing processing time and cost by using a two-dimensional vision system.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If three-dimensional vision systems with large file sizes are used for object detection, then object detection capability is improved, but computational cost and processing time increase significantly

Engineering Contradiction:
Improveobject detection capabilityVSAvoidcomputational cost
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent divides the image processing task into segments: first identifying road regions using prior maps, then extracting features only from those specific regions. This segmentation approach maintains object detection capability while reducing the overall computational burden by avoiding processing of entire three-dimensional images.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent extracts only the necessary features from specific image regions rather than performing feature extraction on entire three-dimensional images. By taking out and processing only relevant road regions identified through prior maps, the system achieves effective object detection with reduced computational cost.

Inventive Principle:
Principle #2Taking out (Extraction)

2Measurement precision

If three-dimensional images with large file sizes are used, then object detection accuracy is improved, but processing time increases making real-time operation infeasible

Engineering Contradiction:
Improveobject detection accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent segments the processing workflow to identify road regions first using prior maps, then performs feature extraction only on those segmented regions. This approach maintains detection accuracy for relevant objects while significantly reducing processing time compared to analyzing entire three-dimensional images.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary actions by generating and using prior maps to identify road regions before performing detailed feature extraction. This preliminary identification step allows the system to focus computational resources on relevant areas, reducing overall processing time while maintaining accuracy.

Inventive Principle:
Principle #10Preliminary action

3Measurement precision

If feature extraction is performed on entire three-dimensional images, then comprehensive object detection is achieved, but computational expense increases making the system commercially infeasible

Engineering Contradiction:
Improvecomprehensive object detectionVSAvoidcommercial feasibility
Core Design Contradiction:
Measurement precisionVSEase of manufacture

Solution Approach 1:

The patent extracts and processes only the essential features from identified road regions rather than performing comprehensive feature extraction on entire three-dimensional images. This extraction approach maintains sufficient object detection capability while reducing computational expenses to commercially feasible levels.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies partial action by performing feature extraction only on road regions identified through prior maps rather than on entire images. This partial processing approach achieves sufficient object detection for safety applications while reducing computational costs to commercially viable levels.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS10970561B2Detecting roadway objects in real-time images
Publication Date: 2021.04.06 TOYOTA JIDOSHA KK
  • US10970561B2 patent drawing
  • US10970561B2 patent drawing
  • US10970561B2 patent drawing

AI summary

The disclosure includes a method that receives a real-time image of a road from a camera sensor communicatively coupled to an onboard computer of a vehicle. The method includes dividing the real-time image into superpixels. The method includes merging the superpixels to form superpixel regions. The method includes generating prior maps from a dataset of road scene images. The method includes drawing a set of bounding boxes where each bounding box surrounds one of the superpixel regions. The method includes comparing the bounding boxes in the set of bounding boxes to a road prior map to identify a road region in the real-time image. The method includes pruning bounding boxes from the set of bounding boxes to reduce the set to remaining bounding boxes. The method may include using a categorization module that identifies the presence of a road scene object in the remaining bounding boxes.