Vehicle External Environment Recognition Using Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing external environment recognizing devices for vehicles face challenges in accurately detecting the position of moving objects, particularly due to image distortion from fisheye lenses and the lack of distance measurement capabilities, leading to inaccurate positional relationships and complex system structures when additional sensors are used.
Innovation Solution
An external environment recognizing device that uses a combination of image processors, object detectors, and bird's-eye view image processors to accurately identify the position of moving objects without additional sensors, employing optical flow and difference calculations from multiple images to determine object positions and distances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a camera with fisheye lens is used to capture images around the vehicle, then a wide field of view is obtained, but image distortion increases making it difficult to accurately identify the position of moving objects
Solution Approach 1:
The patent transforms the distorted 2D fisheye image coordinates into a 3D spatial coordinate system by introducing depth information from optical flow magnitude. This dimensional transformation allows accurate position identification despite the 2D image distortion, as the optical flow magnitude provides the missing depth dimension needed to reconstruct real-world positions.
Solution Approach 2:
The patent introduces optical flow as an intermediary between the distorted image coordinates and the actual object positions. The optical flow vector (comprising direction and magnitude) serves as a mediator that bridges the gap between the distorted 2D image space and the accurate 3D physical space, enabling precise position identification without directly correcting the image distortion.
2Measurement precision
If a measurement sensor is added to measure distance to detected objects, then distance measurement capability is obtained, but the number of sensors increases resulting in complex system structure and increased costs
Solution Approach 1:
The patent makes the existing camera system multi-functional by extracting not only object detection information but also distance measurement information from the same image data. The optical flow magnitude, originally just another feature from the image, is repurposed as a depth cue, allowing the single camera to perform both detection and ranging functions that would traditionally require separate sensors.
Solution Approach 2:
The patent enables the camera system to measure distance using its own inherent properties (image data and optical flow) without external assistance. By analyzing the magnitude of optical flow vectors derived from the image sequence, the system self-generates depth information that would otherwise require dedicated measurement sensors, making the system self-sufficient for both detection and ranging.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This configuration enables reliable and accurate detection of moving object positions and distances, simplifying the system structure and reducing costs by eliminating the need for additional sensors.
Implementation Method 1
the first object detector detects the moving object based on optical flow calculated from a plurality of the cylindrical surface projection images generated from the plurality of original images obtained at different times
Implementation Method 2
a vehicle circumference monitoring device taught by Patent Literature 2 is configured to detect a moving object and a stationary object with optical flow detected from two original images captured at different times and a result of difference calculation between bird's-eye view images obtained through coordinate transformation of the two original images
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A position of a moving object is reliably detected with high accuracy by using only an image around a vehicle. A rear camera (12) mounted on a vehicle (10) obtains an original image (I) around the vehicle (10), a movement region detector (54) detects a moving object from the original image (I), and a difference calculator (58) detects the moving object from a bird's-eye view image (J) of the vehicle (10) generated by a bird's-eye view image processor (56). A moving object position identifying part (62) identifies a position of the moving object based on a distance from the vehicle (10) to the moving object detected by the movement region detector (54) or the difference calculator (58), a lateral direction position (FXk) of the moving object, and an actual width (Wk) of the moving object detected by the movement region detector (54) when a detected object determination part (60) determines that the moving objects detected by the movement region detector (54) and the difference calculator (58) are the same moving object.