Vehicle Camera Image Rectification via Windshield Refraction Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera-based systems for vehicles struggle with accurately estimating object distances due to windshield pane distortions, which are not effectively modeled or separated from camera effects in current calibration methods.
Innovation Solution
A method for image rectification that separates the optical effects of the windshield pane and the camera, using intermediate image data calculated based on camera parameters and windshield pane parameters, allowing for fast computation and accurate modeling of windshield pane distortions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the camera is mounted behind the windshield pane, then the camera lenses are protected from blocking by dirt or rain, but the optical properties of the windshield pane introduce significant image deviations and distortions
Solution Approach 1:
The patent introduces an intermediary calibration process that models the windshield pane as a separate optical element with its own parameters (refractive index, thickness, curvature). By treating the windshield as a mediator between the camera and the external scene, the system can mathematically compensate for its distorting effects through ray tracing calculations that account for refraction at the air-glass interfaces.
Solution Approach 2:
The patent segments the optical system into distinct components: the camera optics and the windshield pane. Each component is calibrated separately with its own parameter set, allowing independent optimization and compensation. This segmentation enables the system to handle each optical element's specific characteristics without the complexity of treating them as a monolithic system.
2Measurement precision
If existing calibration methods treat the camera and windshield pane as one optical system, then calibration can be performed, but the process becomes computationally intensive and requires new calibration for every camera-windshield combination
Solution Approach 1:
The patent divides the calibration process into two independent stages: camera calibration and windshield calibration. The camera is calibrated first to determine its intrinsic parameters, then the windshield is calibrated separately using the camera's parameters to determine the windshield's optical properties. This segmentation allows pre-calibration of the windshield that can be reused across different cameras mounted in the same vehicle.
Solution Approach 2:
The patent performs preliminary calibration of the windshield pane parameters (refractive index, thickness, curvature) in advance, before actual operation. Once these parameters are determined, they can be stored and reused for multiple calibration scenarios, eliminating the need to re-calibrate the entire system for every new camera installation or environmental condition.
3Reliability
If the camera is mounted behind the windshield pane, then the camera is protected from dirt and rain, but accurate distance estimation becomes severely obscured by windshield pane distortions
Solution Approach 1:
The patent uses ray tracing as an intermediary computational process that models the optical path through the windshield pane. By calculating the refraction effects at each interface using the determined windshield parameters, the system can back-project the distorted image rays to their true spatial origins, thereby compensating for the windshield's distorting effect on distance estimation.
Solution Approach 2:
The patent replaces the need for physical repositioning or removal of the camera with a computational approach. Instead of mechanically adjusting the camera to avoid windshield interference, the system uses software-based ray tracing and coordinate transformation to mathematically remove the windshield's optical effects, achieving accurate distance measurement while maintaining the camera's protected position.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The method provides a computationally fast and accurate rectification of images and image points, effectively removing windshield pane distortions and improving distance estimation accuracy in camera-based systems for vehicles.
Implementation Method 1
the optical properties of the windshield pane into the projection geometry of the complete optical system. Windshield pane influences can introduce a substantially different behavior of the optical paths
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for rectification of images and/or image points acquired by at least one camera (2) of a camera-based system (1) of a vehicle (14) with a windshield pane (6). First, a raw image of a scene is acquired with the camera (2). Raw image data is selected from the raw image, wherein the raw image data is the full raw image, a part of the raw image or a plurality of raw image points of the raw image. Then, intermediate image data is calculated based on the raw image data and on camera parameters, wherein the intermediate image data comprises an intermediate image or a plurality of intermediate image points and wherein the intermediate image data resembles an image or image points of the scene obtained by a pinhole camera (8) through the windshield pane (6). Finally, a set of points in space of the scene is calculated, wherein the set of points corresponds to pixels of the intermediate image or to the intermediate image points. Said calculation is performed using a parallel shift (11) of the optical path (7) induced by the windshield pane (6) based on windshield pane parameters. The invention further relates to a camera-based system (1) for a vehicle (14) and to a vehicle (14).