Calibration of camera orientation based on gaze tracking and object detection

Abstract

A system and method for monitoring a camera in a vehicle, the system comprising a memory and at least one processor which receives an image 6502, captured by the camera, of an interior of the vehicle, and extracts non-semantic features of the interior from the image 6504. Spatial transformation data is calculated 6506 and describes a displacement of extracted features compared with reference data, which may be determined from 3D locations within a CAD file, describing reference features of the interior when the camera is in a reference orientation. An orientation of the camera relative to the reference orientation is calculated based on the spatial transformation data 6508. Optionally, the extracting comprises segmenting the image based on a mask applied to the image, and extracting based on the segmented image. Also provided is a non-transitory computer-readable device having instructions stored thereon for implementing the method. The system allows for calibration of the camera. [Fig. 65]