Distance sensor based roadside camera calibration in ADAS applications

Abstract

A roadside camera calibration system based on a distance sensor includes one or more cameras having overlapping road fields of view and a distance sensor. Control logic stored within and executed by the controller includes: control logic for capturing an image and distance sensor data; control logic to filter the data to focus on first and second regions of interest (ROI); control logic for filtering the data to focus on first and second motion of interest (MOI); control logic to filter the data to focus on first and second objects of interest (OOI); control logic to filter the data to focus on first and second positions of interest (POI); and control logic for defining, as a matching object, objects detected by the camera and the sensor and simultaneously satisfying the first and second ROI, MOI, OOI, and POI filters. The control logic calibrates the camera by applying the correction factor based on the matching object.

Description

Technical Field The present disclosure relates to infrastructure-based roadside cameras and systems and methods for calibrating such roadside cameras. Background Art Camera-based advanced driver assistance system (ADAS) applications require calibration between roadside cameras and the real world to perform position estimation in real-world coordinates. Typical calibration methods involve manually surveying an area to determine the real-world positions that can be detected by the roadside camera being calibrated. This manual survey and calibration procedure is time-consuming, cumbersome, inefficient, and complex. Therefore, while current systems and methods for calibrating roadside cameras are effective and achieve their intended purposes, there remains a need for new and improved systems and methods for calibrating roadside cameras used in ADAS applications that increase efficiency, reduce complexity, and reduce the amount of time required to perform calibration. Summary of the invention According to several aspects of the present disclosure, a roadside camera calibration system based on a distance sensor includes one or more roadside cameras. The roadside camera is fixedly mounted to an infrastructure and has a first field of view of at least a portion of a road. The system also includes one or more distance sensors. The distance sensor is fixedly mounted to the infrastructure and has a second field of view of at least a portion of a road. The first and second fields of view overlap at least partially. The system also includes one or more controllers. Each of the one or more controllers has a processor, a memory, and one or more input / output (I / O) ports. The I / O port communicates with the one or more distance sensors and the one or more roadside cameras. The memory stores programmed control logic. The processor executes the programmed control logic. The programmed control logic includes at least first, second, third, fourth, fifth, sixth, seventh, and eighth control logics. The first control logic causes each of the one or more roadside cameras to capture image data, and causes each of the distance sensors to capture distance sensor data. The second control logic filters the image data to focus on a first region of interest (ROI), and filters the distance sensor data to focus on a second ROI. The third control logic filters the image data to focus on a first motion of interest (MOI), and filters the distance sensor data to focus on a second MOI. The fourth control logic filters the image data to focus on a first object of interest (OOI) and filters the distance sensor data to focus on a second OOI. The fifth control logic filters the image data to focus on a first location of interest (POI) and filters the distance sensor data to focus on a second POI. The sixth control logic selectively determines whether the objects detected by the roadside camera and the distance sensor satisfy the first and second ROI, MOI, OOI and POI filters, and defines the objects that satisfy the first and second ROI, MOI, OOI and POI filters as matching objects. The seventh control logic saves the matching objects in a memory for calibration. The eighth control logic calibrates the roadside camera based on the matching objects by applying correction factors so that the roadside camera accurately and precisely reports the position and movement of the matching objects for ADAS applications. In another aspect of the present disclosure, the second control logic further includes: control logic for constraining image data from a roadside camera to a sub-region or ROI within the first field of view in which image data captured by the roadside camera is reliable and accurate; and control logic for constraining distance sensor data from a distance sensor to a sub-region or ROI within the second field of view in which distance sensor data captured by the distance sensor is reliable and accurate. The ROIs within the first and second fields of view are substantially the same. In yet another aspect of the present disclosure, the third control logic further includes control logic for constraining the image data to focus on a first MOI. The first MOI includes predictable and consistent motion of an object along a road. The third control logic further includes control logic for constraining the distance sensor data to focus on a second MOI. The second MOI includes predictable and consistent motion of an object along a road. In yet another aspect of the present disclosure, the predictable and consistent movement of objects along the road also includes movement in a set of predetermined directions, linear or straight-line movement, and movement at predetermined accelerations and velocities. In yet another aspect of the present disclosure, the fourth control logic further includes control logic to constrain the image data to define a first OOI by selecting objects that satisfy the first ROI filter and the first MOI filter and can be directly and unobstructedly viewed by the roadside camera. The fourth control logic further includes control logic to constrain the image data to define a second OOI by selecting objects that satisfy the second ROI filter and the second MOI filter and can be directly and unobstructedly viewed by the roadside camera. In yet another aspect of the present disclosure, the fifth control logic further includes: control logic for constraining the image data by selecting only the first OOI at a position within the first ROI having a predefined fixed or moving position, and control logic for constraining the image data by selecting only the second OOI at a position within the second ROI having a predefined fixed or moving position. The predefined fixed or moving positions include: the position of lane lines, the position of road signs, trees, traffic signals, telecommunication poles, fire hydrants and / or curbs, and the position of movable or moving OOIs. In another aspect of the present disclosure, the sixth control logic also includes control logic for defining an object as a matching object by comparing objects that simultaneously satisfy the first and second ROI, MOI, OOI and POI filters and verifying whether one or more objects that simultaneously satisfy the first and second ROI, MOI, OOI and POI filters are the same object detected by the roadside camera and the distance sensor. In yet another aspect of the present disclosure, the eighth control logic further includes control logic for applying a correction factor to the image data captured by the roadside camera. The correction factor is based on the known physical and geometric position of the roadside camera relative to the known physical and geometric position of the distance sensor, and obtains the point position in the real world coordinate system and translates the point to the camera coordinate system according to the calibration matrix K[I3|03] based on the world homography matrix "H": 1. Where R is the rotation amount, and C is the distance from the projection center of the world coordinate system to the camera coordinate system; 2. where α x=fk x And α y =-fk y ; and x pix =u′ / w′ and y pix =v′ / w′, so that: 3. where α x and α x is the focal length (in pixels), x0 and y0 are the coordinates of the image center (in pixels), “s” is the tilt parameter describing the viewing angle and position of the roadside camera, and K[I3|03] is the calibration matrix. In another aspect of the present disclosure, based on the known relative positions of the roadside camera and the distance sensor, and based on the radial range or distance r between the roadside camera and the object in the world coordinate system d And the height h1 of the roadside camera from the ground where the roadside camera is fixedly installed further adjusts the calibration matrix K[I3|03] so that: In another aspect of the present disclosure, a method for calibrating a roadside camera based on a distance sensor includes: capturing image data with one or more roadside cameras, wherein the roadside cameras are fixedly mounted to infrastructure and have a first field of view of at least a portion of a road; and capturing distance sensor data with one or more distance sensors, wherein the distance sensors are fixedly mounted to infrastructure and have a second field of view of at least a portion of the road. The first and second fields of view overlap at least partially. The method also includes executing program control logic stored in a memory of one or more controllers that communicate with the one or more roadside cameras and the one or more distance sensors. Each of the one or more controllers has a processor, a memory, and one or more input / output (I / O) ports. The I / O ports communicate with the one or more distance sensors and the one or more roadside cameras. The processor executes program control logic, including: filtering image data to focus on a first region of interest (ROI), and filtering distance sensor data to focus on a second ROI; filtering image data to focus on a first motion of interest (MOI), and filtering distance sensor data to focus on a second MOI; filtering image data to focus on a first object of interest (OOI), and filtering distance sensor data to focus on a second OOI; filtering image data to focus on a first location of interest (POI), and filtering distance sensor data to focus on a second POI. The method also includes executing control logic for selectively determining whether an object detected by a roadside camera and by a distance sensor satisfies first and second ROI, MOI, OOI, and POI filters and defining the object as a matching object. The method also includes executing control logic for saving the matching object in a memory for calibration, and for calibrating the roadside camera based on the matching object by applying a correction factor so that the roadside camera accurately and precisely reports the position and motion of the matching object for use in an ADAS application. In yet another aspect of the present disclosure, the method includes: constraining image data from a roadside camera to a sub-region or ROI within a first field of view in which image data captured by the roadside camera is reliable and accurate; and constraining distance sensor data from a distance sensor to a sub-region or ROI within a second field of view in which distance sensor data captured by the distance sensor is reliable and accurate. The ROIs within the first and second fields of view are substantially the same. In another aspect of the present disclosure, the method includes constraining the image data to focus on a first MOI. The first MOI includes predictable and consistent movement of objects along the road. The method also includes constraining the distance sensor data to focus on a second MOI. The second MOI includes predictable and consistent movement of objects along the road. In yet another aspect of the present disclosure, constraining the images and sensor data further includes predictable and consistent movement of select objects along the road, including movement in a set of predefined directions, linear or straight-line movement, and movement at predefined accelerations and velocities. In yet another aspect of the present disclosure, the method includes: constraining image data to define a first OOI by selecting objects that satisfy a first ROI filter and a first MOI filter and can be directly and unobstructedly viewed by a roadside camera; and constraining distance sensor data to define a second OOI by selecting objects that satisfy a second ROI filter and a second MOI filter and can be directly and unobstructedly viewed by the roadside camera. In yet another aspect of the present disclosure, the method includes constraining image data by selecting only first OOIs at locations within a first ROI having a predefined fixed or moving location, and constraining range sensor data by selecting only second OOIs at locations within a second ROI having a predefined fixed or moving location. The predefined fixed or moving locations include: locations of lane lines, locations of road signs, trees, traffic signals, telecommunication poles, fire hydrants and / or curbs, and locations of movable or moving OOIs. In yet another aspect of the present disclosure, the method includes defining an object as a matching object by comparing objects that simultaneously satisfy first and second ROI, MOI, OOI, and POI filters and verifying whether one or more objects that simultaneously satisfy the first and second ROI, MOI, OOI, and POI filters are the same object detected by a roadside camera and a distance sensor. In yet another aspect of the present disclosure, the method includes applying a correction factor to image data captured by a roadside camera. The correction factor is based on the known physical and geometric position of the roadside camera relative to the known physical and geometric position of the distance sensor, and takes a point position in a real world coordinate system and translates the point to a camera coordinate system according to a calibration matrix K[I3|03] based on a world homography matrix "H": 1. Where R is the rotation amount, and C is the distance from the projection center of the world coordinate system to the camera coordinate system; 2. where α x =fk x And α y =-fk y ; and x pix =u′ / w′ and y pix =v′ / w′, so that: 3. where α x and αx is the focal length (in pixels), x0 and y0 are the coordinates of the image center (in pixels), “s” is the tilt parameter describing the viewing angle and position of the roadside camera, and K[I3|03] is the calibration matrix.

[0021] In another aspect of the present disclosure, the method further includes: based on the known relative positions of the roadside camera and the distance sensor, and based on the radial range or distance r between the roadside camera and the object in the world coordinate system d And the height h1 of the roadside camera from the ground where the roadside camera is fixedly installed further adjusts the calibration matrix K[I3|03] so that: In another aspect of the present disclosure, a method for calibrating a roadside camera based on a distance sensor includes: capturing image data with one or more roadside cameras, which are fixedly mounted to infrastructure and have a first field of view of at least a portion of a road. The method also includes: capturing distance sensor data with one or more distance sensors, which are fixedly mounted to infrastructure and have a second field of view of at least a portion of a road. The first and second fields of view overlap at least partially. The method also includes: executing program control logic stored in a memory of one or more controllers that communicate with one or more roadside cameras and one or more distance sensors. Each of the one or more controllers has a processor, a memory, and one or more input / output (I / O) ports. The I / O port communicates with one or more distance sensors and with one or more roadside cameras. The processor executes the program control logic, including control logic for performing the following operations: constraining image data from a roadside camera to a sub-region or region of interest (ROI) within the first field of view, in which the image data captured by the roadside camera is reliable and accurate; and constraining the distance sensor data to a sub-region or ROI within the second field of view, in which the distance sensor data captured by the distance sensor is reliable and accurate. The ROIs within the first and second fields of view are substantially the same. The method further includes: executing control logic for constraining image data to focus on a first motion of interest (MOI). The first MOI includes predictable and consistent motion of an object along a road. The method further includes: executing control logic for constraining distance sensor data to focus on a second MOI. The second MOI includes predictable and consistent motion of an object along a road, including motion along a set of predefined directions, linear or straight motion, and motion at predefined accelerations and speeds. The method further includes executing control logic for constraining image data to define a first object of interest (OOI) by selecting an object that satisfies the first ROI filter and the first MOI filter and can be viewed directly and unobstructed by a roadside camera. The method further includes: executing control logic for constraining image data by the first OOI filter by only selecting a first OOI located at a position within the first ROI having a predefined fixed or moving position, and for constraining distance sensor data by the second OOI filter by only selecting a second OOI located at a position within the second ROI having a predefined fixed or moving position and can be viewed directly and unobstructed by a distance sensor. ,The predefined fixed or mobile locations include: the locations of lane lines, the locations of road signs, trees, traffic signals, telecommunication poles, fire hydrants and / or curbs, and the locations of movable or moving OOIs.The method also includes: executing control logic for filtering image data through a first location of interest (POI) filter to focus on a first POI and filtering distance sensor data through a second POI filter to focus on a second POI. The method also includes: executing the following control logic: defining an object as a matching object by selectively determining whether an object detected by a roadside camera and by a distance sensor satisfies both the first and second ROI, MOI, OOI, and POI filters and by comparing objects that simultaneously satisfy the first and second ROI, MOI, OOI, and POI filters and verifying whether one or more objects that simultaneously satisfy the first and second ROI, MOI, OOI, and POI filters are the same object detected by the roadside camera and the distance sensor. The method also includes: executing the following control logic: saving the matching object in a memory for calibration, and calibrating the roadside camera by applying a correction factor based on the matching object so that the roadside camera accurately and precisely reports the position and motion of the matching object for use in ADAS applications. The correction factors are based on the known physical and geometric position of the roadside camera relative to the known physical and geometric position of the distance sensor, and take the point position in the real world coordinate system and translate it to the camera coordinate system according to the calibration matrix K[I3|03] based on the world homography matrix "H":. 1. Where R is the rotation amount, and C is the distance from the projection center of the world coordinate system to the camera coordinate system; 2. where α x =fk x And α y =-fk y ; and x pix =u′ / w′ and y pix =v′ / w′, so that: 3. where α x and α x is the focal length (in pixels), x0 and y0 are the coordinates of the image center (in pixels), “s” is the tilt parameter describing the viewing angle and position of the roadside camera, and K[I3|03] is the calibration matrix. In another aspect of the present disclosure, the method further includes: based on the known relative positions of the roadside camera and the distance sensor, and based on the radial range or distance r between the roadside camera and the object in the world coordinate system d And the height h1 of the roadside camera from the ground where the roadside camera is fixedly installed further adjusts the calibration matrix K[I3|03] so that: Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way. Figure 1 is a schematic block diagram of a system for calibrating a roadside camera using a range sensor for an advanced driver assistance system (ADAS) application according to an exemplary embodiment; Figure 2 According to an exemplary embodiment, Figure 1 Schematic diagram of camera and world reference frames of a system for calibrating roadside cameras using a distance sensor for ADAS applications; Figure 3 According to an exemplary embodiment, Figure 1 and Figure 2 A schematic diagram of coordinate adjustment of a system for calibrating a roadside camera using a distance sensor for an ADAS application is shown; Figure 4 is a perspective top view of a plurality of vignettes or frames defining an example embodiment of the Figure 1 Aspects of the field of view of a roadside camera and a range sensor of a system for calibrating a roadside camera using a range sensor for an ADAS application are shown; and Figure 5 is a flow chart of a method for calibrating a roadside camera using a range sensor for ADAS applications according to an exemplary embodiment. DETAILED DESCRIPTION The following specific embodiments are merely exemplary and are not intended to limit application and use. In addition, it is not subject to the explicit or implicit theory presented in the aforementioned technical field, background technology, invention content or the following specific embodiments. As used herein, the term "module" refers to hardware, software, firmware, electronic control components, processing logic and / or processor devices used alone or in combination, including but not limited to: application-specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated or grouped) and memories that execute one or more software or firmware programs, combinational logic circuits, and / or other suitable components that provide the described functions. Embodiments of the present disclosure may be described herein in terms of functional and / or logical block components and various processing steps. It should be understood that such block components may be implemented by multiple hardware, software and / or firmware components configured to perform specified functions. For example, embodiments of the present disclosure may employ various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, lookup tables, etc., which may perform various functions under the control of one or more microprocessors or other control devices. In addition, those skilled in the art will appreciate that embodiments of the present disclosure may be practiced in conjunction with a variety of systems, and the systems described herein are merely exemplary embodiments of the present disclosure. For the sake of brevity, technologies related to signal processing, data fusion, signaling, control and other functional aspects of the system (as well as the various operating components of the system) may not be described in detail herein. In addition, the connecting lines shown in the various figures contained herein are intended to represent example functional relationships and / or physical couplings between the various elements. It should be noted that alternative or additional functional relationships or physical connections may exist in the embodiments of the present disclosure. Now refer to Figure 1 , schematically illustrates a system 10 for performing automatic calibration of a roadside camera 12 using a distance sensor 14. The system 10 calibrates a roadside camera 12 by accurately and precisely determining the distance between the camera and a data point or object 16 in the real world and within a first field of view 18 of the roadside camera 12. Although the object 16 may be a fixed object or feature, such as a tree, a light pole, a parking meter, a traffic cone, a curb, a lane line, a flower pot, a building, etc., it should be understood that the object 16 may also be a movable or moving object or feature such as a person, an animal, a vehicle 19, etc. without departing from the scope or intent of the present disclosure. The system 10 also includes one or more distance sensors 14, each of the one or more distance sensors 14 having a second field of view 24 that substantially overlaps the first field of view 18 of the roadside camera 12. The roadside camera 12 and the one or more sensors 14 each communicate with one or more control modules or controllers 26. In several aspects, the roadside camera 12 may include any of a variety of different types of cameras without departing from the scope or intent of the present disclosure. For example, the roadside camera 12 may include a camera capable of capturing optical information of various wavelengths of light (including, but not limited to, infrared, visible light, ultraviolet light, etc.). The roadside camera 12 is also capable of capturing a series of static image data, video graphics data, or a combination of static data and video graphics data. In several aspects, the roadside camera 12 is set at one or more locations along the road 28, including, but not limited to, a fixed location 30 near an intersection 32. The fixed location 30 may include infrastructure such as a light pole, a traffic signal bracket, a crosswalk signal bracket, a utility pole, a building, or other such infrastructure in or on the fixed location 30. In further aspects, without departing from the scope or intent of the present disclosure, the sensor 14 can be any of a variety of different types of distance sensors 14 or a combination of distance sensors 14. For example, the sensor 14 can be an additional camera, a light detection and ranging (LiDAR) sensor, a radio detection and ranging (RADAR) sensor, a sound navigation and ranging (SONAR) sensor, an ultrasonic sensor, or a combination thereof. In addition, the distance sensor 14 can have the ability to communicate with a global positioning system (GPS) to more accurately and precisely set the position of the distance sensor 14 for calibration of the roadside camera 12, as will be further described in detail herein. Each controller 26 is a non-general purpose electronic control device having a pre-programmed digital computer or processor 34, a non-transitory computer readable medium or memory 36 for storing data such as control logic, software applications, instructions, computer code, data, look-up tables, and one or more input / output (I / O) ports 38. The computer readable medium or memory 36 includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disk (CD), a digital compact disk, a video compact disk (DVD), a solid-state memory, or any other type of memory. "Non-transitory" computer readable medium or memory 36 does not include wireless, optical, or other communication links that transmit electrical or other signals. Non-transitory computer readable medium or memory 36 includes media that can permanently store data and media that can store data and then rewrite it, such as any type of program code, including source code, object code, and executable code. The processor 34 is configured to execute code or instructions. In some examples, the controller 26 can be a dedicated wireless or Wi-Fi controller. The I / O port 38 is configured to communicate in a wired or wireless manner using a Wi-Fi protocol under IEEE 802.11x, a Bluetooth communication protocol, a radio frequency (RF) protocol, or the like. In several aspects, the controller 26 includes one or more applications 40. An application 40 is a software program configured to perform a specific function or group of functions. The application 40 may include one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, related data, or a portion thereof adapted to be implemented in a suitable computer-readable program code. The application 40 may be stored within the memory 36 or in an additional or separate memory 36. Examples of the application 40 include an audio or video streaming service, an audio or visual processing service, a roadside camera 12 calibration application 40, and the like. Even when disposed on a fixed structure as described herein, the roadside camera 12 is calibrated in order to properly operate and provide accurate and precise physical location information of the object 16 to an advanced driver assistance system (ADAS) communicating with the roadside camera 12. Calibration is particularly important for newly installed roadside cameras 12 and roadside cameras 12 whose physical location has changed. In some examples, the physical, geometric location of the roadside camera 12 may change due to weather (high wind speeds, snow, hail, etc.), collision of a vehicle 19 with a roadside camera 12 mounted on infrastructure, construction or modification of the road 28, etc. For a roadside camera 12 whose physical, geometric location has changed, it may not be possible to accurately report the location of the object 16 within the first field of view 18. Therefore, in order to correct for changes in the physical location of the roadside camera 12, especially for a roadside camera 12 used in an ADAS process, a calibration process is performed by conducting a survey to determine the physical location of the object 16 relative to the roadside camera 12 and within the first field of view 18 of the roadside camera 12. Now turn to Figure 2 And continue to refer to Figure 1 , a coordinate system for calibrating a roadside camera 12 using the system 10 of the present disclosure is shown in more detail. Specifically, a camera coordinate system 50 and a world coordinate system 52 are shown in more detail. The camera coordinate system 50 is depicted in an exemplary but non-limiting pose corresponding to the physical position and angle of the first field of view 18. The camera coordinate system 50 is applied in an image plane 54, wherein a plurality of image points 56 are determined in the image plane 54. Each of the plurality of image points 56 is defined by a coordinate in the camera coordinate system 50 such that x i is the x position, y i is the Y position, and f is the focal length from the roadside camera 12. That is, f is actually the Z position of the image point 56. In several respects, the image point 56 corresponds to the physical position of the object 16 in the real world. World coordinate system 52 includes the physical location of object 16, where X s is the X position, Y s is the Y position, and Z sis the Z position of one of the objects 16 in the world coordinate system 52. In several aspects, a transformation is performed between the camera coordinate system 50 and the world coordinate system 52 to generate a world camera homography matrix "H" that takes a point position in the real world coordinate system 52 and translates that point to the camera coordinate system 50 according to the calibration matrix: 1. Where R is the rotation amount, and C is the distance from the projection center of the world coordinate system 52 to the camera coordinate system 50. 2. where α x =fk x And α y =-fk y ; and x pix =u′ / w′ and y pix =v′ / w′, so that: 3. where α x and α y is the focal length (in pixels), x0 and y0 are the coordinates of the image center (in pixels), “s” is the tilt parameter describing the viewing angle and position of the roadside camera, and K[I3|03] is the calibration matrix. Now turn to Figure 3 And continue to refer to Figure 1 and Figure 2 , the world coordinates of the distance sensor 14 and the roadside camera 12 are known and schematically shown as C0 (world coordinate of the roadside camera 12) and R0 (world coordinate of the distance sensor 14). In the example where the roadside camera 12 and the distance sensor 14 are located at the same location, it should be understood that C0=R0. The distance sensor 14 is used to detect the radial extent of the object in the second field of view 24 and calculate the coordinate in the world coordinate system 52 using the radius r and the world coordinate r0 of the distance sensor 14. In several aspects, the radius r is the horizontal distance or lateral distance from the fixed position of the roadside camera 12 to the object 16 at the known position in the world coordinate system 52. The vertical height h1 of the roadside camera 12 defines the height above the ground where the roadside camera 12 is fixedly installed. Applying the Pythagorean theorem, h1 defines one adjacent side of the right triangle, r defines the second adjacent side of the right triangle, and r d is the hypotenuse of a right triangle. In several ways, r d is the radial range or radial distance between the roadside camera 12 and the object 16 in the world coordinate system 52. Therefore, it should be understood that And therefore In the example where the distance sensor 14 and the roadside camera 12 are located at the same location, the radius r, the height h1 and the radial range or radial distance r d The same is true for each of the distance sensor 14 and the roadside camera 12. However, it should be understood that the distance sensor 14 and the roadside camera 12 need not be located at the same location. In several aspects, the distance sensor 14 can be located near the roadside camera 12 or can be displaced a considerable distance from the roadside camera 12 without departing from the scope or intent of the present disclosure. That is, in some examples, the distance sensor 14 and the roadside camera 12 can be located at the same corner of the intersection 32 and / or fixedly mounted to the same physical infrastructure component and at or near the same physical location of the physical infrastructure component. In other examples, the distance sensor 14 and the roadside camera 12 can be located at opposite corners of the intersection 32, at different heights, etc. When the distance sensor 14 and the roadside camera 12 are not located at the same location, that is, when the distance sensor 14 and the roadside camera 12 are displaced by an arbitrary distance, a mathematical transformation can be used to align the first field of view 18 and the second field of view 24 so that an object 16 detected in each of the first field of view 18 and the second field of view 24 is consistently located by the distance sensor 14 and the roadside camera 12 of the system 10. To accurately calibrate the roadside camera 12, the system 10 executes control logic stored in the memory 36 to transform data captured by one or more of the roadside camera 12 and the range sensor 14 via the world camera homography matrix "H" to find matching objects 16 within the first and second fields of view 18, 24. The system 10 is then calibrated using representative coordinates of the matching objects 16 in the camera coordinate system 50 and the corresponding calculated coordinates in the world coordinate system 52. It should be understood that roadside cameras 12 and range sensors 14 (e.g., RADAR and LiDARS) utilize different mechanisms to sense or "see" objects 16. Therefore, data corresponding to a given object 16 and obtained by each different sensor 14 or roadside camera 12 may not directly match data obtained by other sensors 14 or roadside cameras 12. However, this limitation can be overcome by utilizing contextual information (e.g., the geometry of the road 28) and focusing only on a subset of all possible scenes to simplify the data set and increase the probability of finding a matching object 16. Therefore, the system 10 executes control logic to reduce the number of detected data points or objects 16 so that the matching process is only performed on a subset of all detected objects 16. Now turn to Figure 4 And continue to refer to Figures 1 to 3, several vignettes or frames 100, 102, and 104 are shown. Based on the geometry of the road 28 and known allowed motions of the vehicle 19, regions of interest (ROI), motions of interest (MOI), objects of interest (OOI), and relative positions of interest (POI) are specified as constraints in the first field of view 18 and the second field of view 24 of the roadside camera 12 and the range sensor 14, respectively. The ROI is a predefined subset or sub-region of the first field of view 18 and the second field of view 24 that includes data for which the system 10 has a high degree of confidence that the objects 16 within the ROI are likely to assist rather than hinder the calibration process of the roadside camera 12. In several aspects, the ROI includes an area in which the roadside camera 12 and the distance sensor 14 have high accuracy and precision, such that the detected objects 16 therein are at a well-defined location with well-defined motion characteristics. It should be understood that the ROI is an area extending from the location of the roadside camera 12 and the location of the distance sensor 14 to a predetermined high accuracy, high precision distance therefrom. When the area of ​​the predetermined high accuracy, high precision distance is exceeded, the detected objects 16 are more likely to be obscured, occluded, or detected less accurately due to optical interference, electromagnetic interference, roadside camera 12 focal length, and other such factors. Similarly, the MOI includes the motion of the object 16 within the first field of view 18 and the second field of view 24 that is highly predictable, consistent, and unlikely to adversely affect the calibration process of the roadside camera 12. That is, the MOI may include, but is not necessarily limited to: the object 16 moving in a straight line or in a specific predefined direction, etc. In contrast, a vehicle 19 moving erratically, performing multiple lane changes, accelerating or decelerating suddenly, or performing other such actions is unlikely to produce good data for calibration purposes. Therefore, such irregularly moving vehicles or other such objects 16 do not define an MOI. OOIs include objects 16 within the first field of view 18 and the second field of view 24 that are easily viewed by the roadside camera 12 and the distance sensor 14 without being obstructed, obfuscated, etc. That is, OOIs are objects 16 that are not obscured or partially obscured by other objects 16, infrastructure, adverse weather conditions, or other optical, electromagnetic, or audio impairments or conditions. Because obscured or partially obscured objects 16 are less "visible" to the roadside camera 12 and / or the distance sensor 14, such obscured or partially obscured objects 16 are less clearly defined in the data captured by the roadside camera 12 and / or the distance sensor 14. Therefore, data associated with such obscured or partially obscured objects 16 is less reliable than similar data associated with unobstructed objects 16 used for calibration of the roadside camera 12. POIs include the locations of objects 16 of particular interest to the calibration process of the roadside camera 12. Such POIs may include locations within the ROI, such as lane lines, road 28 signs, trees, traffic signals, telecommunication poles, fire hydrants, curbs, or other such objects 16 with well-defined locations, etc. Such POIs provide well-defined, easily recorded, and reliable data for comparison between roadside camera 12 data and distance sensor 14 data for calibration purposes. However, it should be understood that although POIs have been described above as generally being immovable locations of infrastructure, other movable locations may also be POIs without departing from the scope or intent of the present disclosure. For example, a POI may include the location of an object 16 that can be detected by both the roadside camera 12 and the distance sensor 14 within the first field of view 18 and the second field of view 24, and the location has been identified as an OOI within the ROI, and / or has an MOI. In several aspects, based on ROI, MOI, OOI and POI, the system 10 focuses on a specific section of the road 28, a specific object 16 (e.g., only cars, etc.), and the object 16 is located in a simple manner. For example, the simplified terminology can be the left and right sides of a two-lane road 28. To simplify further, the system 10 executes control logic that identifies and matches left and right moving objects 16. The system 10 then filters complex scenes by specifying motion constraints. In some examples, motion constraints may include, but are not limited to: only straight-line motion, only motion in a specific direction, discarding data containing lane changes, etc. However, other constraints may be used without departing from the scope or intent of the present disclosure. In addition, in order to avoid occlusion-related issues where an object 16 in the foreground occludes or partially occludes an object 16 behind it, the system 10 only focuses on objects 16 "in front" of the ROI or within the ROI. That is, the system 10 focuses on objects 16 that have no other objects 16 in front of them. In more complex scenarios, such as where the road 28 has multiple lanes of traffic going in more than two directions at a given intersection 32, advanced algorithms such as bipartite graph or Hungarian matching algorithms are utilized to improve the accuracy and precision of matching objects 16 in data from multiple sensor sources, including data from roadside cameras 12 and range sensors 14. Referring again more specifically to the first, second and third frames 100, 102 and 104, the first frame 100 is a depiction of the road 28 with a plurality of first objects 16' arranged thereon. Each first object 16' moves in accordance with a direction arrow 106. A second group of objects 16" is also shown, and the second group of objects 16" is located behind the first objects 16 relative to the direction of movement 106 of the first objects 16'. It should be understood that there is nothing in front of the first objects 16' and that the first objects 16' are moving generally toward the roadside camera 12 and the distance sensor 14. The second frame 102 is a depiction of the first field of view 18 of the roadside camera 12, and contains road markings and the first object 16', but in an oblique perspective corresponding to the pose of the roadside camera 12 relative to the real world and the world coordinate system 52. Similarly, the third frame 104 is a depiction of the second field of view 24 of the range sensor 14. In the example shown in the third frame 104, the range sensor 14 is a RADAR sensor, and although the RADAR sensor has detected the same first object 16', they are depicted as very small points 108 that define the radar cross section of the first object 16'. Therefore, the data detected by each of the roadside camera 12 and the RADAR sensor (range sensor 14) are different from each other, but are similarly positioned. It should be understood that the classification of objects 16 in the roadside camera 12 data can be used with radar cross section (RCS) data in the RADAR sensor data to confirm that the objects 16 match. For example, the RCS of a truck may be significantly greater than the RCS of a car. Similarly, cars and trucks can be distinguished in the data of the roadside camera 12. Combining information from the roadside camera 12 and the range sensor 14 (such as a RADAR sensor) not only improves the accuracy and precision of locating objects 16 within the first field of view 18 and the second field of view 24, but also improves the accuracy and precision of the movement of such objects 16 and confirming that the objects 16 match. Now turn to Figure 5 And continue to refer to Figures 1 to 4 , a flow chart depicting a method 200 for calibrating a roadside camera 12 using the system 10 of the present disclosure is shown. The method begins at block 202. At block 204, the system 10 executes control logic that causes the roadside camera 12 to obtain image and / or video graphics data of a first field of view 18. At block 206, the system 10 executes control logic that applies a first ROI filter to the image and / or video graphics data from the roadside camera 12. The first ROI filter constrains the image and / or video graphics data from the roadside camera 12 to a predefined sub-region or ROI of the first field of view 18. The method 200 then proceeds to block 208. At block 208, the system 10 executes control logic that applies a first MOI filter to the image and / or video graphics data within the ROI of the first field of view 18. The first MOI filter further constrains the data contained within the image and / or video graphics data by selecting only highly predictable and consistent motions of the object 16. In some examples, the first MOI filter selects motion detected along a straight line, along a specific predefined direction, etc. The method 200 then proceeds to block 210. At block 210, the system 10 executes control logic to apply a first OOI filter to the image and / or video graphics data within the ROI and corresponding to the MOI filtered data. The first OOI filter further constrains the data within the image and / or video graphics data by selecting only objects 16 within the ROI that have an MOI and are easily viewed by the roadside camera 12 without being obscured, obscured, etc. That is, an OOI is an object 16 that is not obscured or partially obscured by other objects 16, infrastructure, adverse weather conditions, or other optical, electromagnetic, or audio impairments or conditions. The method 200 then proceeds to block 212. At block 212, the system 10 executes control logic that applies a first POI filter or relative positioning filter to the image and / or video graphics data from the roadside camera 12. The first POI filter further constrains the data within the image and / or video graphics data by selecting only objects 16 within the ROI that have an MOI and have been identified as OOIs, and further identifies the locations of objects 16 that are of particular interest to the roadside camera 12 calibration process. Such POIs may include locations within the ROI, such as lane lines, road 28 signs, trees, traffic signals, utility poles, fire hydrants, curbs, or other such objects 16 with well-defined locations, etc. Such POIs provide well-defined, easily recorded, and reliable data for comparison between the roadside camera 12 data and the distance sensor 14 data for calibration purposes. At block 214, the system 10 executes control logic that causes the range sensor 14 to obtain range sensor data for the second field of view 24. At block 216, the system 10 executes control logic that applies a second ROI filter to the range sensor 14 data. The second ROI filter constrains the range sensor 14 data to a predefined sub-region or ROI of the second field of view 24. The method 200 then proceeds to block 218. At block 218, the system 10 executes control logic that applies a second MOI filter to the range sensor 14 data within the ROI of the second field of view 24. The second MOI filter further constrains the data contained within the range sensor 14 data by selecting only highly predictable and consistent motions of the object 16. In some examples, the second MOI filter selects motion detected along a straight line, along a particular predefined direction, etc. The method 200 then proceeds to block 220. At block 220, the system 10 executes control logic that applies a second OOI filter to the distance sensor 14 data within the ROI and corresponding to the MOI filtered data. The second OOI filter further constrains the data within the distance sensor 14 data by selecting only objects 16 within the ROI that have an MOI and are easily viewed by the roadside camera 12 without being obscured, obfuscated, etc. That is, an OOI is an object 16 that is not obscured or partially obscured by other objects 16, infrastructure, adverse weather conditions, or other optical, electromagnetic, or audio impairments or conditions. The method 200 then proceeds to block 222. At block 222, the system 10 executes control logic that applies a second POI filter or relative positioning filter to the distance sensor 14 data. The second POI filter further constrains the data within the distance sensor 14 data by selecting only objects 16 within the ROI that have an MOI and have been identified as an OOI, and further identifies the locations of objects 16 of particular interest to the roadside camera 12 calibration process. Such POIs may include locations within the ROI, such as lane lines, road 28 signs, trees, traffic signals, telecommunication poles, fire hydrants, curbs, or other such objects 16 with well-defined locations, etc. Such POIs provide well-defined, easily recorded, and reliable data for comparison between the roadside camera 12 data and the distance sensor 14 data for calibration purposes. It should be understood that although blocks 204 to 222 have been described sequentially herein, the first and second ROI filters may be executed in parallel, the first and second MOI filters may be executed in parallel, the first and second OOI filters may be executed in parallel, and the first POI filter and the second POI filter may be executed in parallel without departing from the scope or intent of the present disclosure. Likewise, the first and second ROI, MOI, OOI, and POI filters may be performed in an order different than that described herein, or may be performed entirely simultaneously and / or in parallel without departing from the scope or intent of the present disclosure. At block 224, the system 10 executes matching control logic to determine whether an object 16 identified as satisfying the first ROI, MOI, OOI, and POI filters and the second ROI, MOI, OOI, and POI filters has been detected. When it is determined that an object 16 that satisfies both the first and second ROI, MOI, OOI, and POI filters has been detected, the method 200 proceeds to block 226, where one or more objects 16 that have satisfied one or more of the first ROI, MOI, OOI, and POI filters and also satisfy one or more of the second ROI, MOI, OOI, and POI filters are stored in the memory 36 for calibration of the roadside camera 12. In several aspects, in order to properly calibrate the roadside camera 12, the first and second ROI, MOI, OOI, and POI filters satisfy a plurality of different radial ranges or radial distances r from each of the roadside camera 12 and the distance sensor 14. d A plurality of different radial ranges or radial distances r from each of the roadside camera 12 and the distance sensor 14 d By avoiding the possibility of calibration or substantially concentrating the calibration in a first radial range or distance r d The quality and accuracy of the calibration is improved while neglecting the first radial range or distance r d The calibration is actually incorrect for other possible radial ranges or distances r d Therefore, by satisfying the first and second ROI, MOI, OOI and POI filters and being located at various different radial ranges or distances r from the roadside camera 12 and the distance sensor 14 d In a further example, when the roadside camera 12 and the distance sensor 14 are not co-located, the physical size and shape of the object 16 detected and satisfying the first and second ROI, MOI, OOI and POI filters may lead to uncertainty as to whether the detected object 16 is actually a matching object 16. Therefore, to ensure proper matching, at least a portion of the matching control logic includes a time synchronization algorithm or process. The time synchronization algorithm or process can be any of a variety of processes that take into account the physical relative positions of the roadside camera 12 and the distance sensor 14, the image or data capture frequency or periodicity, etc., and the directionality, acceleration and velocity of the object 16 detected and satisfying the first and second ROI, MOI, OOI and POI filters (actually the same object 16 detected in the first and second ROI, MOI, OOI and POI filters). In some examples, the time synchronization algorithm can be executed within the controller 26, or executed within a separate and different time server, or executed within a different time server application, subroutine or control logic within the controller 26. Once the matching objects 16 are determined, the method 200 proceeds to block 228 where the method 200 calibrates the roadside camera 12 by applying a correction factor to the position information of each object 16 detected within the first field of view 18 and satisfying the first ROI, MOI, OOI, and POI filters. The correction factor may be a mathematical weight or other such multiplication factor that takes into account the relative positions of the distance sensor 14 and the roadside camera 12 with respect to the matching object 16. Referring again to block 224, when no matching object 16 is found, the method 200 proceeds to block 230, where the detected object 16 that does not satisfy the first and second ROI, MOI, and POI filters is discarded. From blocks 228 and 230, the method 200 proceeds to block 232, where the method 200 ends. From block 232, the method 200 returns to block 202, where the method 200 is run again. It should be understood that the method 200 can be executed or run automatically, recursively, continuously, periodically, or when a specific condition occurs (e.g., manually initiated by a person) without departing from the scope or intent of the present disclosure. It should also be noted that although the method 200 can be run as described above, in many examples, the method 200 is only executed when the distance sensor 14 is used to calibrate the roadside camera 12. That is, in many cases, the roadside camera 12 only needs to be calibrated occasionally, or when certain conditions occur, such as when a newly installed roadside camera 12 is being calibrated, or the physical, geometric position of the roadside camera 12 changes due to weather changes (high wind speed, snow, hail, etc.), collisions between vehicles 19 and roadside cameras 12 installed on infrastructure, construction or reconstruction of the road 28, etc. In addition, although the calibration process has been described above as being used to calibrate a roadside camera 12, it should be understood that systems 10 and methods 200 that are substantially the same or similar to those described herein may also be used to calibrate distance sensors 14 and cameras 12, etc., without departing from the scope or intent of the present disclosure. The disclosed system 10 and method 200 for calibrating a roadside camera 12 via a distance sensor 14 (or calibrating a distance sensor 14 via a roadside sensor 12) provide a number of advantages, including improved efficiency, reduced complexity, and reduced time and effort required to perform calibration of a roadside camera 12 and / or distance sensor 14 for camera-based advanced driver assistance system (ADAS) applications compared to a manual calibration process. The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A roadside camera calibration system based on a distance sensor, comprising: one or more roadside cameras fixedly mounted to the infrastructure and having a first field of view of at least a portion of the road; one or more distance sensors fixedly mounted to the infrastructure and having a second field of view of at least a portion of the roadway, wherein the first field of view and the second field of view at least partially overlap; and one or more controllers, each of the one or more controllers having a processor, a memory, and one or more input / output (I / O) ports, the I / O ports communicating with the one or more distance sensors and the one or more roadside cameras; The memory stores program control logic; The processor executes the program control logic; The program control logic includes: first control logic for causing each of the one or more roadside cameras to capture image data, and for causing each of the range sensors to capture range sensor data; a second control logic that filters the image data to focus on a first region of interest (ROI) and filters the range sensor data to focus on a second ROI; a third control logic that filters the image data to focus on a first motion of interest (MOI) and filters the range sensor data to focus on a second MOI; a fourth control logic to filter the image data to focus on a first object of interest (OOI) and to filter the range sensor data to focus on a second OOI; a fifth control logic that filters the image data to focus on a first location of interest (POI) and filters the range sensor data to focus on a second POI; a sixth control logic, selectively determining whether the object detected by the roadside camera and the distance sensor satisfies the first and second ROI, MOI, OOI and POI filters, and defining the object as a matching object; A seventh control logic, storing the matching object in the memory for calibration; and An eighth control logic calibrates the roadside camera by applying a correction factor based on the matching object, so that the roadside camera accurately and precisely reports the position and movement of the matching object for use in ADAS applications.

2. The system according to claim 1, wherein: The second control logic further includes: control logic for constraining image data from the roadside camera to a sub-region or ROI within the first field of view in which the image data captured by the roadside camera is reliable and accurate; control logic to constrain range sensor data from the range sensor to a sub-region or ROI within the second field of view in which the range sensor data captured by the range sensor is reliable and accurate; and The ROIs within the first field of view and the second field of view are substantially the same.

3. The system according to claim 1, wherein: The third control logic further includes: control logic constraining the image data to focus on the first MOI, wherein the first MOI includes predictable and consistent movement of objects along the roadway; and Control logic constrains the distance sensor data to focus on the second MOI, wherein the second MOI includes predictable and consistent movement of objects along the roadway.

4. The system according to claim 3, wherein: The predictable and consistent movement of the object along the path also includes: Movement in a set of predefined directions, linear or rectilinear motion, and movement at predefined accelerations and velocities.

5. The system according to claim 1, wherein: The fourth control logic further includes: constraining the image data to define control logic for a first OOI by selecting objects that satisfy the first ROI filter, the first MOI filter, and are directly and unobstructedly viewable by the roadside camera; and The range sensor data is constrained to define control logic for a second OOI by selecting objects that satisfy the second ROI filter, the second MOI filter, and are directly and unobstructedly viewable by the range sensor.

6. The system according to claim 5, wherein: The fifth control logic further includes: a control logic for constraining the image data by selecting only first OOIs located within first ROIs having predefined fixed or moving positions; and constraining the control logic of the range sensor data by selecting only the second OOI at locations within the second ROI having a predefined fixed or moving location; and The predefined fixed or mobile locations include: the locations of lane lines, road signs, trees, traffic signals, telecommunication poles, fire hydrants and / or curbs, and the locations of movable or moving OOIs.

7. The system according to claim 1, wherein: The sixth control logic further includes: Control logic for defining the object as a matching object by comparing objects that simultaneously satisfy the first and second ROI, MOI, OOI, and POI filters and verifying whether one or more objects that simultaneously satisfy the first and second ROI, MOI, OOI, and POI filters are the same object detected by the roadside camera and the distance sensor.

8. The system according to claim 7, wherein: The eighth control logic further includes: Control logic that applies the correction factors to image data captured by the roadside camera, wherein the correction factors are based on the known physical and geometric position of the roadside camera relative to the known physical and geometric position of the distance sensor, and obtains the point position in the real world coordinate system and translates the point to the camera coordinate system according to the calibration matrix K[I3|03] based on the world homography matrix "H":

1. Where R is the rotation amount, and C is the distance from the projection center of the world coordinate system to the camera coordinate system; 2. where α x =fk x And α y =-fk y ; and x pix =u′ / w′ and y pix =v′ / w′, so that:

3. where α x and α y is the focal length in pixels, x0 and y0 are the coordinates of the center of the image in pixels, "s" is the tilt parameter describing the view and position of the roadside camera, and K[I3|03] is the calibration matrix.

9. The system according to claim 8, wherein: Based on the known relative positions of the roadside camera and the distance sensor, and based on the radial range or distance r between the roadside camera and the object in the world coordinate system d The height h1 of the roadside camera from the ground on which the roadside camera is fixedly installed further adjusts the calibration matrix K[I3|03], so that:

10. A roadside camera calibration method based on a distance sensor, comprising: capturing image data with one or more roadside cameras fixedly mounted to the infrastructure and having a first field of view of at least a portion of a roadway; capturing range sensor data with one or more range sensors fixedly mounted to the infrastructure and having a second field of view of at least a portion of the roadway, wherein the first and second fields of view at least partially overlap; executing programmed control logic stored in a memory of one or more controllers in communication with the one or more roadside cameras and the one or more range sensors, each of the one or more controllers having a processor, a memory, and one or more input / output (I / O) ports; The I / O port communicates with the one or more distance sensors and the one or more roadside cameras; the processor executes the program control logic; the program control logic includes control logic for performing the following operations: constraining image data from the roadside camera to a sub-region or region of interest (ROI) within the first field of view, in which the image data captured by the roadside camera is reliable and accurate; constraining the range sensor data from the range sensor to a sub-region or ROI within the second field of view in which the range sensor data captured by the range sensor is reliable and accurate, wherein the ROIs within the first and second fields of view are substantially identical; constraining the image data to focus on a first motion of interest (MOI), wherein the first MOI comprises predictable and consistent motion of an object along the road; constraining the range sensor data to focus on a second MOI, wherein the second MOI includes predictable and consistent movement of objects along the roadway, including movement in a set of predefined directions, linear or straight-line movement, and movement at predefined accelerations and velocities; constraining the image data to define a first object of interest (OOI) by selecting objects that satisfy a first ROI filter, a first MOI filter, and that are directly and unobstructedly viewable by the roadside camera; constraining the image data by the first OOI filter by selecting only first OOIs located at positions within a first ROI having a predefined fixed or moving position; constraining, by the second OOI filter, the range sensor data by selecting only second OOIs that are located at positions within a second ROI having a predefined fixed or moving position and that are directly and unobstructedly viewable by the range sensor; The predefined fixed or mobile locations include: the location of lane lines, the location of road signs, trees, traffic signals, telecommunication poles, fire hydrants and / or curbstones, and the location of movable or moving OOIs; filtering the image data through a first location of interest (POI) filter to focus on a first POI, and filtering the range sensor data through a second POI filter to focus on a second POI; selectively determining whether the objects detected by the roadside camera and the distance sensor satisfy the first and second ROI, MOI, OOI and POI filters at the same time, and defining the objects as matching objects by comparing the objects satisfying the first and second ROI, MOI, OOI and POI filters at the same time and verifying whether one or more objects satisfying the first and second ROI, MOI, OOI and POI filters at the same time are the same objects detected by the roadside camera and the distance sensor; storing the matching object in the memory for calibration; and Calibrate the roadside camera based on the matching object by applying correction factors so that the roadside camera accurately and precisely reports the position and motion of the matching object for use in ADAS applications, wherein the correction factors are based on the known physical and geometric position of the roadside camera relative to the known physical and geometric position of the distance sensor, and obtain the point position in the real world coordinate system and translate the point to the camera coordinate system according to the calibration matrix K[I3|03] based on the world homography matrix "H":

1. Where R is the rotation amount, and C is the distance from the projection center of the world coordinate system to the camera coordinate system; 2. where α x =fk x And α y =-fk y ; and x pix =u′ / w′ and y pix =v′ / w′, so that:

3. where α x and α y is the focal length in pixels, x0 and y0 are the coordinates of the center of the image in pixels, "s" is the tilt parameter describing the view and position of the roadside camera, and K[I3|03] is the calibration matrix.

Images