Automatic calibration method for a monocular camera

Abstract

The present invention relates to a method for the automatic extrinsic calibration of a monocular camera, designed to estimate its position and orientation in space without the use of physical calibration devices. The method comprises acquiring at least one image of a scene including a ground surface, generating a dense depth map using a deep neural network, reconstructing a three-dimensional mesh of the scene, identifying a dominant plane corresponding to the ground, and estimating the camera's position relative to this plane. The method can be applied to fixed or mobile cameras and extends to the metric calibration of a reconstructed 3D model, notably by resizing using extrinsic parameters or visual references such as the average size of detected objects. It also enables the precise localization of objects in the scene by geometric projection onto the ground model.The invention is particularly applicable to the fields of video surveillance, autonomous navigation, urban analysis and 3D modeling.

Description

Title of the invention: Automatic calibration method for a monocular camera. Technical field

[0001] The present invention relates to a method for calibrating an extrinsic parameter of a monocular camera. In particular, it concerns the automatic estimation of the position and orientation of a camera from images of a real scene, without the need for specific calibration devices or on-site physical measurements. Technical background

[0002] Extrinsic calibration of a camera consists of determining its position and orientation in a three-dimensional reference frame, generally relative to an observed scene. This operation is essential for many computer vision applications, such as 3D reconstruction, object geolocation, multi-sensor data fusion, or autonomous navigation.

[0003] Traditional calibration methods rely on the use of physical targets or known geometric patterns (such as checkerboards, spheres, or ArUco markers) arranged in the scene. These targets allow for the establishment of precise correspondences between 2D points detected in the image and their known 3D coordinates in space. From these correspondences, the extrinsic parameters of the camera can be estimated using projective geometry techniques. Although accurate, these methods have several limitations: they require specific scene preparation, on-site human intervention, and are not suitable for uncontrolled environments or permanent installations (e.g., surveillance cameras in urban or highway environments).

[0004] Other methods rely on three-dimensional reconstruction techniques, geometric analysis, or deep learning that exploit only the images captured by the camera, without recourse to physical targets.

[0005] WO 2025 / 012344 Al describes a method for calibrating a stationary monocular camera using a 3D model of the scene previously generated with a mobile camera. Correspondences are established between pixels of an image captured by the stationary camera and elements of the 3D model, allowing the camera's exposure parameters to be estimated. This approach relies on a preliminary mapping phase of the scene, often carried out by a drone or a vehicle equipped with a mobile camera.

[0006] US 2024 / 0371036 Al proposes an on-board calibration method in which a monocular camera mounted on a vehicle captures an image of the road. A neural network estimates a depth map, from which a road profile is reconstructed. This profile, modeled by a polynomial function, allows the image coordinates of an object to be projected onto the real scene, and its three-dimensional position to be deduced. This method is particularly well-suited to road environments but assumes a structured scene and a continuous ground plane.

[0007] US 2021 / 0407117 Al describes an approach for extracting the ground plane from monocular images, using a self-supervised neural network to generate a depth map. A 3D mesh is reconstructed from this map, and surface normals are calculated for each polygon. Upward-pointing normals are interpreted as belonging to the ground, thus allowing the identification of a dominant plane. This method enables unsupervised geometric segmentation of the ground, but remains sensitive to the quality of the reconstruction and the structure of the scene.

[0008] These methods have several limitations. Some require a preliminary active mapping phase of the scene, involving the use of a mobile camera or a third-party device, which limits their applicability in contexts where access to the scene is restricted or unplanned. Others rely on strong assumptions about the scene's structure, such as the presence of a continuous road profile or well-defined flat surfaces, which makes them less robust in unstructured, natural, or sparsely populated environments. Furthermore, most monocular approaches do not allow for direct estimation of the scene's absolute scale, which limits the accuracy of the resulting extrinsic calibration. Summary of the invention

[0009] A first aspect of the invention relates to a method for the automatic calibration of at least one extrinsic parameter of a monocular camera, said method comprising the following steps: - Acquisition of at least one image of a scene including a ground surface; - Generation of a dense depth map using a deep neural network applied to said image or to a sequence of images; - Reconstruction of a three-dimensional mesh of the scene from the depth map; - Identification of a dominant plane corresponding to the ground surface, using an algorithm for grouping local grid normals, weighted according to position in the image and depth; - Estimation of the camera's position and orientation relative to the dominant plane.

[0010] According to some embodiments, the weighting applied during clustering: - Varies with the position of the points in the image, in particular the weighting factor increases for points in the image located in the lower part of the image, and / or; - Is inversely proportional to the depth.

[0011] According to some embodiments, the monocular camera is mobile, and the successive scenes are visually matched between successive images acquired using a point set matching algorithm

[0012] A second aspect of the invention relates to a method for calibrating a three-dimensional model of a scene, comprising: - Estimation of a three-dimensional model from a sequence of image(s) acquired by a monocular camera; - Resizing the three-dimensional model using an extrinsic camera parameter, said extrinsic parameter being calibrated using a method according to any one of the embodiments of the first aspect of the invention.

[0013] According to some embodiments, the resizing of the three-dimensional model is carried out using a scale factor determined from moving objects detected in the scene, by comparing their apparent size to a known theoretical average size.

[0014] According to some embodiments, the resizing is performed using: - the known height of the camera relative to the ground; - of a known actual distance between two points on the ground; - an average size of recurring objects in the scene, preferably vehicles and / or pedestrians, the average size being established using a prior statistical study of the size of recurring objects from a sequence of images acquired over a determined time period

[0015] A second aspect of the invention relates to a method for locating an object in a scene, comprising the following steps: - Estimation and calibration of a 3D model using a method according to any one of the embodiments of the second aspect of the invention; - Projection of at least one point of the object onto the ground surface by ray tracing from the camera, determining the intersection with the 3D model of the ground.

[0016] Another aspect of the invention relates to a data processing device comprising means for implementing a process according to any one of the modes of the first, second and / or third aspect of the invention.

[0017] Another aspect of the invention relates to a computer program comprising instructions which, when executed by at least one processor, cause said processor to implement a process according to any one of the modes of the first, second and / or third aspect of the invention.

[0018] Another aspect of the invention relates to a non-transient storage medium containing instructions to execute a process according to any one of the modes of the first, second and / or third aspect of the invention.

[0019] Another aspect of the invention relates to a monocular camera configured to acquire an image of a scene including a ground surface, said camera being associated with a processing device as described above.

[0020] Urban and / or road surveillance system comprising at least one monocular camera as described above and a data processing device as described above. Brief description of the drawings

[0021] [Fig-1] is a flowchart of a method for calibrating an extrinsic parameter of a monocular camera.

[0022] [Fig.2] is a schematic representation of an image of an example scene acquired by a monocular camera such as a surveillance monocular camera.

[0023] [Fig.3] is a schematic representation of a three-dimensional cloud mesh of points of the scene in the image of [Fig.2].

[0024] [Fig.4] is a schematic representation of the groups of normals to the planes dominant detected from the mesh of [Fig.3]. Detailed description of the implementation methods

[0025] A first aspect of the invention, with reference to [Fig. 1] to [Fig. 4], is a method for automatically calibrating at least one extrinsic parameter of a monocular camera. The method comprises the following steps: - acquisition of at least one image of a scene including a ground surface ([Fig.2]); - generation of a dense depth map using a deep neural network applied to said image or to a sequence of images ([Fig.2]); - reconstruction of a three-dimensional mesh of the scene from the depth map ([Fig.3]); - identification of a dominant plane corresponding to the ground surface, using an algorithm for grouping local normals of the mesh, weighted according to position in the image and depth ([Fig.4]); - estimation of the position and orientation of the camera relative to the dominant plane.

[0026] In the context of the invention, the extrinsic parameters refer to the position (translation) and orientation (rotation) of the camera in three-dimensional space. The depth map is a frame-by-frame representation of the distance between each pixel and the camera, expressed either as absolute values ​​or as inverse depth values. It can be generated by a deep neural network trained in self-supervised or supervised mode, such as Monodepth2, DPT, or any other monocular depth prediction model. These networks exploit visual cues of perspective, texture, and motion to infer the 3D structure of the scene. The three-dimensional mesh is then reconstructed from this map, by triangulation or interpolation, enabling explicit geometric modeling of the scene, suitable for subsequent processing such as plane segmentation or object localization.

[0027] The identification of the dominant plane(s) relies on the analysis of the local mesh normals, calculated from the vectors defining the triangular faces. The normals, consistently oriented upwards in a camera or world frame, are clustered to detect an extended flat surface, interpreted as the ground. This detection is refined by geometric clustering techniques, such as hierarchical clustering or density clustering (e.g., DBSCAN - Density-Based Spatial Clustering of Applications with Noise), applied to the normal vectors or points of the mesh. As illustrated in [Fig. 4], the method according to the invention makes it possible to discriminate between floors on several levels, such as in a shopping mall.

[0028] The method can incorporate spatial weighting during clustering, where points located in the lower part of the image receive a higher weight, reflecting their increased probability of belonging to the ground. Similarly, a weighting inversely proportional to depth can be applied to favor points close to the camera, which are generally more reliable. These weightings allow the clustering algorithm to be directed towards the most relevant areas, improving the robustness of ground plane detection, even in the presence of noise, extraneous objects, or complex geometries, such as multiple levels or floors ([Fig. 4]).

[0029] The camera pose estimation is deduced from the geometry of the detected plane by determining the transformation (rotation and translation) that aligns the dominant plane with a reference plane, such as the horizontal plane of the world. This method allows for automatic calibration, without human intervention or physical devices, and is applicable to real-world scenes, even unstructured ones, including those with topographical variations or moving elements.

[0030] According to some embodiments, the weighting applied during clustering varies with the position of the points in the image, in particular the weighting factor The weighting factor increases for points in the lower part of the image and / or is inversely proportional to depth. The weighting applied during clustering varies dynamically depending on the position of the points in the image and their estimated depth. In particular, the weighting factor increases for points located in the lower part of the image, an area where ground is statistically more present in scenes captured by monocular cameras, whether fixed or mounted. Furthermore, this weighting can also be inversely proportional to depth, so that points close to the camera—generally more reliable in terms of geometric reconstruction—have a greater influence on the segmentation process.

[0031] This dual spatial and metric weighting allows the clustering algorithm to be directed towards the regions most likely to represent the ground. Clustering can be performed on surface normals extracted from the 3D mesh, or directly on the position vectors of the reconstructed points. Algorithms such as DBSCAN, Mean Shift, or variants of weighted K-means can be used to group points according to their orientation and local density. In some cases, a pre-selection of points is performed using confidence maps or segmentation masks from the neural network, in order to reduce the impact of extraneous objects or reconstruction noise.

[0032] The weighting can also incorporate contextual factors, such as temporal consistency in a video sequence, or the stability of normals in a given area. By combining these criteria, the method significantly improves the robustness of ground plane detection, even in complex, irregular, or partially obstructed environments. This approach makes it possible to improve the reliability of camera pose estimation, based on a more stable geometric basis that is more representative of the actual scene.

[0033] According to some embodiments, the monocular camera is mobile, and successive scenes are visually matched between successive images acquired using a point-set matching algorithm. These points can, for example, be detected using robust visual descriptors (such as SIFT, ORB, or SuperPoint), and then matched between successive images using local or global matching techniques, possibly enhanced by neural networks specialized in detectorless matching.

[0034] The method can be extended to cameras mounted on vehicles, drones, or pedestrians, where the camera is subject to continuous movement. Visual matching between successive images makes it possible to link the local reconstructions in a coherent spatial reference frame by estimating the rigid transformations between successive poses. This estimation is based on SLAM (Simultaneous) techniques. Localization and Mapping) monocular, which combine visual feature detection, motion estimation (egomotion) and incremental scene reconstruction.

[0035] Monocular SLAM makes it possible to construct a time chain of camera poses, even in the absence of depth sensors or GPS. Although the absolute scale of the scene remains indeterminate in a purely monocular setting, the relative structure is preserved, which makes it possible to align local reconstructions and aggregate geometric information over the entire sequence.

[0036] These embodiments can make the process applicable to dynamic videos, such as those from mobile cameras, and allow continuous or periodic extrinsic calibration, suitable for evolving environments or embedded systems requiring regular updating of their perception geometry.

[0037] A second aspect of the invention relates to a method for calibrating a three-dimensional model of a scene, comprising: - the estimation of a three-dimensional model from a sequence of image(s) acquired by a monocular camera, said estimation being able to rely on structural reconstruction techniques from motion (Structure from Motion) or monocular SLAM, allowing to generate a relative 3D representation of the scene, typically in the form of a point cloud or polygonal mesh; - resizing the three-dimensional model using an extrinsic camera parameter, such as its height relative to the ground or its position in a known reference frame, said extrinsic parameter being previously calibrated using a method according to any one of the embodiments of the first aspect of the invention.

[0038] This embodiment allows the conversion from a relative 3D model, that is, one expressed on an arbitrary scale without physical units, to a metric model, in which distances, dimensions, and positions are expressed in an absolute unit (for example, meters). This conversion is made possible by the introduction of a scaling factor derived from the extrinsic calibration of the camera. Indeed, estimating the camera's position and orientation relative to a reference plane, such as the ground, provides a reliable geometric basis for anchoring the reconstructed model in a physical reference frame. The resizing can be performed by affine or homogeneous transformation of the 3D model, applying a global or local scaling factor, depending on the characteristics of the scene.This resulting metric model can be used for precise physical measurements, geometric analyses, or applications for tracking and locating objects in real-world environments.

[0039] According to some embodiments, the resizing of the three-dimensional model is carried out using a scale factor determined from objects Mobile objects detected in the scene are scaled by comparing their apparent size to a known theoretical average size. These objects are identified in images using visual detection techniques, possibly assisted by convolutional neural networks or semantic segmentation algorithms. Once detected, their apparent size in the reconstructed 3D model is measured, typically by extracting the height between the lowest and highest points of their 3D envelope. This measurement is then compared to a known theoretical average size, derived from statistical data or embedded knowledge bases, to estimate an overall scaling factor applicable to the entire model.

[0040] As an example, the average height of a pedestrian (approximately 170 cm) or a light vehicle (approximately 1.5 m to 1.8 m) can be used as a reference. From a measurement of the apparent height of these objects in the reconstructed 3D model, it is possible to deduce a scale factor that allows the relative dimensions of the model to be converted into metric units. This approach has the advantage of being fully automated, without requiring human intervention or prior knowledge of the scene. It relies on the exploitation of statistical regularities in the observed environment and can be enhanced by aggregating measurements on several similar objects detected in different images or at different times. The scale factor thus obtained allows the 3D model to be reliably calibrated, even in the absence of fixed physical reference points or direct measurements, making the system particularly well-suited to dynamic urban or road environments..

[0041] According to some embodiments, the resizing is carried out using known or estimated geometric information in the scene, allowing the reconstructed model to be anchored in a metric reference frame, in particular it is carried out using: - the known height of the camera relative to the ground; - of a known real distance between two points on the ground; or - of an average size of recurring objects in the scene, preferably vehicles and / or pedestrians, the average size being established using a prior statistical study of the size of recurring objects from a sequence of images acquired over a determined time period.

[0042] These three approaches are complementary. The first consists of using the known height of the camera relative to the ground, measured during installation or estimated from manufacturer data. This height, combined with the camera's orientation and the geometry of the detected ground plane, allows for the deduction of an overall scale factor. The second approach relies on knowing the actual distance between two points on the ground visible in the scene, such as road markings, pedestrian crossings, or fixed infrastructure elements. This distance can be used as a reference to adjust the dimensions of the reconstructed model. Finally, a The third approach exploits the average size of recurring objects present in the scene, such as vehicles or pedestrians. This average size can be determined from a prior statistical study, carried out on a sequence of images acquired over a defined period of time, allowing for the aggregation of observations and the smoothing of individual variations.

[0043] These embodiments offer great flexibility in the information sources used to calibrate the scale, adapting to the constraints of the acquisition context. For example, in the case of a fixed camera installed at a known height, calibration can be performed immediately after ground plane detection. In urban or road environments, the presence of road markings or standard objects allows for the use of visual reference measurements without human intervention. The use of recurring objects, on the other hand, allows for progressive and autonomous calibration, based on continuous observation of the scene. By combining these different sources, the method improves the accuracy and robustness of the resizing, while maintaining adaptability to varied scenes, whether structured or unstructured, static or dynamic.

[0044] A third aspect of the invention relates to a method for locating an object in a scene, comprising the following steps: - estimation and calibration of a 3D model using a method according to any one of the embodiments of the second aspect of the invention, so as to obtain a three-dimensional metric and spatially coherent representation of the observed environment; - projection of at least one point of the object onto the ground surface by ray tracing from the camera, determining the intersection with the 3D model of the ground.

[0045] This method allows for the precise localization of an object (such as a pedestrian, a vehicle, or luggage) within the scene, using physical coordinates, i.e., in a three-dimensional metric reference frame. Ray casting consists of tracing a vector from the optical center of the camera to a point of interest in the image, generally located on the detected object, such as its center of mass or a feature point. This vector is then intersected with the 3D ground model, typically represented as a polygonal mesh or a fitted flat surface. The intersection provides an estimate of the object's ground position, expressed in the same reference frame as the 3D model.

[0046] This technique enables robust localization, even in the absence of depth sensors or physical beacons, and is applicable to dynamic or partially structured scenes. It is particularly useful in surveillance, object tracking, or intrusion detection contexts, where precise knowledge of an object's position relative to its environment is essential. The method can can be extended to multi-object localization, trajectory prediction or behavior analysis, by combining projected positions with temporal information from a video sequence.

[0047] The invention also covers the following aspects: - a data processing device comprising means for implementing a process according to any one of the embodiments of the first aspect of the invention: this device may be a computer, a server, or an embedded module, integrating image processing, 3D reconstruction, and calibration algorithms. - a computer program comprising instructions for executing any one of the processes described above; - a non-transient storage medium containing instructions to execute any of the processes described above.

[0048] Another aspect of the invention relates to a monocular camera configured to acquire an image of a scene including a ground surface, associated with a data processing device as described above.

[0049] Another aspect of the invention relates to an urban and / or road surveillance system integrating a camera and a data processing device as described above, for crowd modeling, traffic violation detection or automatic calibration of video surveillance cameras.

[0050] The various aspects of the invention can find applications in video surveillance systems, urban analysis, intelligent transport, road safety or even people flow management.

Claims

Demands

1. A method for automatically calibrating at least one extrinsic parameter of a monocular camera, said method comprising the following steps: - Acquisition of at least one image of a scene including a ground surface; - Generation of a dense depth map using a deep neural network applied to said image or to a sequence of images; - Reconstruction of a three-dimensional mesh of the scene from the depth map; - Identification of a dominant plane corresponding to the ground surface, using an algorithm for grouping the local normals of the mesh, weighted according to position in the image and depth; - Estimation of the position and orientation of the camera with respect to said dominant plane.

2. The method according to claim 1, wherein the weighting applied during clustering: - Varies with the position of the points in the image, in particular the weighting factor increases for points in the image located in the lower part of the image, and / or; - Is inversely proportional to the depth.

3. A method according to any one of claims 1 to 2, wherein the monocular camera is mobile, and successive scenes are visually matched between successive images acquired using a point-set matching algorithm

4. A method for calibrating a three-dimensional model of a scene, comprising: - Estimating a three-dimensional model from a sequence of image(s) acquired by a monocular camera; - Resizing the three-dimensional model using an extrinsic parameter of the camera, said extrinsic parameter being calibrated using a method according to any one of claims 1 to 3.

5. Calibration method according to claim 4, wherein the resizing of the three-dimensional model is carried out using of a scale factor determined from moving objects detected in the scene, by comparing their apparent size to a known theoretical average size.

6. A calibration method according to claim 4, wherein the resizing is performed using: - the known height of the camera relative to the ground; - a known actual distance between two points on the ground; - an average size of recurring objects in the scene, preferably vehicles and / or pedestrians, the average size being established using a prior statistical study of the size of recurring objects from a sequence of images acquired over a predetermined time period

7. A method for locating an object in a scene, comprising the following steps: - Estimating and calibrating a 3D model using a method according to any one of claims 4 to 6; - Projecting at least one point of the object onto the ground surface by ray tracing from the camera, determining the intersection with the 3D model of the ground.

8. Data processing device comprising means for implementing a method according to any one of claims 1 to 7.

9. A computer program comprising instructions which, when executed by at least one processor, cause said processor to implement a method according to any one of claims 1 to 7.

10. Non-transient storage medium containing instructions to perform a process according to any one of claims 1 to 7.

11. Monocular camera configured to acquire an image of a scene including a ground surface, said camera being associated with a processing device according to claim 10.

12. Urban and / or road surveillance system comprising at least one camera according to claim 11 and a data processing device according to claim 8.

13. Use of the system according to claim 12 for crowd modeling, traffic violation detection or automatic calibration of CCTV cameras.

Images