Calibration method for surveying instrument and system for calibrating surveying instrument

Abstract

A method of calibrating a camera, the camera comprising an image sensor and a diffraction stop projecting light onto the image sensor, the method comprising: providing a camera model mathematically describing the camera; placing the camera at a first rotation position relative to the plurality of targets, and recording a first image of the plurality of targets; rotating the camera to a second rotation position relative to the target, and recording a second image; determining a first set of viewing angles of the target from the first image and a second set of viewing angles from the second image using a camera model; determining a function based on the first and / or second set of viewing angles and a set of difference angles representing a difference between the first and second viewing angles; determining camera model errors according to the function, and comparing the errors with a limit value; and if the error is not lower than the limit value, updating the camera model based on the error, and repeating the steps of determining and comparing. The invention releases insight from geographic data and also relates to improvements in sustainability and environmental development: we together create a safe and livable world.

Description

Technical Field

[0001] The invention relates to a method for calibrating a camera of a computer vision system, in particular a camera of a surveying instrument, wherein the camera has a diffraction stop as an objective lens. The invention also relates to a system or a device for calibrating a camera. The invention unlocks insights from geographic data and also relates to improvements in sustainability and environmental development: together we create a safe and liveable world. Background Art

[0002] Systems and methods for measuring the position of remote objects are already known, for example in the field of surveying and / or monitoring of structures. Conventionally, the cameras of such systems use lenses or refractive optical elements in the camera objective.

[0003] WO 2019 / 143249 A1, WO 2019 / 143250 A1 and WO 2021 / 167452 A2 disclose an apparatus and method for monitoring the position of an object, such as a structure, over time. The disclosed system includes a beacon or target located on the object and monitored by a camera to detect changes or fluctuations in the position of the target over time.

[0004] WO 2019 / 143250 A1 and WO 2021 / 167452 A2 disclose a system using a non-refractive optical element such as a pinhole as a camera objective. It has been observed that this provides a number of advantages over lens-based camera systems, including reduced optical distortion, improved depth of field, and reduced thermal sensitivity. For such systems, thermal sensitivity, for example due to thermal expansion, is believed to be primarily affected by the thermal expansion coefficient of the housing of the camera, rather than by thermal effects on the objective itself, as is the case with lens-based systems.

[0005] However, the observation accuracy of camera-based surveying instruments is affected not only by temperature-related contraction and expansion of the camera housing and other components of the camera, but also by mechanical manufacturing tolerances, such as the position of the lens or aperture relative to the optical sensor.

[0006] Therefore, like any other measurement system, a system using non-refractive optical elements or diffraction stops as camera objectives needs to be calibrated to provide accurate measurement results. Calibration can be used to compensate for deviations from expected values ​​of the physical parameters that cause observation errors.

[0007] However, conventional survey instrument calibration methods are not satisfactory because they are primarily aimed at compensating for optical errors in the lens. Using conventional calibration methods, the instrument being calibrated observes a scene with defined measurement points. When using these methods, at least one of the following requirements must be met:

[0008] The location of the measurement points in the scene relative to the survey instrument must be known accurately;

[0009] The relative directions of the measurement points must be known accurately.

[0010] Therefore, conventional calibration methods require the exact known absolute positions of many points in the pattern that are in the field of view of the surveying instrument, or the exact known orientations of many points in the scene. This means that in order for the instrument being calibrated to achieve very high accuracy, the pattern itself also needs to be calibrated to very high accuracy. This is difficult to achieve.

[0011] Furthermore, conventional methods are designed for calibrating lens-based optical systems, which have their own set of dynamic errors that limit the ultimate accuracy that can be achieved. In measurement systems that use objectives based on non-refractive optical elements or diffraction stops instead of lenses, these errors are absent or at least differ to a greater extent.

[0012] The presence of such errors has a detrimental effect on the accuracy of the monitored structure and the resulting uncertainties. Due to these uncertainties, the structure may be affected by early maintenance projects or by over-parameterization during the design phase, instead of only by factors that are strictly necessary. This may have negative consequences for the environment. Furthermore, in cases where the movements indicated by the measurements are smaller than the actual movements, such an underestimation of the structural health may also have significant safety consequences.

[0013] Diffraction-stop based cameras do not have any lens-based dynamic distortion, making it possible to measure the position changes of a target with very high accuracy. However, to achieve this high accuracy, very precise calibration methods are required.

[0014] Therefore, there is a need to define a calibration method for surveying instruments based on diffraction stops. Summary of the invention

[0015] The problem addressed by the present invention relates to providing a method and a system which enable the calibration of a camera using a non-refractive optical element such as a diffraction stop or a pinhole as a camera objective for projecting an image onto an image sensor.

[0016] More specifically, the object of the invention is to overcome the disadvantages of the prior art and to allow calibration of a camera provided with a diffraction stop for projecting an image onto an image sensor.

[0017] According to a first aspect of the present invention, there is provided a method for calibrating a camera of a computer vision system, the camera comprising an image sensor and a diffraction stop arranged to project light onto the image sensor, the method comprising the following steps:

[0018] (a) Distribute multiple targets within the camera's field of view;

[0019] (b) providing a camera model representing a mathematical model of the camera;

[0020] (c) placing the camera at a first rotational position relative to the target, and recording a first image of the plurality of targets using the camera at the first rotational position;

[0021] (d) determining a first set of calculated viewing angles from the first image using the camera model, the first set of calculated viewing angles comprising a first calculated viewing angle of each target relative to the camera;

[0022] (e) rotating the camera relative to the target to a second rotational position, and recording a second image of the plurality of targets using the camera at the second rotational position;

[0023] (f) determining a second set of calculated viewing angles from the second image using the camera model, the second set of calculated viewing angles comprising a second calculated viewing angle of each target relative to the camera;

[0024] (g) determining a function or graph based on the first set of calculated observation angles and / or the second set of calculated observation angles and a set of difference angles representing the difference between the first calculated observation angle and the second calculated observation angle for each target;

[0025] (h) determining a camera model error according to the function or graph;

[0026] (i) comparing the camera model error with a predetermined limit;

[0027] (j) if the camera model error is below the limit, determining that the camera model is acceptable;

[0028] (k) If the camera model error is not lower than the limit, an updated camera model is calculated based on the camera model error, and steps d), f), g), h), i) and j) or k) are repeated based on the updated camera model.

[0029] The camera model comprises a mathematical representation of the camera, in particular a mathematical model containing parameter values ​​describing the relationship between the diffraction stop and the pixels of the image sensor. As an initial model, a mathematical model representing the design specifications of the camera can be used. Alternatively, a camera model determined in a previous calibration can be used.

[0030] The method is an iterative method of calculating an updated, ie calibrated, camera model, wherein the calculation is performed repeatedly until the camera model error or parameter offset value is within predetermined limits or thresholds.

[0031] The specific amount by which the camera is rotated between two measurements, ie when moving from a first rotational position to a second rotational position, is not important and does not need to be known. The measured change in the viewing angle of each target forms the basis for the calculation of the calibration parameters.

[0032] The rotation from the first rotational position to the second rotational position preferably comprises a substantially pure rotational movement. However, due to reasons such as the axis of rotation not being located at the nominal position of the diffraction stop, the movement may also comprise a translational movement. The influence of the translational movement on the calibration result may be minimized by positioning the target at a sufficiently large distance. Alternatively, the translational movement may be determined and corrected by an algorithm during the calibration process.

[0033] The order of the steps listed above is not required, however, as will be appreciated by those skilled in the art, the order of the steps may be changed. For example, although step (d) of determining the first set of calculated viewing angles is mentioned before step (e) of rotating the camera to the second rotational position, the step of rotating to the second rotational position and recording the second image may be performed before determining the first set and the second set of calculated viewing angles.

[0034] Compared to conventional calibration methods, with the calibration method of the present disclosure, there is no need for precisely known positions or orientations of calibration patterns such as target arrays. The calibration method is based on the fact that the observed change in viewing angle caused by the rotation of the camera should be equal for all targets, regardless of the orientation or position of each target.

[0035] The optical axis of the camera is defined in a conventional manner. The optical axis of the camera can be represented by an imaginary line passing through the center of the diffraction stop and orthogonal to the plane of the image sensor.

[0036] The rotation from the first rotational position to the second rotational position is typically a rotation of about 1 gon, 1 degree or a few degrees. Although in principle the camera can be rotated by a greater angle between measurements, as described in further detail below, it has been found to be advantageous to arrange the targets at positions close to the periphery of the camera's field of view. The amount of rotation should then be chosen so that targets at opposite peripheries can both be detected by the camera in the first rotational position and in the second rotational position.

[0037] The actual or true viewing angle is generally defined as the separation angle of the target relative to the optical axis of the camera. That is, the viewing angle can be defined as the angle formed between an imaginary line extending from the target to the center of the aperture and the optical axis of the camera.

[0038] The calculated viewing angle is the angle calculated using an algorithm that takes as input the projection of the target on the image sensor and the camera model. Such an algorithm may use techniques as further detailed below.

[0039] The actual viewing angle and the calculated viewing angle are not necessarily the same, but may differ due to errors in the accuracy of the camera model and the algorithm.

[0040] The rotation from the first rotational position to the second rotational position may be substantially in a horizontal plane.

[0041] The plurality of targets may be arranged such that at least a portion thereof lies within a horizontal plane.

[0042] As described above, the method of the present disclosure is designed for calibrating cameras based on diffraction apertures, such as pinhole cameras. The method can be applied in a similar manner to cameras using other types of diffraction apertures or non-refractive elements, such as the cameras described in WO 2019 / 143250 A1 and WO 2021 / 167452 A2.

[0043] The method is not limited to use with cameras or instruments for surveying applications, but can also find use in other computer vision systems, not only for geodetic applications, but also for a variety of other applications, such as systems for machine alignment of components.

[0044] The target can be an active target or beacon, such as a self-luminous light emitting diode (LED). Alternatively, the target can be a passive target, such as a conventional survey prism, a retroreflector, or a hollow reflector that reflects light emitted by the camera and / or other light sources. The target should be small enough to be effectively considered a point source when viewed by the camera. The condition that the target is an effective point source sets the minimum distance between the camera and the target during the calibration measurement.

[0045] The camera also typically includes a processing unit configured to determine the first and second viewing angles and a change in viewing angle, i.e., the difference between the first and second viewing angles for each target. The processing unit may be located within the camera housing, in which the image sensor is also arranged, or the processing unit may be located outside the housing.

[0046] The camera is usually located on a rotating stage, which can be a universal rotating stage or a custom rotating stage, or the camera is located on a tripod during measurement, which allows the camera to be rotated in the horizontal plane and preferably also in the vertical plane, for example from a first rotation position to a second rotation position. The tripod can also allow the camera to be tilted relative to the horizontal plane.

[0047] The influence of different types of camera rotation and / or tilt between different positions of the recorded images, as well as the deviation of the aperture position from its nominal position on the viewing angle calculated from the recorded images will be discussed in reference to Figures 4A-4C This is described in more detail in the detailed description given.

[0048] The plurality of targets preferably comprises at least three targets. Generally speaking, the number of targets is about 10-20 targets, or even about 100 targets. The number of targets used can be set based on what is considered to be an acceptable processing time and / or processing load.

[0049] The targets can all be arranged at substantially the same distance from the camera. This has the advantage that all targets can be mounted to a single structural element, such as a beam or an arc structure, and all targets appear with similar brightness on the image sensor.

[0050] However, it is not required that all targets be arranged at the same distance from the camera. The method can be applied to multiple targets located at substantially any distance from the camera.

[0051] With regard to the distance between the target and the camera, the following factors may be considered: It is preferred that the targets be placed at a distance such that they appear to be essentially point sources, but still close enough that the projection on the image sensor is bright enough to be resolved by the image processing algorithm. In addition, the target should be at the minimum distance such that any errors in the calculated viewing angle due to the center of rotation not being exactly at the center of the diffraction stop are within acceptable limits.

[0052] The step of determining a function or graph based on the first and / or second set of calculated observation angles and a set of difference angles may include providing a graphical representation of the difference angle as a function of the first and / or second calculated observation angles, and / or determining a mathematical function of the difference angle relative to the first and / or second calculated observation angles. Such a function or graph may be determined using curve fitting and may be represented as Δα=F(α 1 ), or equivalently expressed as Δα=F(α 2 ).

[0053] In this figure, the X-axis usually represents the first rotation position (α 1 ) or the second rotation position (α 2 ) for each target, and the Y axis represents the difference angle Δα for each target. The difference angle is calculated as the difference between the calculated observation angles for each target at the first rotation position and the second rotation position, that is, Δα=α 1 -α 2 Or similarly Δα=α 2 -α 1.

[0054] From the function or graph, the camera model error may be determined as an offset value of one or more parameters of the camera model.

[0055] If the camera model is considered to provide a mathematical representation of the position of the diffraction stop relative to the image sensor in a Cartesian coordinate system, then the coordinates of the pixel image sensor can be expressed in x, y, z, with the nominal position of the center of the diffraction stop as the origin. In this representation, the z-axis is taken as the axis starting from the origin along the optical axis through the nominal position of the stop. According to the definition of the Cartesian coordinate system, the x-axis and y-axis are parallel to the plane of the image sensor and perpendicular to the z-axis, respectively. Typically, the x-axis can be in the horizontal plane, while the y-axis is in the vertical direction.

[0056] Using this camera model, the offset of one or more of the x, y, and z coordinates can be determined from the graph as follows:

[0057] The linear (first order) component represents the X offset error (when the movement from the first rotational position to the second rotational position is in the horizontal plane) and / or the Y offset error (when the movement from the first rotational position to the second rotational position is a rotation in the vertical plane).

[0058] - The quadratic (second order) component represents the Z (focus) offset error.

[0059] - A flat graph means there is no X(Y) or Z error. In this case, the calculated and actual viewing angles are equal.

[0060] The numerical value of the camera model error or the parameter offset value can be calculated from the first-order and / or second-order components.

[0061] If the calculated camera model error is not within a predetermined limit or threshold, it will be fed back into the algorithm to provide an updated camera model. Based on this updated camera model, the calculation (or determination) step is repeated until the camera model error is within the limit.

[0062] Therefore, the calculation is iterated until a converged result is obtained in which the camera model error is smaller than a limit value.

[0063] In the above manner, two-dimensional calibration parameters, ie, X and Z or Y and Z, can be determined.

[0064] For many applications of cameras, especially surveying applications, calibration in the horizontal direction is considered to be of the utmost importance. For such applications, it is sufficient to carry out the above method and calibrate the camera model in the horizontal direction by implementing the rotation of the camera from the first rotational position to the second rotational position as a rotation in the horizontal plane. In this case, calibration can be performed on site without removing the camera from the monitoring site and bringing it to a laboratory.

[0065] When calibration is to be performed along the X-axis and the Y-axis, the method can be performed in various ways.

[0066] According to a first version of the method, when performing the method, the movement from the first rotational position to the second rotational position is a rotation in a horizontal plane, and after the computer model error thus determined has converged to a value within preset limits, the camera is rotated 90° about its optical axis (i.e. about the z-axis) and the method steps are repeated, again rotating the camera in the horizontal plane. According to this version, the camera model can be calibrated in the x, y, z directions from the four recorded images.

[0067] According to a second version of the method, the movement from the first rotational position to the second rotational position involves a rotation about two rotational axes, so that four sets of calculated observation angles can be calculated from the two recorded images, which involve angles relative to the x-axis and relative to the y-axis.

[0068] According to a third version of the method, when performing the method, the movement from the first rotational position to the second rotational position is achieved by a rotation in a horizontal plane. Once the computer model error has converged to a value within preset limits, the camera is tilted relative to the horizontal plane, i.e. the camera is rotated about the x-axis (extending in the horizontal plane), and the method is repeated. According to this version, the camera model can be calibrated from three recorded images.

[0069] According to the first version of the method, the method may further comprise the steps of: once the camera model is determined to be acceptable, rotating the camera by substantially 90 degrees around its optical axis; and repeating steps c) to k).

[0070] By rotating the camera 90° around its optical axis, the camera can be calibrated on the two-dimensional plane of the image sensor even if the multiple targets are arranged in a one-dimensional array. This can show or simplify the three-dimensional position of the diffraction stop relative to the image sensor as a two-dimensional problem.

[0071] Other advantages relate to the fact that the target must be placed on the virtual horizon as viewed by the camera when the camera is in a normal, non-tilted orientation.

[0072] After a rotation of substantially 90° about the optical axis, steps c) to k) are repeated. After a rotation of 90°, the camera can be considered to be in a third rotation position and the camera can be rotated in the horizontal plane to a fourth rotation position.

[0073] In other words, repeating steps c) to k) can be expressed as:

[0074] (c') placing the camera at a third rotational position relative to the target, and recording a third image of the plurality of targets using the camera at the third rotational position;

[0075] (d') determining a third set of calculated viewing angles from the third image using the camera model, the third set of calculated viewing angles comprising a third calculated viewing angle of each target relative to the camera;

[0076] (e') rotating the camera to a fourth rotational position relative to the targets, and recording a fourth image of the plurality of targets using the camera at the fourth rotational position;

[0077] (f') determining a fourth set of calculated viewing angles from the fourth image using the camera model, the fourth set of calculated viewing angles comprising fourth calculated viewing angles of each target relative to the camera;

[0078] (g') determining a function or graph based on the third and / or fourth sets of calculated observation angles and a set of difference angles representing the difference between the third calculated observation angle and the fourth calculated observation angle for each target;

[0079] (h') determining the camera model error according to the function or graph;

[0080] (i') comparing the camera model error with a predetermined limit;

[0081] (j') if the camera model error is below the limit, the camera model is determined to be acceptable;

[0082] (k') If the camera model error is not lower than the limit, an updated camera model is calculated based on the camera model error, and steps d'), f'), g'), h'), i') and j') or k') are repeated based on the updated camera model.

[0083] In a first version, at least a portion of the plurality of targets may be arranged in a substantially one-dimensional row as viewed from the camera, the one-dimensional row and the camera being located substantially in a plane. Step (f) of rotating the camera from the first rotational position to the second rotational position may be performed substantially in said plane.

[0084] The one-dimensional line does not necessarily have to be a straight line, but may also extend along a circle segment, so that all targets are located at an equal distance from the camera. The plane is preferably oriented substantially along a horizontal plane. Thus, the calibration device may be placed in an office or laboratory space, which typically has a larger dimension in the horizontal direction than in the vertical direction. However, conceptually, the plane may equally well be oriented in any other direction.

[0085] As mentioned above, it may be advantageous to arrange the targets at a distance from the camera so that the targets effectively form point sources when viewed through the camera. This sets a minimum distance between the camera, i.e. the diffraction stop of the camera, and the targets. In general, this distance may typically be of the order of a few meters. For example, a distance of 3 meters between each target and the camera may be used.

[0086] The main reason for the minimum distance is that it is not possible to rotate perfectly about the central axis. The central axis should pass through the center of the aperture, but, due to mechanical tolerances, this is not always possible. Therefore, in practice, the camera is not only rotated, but also translated (moved sideways) to some small amount, resulting in an additional change in the viewing angle. Since the rotation is usually small (e.g. 1 to 2 degrees), and since the center of rotation is reasonably well known (to within a millimeter or so), the translation can be considered to be quite small. If the target is far enough away, the translation is negligible relative to the distance, so the angular error resulting from the translation also becomes negligible.

[0087] However, whether this condition is actually met may depend on the algorithm used and what the algorithm is able to correct.

[0088] According to a second version of the method, the plurality of targets are arranged in a two-dimensional arrangement as seen from the camera, and the step (f) of rotating the camera from the first rotational position to the second rotational position comprises a rotation in a horizontal plane and a rotation in a vertical plane.

[0089] According to this embodiment, recording two images, i.e. two measurements, is sufficient to calibrate the camera in the x, y and z axes. Since the movement from the first rotational position to the second rotational position involves a rotation about a vertical axis and a tilt relative to a horizontal plane, the second rotational position differs from the first rotational position in both horizontal and vertical angles, whereby a calculated observation angle relative to the x-axis and relative to the y-axis can be determined for each target from each of the two images, so that the camera model errors, including the x-offset error, the y-offset error and the z-offset error, can be calculated from the two images.

[0090] According to a third version of the method, the plurality of targets are arranged in a two-dimensional arrangement as seen from the camera, and the step (f) of rotating the camera from the first rotational position to the second rotational position is performed substantially in a horizontal plane about a first axis passing through the aperture plane and extending substantially perpendicularly to the horizontal plane, and the first and second calculated observation angles represent angles in the horizontal plane; the method further comprises the following steps:

[0091] (l) determining a third set of calculated viewing angles from the second image using the camera model, the third set of calculated viewing angles comprising a third calculated viewing angle of each target relative to the camera, the third calculated viewing angle representing an angle in a direction substantially perpendicular to the horizontal plane;

[0092] Once the camera model is determined to be acceptable in step (j):

[0093] (m) rotating the camera about a second axis passing through an aperture of the camera, thereby rotating the camera to a third rotational position, the second axis being substantially perpendicular to the first axis and extending in a horizontal plane;

[0094] (n) recording a third image of the plurality of targets using the camera in a third rotational position;

[0095] (o) determining a fourth set of calculated viewing angles from the third image using the camera model, the fourth set of calculated viewing angles comprising fourth calculated viewing angles of each target relative to the camera;

[0096] (p) determining a second function or graph based on the third and / or fourth sets of calculated observation angles and a set of second difference angles representing the difference between the third calculated observation angle and the fourth calculated observation angle for each target; and

[0097] (q) determining a second camera model error based on the second function or graph;

[0098] (r) comparing the second camera model error to a second predetermined limit;

[0099] (s) if the second camera model error is lower than a second limit, determining that the camera model is acceptable;

[0100] (t) If the second camera model error is not lower than the second limit, calculating an updated camera model based on the second camera model error, and repeating steps (l), (o)-(r) or (t) based on the updated camera model.

[0101] In this context, the step (m) of rotating the camera about the second axis may be alternatively described as tilting the camera relative to the horizontal plane. Similar to the general concept described above, the amount of this rotation does not need to be known, and is typically about 1 angstrom, 1 degree, or a few degrees.

[0102] Step (a) advantageously comprises arranging the plurality of targets such that one target is arranged at each lateral periphery of the field of view of the camera and any remaining targets in the plurality of targets are substantially evenly distributed between targets arranged at the lateral peripheries.

[0103] It has been observed that placing two of the targets near either periphery of the camera's field of view is beneficial to the calibration process. Errors in the change in viewing angle measured by the camera, caused by errors in the modeled relative position of the aperture with respect to the center of the image sensor, along the plane of the image sensor surface, are believed to increase with increasing distance from the center of the field of view. Therefore, by placing the targets at or near the edge of the field of view, compensation parameters for this modeled parameter can be efficiently calculated.

[0104] According to some embodiments, when the camera is in a neutral position where its optical axis coincides with the centerline of the calibration measurement device, the two peripherally arranged targets can be located at approximately ±30° from the centerline of the calibration measurement device, that is, approximately ±30° relative to the optical axis of the camera.

[0105] An odd number of targets may be provided, with one target disposed at or near each lateral edge or side of the camera's field of view and one target located substantially at or at least close to the center of the camera's field of view.

[0106] It is considered advantageous to place the target substantially in the center of the field of view, since the error in the measured variation of the viewing angle caused by the error in the modeled focal length is considered to be at its greatest level in the center of the field of view. Therefore, by placing the target at the center of the camera's field of view, the compensation parameter or z-offset of the modeled focal length can be efficiently calculated.

[0107] The method may further comprise the step (h1) of determining calibration parameters as corrections to be applied to the parameters representing the camera model, wherein the calibration parameters are determined such that for each target the determined changes in viewing angle are substantially equal.

[0108] Thus, corrections to the modeling parameters describing the camera model, as defined by the initial or nominal camera model, are determined as correction or compensation offsets for which all targets exhibit substantially the same change in the calculated viewing angle. Preferably, the correction or compensation offset is calculated for each of the three modeling parameters defining the camera model.

[0109] The correction or compensation offset can be calculated using algorithms known in the field of image processing. Algorithms known in the field of image processing are, for example, algorithms using gravity methods. The algorithms may involve iterative algorithms, or direct calculations based on geometric and / or trigonometric considerations, or a combination of direct and iterative methods.

[0110] For example, the algorithmic step may include calculating the center position of a spot recorded by the image sensor, the spot representing the image of the target projected onto the image sensor. The center position may be calculated to sub-pixel accuracy by using a so-called centroid algorithm known to those skilled in the art.

[0111] The camera model may include the three-dimensional positions of the pixels of the image sensor relative to the diffraction stop as parameters.

[0112] A camera model is a mathematical model or representation of a camera and can therefore be based on three parameters. These parameters can be expressed as coordinates in a coordinate system, usually a Cartesian coordinate system. This relatively simple model is considered sufficient for the purpose of calibrating cameras, with the diffraction stop presenting a simplified geometry compared to lens-based systems.

[0113] Furthermore, the camera model may take into account the influence of other optical elements, such as filters, glass cover elements or any other optical elements.

[0114] The method may further comprise the step of bringing the camera to a second temperature and subsequently repeating the steps of any preceding claim at the second temperature.

[0115] The calibration measurement steps described herein may first be performed at a first temperature. Subsequently, the camera is brought to a second temperature different from the first temperature and the calibration measurement steps are repeated at the second temperature. During the different measurements, the camera is kept at a substantially constant temperature. The camera may be brought to the second temperature by heating or cooling, for example, in a climate chamber.

[0116] The method as described above can be performed at the same time as the actual measurement using a computer vision system.

[0117] In this embodiment, the system preferably includes a microprocessor-controlled rotation device for rotating the camera to different positions, as described above. By performing the steps as described above and rotating the camera using the rotation device, the camera can be calibrated while performing an operational survey, thereby observing multiple fixed reference points and multiple movable survey points by observing changes in the viewing angle of the reference points. That is, the fixed reference points can be used as the targets described above.

[0118] Calibrating the camera during the survey is advantageous, since the calibration parameters of the camera model may change due to environmental influences such as temperature and component aging. When using only a single rotating device, preferably configured to rotate the camera about the Y-axis of the aperture, i.e. in the horizontal plane, and at least three virtual static targets distributed substantially along the horizon, at least the focal length of the camera can be determined. In a practical embodiment, the focal length is most sensitive to ambient temperature changes and therefore needs to be recalibrated during the survey. Generally, in order to use such an arrangement, the x-axis offset should also be calibrated.

[0119] Furthermore, this approach is even more advantageous because it eliminates measurement errors caused by dust or other contamination in the optical path. For example, a speck of dust on the optical window may block light from the survey target for one rotational position but not for another.

[0120] For some surveying applications, it is known that the survey points move very slowly. Therefore, the survey points can be considered to be stationary for a relatively short period of time. Therefore, the survey target will not move significantly during the time required to determine the first and second sets of observation angles for each survey and reference point. In this way, fixed reference points and movable survey points can be used as targets as described above.

[0121] In another embodiment of the present invention, the system includes a passive rotation device. Such a device may be, for example, a bimetallic coil spring to rotate the camera around the aperture in response to changes in ambient temperature. Preferably, such a passive rotation device is configured to rotate the camera around the Y axis of the aperture, thereby enabling determination of changes in focal length due to changes in ambient temperature.

[0122] Such a passive rotary device can be advantageous in power-constrained applications because it requires no electricity. This is beneficial for applications such as remote, battery- or solar-powered IoT sensors.

[0123] Although the targets are described above as being preferably arranged substantially in a horizontal plane, i.e. along the horizon as seen from the camera, and where applicable along a vertical axis substantially in a vertical plane passing through the aperture of the camera, this is not essential. Although such an arrangement may be advantageous in terms of computational efficiency, making the calculations easier, the method can equally well be used for targets arranged away from the horizontal and / or vertical axis. In this case, the fact that the target is located at a (considerable) distance from the horizontal and / or vertical axis may be taken into account in the calculations, for example by additional mathematical conversions or terms in the equations.

[0124] The calibration method of the first aspect may be performed using the system of the second aspect of the present invention.

[0125] According to a second aspect, there is provided a system for calibrating a camera of a computer vision system, the camera comprising an image sensor and a diffraction stop arranged to project incident light onto the image sensor, the system comprising:

[0126] multiple targets to be viewed by the camera;

[0127] a mounting device for mounting the camera so that when the camera is mounted on the mounting device, the plurality of targets are within the camera's field of view, the mounting device enabling the camera to rotate between a first rotational position and a second rotational position relative to the plurality of targets; and

[0128] One or more processors configured to:

[0129] (b) providing a camera model representing a mathematical model of the camera;

[0130] (c) recording a first image of the plurality of targets with the camera in a first rotational position;

[0131] (d) determining a first set of calculated viewing angles from the first image using the camera model, the first set of calculated viewing angles comprising a first calculated viewing angle of each target relative to the camera;

[0132] (e) recording a second image of the plurality of targets with the camera in a second rotational position;

[0133] (f) determining a second set of calculated viewing angles from the second image using the camera model, the second set of calculated viewing angles comprising a second calculated viewing angle of each target relative to the camera;

[0134] (g) determining a function based on the first set of calculated observation angles and / or the second set of calculated observation angles and a set of difference angles representing the difference between the first calculated observation angle and the second calculated observation angle for each target;

[0135] (h) determining a camera model error according to the function;

[0136] (i) comparing the camera model error with a predetermined limit;

[0137] (j) if the camera model error is below the limit, determining that the camera model is acceptable;

[0138] (k) If the camera model error is not lower than the limit, an updated camera model is calculated based on the camera model error, and steps d), f), g), h), i) and j) or k) are repeated based on the updated camera model.

[0139] Specifically, the system is configured to execute the method of the first aspect described above.

[0140] The image sensor is configured to receive and detect reflections and / or light beams generated by a target located within the field of view of the camera. To this end, the image sensor may preferably be a two-dimensional sensor.

[0141] The one or more processing units may be located in or at the camera, for example in the inner space of the camera, close to the image sensor. Alternatively, the processing unit may be located remote from the camera. Advantageously, the system comprises a first processor arranged in the camera, which records and stores images; and one or more second processors provided in the computer, which are configured to receive and store images and possibly other data, process the images and any other data, and determine to calculate viewing angles and calibration parameters, i.e., an updated camera model.

[0142] The mounting device advantageously comprises a microprocessor-controlled rotation device. The microprocessor-controlled rotation device can be configured to rotate the camera to move the camera to or between the above-mentioned different rotation positions. As described above, this enables the method to be performed during a survey, i.e., to be effective during field measurements.

[0143] Such a device may be a motor, a servo mechanism, a solenoid or some other device that can be controlled by a microprocessor. Such a device may include an angle encoder or some other angle feedback device. Such a device may also be combined with some device to limit the rotation angle.

[0144] The rotating means may be configured to rotate the aperture independently about a single axis or a plurality of axes.Preferably, the rotating means is configured to rotate the camera about the Y-axis of the aperture.

[0145] The rotating device may, for example, be used to rotate a camera configured to survey a plurality of fixed reference points and a plurality of movable survey points.

[0146] In another embodiment, the camera includes a passive rotation device. Such a device may be, for example, a bimetallic coil spring to rotate the camera about the aperture in response to changes in ambient temperature. Preferably, such a passive rotation device is configured to rotate the camera about the Y axis of the aperture, thereby enabling the determination of changes in focal length due to changes in ambient temperature.

[0147] The mounting arrangement may be configured to enable the camera to be rotated about a vertical axis and / or a horizontal axis.

[0148] The one or more processors may be configured to perform one or more of steps b) to t) described above with respect to the method of the first aspect.

[0149] The targets to be measured may be arranged according to any of the arrangements or distributions described above with respect to the method of the first aspect.

[0150] According to a third aspect of the present disclosure, a computer program product is provided, the computer program product comprising instructions, which when executed by a computer causes the computer to perform the following steps:

[0151] (b) providing a camera model representing a mathematical model of a camera, the camera comprising an image sensor and a diffraction stop arranged to project incident light onto the image sensor;

[0152] (d determining, from a first image of the plurality of targets recorded with a camera at a first rotational position relative to the plurality of targets, using a camera model, a first set of calculated viewing angles from the first image, the first set of calculated viewing angles comprising a first calculated viewing angle of each target relative to the camera;

[0153] (f) determining, from a second image of the plurality of targets recorded with the camera in a second rotational position, using the camera model, a second set of calculated viewing angles from the second image, the second set of calculated viewing angles comprising a second calculated viewing angle of each target relative to the camera;

[0154] (g) determining a function based on the first set and / or the second set of calculated observation angles and a set of difference angles representing the difference between the first calculated observation angle and the second calculated observation angle for each target;

[0155] (h) determining a camera model error according to the function;

[0156] (i) comparing the camera model error with a predetermined limit;

[0157] (j) if the camera model error is below the limit, determining that the camera model is acceptable;

[0158] (k) If the camera model error is not lower than the limit, an updated camera model is calculated based on the camera model error, and steps d), f), g), h), i) and j) or k) are repeated based on the updated camera model.

[0159] Here, the instructions may cause one or more processors of the computer to perform any one or more of the steps.

[0160] The instructions of the computer program product may also be configured to cause the one or more processors to control movement of the camera between the different rotational positions described above.

[0161] In summary, a method, system and computer program product for calibrating a camera provided with a diffraction stop as an objective lens are provided. With the calibration method, relevant physical parameters of a camera model can be determined with relatively simple and non-precise assembly.

[0162] According to the general concept of the present disclosure represented by the first aspect described above, by placing a camera and a plurality of targets in a plane, the camera can be calibrated in a direction along the plane by recording two images of the plurality of targets, wherein between the recording of the two images the camera has been rotated in the plane relative to the axis passing through the diffraction stop of the camera. The rotation is small, typically around 1 or a few degrees. For each of the two recorded images, the calculated observation angle of each target recorded on the image sensor is determined using a camera model. Calibration parameters are determined based on the difference in the calculated observation angle between the first and second images of each target. With an accurately calibrated camera model, all targets should have the same difference in the calculated observation angle. If this is not the case, calibration parameters are determined and fed back into the camera model and the calculation is iterated until the calibration of the camera is deemed acceptable.

[0163] This general concept enables calibration along a first axis of the image sensor in a direction along a plane and within the focal length of the camera.Calibration along a second axis of the image sensor perpendicular to the first axis can be performed according to any of three different alternatives of the method as described above.

[0164] The method described enables accurate calibration of survey instruments based on diffraction stops, which facilitates monitoring / surveying of structures with much higher accuracy than instruments using refractive lenses.

[0165] It has been found that the method of the present disclosure helps to achieve an error of the relative viewing angle of a camera with a pinhole or other type of diffraction stop close to 1 milligant or 1 millidegree. This is about 50 times better than the achievable error of a surveying instrument with a glass lens, where the error is mainly caused by the deformation of the lens. BRIEF DESCRIPTION OF THE DRAWINGS

[0166] The embodiments of the present disclosure will be described below with reference to the accompanying drawings. However, the embodiments of the present disclosure are not limited to specific embodiments, but should be interpreted as including all modifications, changes, equivalent devices and methods and / or alternative embodiments of the present disclosure.

[0167] As used herein, the terms “having”, “may have”, “include” and “may include” indicate the existence of corresponding features (such as values, functions, operations or parts, etc.), and do not exclude the existence of additional features.

[0168] As used herein, the terms "A or B", "at least one of A or / and B", or "one or more of A or / and B" include all possible combinations of the listed items. For example, "A or B", "at least one of A and B", or "at least one of A or B" means: (1) including at least one A, (2) including at least one B, or (3) including at least one A and at least one B.

[0169] As used herein, terms such as "first" and "second" may modify various elements, regardless of the order and / or importance of the corresponding elements, and do not limit the corresponding elements. These terms may be used to distinguish one element from another element. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0170] It should be understood that when one element (e.g., a first element) is "operably or communicatively" coupled or "connected" to another element (e.g., a second element), the element may be directly coupled to the other element, and there may be an intermediate element (e.g., a third element) between the element and the other element. Conversely, it should be understood that when one element (e.g., a first element) is "directly coupled" or "directly connected" to another element (e.g., a second element), there is no intermediate element (e.g., a third element) between the element and the other element.

[0171] The expression "configured to (or arranged to)" used in the present disclosure may be used interchangeably with "suitable for", "capable of...", "designed to", "suitable for", "manufactured to", or "capable of", depending on the context. The term "configured to (arranged to)" does not necessarily mean "specially designed to" at the hardware level. On the contrary, in some contexts, the expression "the device is configured to..." may mean that the device "is capable of ..." with other devices or components.

[0172] The terms used in the various embodiments of the present disclosure are to illustrate specific embodiments and are not intended to limit the present disclosure. Except as otherwise indicated in the context, the singular form as used herein is also intended to include the plural form. Except as otherwise defined, all terms (including technical or scientific terms) as used herein have the same meaning as the meaning generally understood by those of ordinary skill in the relevant art. The terms defined in the general dictionary should be interpreted as having the same or similar meaning as the contextual meaning of the relevant technology, and should not be interpreted as having ideal or exaggerated meanings, unless they are clearly defined in this article. Depending on the circumstances, even the terms defined in the present disclosure should not be interpreted as excluding embodiments of the present disclosure.

[0173] In determining the scope of protection afforded by the claims herein, any elements equivalent to the elements specified in the claims should be appropriately considered.

[0174] The present invention will now be described in more detail with reference to the accompanying drawings, in which:

[0175] Figures 1A-1C Several schematic configurations of systems for monitoring the position and / or movement of an object known in the prior art are shown;

[0176] Figure 2 A system schematically illustrates a general embodiment of a camera to be calibrated according to the present disclosure;

[0177] Figure 3 shows an overview of the functionality of a camera that can be calibrated using the method of the present invention;

[0178] Figure 4A , 4B 4C are schematic diagrams showing features of a camera model used for calibration and errors in calibrated aperture positions according to an embodiment of the present invention;

[0179] Figure 5 schematically illustrates a setup for a calibration measurement according to an embodiment of the present invention;

[0180] Figure 6 schematically illustrates a scene observed by a camera during a calibration measurement according to an embodiment of the invention;

[0181] Figure 7 A flow chart showing a calibration method according to an embodiment of the present invention;

[0182] Figure 8 shows a two-dimensional arrangement of targets according to some embodiments of the present invention;

[0183] Fig. 9 A flow chart showing a calibration method according to a further embodiment of the present invention;

[0184] Fig.10 A flow chart showing a calibration method according to a further embodiment of the present invention;

[0185] Fig.11 A flow chart showing a calibration method according to a further embodiment of the present invention;

[0186] Figure 12-17 Examples of simulated measurements at different steps of the calibration process are shown. DETAILED DESCRIPTION

[0187] Generally speaking, the present disclosure relates to systems for calibrating a survey object or tracking the motion of an object by tracking one or more targets or survey reflectors attached to the object.

[0188] Although the illustrated embodiments are described with respect to a camera having an optical input system formed by a non-refractive element in the form of a diffraction stop or pinhole at the camera objective, i.e. the camera objective, it will be appreciated that the non-refractive element may also be any other non-refractive element, such as the non-refractive element described in WO 2021 / 167452 A2. Similarly, although the described embodiments use a prism as a survey reflector, it will be appreciated that a different reflective element may also be used, such as another type of prism or a hollow mirror.

[0189] Figure 1A An exemplary arrangement of a system for monitoring an object 3 is shown. The system comprises a sensor device, such as a camera 7. The system also comprises a plurality of targets 1, here in the form of survey reflectors, which are attached to the object 3 at a plurality of locations. The object 3 is shown to comprise one or more buildings to which the targets 1 are fixed. However, the object 3 may also be any other structure, such as a tower, a tunnel ( Figure 1B ) or bridge ( Figure 1C ), it can also be a means of transportation (such as a ship on land), or a natural object, such as a large rock.

[0190] The object 3 is monitored by monitoring or measuring the position of the survey reflector 1. By monitoring their position over time, the movement of the whole or part of the object 3 can be detected. Preferably, the amount, degree and / or direction of the movement can also be determined. Thereby, the state of the object 3, such as stability or integrity, or mechanical properties, can be monitored.

[0191] One camera 7 is shown. However, the system may comprise more than one camera 7.

[0192] exist Figure 1AIn the illustration of , a camera 7 is arranged to generate a divergent light beam 5 and emit it to a plurality of survey reflectors 1. The survey reflectors 1 reflect the portion of the divergent light beam 5 impinging thereon, thereby forming a reflected light beam 6, which is reflected back to the camera 7. The light beam 5, which is generally substantially conical, has a solid angle Ω1 covering the field of view of the camera 7. Thereby, a plurality of survey reflectors 1 can be monitored substantially simultaneously. Alternatively, as known to those skilled in the art, the targets may be active targets, such as light emitting diodes LED, each LED emitting a light beam 6 which is recorded by the camera. In such an embodiment, the camera does not need to be arranged to generate and emit a divergent light beam 5. It should be understood that the calibration method according to the present disclosure can be similarly applied to cameras 7 including a light source emitting a light beam 5 and cameras 7 without a light source.

[0193] Figure 1B An embodiment is shown in a tunnel 3. A railway with railway sleepers 12 passes through the tunnel 3. Both the tunnel walls and the railway sleepers 12 are provided with survey reflectors or targets 1. A camera 7 is arranged to observe all survey reflectors or targets 1 in its field of view.

[0194] Figure 1C An embodiment is shown on a bridge 3. The bridge 3 is provided with a plurality of survey reflectors or targets 1. A camera 7 is arranged to observe all survey reflectors 1.

[0195] Figure 2 A schematic diagram of the measurement principle of a system 20 for monitoring multiple positions on an object 3 is provided, to which the calibration method described herein can be applied. For ease of illustration, Figure 2 The system 20 is shown as monitoring a target or survey reflector 21. However, for example Figures 1A to 1C As shown, the camera 27 may be arranged to monitor a plurality of survey reflectors. In the embodiment shown, the survey reflectors 21 are formed by prisms. Alternatively, other types of reflectors may be used, such as hollow reflectors.

[0196] The system 20 comprises a camera 27 and a processing unit 29, which may be contained or arranged within the camera 27. Alternatively, it may be arranged remotely from the camera 27. In addition to the processor unit within the camera, one or more processors may be provided externally to the camera for calibration purposes.

[0197] In the embodiment shown, the camera 27 comprises a light source 22 that emits a divergent light beam 25. The light source 22 typically comprises a light emitting diode (LED). The light beam 25 has a first solid angle Ω1, which may be large enough to cover substantially the entire field of view of the camera 27. Thus, all surveying reflectors 21 located within the field of view of the camera are illuminated by the light beam 25 without moving, rotating or scanning the camera or the light beam (with the possible exception of one or more surveying reflectors being obstructed by obstacles, such as in the case of Figure 1A Pedestrians or vehicles in the case of monitoring buildings as shown, or in Figure 1B The monitoring tunnel case shown is a train).

[0198] The surveying reflector 21 reflects the portion of the light beam 25 it receives, thereby forming a reflected light beam 26 , which is reflected back to the camera 27 .

[0199] Alternatively, as described above, instead of using a survey reflector 21, an active target or beacon may be used, typically comprising an LED, which itself emits a light beam 26. In such an embodiment, the camera need not include a light source 22. As will be appreciated by those skilled in the art, the light beam emitted by such an active target is received and processed by the camera 27 in the same manner as described herein with respect to the light beam 25.

[0200] The device 20 further comprises an image sensor 24 arranged to receive light, namely the portion 261 of the reflected light beam 26 entering the camera 27. As a result of receiving the reflected light 261, the image sensor 24 generates data, which in a preferred embodiment is in the form of a two-dimensional image.

[0201] The light beam 25 may be amplitude modulated so that its amplitude exhibits a defined variation over time. Alternatively and / or additionally, other types of encoding may be applied to the first light beam. When an active beacon or target is used, the light emitted thereby may be similarly encoded. Environmental influences on the measurement, such as interference from ambient light, may be reduced by applying appropriate filtering techniques during image processing of the image data.

[0202] The details and positions of the light source 22 and any possible additional light sources, the various types of encoding applied to the light beam 25 and any possible additional light beams and their processing during image processing are described in detail in WO 2021 / 167452 A2.

[0203] Between the image sensor 24 and the first light source 22, or at least its emitting surface, a body 28 is arranged, which in the embodiment shown is substantially planar and forms part of the camera housing. The body 28 is opaque to light and comprises an optical injection system, in the embodiment shown, in the form of a diffraction stop or pinhole 23, forming the objective of the camera.

[0204] Although the description in this article will focus on an optical injection system formed by a pinhole, other types of diffraction elements, in particular the diffraction elements described in WO 2019 / 143250 A1 and WO 2021 / 167452 A2, may be equally applicable.

[0205] The processing unit 29 is configured to determine the position or azimuth of each survey reflector from the data, typically by image processing the data provided by the image sensor, and to detect movement of one or more of the plurality of survey reflectors based on a comparison of the determined position or azimuth of each survey reflector with its previously determined position or azimuth.

[0206] Figure 3 An example of a camera 7 is shown. The exemplary camera 7 has a processing unit 9 connected to a non-refractive optical device 101, an image sensor 120, a clock 123, a memory 15, one or more position and / or orientation measurement components 16, an output unit 17, an input unit (or user interface) 19, an electronic networking module 109, and one or more light sources 102. The non-refractive optical device 101 is shown as being connected to the image sensor 120. The latter "connection" does not necessarily have to be a physical connection. Here, "connection" is intended to refer to the situation where the non-refractive optical device 101 is arranged to receive ambient light so that the received ambient light is received by the image sensor 120. It can be understood from the embodiments described above that it is not required Figure 3 All functional elements shown in are present.

[0207] All connections intended for data transmission may be physical connections (wiring), but, alternatively, they may be wireless and based on the transmission of electromagnetic / optical radiation.

[0208] The non-refractive optical device 101 may be any type of diffraction stop mentioned above, such as one or more pinholes. The diameter of the pinhole may be in the range of 50 μm to 400 μm. Alternatively, as mentioned above, the non-refractive optical device may be replaced by a lens, which is preferably a thin lens, thereby allowing temperature modulation with low computational effort.

[0209] The processing unit 9 may be any suitable processing unit known in the art.

[0210] The image sensor 120 preferably comprises a set of photosensitive elements (pixels) arranged in a two-dimensional matrix, such as a CCD sensor or a CMOS sensor, which forms the image plane of the camera. The image sensor 120 is arranged to receive the light beams 6 that have entered through the diffraction aperture 101. Each light beam 6 will be focused on a subset of the photosensitive elements. Each such subset corresponds to the solid angle of an incident light beam 6, i.e. the angle of incidence in the horizontal plane and the angle of incidence in the vertical plane relative to the earth. Of course, the angle of incidence can also be measured relative to another object outside the earth, such as a geostationary orbit satellite. As long as the camera 7 and the survey reflector 1 remain in fixed positions, these subsets are static for each survey reflector 1.

[0211] In an alternative embodiment, a line sensor can be used in combination with an optical slit as an objective lens instead of using a pinhole, as described in WO 2019 / 143250 A1.

[0212] Optionally, a temperature control system 103 may be provided to reduce thermal influences on the measurement data. The heat capacity of the non-refractive optics 101 is lower when compared to a camera 7 using a lens system instead of the non-refractive optics 101. Thermal stability may be improved by implementing a temperature control system in the form of a thermostat 103. Figure 3 An embodiment is shown with a reversible Peltier element 103 (i.e. configured for cooling and heating) for the non-refractive optical device 101. The Peltier element 103 is connected to the processing unit 9 and its temperature is controlled by the processing unit 9 so that the non-refractive optical device 101 is kept at a predetermined temperature. Alternatively, the thermal stability may be improved by the design of the camera housing, in particular by the materials used and / or by measuring the temperature at different locations therein and using a model that takes into account the thermal effects during the processing of the data from the image sensor.

[0213] In the following, some general aspects of the above described systems and methods of operating the same will be summarized.

[0214] The image sensor 24, 120 converts the received light beam 6 into an image. The image is a set of electronic signals, referred to herein as pixel signals. Each pixel signal is generated by a photosensitive element and has a value that depends on the light intensity of the light received by the photosensitive element. Therefore, the pixel signal can also be related to the object 3 to which the survey reflector 1 is attached and its surroundings.

[0215] The image sensor is positioned so that light entering the camera through the non-refractive element forms a diffraction pattern on the image sensor. This diffraction pattern depends on the characteristics of the non-refractive element and appears as dark or bright areas on the image sensor depending on the distance and angle of each pixel of the image sensor to the non-refractive element. By integrating multiple data frames, each data frame comprising multiple pixels, typically at least 100 pixels, high-resolution measurements can be obtained.

[0216] The processing unit 9 is arranged to receive the pixel signals from the image sensor 120 and store them in the memory 15. The pixel signals may be stored by the processing unit 9 as a single picture, preferably with a time stamp and / or a position stamp indicating the position of the camera 7. However, preferably, the pixel signals are stored by the processing unit 9 as a series of pictures together constituting a video, wherein each picture is provided with a time stamp and / or a position stamp indicating the position of the camera 7.

[0217] As known to those skilled in the art, the clock 23 provides clock signals to the processing unit 9. The clock signals are used for normal processing of the processing unit 9. The processing unit 9 can base time stamping on these clock signals. However, the camera 7 can also be equipped with a GNSS unit that receives time signals from satellites, or can receive time signals from another suitable source.

[0218] The memory 15 may include different types of sub-memories, such as ROM (read only memory) / flash memory that stores appropriate program instructions and data to run the processing unit 9. In addition, the memory includes an appropriate RAM (random access memory) type memory that is used to store temporary data, such as data received from the image sensor 120. The memory 15 may also include a cache type memory. Some or all of the sub-memories may be physically located away from other components. The processing unit 9 may also be arranged to send all pixel signals to a remote unit via the electronic networking module 20 for external storage and processing. A local copy of these pixel signals can then (but not necessarily) be stored in a local memory 15 within the camera 7.

[0219] The memory 15 stores initial position data indicating the initial position of the camera 7. Such initial position data may be established using a theodolite and then stored by the user. Such initial position data may also be generated by measurements made by the camera 7 itself. For example, the camera 7 may collect continuous pictures from a known "flash" light source mounted on a high-altitude traffic obstacle sign with a known position. Such obstacle signs may be arranged within a defined vertical distance on a tall structure, thereby allowing triangulation. The memory 15 also stores a camera ID, which identifies the camera 7 and is used by the processing unit 9 in external communications with other devices to identify itself to these other external devices.

[0220] The position and / or orientation measuring means 16 may comprise one or more accelerometers and / or gyroscopes / gyroscopes, as known to those skilled in the art. They may also comprise the above-mentioned GNSS unit. Such accelerometers and / or gyroscopes / gyroscopes measure the motion of the camera itself and derive an updated camera position and orientation from such measurements. The processing unit 9 then stores the updated camera position and / or orientation in the memory 15. In this way, when measuring the position of one or more survey reflectors 1, the changing camera position and / or orientation may be taken into account. The accuracy may be in the order of a few thousandths of a degree. Tests have shown a peak-to-peak value of 2 milli-degrees. In addition, the three-axis accelerometer assembly is also capable of measuring the direction of the earth's gravity when stationary. A three-dimensional gyroscope assembly with sufficient performance is capable of measuring the direction of the earth's axis of rotation (also when stationary).

[0221] The output unit 17 may include one or more sub-output units, such as a display and a speaker.

[0222] The input unit 19 may include one or more sub-input units, such as a keyboard and a microphone. The display and the keyboard may be made into two different touch screens. However, they may also be implemented as a single touch screen.

[0223] The electronic networking module 20 may include LTE (Long Term Evolution), Ethernet, WiFi, Bluetooth, power line communication, low power wide area network (such as LoRa TM and Sigfox TM ) and one or more of NFC (Near Field Communication) modules. Technologies known in the field of IoT (Internet of Things) and any proprietary communication protocols can be used.

[0224] The at least one light source 102 comprises at least one light source, such as a light emitting diode (LED) light source configured to generate light. The processing unit 9 is arranged to control each LED light source such that they generate a light beam.

[0225] The camera 7, 27 is usually arranged in a fixed position so that it is static. Here, the static position is known and stored in a memory 15 accessible to the processing unit 9 of the camera.

[0226] When all survey reflectors or targets 1 , 21 are installed, they have an initial position / azimuth which can be stored in the memory 15 of the camera.

[0227] Thus, when the system starts up, the camera knows all the initial positions / azimuths of the survey reflectors / targets corresponding to the initial positions and orientations of the objects 3 to which they are attached.

[0228] The processing unit 9 is arranged to calculate the initial stereo angle of incidence of each reflected light beam 6. That is, the received reflected light beam is imaged by the non-refractive optical device onto one or more photosensitive elements of the image sensor 120. The processing unit 9 determines which photosensitive elements these are and then establishes the stereo angle of incidence of the corresponding light pulse. The techniques for doing so are known to those skilled in the art and do not need to be explained in further detail here.

[0229] When the object 3 is stable, i.e. does not move, the positions of all survey reflectors 1 are also stable. Therefore, the stereo angle of incidence of each reflected light beam on the camera image sensor is fixed. However, as soon as the object 3 or a part thereof moves, this stereo angle of incidence of the reflected light beam 6 changes. The processing unit 9 is arranged to calculate this change in the stereo angle of each light beam 6.

[0230] For example, images recorded by a camera can be processed as described in WO 2019 / 143250 A1 and WO 2021 / 167452 A2 and / or by any method known to a person skilled in the art to determine the position, orientation and / or observation angle of multiple targets observed by the camera.

[0231] According to this disclosure, Figure 4A A schematic diagram of the image sensor 24 and the diffraction stop 23 of a camera 27, seen from the side, and the x, y and z axes of a coordinate system which can be used to define a camera model and to calculate parameter offset values ​​for calibrating the camera model. In a preferred embodiment, the nominal position of the center of the diffraction stop is taken as the origin of the coordinate system. The x-axis and the y-axis are substantially parallel to the surface of the image sensor 24. The z-axis represents the optical axis of the camera, and the distance between the surface of the image sensor 24 at the optical axis and the center of the diffraction stop 23 at the origin, i.e. the distance along the z-axis, represents the focal length.

[0232] Figure 4B Shows the view from the front of the camera Figure 4A , showing the front side 28 of the housing provided with the diffraction stop 23 .

[0233] According to an embodiment, the x-axis may be arranged along a horizontal plane and the y-axis may be arranged along a vertical plane substantially perpendicular to the horizontal plane.

[0234] exist Figure 4C , the diffraction stop 23 and the photon-sensitive surface of the image sensor 24 at the nominal position, as well as the offset of the diffraction stop along each of the three coordinate axes are schematically shown. Figure 4C As schematically shown, the position of the diffraction stop 23 can deviate in three dimensions from the nominal position, ie the intended or designed position.

[0235] Deviations in the x, y and z dimensions are referred to as x offset, y offset and focus error, respectively. For simplicity, the description of the effects of the deviation parameters of the camera model will be limited to targets viewed on approximately horizontal or vertical lines, both of which pass through the optical axis (forming a cross).

[0236] A measurement system with a well-calibrated camera exhibits the following behavior:

[0237] If the camera is rotated around the aperture in the horizontal plane, that is, around Figure 4B If the y-axis rotation ("yaw" movement) shown is performed, the reported horizontal angles of the targets horizontally distributed in the aforementioned finite target set, that is, the calculated observation angles in the horizontal direction, will all change by an amount, which is called the differential angle (or differential angle), which is equal to the rotation angle of the camera.

[0238] If the camera is rotated around the aperture in a vertical plane parallel to the optical axis, that is, around Figure 4B If the x-axis rotation ("pitch" motion) shown is performed, then the vertical angles reported for the vertically distributed targets in the aforementioned limited set of targets will all change by an amount, called the difference angle, which is equal to the rotation angle of the camera.

[0239] The above mentioned deviations have an influence on the measured data. Possible deviations of the image sensor position in the opposite direction to the deviation of the aperture have the same effect.

[0240] Partially independent effects of the deviation:

[0241] If only the x-offset is non-zero and the camera is rotated about the y-axis of the aperture, then the calculated horizontal angle of the horizontally distributed targets in the aforementioned finite set of targets changes by the angle of rotation of the camera plus the so-called angular error (which can also be negative). This error is proportional to the horizontal angle between the orientation of the target and the optical axis.

[0242] If only the y offset is non-zero, and the camera is rotated about the x-axis of the aperture, then the calculated vertical angle of the vertically distributed targets in the aforementioned finite set of targets changes by the angle of rotation of the camera plus the angular error, which is proportional to the vertical angle between the orientation of the target and the optical axis.

[0243] If only the focus error is non-zero and the camera is rotated about the Y axis of the aperture, then the calculated horizontal angle of the horizontally distributed targets in the aforementioned finite set of targets changes by the angle of rotation of the camera plus the difference angle error. The difference angle error is positive or negative, depending on the sign of the focus error. The absolute value of the difference angle error is maximum for targets on the optical axis. The absolute value of the difference angle error decreases with the absolute value of the angle between the horizontal angle of the target direction and the optical axis.

[0244] If only the focus error is non-zero and the camera is rotated about the X-axis of the aperture, then the calculated vertical angle of the vertically distributed targets in the aforementioned finite target set changes by the camera's rotation angle plus the difference angle error. The difference angle error is positive or negative, depending on the sign of the focus error. The absolute value of the difference angle error is maximum for targets on the optical axis. The absolute value of the difference angle error decreases with the absolute value of the vertical angle of the target direction and the angle between the optical axis.

[0245] By (slightly) rotating the camera (eg 1 degree) in the horizontal plane or in the vertical plane parallel to the optical axis (or a combination of both), different angles and potential errors therein can be introduced.

[0246] If the x, y and z offsets are determined by mathematical expressions or by iteration of an algorithm and subsequently compensated for in processing of the measured image data to determine the viewing angle, orientation and / or position of the target, then the remaining differential angle error caused by the aperture offset will be negligible.

[0247] With one or more aperture offsets, targets that are in the camera's field of view but not in the limited directions mentioned above (i.e., not on the x-axis or y-axis) will also exhibit differential angular error, but the relationship between aperture offset and differential angular error is more complex.

[0248] Therefore, in order to facilitate the calculation of the camera model parameters, it is preferred to use only restricted directions of the targets, i.e., using targets arranged along one or both of the x-axis and the y-axis. In addition, it can be understood from the effect of the aperture shift described that if the camera is rotated 90 degrees around its optical axis between measuring the horizontal angle and measuring the vertical angle (not necessarily in this order), then multiple targets distributed within the field of view of the camera can be used, even in only one plane, such as on the ground, to measure horizontal and vertical differential angle errors.

[0249] Figure 5 A system 100 or apparatus for calibration measurement according to an embodiment of the present disclosure is schematically shown. The system comprises a camera 7, which may be a reference Figure 2 and Figure 3The camera 7 is illustrated; a plurality of targets 1 arranged to be observed by the camera 7; and an external processing unit 190, which includes one or more external processors, for example located in a laptop or desktop computer. The external processing unit 190 is connected to or connectable to the camera 7, i.e., to an electronic networking module 109 for receiving recorded images and possible other data from the camera 7, for further processing, such as one or more of the calculation steps for calculating observation angles, difference angles, camera model errors and / or calibration parameter values, as described with respect to various embodiments of the present disclosure. In particular, the one or more processors 190 can be configured, i.e., programmed to perform reference Figure 7 One or more of steps 7030-7100 described above, or in Fig. 9 or Fig.10 One or more steps shown in .

[0250] The camera 7 is arranged on a mounting device, such as a rotating table. The mounting device can be any mounting device as described in the summary of the invention above. Figure 5 The camera 7 is shown rotated from a first rotational position P1 to a second rotational position P2. By recording images of a plurality of targets 1 with the camera in each of these positions, the aperture shift and the calibration parameter values ​​can be calculated.

[0251] As mentioned above, the target 1 can be a reflective target or an active light source. The distance d between the target 1 and the camera 7 can be set based on various parameters, such as the camera, the algorithms used for image processing and calculation of the target position and the viewing angle, practical considerations, such as space limitations when performing calibration in a laboratory environment or in the field, etc.

[0252] exist Figure 5 In FIG, the targets are shown arranged in a plane, such as a horizontal plane, and the camera is also located in the plane. Figure 5 In the embodiment, the targets are further shown as being arranged along a semicircle, with all targets being at substantially the same distance from the camera. However, this is not limiting, and the targets 1 may also be arranged along a straight line. In other embodiments or alternatives of the method, a plurality of targets may be arranged in a two-dimensional arrangement or array, such as Figure 8 shown.

[0253] like Figure 5 As shown, the targets are preferably substantially evenly distributed between a first lateral or peripheral position L1 and a second lateral or peripheral position L2, and a central target is arranged substantially along the optical axis of the camera. Although this arrangement may be preferred, as described in the Summary of the Invention section, other arrangements are possible.

[0254] Generally speaking, the distance d may be a few meters, depending on the distance used in an actual, real-life survey setting. In the example discussed herein, the distance is approximately 3 meters.

[0255] Figure 6 Schematic diagram showing the use of Figure 5 The scene observed by the camera 7 when moving from a first rotational position P1 to a second rotational position P2 during a calibration measurement of the system 100. The solid circles respectively show the observation positions of the multiple targets 1 when the camera is at position P1, and the dashed circles show the observation positions of the multiple targets 1 when the camera is at position P2. In each of the positions P1 and P2, the position of each target can be represented by its calculated observation angle HAi. Figure 6 In FIG. 1 , the calculated observation angle HAi at P2 and the difference angle ΔHAi are shown, where the difference angle ΔHAi represents the change in the calculated observation angle of each target between P1 and P2. The calibration of the camera, i.e., the calculation of the calibration parameter values, can be performed using the following method: Figure 7 The method shown proceeds, starting from an initial camera model and images recorded at P1 and P2, respectively.

[0256] Figure 7 A flow chart showing a method of calibrating a camera 7 of an embodiment of the present disclosure is shown. Although the method is described herein with respect to a camera of a monitoring or surveying system, the method may be similarly applied to cameras for other types of computer vision systems.

[0257] Figure 7 Steps 7010 to 7100 of the method may advantageously be performed on one or more processors, such as the internal processor 109 of the camera and / or one or more external processors 190.

[0258] At the beginning Figure 7 Prior to the flowchart shown in FIG. 1 , multiple targets are distributed within the field of view of the camera. For example, the targets and the camera may be arranged as follows: Figure 5 Arrange as shown.

[0259] In step 7010, an initial camera model is provided. This may be a camera model based on the design of the camera, or a camera model obtained during a previous calibration of the camera. The camera model may include a mathematical model of the position of the pixels of the image sensor relative to the stop 23, i.e. the center of the objective lens of the camera. The model may be based on a reference Figures 4A-4C The camera model can also take into account other parameters and characteristics of the camera.

[0260] In step 7020, the camera 7 is arranged at a first rotation position P1 relative to the target 1, for example Figure 5A position P1 is defined at which a first image of the plurality of targets is recorded.

[0261] In step 7030, a first set of calculated viewing angles is determined from the first image using the camera model and algorithms provided in step 7010, which may be known in the art, as described above. The first set of calculated viewing angles includes a first calculated viewing angle of each target relative to the camera, wherein the camera is in a first rotational position.

[0262] In step 7040, the camera is moved to a second rotational position relative to the plurality of targets, e.g. Figure 5 and recording a second image of the plurality of targets using the camera in a second rotational position.

[0263] In step 7050, similar to step 7030, a second set of calculated viewing angles is determined from the second image using the camera model provided in step 7010. The second set of calculated viewing angles includes a second calculated viewing angle of each target relative to the camera, wherein the camera is in a second rotational position.

[0264] In step 7060, a function or graph is determined based on the first and / or second set of calculated observation angles and a set of difference angles representing the difference between the first calculated observation angle and the second calculated observation angle for each target. An example of such a graph is shown in the step between steps 7060 and 7070.

[0265] In step 7070, the camera model error is determined using a function or graph. The camera model error may be determined as an offset value relative to one or more parameters defining the camera model, such as Figure 4C shown.

[0266] The parameter offset of one or more of the x, y, and z coordinates may be determined as follows:

[0267] The linear (first order) component represents the X offset error (when the movement from the first rotational position to the second rotational position is in the horizontal plane) and / or the Y offset error (when the movement from the first rotational position to the second rotational position is a rotation in the vertical plane).

[0268] - The quadratic (second order) component represents the Z (focus) offset error.

[0269] - A flat function / graph means there is no error in X(Y) or Z. In this case, the calculated and actual viewing angles are equal.

[0270] The camera model error or parameter offset value may be quantified, ie, the numerical value of the error may be calculated based on the first-order and / or second-order components.

[0271] In step 7080, the camera model error is compared to preset limits which are considered to provide measurement data of sufficient accuracy.

[0272] If the camera model error is not within the preset limit, the camera model error is fed back into the camera model and an updated camera model is provided in step 7090. Subsequently, the algorithm or flowchart is re-iterated for steps 7030, 7050, 7060, 7070 and 7080. This re-iteration is performed until the resulting camera model error is below the current limit.

[0273] If the camera model error is less than the current limit, the camera model is determined to be acceptable, that is, calibrated to sufficient accuracy, and step 7100 is executed.

[0274] As described above, the movement of the camera from the first rotational position in 7020 to the second rotational position in 7040 may be performed in one plane, which is typically a horizontal plane. Then, Figure 7 The method described in leads to a calibration in this plane and in the direction of the focus, ie in the x and z directions.

[0275] For some applications or scenarios, calibration along the x and z directions may be sufficient. For example, for many surveying applications, calibration in the horizontal direction is considered to be the most critical.

[0276] Referenced above Figure 7 The described methods can be performed in a laboratory environment as well as in a field measurement environment.

[0277] According to other embodiments, the plurality of targets may be arranged in a two-dimensional arrangement, for example, Figure 8 The arrangement shown comprises two one-dimensional arrangements along the x-axis and the y-axis, respectively, i.e. arranged substantially perpendicularly and crosswise relative to each other. In such an embodiment, the movement of the camera from the first rotational position to the second rotational position may comprise a rotation about the x-axis and a rotation about the y-axis, which enables the determination of a calculated viewing angle relative to the x-axis and the y-axis, thereby enabling calibration in all three dimensions using the first and second recorded images.

[0278] exist Fig. 9 This method is shown in the flowchart of FIG. Steps 9010, 9020, 9030, 9040, 9050, 9080, 9090 and 9100 are similar to Figure 7 Steps 7010, 7020, 7030, 7040, 7050, 7080, 7090 and 7100 are not described in detail here.

[0279] In step 9060, the first and second sets of observation angles each include a horizontal angle component and a vertical angle component. A first function or graph is determined based on the first and / or second set of calculated horizontal components of the observation angles and a set of horizontal difference angles representing the difference between the first calculated horizontal component and the second calculated horizontal component of the observation angle for each target. An example of such two graphs is shown in step 9065 between steps 9060 and 9070.

[0280] In step 9070, a camera model error is determined using the first and second functions or graphs. The camera model error may be determined as an offset value relative to one or more parameters defining the camera model, such as Figure 4C shown.

[0281] The parameter offset of one or more of the x, y, and z coordinates may be determined as follows:

[0282] - The linear (first order) component in the first function or graph represents the X offset error (corresponding to the component of motion in the horizontal plane from the first rotational position to the second rotational position).

[0283] - The linear (first order) component in the second function or graph represents the Y offset error (corresponding to the component of motion in the vertical plane from the first rotational position to the second rotational position).

[0284] - The quadratic (second order) components in the first and second functions or graphs represent the Z (focus) offset error.

[0285] - Flat first and second functions / graphs indicate no errors in X, Y and Z. In this case, the calculated and actual viewing angles are equal.

[0286] In other embodiments, the camera is calibrated along the y direction. After step 7100, the camera model is considered to be calibrated along the x direction. The method for calibration can be as follows: Fig.10 or Fig.11 In this case, the camera model leading to step 7100 can be used as Fig.10 or Fig.11 The input in one of the flowcharts is, namely, the initial camera model.

[0287] In use Fig.10 The flowchart further illustrates the method in which the camera model is calibrated along the y-axis and then along the x-axis, as Figure 7 As shown. Upon arrival Figure 7 At step 7100, the camera is rotated substantially 90 degrees about its optical axis and calibration is continued, as shown in FIG. Fig.10 The flowchart is shown in the following figure. Figure 7The method is repeated in the same manner, with the camera rotated 90 degrees. This enables the camera to be calibrated along the x-axis and y-axis using a one-dimensional target array, such as Figure 5 shown.

[0288] In step 1010, an initial camera model is provided. This may be from Figure 7 Flowchart of the method to obtain the camera model.

[0289] In step 1020 , the camera 7 is placed in a third rotational position P3 relative to the target 1 , in which step third images of the plurality of targets are recorded.

[0290] In step 1030, a third set of calculated viewing angles is determined from the third image using the camera model provided in step 1010. The third set of calculated viewing angles includes third calculated viewing angles of each target relative to the camera, wherein the camera is in a third rotational position.

[0291] In step 1040, the camera is moved to a fourth rotational position, such as position P4, which is similar to Figure 5 A rotation from a first rotational position P1 to a second rotational position P2 is shown. A fourth image of the plurality of targets is recorded using the camera in a fourth rotational position.

[0292] In step 1050, a fourth set of calculated viewing angles is determined from the fourth image using the camera model.

[0293] In step 1060, a function or graph is determined based on the third and / or fourth sets of calculated observation angles and a set of difference angles representing the difference between the third calculated observation angle and the fourth calculated observation angle for each target.

[0294] In step 1070, a function or graph is used to generate a Figure 7 The camera model error is determined in the manner of method step 7070 .

[0295] In step 1080, the camera model error is compared to a preset limit value, which indicates an acceptable camera model error.

[0296] If the camera model error is not within the preset limit, the camera model error is fed back into the camera model and an updated camera model is provided in step 9090. Subsequently, the algorithm or flowchart is re-iterated for steps 1030, 1050, 1060, 1070 and 1080. This re-iteration is performed until the resulting camera model error is below the current limit.

[0297] If the camera model error is less than the current limit, the camera model is determined to be acceptable, that is, calibrated to a sufficient accuracy, and step 1100 is executed.

[0298] In a method according to another embodiment, using Fig.11 The flowchart further illustrates that the camera model is calibrated along the y-axis and then along the x-axis, as shown in Figure 7 As shown. Upon arrival Figure 7 At step 7100, the camera is rotated relative to the horizontal plane, that is, around the x-axis, step 1140, and calibration is continued, as shown in FIG. Fig.11 In this method, a two-dimensional arrangement of targets is used, such as Figure 8 shown.

[0299] In step 1110, an initial camera model is provided. This may be from Figure 7 Flowchart of the method to obtain the camera model.

[0300] In step 1120, using the image previously recorded at the second rotational position, a third set of calculated viewing angles is determined in step 1130, the third set of calculated viewing angles comprising the viewing angles of each target relative to the y-axis. Fig.11 is shown as Figure 7 The calibration along the x-axis is shown as a separate step performed after completion, but the third set of calculations to calculate the viewing angle can be done with Figure 7 The second set of calculations in step 7050 are performed simultaneously with the determination of the observation angle.

[0301] In step 1140, the camera is moved from the second rotational position to a third rotational position by rotating about the x-axis, i.e., relative to the horizontal plane. This can also be represented as the camera being tilted slightly downward or upward. The amount of rotation is preferably similar to the amount of rotation when moving from the first rotational position to the second rotational position. A third image of the plurality of targets is recorded using the camera in the fourth rotational position.

[0302] In step 1150 , a fourth set of calculated viewing angles is determined from the fourth image using the camera model.

[0303] In step 1160, a function or graph is determined based on the third and / or fourth sets of calculated observation angles and a set of difference angles representing the difference between the third calculated observation angle and the fourth calculated observation angle for each target.

[0304] In step 1170, a function or graph is used similar to reference Figure 7 The camera model error is determined in the manner of method step 7070 .

[0305] In step 1180, the camera model error is compared to a preset limit value, which indicates an acceptable camera model error.

[0306] If the camera model error is not within the preset limit, the camera model error is fed back into the camera model and an updated camera model is provided in step 1190. Subsequently, the algorithm or flowchart is re-iterated for steps 1130, 1150, 1160, 1170 and 1180. This re-iteration is performed until the resulting camera model error is below the current limit.

[0307] If the camera model error is less than the current limit, the camera model is determined to be acceptable, i.e., calibrated to sufficient accuracy, step 11100.

[0308] Thus, according to this method, the two-dimensional distribution of the targets can be used to calibrate the camera in all three dimensions based on the three recorded images.

[0309] In the method described above, calibration is described as calibrating the camera model along the horizontal axis or x-axis before calibrating along the vertical axis or y-axis. However, it should be understood that the order is not important and calibration can also be performed first along the vertical axis and then along the horizontal axis.

[0310] Figure 12-17 Graphs showing the differential angle error of a target simulated at different stages of calibration of a camera model in three dimensions x, y and z according to the method described herein are shown. This provides an illustration of the results of the calibration steps described herein. The calibration is simulated for x-offset and y-offset.

[0311] The simulations were performed using the following settings and assumptions:

[0312] The images are slightly noisy (intended to simulate images from a real camera).

[0313] All offsets (in x, y, and z = focal length) were set to 100 microns.

[0314] To generate different angles, the camera is rotated by an angle of 1 gon.

[0315] For leveling and calibration, five targets are placed on the horizon approximately 15 meters apart from each other, with one target in the middle.

[0316] For vertical measurement and calibration, five targets were arranged to appear in a vertical line, with all targets at a horizontal angle of approximately 0 Å (in other words, in the middle of the field of view), approximately 10 Å apart from each other, with one target in the middle.

[0317] Targets are placed at imperfectly regular locations to provide a realistic simulation of calibration.

[0318] The results are described as follows:

[0319] Fig.12 A plot of the differential angle error for each of the targets 1-5 is shown versus their respective calculated viewing angles. Here, the horizontal differential angle error is calculated from the measured horizontal viewing angles, prior to any calibration.

[0320] Fig.13 A plot of the angular error versus the calculated observation angle after correcting the x-offset to make the dashed trend line horizontal is shown. The offset between the center of the diffraction stop and the center of the image sensor (so defined) is actually compensated along an axis, such as the x-axis. This axis is parallel to the sensor plane and is actually parallel to the plane in which the target and the camera are arranged.

[0321] Fig.14 The graph is shown after focus error correction, which flattens the imaginary curvature through the point.

[0322] Fig.15 Shown in Fig.13 A plot of the vertical error angle versus the observation angle after horizontal calibration. Here, the vertical error angle is calculated from the measured vertical observation angle.

[0323] Fig.16 The graph is shown after y-offset compensation has been made, which makes the trend line horizontal.

[0324] Fig.17 The horizontal difference angle error is calculated again from the measured horizontal observation angle for verification. Fig.11 Comparison of the graphs of shows that the graph is now essentially flat, with no first or second order components.

[0325] In this simulated case, if this gives a better compromise between horizontal and vertical angular errors, a second adjustment of the focus could be made based on measurements in the vertical plane. Fig.17 It can be seen that this is not necessary in the simulation case, as an acceptable focus offset calibration is already achieved from the horizontal calibration.

[0326] Thus, in summary, the present disclosure provides a method and system that can calibrate a diffraction stop based camera for surveying and other computer vision purposes. The present disclosure provides different alternatives for achieving calibration of a camera model along three different dimensions, which can be performed using a simple arrangement of targets. Neither the exact position of the target nor the amount of rotation of the camera need to be known.

[0327] It will be apparent to those skilled in the art that the scope of the present invention is not limited to the examples discussed above, and that various modifications and variations can be made thereto without departing from the scope of the present invention as defined by the appended claims. Although the present invention is shown and described in detail in the drawings and the specification, such illustration and description are merely illustrative or exemplary and not restrictive. The present invention is not limited to the disclosed embodiments, but includes any combination of the disclosed embodiments that can bring advantages.

[0328] By studying the drawings, the specification and the appended claims, those skilled in the art can understand and effect variations to the disclosed embodiments in practicing the claimed invention.The features of the above embodiments and aspects can be combined unless their combination results in an obvious technical conflict.

Claims

1. A method for calibrating a camera of a computer vision system, the camera comprising an image sensor and a diffraction stop arranged to project light onto the image sensor, the method comprising the following steps: (a) distributing a plurality of targets within the field of view of the camera; (b) providing a camera model representing a mathematical model of the camera; (c) placing the camera at a first rotational position relative to the targets and recording a first image of the plurality of targets using the camera at the first rotational position; (d) determining a first set of calculated viewing angles from the first image using the camera model, the first set of calculated viewing angles comprising a first calculated viewing angle of each target relative to the camera; (e) rotating the camera relative to the targets to a second rotational position, and recording a second image of the plurality of targets using the camera in the second rotational position; (f) determining a second set of calculated viewing angles from the second image using the camera model, the second set of calculated viewing angles comprising a second calculated viewing angle of each target relative to the camera; (g) determining a function or graph based on the first set of calculated observation angles and / or the second set of calculated observation angles and a set of difference angles representing the difference between the first calculated observation angle and the second calculated observation angle for each target; (h) determining a camera model error from the function or graph; (i) comparing the camera model error with a predetermined limit; (j) if the camera model error is below the limit, determining that the camera model is acceptable; (k) If the camera model error is not lower than the limit value, calculating an updated camera model based on the camera model error, and repeating steps d), f), g), h), i) and j) or k) based on the updated camera model.

2. The method according to claim 1, further comprising: - once the camera model is determined to be acceptable, rotating the camera by substantially 90 degrees about its optical axis; and - Repeat steps c) to k).

3. The method according to claim 2, wherein: At least a portion of the plurality of targets are arranged in a substantially one-dimensional row as viewed from the camera, the one-dimensional row and the camera being located substantially in one plane.

4. The method of claim 3, wherein step (f) of rotating the camera from the first rotational position to the second rotational position is performed substantially within the plane.

5. The method according to claim 1, wherein: The plurality of targets are arranged in a two-dimensional arrangement as viewed from the camera, and wherein the step (f) of rotating the camera from the first rotational position to the second rotational position comprises a rotation in a horizontal plane and a rotation in a vertical plane.

6. The method according to claim 1, wherein: The plurality of targets are arranged in a two-dimensional arrangement as viewed from the camera, and wherein the step (f) of rotating the camera from the first rotational position to the second rotational position is performed substantially in a horizontal plane about a first axis, the first axis passing through the plane of the aperture and extending substantially perpendicular to the horizontal plane, and the first calculated observation angle and the second calculated observation angle represent angles in the horizontal plane; The method further comprises: (l) determining a third set of calculated viewing angles from the second image using the camera model, the third set of calculated viewing angles comprising a third calculated viewing angle of each target relative to the camera, the third calculated viewing angle representing an angle in a direction substantially perpendicular to a horizontal plane; Once it is determined in step (j) that the camera model is acceptable, the method further comprises the following steps: (m) rotating the camera about a second axis passing through an aperture of the camera to rotate the camera to a third rotational position, the second axis being substantially perpendicular to the first axis and extending in a horizontal plane; (n) recording a third image of the plurality of targets with the camera in the third rotational position; (o) determining a fourth set of calculated viewing angles from the third image using the camera model, the fourth set of calculated viewing angles comprising fourth calculated viewing angles of each target relative to the camera; (p) determining a second function or graph based on the third set of calculated observation angles and / or the fourth set of calculated observation angles and a set of second difference angles representing the difference between the third calculated observation angle and the fourth calculated observation angle for each target; and (q) determining a second camera model error from the second function or graph; (r) comparing the second camera model error with a second predetermined limit; (s) if the second camera model error is lower than the second limit, determining that the camera model is acceptable; (t) If the second camera model error is not lower than a second limit, calculating an updated camera model based on the second camera model error, and repeating steps (l), (o)-(r) and (s) or (t) based on the updated camera model.

7. A method as described in any of the preceding claims, wherein step (a) includes arranging the plurality of targets so that one target is arranged at each lateral periphery of the field of view of the camera, and any remaining targets in the plurality of targets are substantially evenly distributed between the targets arranged at the lateral peripheries.

8. The method according to any one of the preceding claims, further comprising: (h1) determining calibration parameters as corrections to be applied to parameters of a mathematical model representing the camera, wherein the calibration parameters are determined such that the difference angles are substantially equal for each target.

9. The method according to any of the preceding claims, wherein the camera model comprises the following data as parameters: The three-dimensional positions of pixels of the image sensor relative to the diffraction stop.

10. A method according to any one of the preceding claims, wherein the method is performed simultaneously with the actual measurement using a computer vision system.

11. The method according to any one of the preceding claims, further comprising: The camera is brought to a second temperature and the steps of any of the preceding claims are repeated at the second temperature.

12. A system for calibrating a camera of a computer vision system, the camera comprising an image sensor and a diffraction stop arranged to project incident light onto the image sensor, the system comprising: a plurality of targets to be viewed by the camera; a mounting device for mounting the camera so that when the camera is mounted on the mounting device, the plurality of targets are within the camera's field of view, the mounting device enabling the camera to rotate between a first rotational position and a second rotational position relative to the plurality of targets; and One or more processors, the processors configured to: (b) providing a camera model representing a mathematical model of the camera; (c) recording a first image of the plurality of targets with the camera in a first rotational position; (d) determining a first set of calculated viewing angles from the first image using the camera model, the first set of calculated viewing angles comprising a first calculated viewing angle of each target relative to the camera; (e) recording a second image of the plurality of targets with the camera in a second rotational position; (f) determining a second set of calculated viewing angles from the second image using the camera model, the second set of calculated viewing angles comprising a second calculated viewing angle of each target relative to the camera; (g) determining a function or graph based on the first set of calculated observation angles and / or the second set of calculated observation angles and a set of difference angles representing the difference between the first calculated observation angle and the second calculated observation angle for each target; (h) determining a camera model error from the function; (i) comparing the camera model error with a predetermined limit; (j) if the camera model error is below the limit, determining that the camera model is acceptable; (k) If the camera model error is not lower than the limit, calculating an updated camera model based on the camera model error, and repeating steps d), f), g), h), i) and j) or k) based on the updated camera model.

13. The system of claim 12, wherein the mounting means comprises a microprocessor controlled rotation means.

14. A system according to claim 12 or 13, wherein the mounting arrangement is configured to enable the camera to be rotated about a vertical axis and / or a horizontal axis.

15. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the following steps: (b) providing a camera model representing a mathematical model of a camera, the camera comprising an image sensor and a diffraction stop arranged to project incident light onto the image sensor; (d) determining, from a first image of the plurality of targets recorded with a camera at a first rotational position relative to the plurality of targets, using the camera model, a first set of calculated viewing angles from the first image, the first set of calculated viewing angles comprising a first calculated viewing angle of each target relative to the camera; (f) determining, from a second image of the plurality of targets recorded with the camera in a second rotational position, a second set of calculated viewing angles using the camera model from the second image, the second set of calculated viewing angles comprising a second calculated viewing angle of each target relative to the camera; (g) determining a function based on the first set of calculated observation angles and / or the second set of calculated observation angles and a set of difference angles representing the difference between the first calculated observation angle and the second calculated observation angle for each target; (h) determining a camera model error from the function; (i) comparing the camera model error with a predetermined limit; (j) if the camera model error is below the limit, determining that the camera model is acceptable; (k) If the camera model error is not lower than the limit value, calculating an updated camera model based on the camera model error, and repeating steps d), f), g), h), i) and j) or k) based on the updated camera model.

Images