Calibration method for survey equipment and system for calibrating survey equipment

Abstract

A method of calibrating a camera including an image sensor and a diffractive aperture that projects light onto the image sensor, the method comprising: providing a camera model that mathematically describes the camera; positioning the camera at a first rotational position with respect to a plurality of targets and recording a first image of the plurality of targets; rotating the camera to a second rotational position with respect to the targets and recording a second image; using the camera model to determine a first set of observation angles of the targets from the first image and a second set of observation angles from the second image; determining a function based on the first set of observation angles and / or the second set of observation angles and a set of difference angles representing the difference between the first and second observation angles; determining camera model errors from the function and comparing them to limits; if the errors are not less than the limits, repeating the steps of updating, determining and comparing the camera model based on the errors. By revealing insights from Geo-Data, the present invention further relates to improving sustainability and environmental development, creating a world that is safe and suitable for living together.

Description

Technical Field

[0001] The present invention relates to a method for calibrating a camera of a computer vision system, particularly a camera of a survey device, wherein the camera is provided with a diffractive aperture as an objective lens. The present invention can further relate to a system or setup for calibrating the camera. By revealing insights from Geo-Data, the present invention further relates to improving sustainability and environmental development, creating a world that is both safe and suitable for habitation.

[0002] Background Art Systems and methods for measuring the position of a remote object have been known for a long time, for example in the field of structural survey and / or monitoring. Conventionally, the cameras of such systems use lenses or refractive optical elements in the camera objective lens.

[0003] WO 2019 / 143249, WO 2019 / 143250 and WO 2021 / 167452 disclose devices and methods for monitoring the position of an object, such as a structure, over time. The disclosed systems include beacons or targets that are located on the object and are monitored by a camera to detect changes or variations in the position of the target over time.

[0004] WO 2019 / 143250 and WO 2021 / 167452 disclose systems that use a non-refractive optical element, such as a pinhole, as the camera objective lens. It has been observed that this provides several advantages over lens-based camera systems, including reduction of optical distortion, improvement of depth of field, and reduction of thermal sensitivity. In the case of such systems, it has been found that the thermal sensitivity caused, for example, by thermal expansion is mainly affected by the thermal expansion coefficient of the camera housing and not by the thermal effects for such purposes as in the case of lens-based systems.

[0005] However, the observation accuracy of camera-based survey equipment is troubled not only by the temperature-dependent shrinkage and expansion of the camera housing and other components of the camera, but also by mechanical manufacturing tolerances such as the position of the lens or aperture with respect to the optical sensor.

[0006] Therefore, similar to other measurement systems, systems that use non-refractive optical elements or diffractive apertures as camera objectives also need to be calibrated to provide accurate measurement results. Deviations of physical parameters from expected values that cause observation errors can be compensated for by calibration.

[0007] However, conventional survey equipment calibration methods have been shown to be insufficient because they mainly aim to compensate for the optical errors of the lens. In conventional calibration methods, the equipment under calibration observes the scene at defined measurement points. In these methods, at least one of the following requirements must be met. · The position of the measurement points in the scene with respect to the survey equipment must be accurately known. · The relative directions of the measurement points must be accurately known.

[0008] Therefore, conventional calibration methods require the accurately known absolute positions of many points within a pattern located within the field of view of the survey equipment, or the accurately known directions to many points within the scene. This means that the pattern itself also needs to be calibrated to a very high accuracy in order to achieve a very high accuracy of the calibrated equipment. This is difficult to achieve.

[0009] Furthermore, conventional methods are designed to calibrate lens-based optical systems that have a set of unique dynamic errors that limit the ultimate accuracy that can be achieved. These errors do not exist, or at least are substantially different, in survey systems that use objectives based on non-refractive optical elements or diffractive apertures instead of lenses.

[0010] The existence of such errors affects the accuracy of structural monitoring and the resulting uncertainties. As a result of these uncertainties, the structure may be subject to premature maintenance projects or over-parameterization during the design phase, rather than what is strictly required. This can have an adverse impact on the environment. Furthermore, such an underestimation of structural integrity can also lead to significant safety consequences if the measurement results indicate less movement than actual.

[0011] Diffraction aperture-based cameras can measure target position changes with very high accuracy because there is no lens-based dynamic distortion. However, to achieve this high-precision measurement, a very accurate calibration method is required.

[0012] Therefore, it is necessary to define a calibration method for diffraction aperture-based survey equipment.

[0013] Summary of the Invention The problem addressed by this specification includes providing a method and system that enable calibration of a camera using a non-refractive optical element, such as a diffraction aperture or a pinhole, as a camera objective lens for projecting an image onto an image sensor.

[0014] More specifically, an object of the present invention is to overcome the drawbacks of the prior art and enable calibration of a camera provided with a diffraction aperture for projecting an image onto an image sensor.

[0015] According to a first aspect of the present invention, there is provided a method for calibrating a camera of a computer vision system, the camera including an image sensor and a diffraction aperture configured to project light onto the image sensor, the method comprising: (a) distributing a plurality of targets within the field of view of the camera; (b) providing a camera model representing a mathematical model of the camera; (c) Placing the camera at a first rotational position with respect to the target and recording a first image of a plurality of targets with the camera at the first rotational position; (d) Determining a first set of observed angles calculated from the first image using a camera model, the first set of calculated observed angles including the first calculated observed angle for each target with respect to the camera; (e) Rotating the camera to a second rotational position with respect to the target and recording a second image of the plurality of targets with the camera at the second rotational position; (f) Determining a second set of observed angles calculated from the second image using the camera model, the second set of calculated observed angles including the second calculated observed angle for each target with respect to the camera; (g) Determining a function or plot based on the first set and / or the second set of calculated observed angles and a set of difference angles, the difference angles representing the difference between the first calculated observed angle and the second calculated observed angle for each target; (h) Determining a camera model error from the function or plot; (i) Comparing the camera model error with a predetermined limit; (j) If the camera model error is less than the limit, determining that the camera model is acceptable; (k) If the camera model error is not less than the limit, calculating an updated camera model based on the camera model error and repeating steps d), f), g), h), i) and j) or k) based on the updated camera model.

[0016] The camera model includes a mathematical representation of the camera, in particular, a mathematical model including parameter values that describe the relationship between the diffraction aperture and the pixels of the image sensor. As an initial model, a mathematical model representing the design specifications of the camera can be used. Alternatively, a camera model determined in a previous calibration can be used.

[0017] This method is an iterative method for calculating an updated, i.e., calibrated, camera model, and the calculation is repeated until the camera model error, or the parameter offset value, falls within a predetermined limit or threshold.

[0018] The specific amount by which the camera is rotated between two measurements, i.e., when moving from a first rotational position to a second rotational position, is not important and need not be known. It is the measured change in the observation angle of each target that forms the basis for the calculation of the calibration parameters.

[0019] The rotation from the first rotational position to the second rotational position preferably includes a substantially pure rotational movement. However, for example, due to the rotational axis not being positioned at the nominal position of the diffraction aperture, the movement can also include a translational movement. The influence of the translational movement on the calibration result can be minimized by placing the target at a sufficiently large distance. Alternatively, the translational movement may be determined and corrected algorithmically during the calibration process.

[0020] The order of the above steps is not essential and can be changed as will be understood by those skilled in the art. For example, although step (d) of determining the first set of calculated observation angles is mentioned before step (e) of rotating the camera to the second rotational position, the step of rotating to the second rotational position and recording the second image can be performed before determining the first set of calculated observation angles and the second set of calculated observation angles.

[0021] In contrast to conventional calibration methods, in the calibration method according to the present disclosure, an exactly known position or orientation of a calibration pattern such as a target array is not required. The calibration method is based on the fact that the observed change in the observation angle due to the rotation of the camera must be equal for all targets, regardless of the direction or position of each target.

[0022] The optical axis of the camera is defined by a conventional method. The optical axis of the camera can be represented by an imaginary line passing through the center of the diffraction aperture and perpendicular to the plane of the image sensor.

[0023] The rotation from the first rotational position to the second rotational position is typically a rotation of about 1 gon, 1 degree, or several degrees. In principle, the camera can be rotated by a larger angle between measurements, as will be further explained below, but it has been found advantageous to place the target near the periphery of the camera's field of view. The amount of rotation should then be selected such that the target on the opposite side of the surroundings is detected by the camera at both the first and second rotational positions.

[0024] The actual, or real, observation angle is generally defined as the separation angle of the target with respect to the optical axis of the camera. That is, the observation angle can be defined as the angle formed by the imaginary line extending from the target to the center of the aperture and the optical axis of the camera.

[0025] The calculated observation angle is the angle calculated using an algorithm that takes as input the projection of the target on the image sensor and the camera model. Such an algorithm can use techniques as will be further outlined below.

[0026] The actual observation angle and the calculated observation angle are not necessarily the same and may differ due to errors in the camera model or the accuracy of the algorithm.

[0027] The rotation from the first rotational position to the second rotational position can be substantially in the horizontal plane.

[0028] The plurality of targets may be arranged such that at least a part of them is located in the horizontal plane.

[0029] As described above, the method according to the present disclosure is designed to calibrate diffraction aperture-based cameras such as pinhole cameras. This method can be similarly applied to cameras using other types of diffraction apertures or non-refractive elements, as described in International Publication No. WO 2019 / 143250 and International Publication No. WO 2021 / 167452.

[0030] This method is not limited to use in cameras or devices for survey purposes, and can be similarly used not only in surveying applications, but also in various other applications, for example, in other computer vision systems intended for use in systems for mechanical alignment of components.

[0031] The target may be an active target or beacon that emits light itself, such as a light-emitting diode, LED. Alternatively, the target may be a passive target such as a conventional survey prism, retroreflector, or hollow mirror that reflects light emitted by the camera and / or other light sources. The target must be small enough to be effectively considered a point source when observed by the camera. The condition of the target being an effective point source sets the minimum distance between the camera and the target during the calibration measurement.

[0032] The camera generally further includes a processing unit configured to determine the first and second observation angles, as well as the change in the observation angle, i.e., the difference between the first and second observation angles of each target. The processing unit can be arranged within the camera housing where the image sensor is also arranged, or outside the housing.

[0033] The camera is typically placed on a rotation stage, which may be either a general-purpose rotation stage or a custom-made rotation stage, or on a tripod during measurement. The rotation stage or tripod enables rotation of the camera in the horizontal plane and preferably also in the vertical plane, for example, to be rotated from a first rotation position to a second rotation position. The tripod can further enable tilting of the camera with respect to the horizontal plane.

[0034] The effects of different types of rotation and / or tilt of the camera between different positions where the image is recorded, and the deviation of the aperture position from its nominal position with respect to the observation angle calculated from the recorded image, are explained in detail with respect to FIGS. 4A to 4C.

[0035] The plurality of targets preferably includes at least three targets. Generally, the number of targets can be from about 10 to 20 targets, or even up to about 100 targets. The number of targets used can be set based on what processing time and / or processing load is considered acceptable.

[0036] The targets may all be arranged at substantially the same distance from the camera. This has the advantage that all the targets can be attached to a single structural element such as a beam or an arc, and all the targets appear with a similar brightness on the image sensor.

[0037] However, it is not necessary for all the targets to be arranged at the same distance from the camera. The method can be applied to a plurality of targets located at substantially any distance from the camera.

[0038] The following considerations may be made regarding the distance between the target and the camera. That is, the target may preferably be arranged at a distance such that it appears as a substantially point light source, even though the projection on the image sensor is bright enough to be identified by the image processing algorithm. Further, the target should be arranged at the minimum distance at which the error in the calculated observation angle caused by the rotation center not being perfectly centered within the diffraction aperture is within an acceptable range.

[0039] The step of determining a function or plot based on the first set and / or second set of calculated observation angles and the set of difference angles can include providing a graphical representation of the difference angles as a function of the first calculated observation angle and / or the second calculated observation angle, and / or determining a mathematical function of the difference angles with respect to the first calculated observation angle and / or the second calculated observation angle. Such a function or plot may be determined using curve fitting and may be expressed as Δα = F(α1), or equivalently, Δα = F(α2).

[0040] In the plot, generally, the X-axis represents the calculated observation angle of each target at the first (α1) or second (α2) rotational position, and the Y-axis represents the difference angle Δα of each target. The difference angle is calculated as the difference between the calculated observation angles of each target at the first rotational position and the second rotational position, i.e., Δα = α1 - α2 or similarly Δα = α2 - α1.

[0041] From the function or plot, the camera model error can be determined as an offset value for one or more of the parameters of the camera model.

[0042] If the camera model is considered to provide a mathematical representation of the position of the diffraction aperture with respect to an image sensor represented in a Cartesian coordinate system, the coordinates of the pixel image sensor can be represented by x, y, z, with the nominal position of the center of the diffraction aperture as the origin. In such a representation, the z-axis is considered to be the axis passing through the nominal position of the aperture along the optical axis from the origin. The x-axis and y-axis are each parallel to the plane of the image sensor and perpendicular to the z-axis according to the definition of the Cartesian coordinate system. Generally, the x-axis is in the horizontal plane and the y-axis is along the vertical direction.

[0043] Using such a camera model, an offset in one or more of the x, y, and z coordinates can be determined from the plot as follows. The linear (first-order) component indicates the X offset error (when the movement from the first rotational position to the second rotational position is in the horizontal plane) and / or the Y offset error (when the movement from the first rotational position to the second rotational position is a rotation in the vertical plane). The quadratic (second-order) component indicates the Z (focus) offset error. The flat plot indicates no X(Y) or Z error. In this case, all of the calculated observation angles and the actual observation angles are equal.

[0044] The numerical value of the camera model error, or the parameter offset value, can be calculated from the first-order and / or second-order components.

[0045] If the calculated camera model error is not within a predetermined limit or threshold, these are fed back to the algorithm to provide an updated camera model. The calculation (or determination) step is repeated based on this updated camera model until the camera model error is within the limit.

[0046] Therefore, the calculation is repeated until a convergent result is obtained, and the camera model error is less than the limit.

[0047] In the manner described above, the two-dimensional, i.e., X and Z, or Y and Z, calibration parameters can be determined.

[0048] In many applications of the camera, particularly in survey applications, horizontal calibration is considered to be the most important. In such applications, the rotation of the camera from the first rotational position to the second rotational position is realized as a rotation in the horizontal plane, and it may be sufficient to perform the above-described method of horizontally calibrating the camera model. In this case, calibration can be performed on-site, eliminating the need to remove the camera from the monitoring location and take it to a laboratory space.

[0049] When calibration is performed along both the X and Y axes, the method can be performed in various ways.

[0050] According to the first version of this method, the above method is executed in a state where the movement from the first rotational position to the second rotational position is a rotation within a horizontal plane. Thus, after the determined computer model error converges to a value within a preset limit, the camera is rotated by 90° around its optical axis, i.e., around the z-axis, the method steps are repeated, and the camera is rotated again within the horizontal plane. According to this version, the camera model can be calibrated in the x, y, and z directions from four recorded images.

[0051] According to the second version of the method, the movement from the first rotational position to the second rotational position includes rotations around two rotational axes, enabling four sets of calculated observation angles including the angles with respect to the x-axis and the y-axis to be calculated from two recorded images.

[0052] According to the third version of the method, the above method is executed in a state where the movement from the first rotational position to the second rotational position is realized by a rotation within a horizontal plane. When the computer model error converges to a value within a preset limit, the camera tilts with respect to the horizontal plane, i.e., rotates around the x-axis (extending within the horizontal plane), and the method is repeated. According to this version, the camera model can be calibrated from three recorded images.

[0053] According to the first version of this method, the method can further include the step of rotating the camera substantially 90 degrees around its optical axis when it is determined that the camera model is acceptable, and the step of repeating steps c) to k).

[0054] By rotating the camera by 90° around its optical axis, even when a plurality of targets are arranged in a one-dimensional array, the camera can be calibrated over the two-dimensional plane of the image sensor. This enables the three-dimensional position of the diffraction aperture with respect to the image sensor to be illustrated or reduced as a two-dimensional problem.

[0055] A further advantage relates to the fact that the target must be placed only on the virtual horizon, as seen by the camera when the target is in a normal, non-inclined orientation.

[0056] After rotating substantially 90° about the optical axis, steps c) to k) are repeated. After the 90° rotation, the camera can be considered to be in a third rotational position, and the camera can rotate to a fourth rotational position within the horizontal plane.

[0057] In other words, the step of repeating steps c) to k) can be represented as follows. (c’) Placing the camera in a third rotational position with respect to the target and recording a third image of the plurality of targets with the camera in the third rotational position; (d’) Determining a third set of observed angles calculated from the third image using a camera model, the third set of calculated observed angles including the third calculated observed angles of each target with respect to the camera; (e’) Rotating the camera to a fourth rotational position with respect to the target and recording a fourth image of the plurality of targets with the camera in the fourth rotational position; (f’) Determining a fourth set of observed angles calculated from the fourth image using a camera model, the fourth set of calculated observed angles including the fourth calculated observed angles of each target with respect to the camera; (g’) Determining a function or plot based on the third and / or fourth set of calculated observed angles and a set of difference angles, the difference angles representing the difference between the third and fourth calculated observed angles for each target; (h’) Determining a camera model error from the function or plot; (i’) Comparing the camera model error with a predetermined limit; (j’) If the camera model error is less than the limit, determining that the camera model is acceptable; (k’) If the camera model error is not smaller than the limit, calculate a camera model updated based on the camera model error, and repeat steps d’), f’), g’), h’), i’) and j’) or k’) based on the updated camera model.

[0058] In a first version, at least some of the plurality of targets may be arranged in a substantially one-dimensional row as seen from the camera, and the one-dimensional row and the camera are arranged in substantially one plane. The step of rotating the camera from the first rotational position to the second rotational position (e) can be performed substantially within the plane.

[0059] The one-dimensional row need not be straight and may extend along a circular segment such that all targets are arranged at an equal distance from the camera. The plane is preferably oriented substantially along a horizontal plane. Thereby, the calibration setup can generally be arranged within an office or laboratory space having larger horizontal dimensions than vertical dimensions. However, from a conceptual point of view, the plane can be equally well oriented in any other direction.

[0060] As described above, it may be advantageous to place the targets at a distance from the camera that effectively forms a point source when observed by the camera. Thereby, the minimum distance between the camera, i.e., the diffraction aperture of the camera, and the targets is set. Generally, the distance can typically be on the order of several meters. For example, a distance of 3 meters between each target and the camera can be used.

[0061] The main reason for the minimum distance is that it cannot rotate completely around the central axis. The central axis should pass through the center of the opening, but due to mechanical tolerances, this is not always the case. Therefore, in practice, the camera is not only rotated but also translated (lateral movement) by a certain small amount, resulting in an additional change in the viewing angle. The rotation is usually small (e.g., 1.2 degrees), and since the center of rotation is moderately well-known (within 1 mm), the translation can be considered substantially small. When the target is sufficiently far away, the translation can be considered negligible with respect to the distance, and thus the angular error resulting from the translation can also be ignored.

[0062] However, whether this condition is actually met may depend on the algorithm used and what this algorithm can correct.

[0063] According to a second version of the method, a plurality of targets are arranged in a two-dimensional arrangement as seen from the camera, and the step (e) of rotating the camera from a first rotational position to a second rotational position includes a rotation in the horizontal plane and a rotation in the vertical plane.

[0064] According to this embodiment, the recording of two images, i.e., two measurements, is sufficient to calibrate the camera in the x, y, and z axes. Since the movement from the first rotational position to the second rotational position includes both a rotation around the vertical axis and an inclination with respect to the horizontal plane, the second rotational position will be different from the first rotational position with respect to both the horizontal angle and the vertical angle. Thereby, from each of the two images, the calculated viewing angles with respect to the x-axis and the y-axis can be determined for each target, and a camera model error including an x-offset error, a y-offset error, and a z-offset error can be calculated from the two images.

[0065] According to a third version of the method, a plurality of targets are arranged in a two-dimensional arrangement as seen from the camera, and step (e) of rotating the camera from a first rotational position to a second rotational position is performed about a first axis that extends substantially in the horizontal plane through the plane of the aperture and extends substantially perpendicular to the horizontal plane, the first and second calculated observation angles represent angles in the horizontal plane, and the method (l) determining a third set of observation angles calculated from the second image using a camera model, the third set of calculated observation angles including a third calculated observation angle for each target with respect to the camera, the third calculated observation angle representing an angle in a direction substantially perpendicular to the horizontal plane, the method further including when the camera model is determined to be acceptable in step (j), (m) rotating the camera about a second axis passing through its aperture, the second axis being substantially perpendicular to the first axis and extending in the horizontal plane, thereby rotating the camera to a third rotational position, (n) recording a third image of the plurality of targets with the camera in the third rotational position, (o) determining a fourth set of observation angles calculated from the third image using a camera model, the fourth set of calculated observation angles including a fourth calculated observation angle for each target with respect to the camera, (p) determining a second function or plot based on the third and / or fourth set of calculated observation angles and a set of second difference angles, the second difference angle representing the difference between the third calculated observation angle and the fourth calculated observation angle for each target, (q) determining a second camera model error from the second function or plot, (r) comparing the second camera model error with a second predetermined limit, (s) when the second camera model error is less than the second limit, determining that the camera model is acceptable, If the second camera model error is not less than the second limit, calculating a camera model updated based on the second camera model error, and repeating steps (l), (o) to (r) and (s) or (t) based on the updated camera model, are further included.

[0066] In this specification, the step (m) of rotating the camera around the second axis can alternatively be described as tilting the camera with respect to the horizontal plane. Similar to the general concepts described above in this specification, the amount of this rotation need not be known either, and is generally about 1 gon, 1 degree, or several degrees.

[0067] Step (a) preferably includes arranging a plurality of targets such that one target is arranged at each lateral peripheral portion of the field of view of the camera, and any remaining targets of the plurality of targets are substantially uniformly distributed between the targets arranged at the lateral peripheral portions.

[0068] It has been observed that arranging two targets around any of the fields of view of the camera is advantageous for the calibration procedure. It is known that the error in the change in the observed angle measured by the camera, caused by the error in the modeled relative position of the aperture with respect to the center of the image sensor along the plane of the image sensor surface, increases with the distance from the center of the field of view. Therefore, by arranging the targets at or near the edge of the field of view, the compensation parameters for this modeling parameter can be efficiently calculated.

[0069] According to an embodiment, the two targets arranged around may be arranged at about ±30° from the center line of the calibration measurement setup, that is, with respect to the optical axis of the camera when the camera is arranged in a neutral position where its optical axis coincides with the center line of the calibration measurement setup.

[0070] An odd number of targets may be provided, with one target disposed at or near each lateral edge or side of the camera's field of view, and one target disposed at or near the center of the camera's field of view.

[0071] Placing the target substantially at the center of the field of view has been found to be advantageous because it has been found that the error in the measured change in the viewing angle caused by the modeled focal length error exhibits its maximum level at the center of the field of view. Thereby, the modeled focal length, i.e., the compensation parameter for the z-offset, can be efficiently calculated by placing the target at the center of the camera's field of view.

[0072] The method can further include a step (h1) of determining the calibration parameter as a correction applied to the parameters representing the camera model, the calibration parameter being determined such that the determined change in the viewing angle is substantially equal for each target.

[0073] Thus, the correction to the modeling parameters defined by the initial or nominal camera model that describes the camera model is determined as a correction or compensation offset that results in substantially the same change in the calculated viewing angle for all targets. Preferably, the correction or compensation offset is calculated for each of the three modeling parameters that define the camera model.

[0074] The correction or compensation offset can be calculated using algorithms known in image processing. Algorithms known in the technical field of image processing are, for example, algorithms that use the gravity method. The algorithm can include any of an iterative algorithm, or a direct calculation based on geometric and / or trigonometric considerations, or a combination of a direct method and an iterative method.

[0075] For example, the algorithm step can include calculating the center position of the blob recorded by the image sensor, where the blob represents an image of the target projected onto the image sensor. The center position can be calculated to sub-pixel accuracy using, for example, the so-called centroid algorithm, as is known to those skilled in the art.

[0076] The camera model can include, as parameters, the three-dimensional positions of the pixels of the image sensor with respect to the diffraction aperture.

[0077] Thus, the camera model, which is a mathematical model or representation of the camera, can be based on three parameters. These can be represented as coordinates in a coordinate system, typically a Cartesian coordinate system. This relatively simple model has been found to be sufficient for the purpose of calibrating the camera, and the diffraction aperture renders a simplified geometric shape compared to a lens-based system.

[0078] Furthermore, the camera model can take into account the effects of other optical elements, such as filters, glass cover elements, or any other optical elements.

[0079] The method can further include the step of bringing the camera to a second temperature and subsequently repeating the steps of any one of claims 1 to 10 at the second temperature.

[0080] The calibration measurement steps described herein may first be performed at a first temperature. Subsequently, the camera is brought to a second temperature different from the first temperature, and the calibration measurement steps are repeated at the second temperature. During the different measurements, the camera is maintained at a substantially constant temperature. The camera can be brought to the second temperature, for example, by heating or cooling in a climate chamber.

[0081] The methods described herein can be performed while making actual measurements using a computer vision system.

[0082] In this embodiment, the system preferably includes a microprocessor-controlled rotating device configured to rotate the camera to different positions as described above in this specification. By performing the steps as described above in this specification and rotating the camera using the rotating device, an operation survey is performed by observing the change in the observation angle of the reference point, and the camera can be calibrated while observing a number of stationary reference points and a number of movable survey points. That is, the stationary reference points can be used as the targets described above in this specification.

[0083] Calibrating the camera during the survey is advantageous because the calibration parameters of the camera model can change not only due to environmental influences such as temperature but also due to the aging deterioration of the components. At least the focal length of the camera can be determined by a single rotating device configured to rotate the camera preferably around the Y-axis of the aperture, i.e., in the horizontal plane, and at least three substantially stationary targets distributed substantially along the horizontal line. In an actual embodiment, the focal length is the most sensitive to changes in the environmental temperature, so it needs to be recalibrated during the survey. Generally, the x-axis offset can also be calibrated for using this setup.

[0084] Furthermore, such a method can be even more advantageous as it can eliminate measurement errors due to dirt or other contamination in the optical path. For example, a dirt speck on the optical viewport may block the light rays from the survey target at one rotational position but not at another rotational position.

[0085] In some survey applications, it is known that the survey points move very slowly. Therefore, the survey points can be considered to be static for a certain extent of substantially short time. Therefore, the survey targets do not move substantially within the time required to determine the first and second sets of the observation angles of each survey and reference point. In this way, the stationary reference points and the movable survey points can be used as the targets described above in this specification.

[0086] In another embodiment of the present invention, the system includes a passive rotation device. Such a device may be, for example, a bimetallic coil spring for rotating the camera around the aperture in response to a change in ambient temperature. Preferably, such a passive rotation device is configured to rotate the camera around the Y-axis of the aperture so as to be able to determine the change in focal length due to the change in ambient temperature.

[0087] Since such a passive rotation device does not require power, it can be advantageous for applications where power is limited. This facilitates applications such as remote battery or Internet sensors powered by solar power.

[0088] In this specification, the target is described as preferably being arranged substantially in a horizontal plane, i.e., along the horizon as seen from the camera and, if applicable, along the vertical axis and along a vertical plane substantially passing through the aperture of the camera, but this is not a requirement. Such an arrangement is advantageous from the viewpoint of computational efficiency and may make the calculations easier, but the method can be used equally well with a target arranged away from the horizontal and / or vertical axes. In such a case, the fact that the target is arranged at a (substantial) distance from the horizontal and / or vertical axes can be taken into account in the calculations, for example, by additional mathematical transformations or terms in the equations.

[0089] The calibration method according to the first aspect may be performed using the system according to the second aspect of the present invention.

[0090] According to a second aspect, a system for calibrating a camera of a computer vision system is provided, the camera including an image sensor and a diffractive aperture configured to project incident light onto the image sensor, the system comprising a plurality of targets observed by the camera, An attachment device for attaching a camera such that when the camera is attached, a plurality of targets are within the field of view of the camera, the attachment device enabling the camera to be rotated between a first rotational position and a second rotational position with respect to the plurality of targets, one or more processors, (b) providing a camera model representing a mathematical model of the camera, (c) recording a first image of the plurality of targets with the camera in the first rotational position, (d) determining a first set of observed angles calculated from the first image using the camera model, the first set of calculated observed angles including a first calculated observed angle for each target with respect to the camera, (e) recording a second image of the plurality of targets with the camera in the second rotational position, (f) determining a second set of observed angles calculated from the second image using the camera model, the second set of calculated observed angles including a second calculated observed angle for each target with respect to the camera, (g) determining a function or plot based on the first set and / or the second set of calculated observed angles and a set of difference angles, the difference angles representing the difference between the first calculated observed angle and the second calculated observed angle for each target, (h) determining a camera model error from the function, (i) comparing the camera model error to a predetermined limit, (j) if the camera model error is less than the limit, determining that the camera model is acceptable, (k) if the camera model error is not less than the limit, calculating an updated camera model based on the camera model error and repeating steps d), f), g), h), i) and j) or k) based on the updated camera model, one or more processors configured as such, and including.

[0091] In particular, the system is configured to execute the method according to the first aspect described herein.

[0092] The image sensor is configured to receive and detect reflections and / or light beams generated by a target located within the field of view of the camera. For this purpose, the image sensor may preferably be a two-dimensional sensor.

[0093] One or more processing units can be arranged close to the image sensor, within or on the camera, for example in the internal space of the camera. Alternatively, the processing unit may be arranged remotely from the camera. Advantageously, the system includes a first processor arranged within the camera for recording and storing images, and one or more second processors provided in a computer, the one or more second processors being configured to receive and store images and possibly other data, process either the images or other data, and determine the calculated observation angles and calibration parameters, i.e. the updated camera model.

[0094] The mounting device may preferably include a microprocessor-controlled rotation device. The microprocessor-controlled rotation device can be configured to rotate the camera so as to move the camera among or between the different rotational positions described above herein. Thereby, as described above herein, the method can be executed live during the survey, i.e. during the measurements on site.

[0095] Such a device may be an electric motor, a servo, a solenoid, or any other microprocessor-controllable device. Such a device can incorporate an angle encoder or some other angle feedback device. Such a device can also incorporate some means for limiting the rotation angle.

[0096] The rotation device may be configured to rotate the aperture independently about a single axis or multiple axes. Preferably, the rotation device is configured to rotate the camera about the Y-axis of the aperture.

[0097] The rotation device can be used, for example, to rotate a camera configured to survey several stationary reference points and several movable survey points.

[0098] In another embodiment, the camera includes a passive rotation device. Such a device may be, for example, a bimetallic coil spring for rotating the camera around an aperture in response to a change in ambient temperature. Preferably, such a passive rotation device is configured to rotate the camera around the Y-axis of the aperture so that a change in focal length due to a change in ambient temperature can be determined.

[0099] The mounting device may be configured to enable rotation of the camera around a vertical axis and / or a horizontal axis.

[0100] One or more processors can be configured to perform one or more of steps b) to t) described hereinabove with respect to the method according to the first aspect.

[0101] The target to be measured may be arranged according to any of the arrangements or distributions described hereinabove with respect to the method of the first aspect.

[0102] According to a third aspect of the present disclosure, a computer program product is provided, the computer program product including instructions which, when executed by a computer, cause the computer to (b) provide a camera model representing a mathematical model of a camera including an image sensor and a diffraction aperture configured to project incident light onto the image sensor; (d) determine, from a first set of observed angles calculated from a first image of a plurality of targets recorded by the camera in a first rotational position with respect to the plurality of targets, using the camera model, the first set of observed angles calculated from the first image, the first set of calculated observed angles including the first calculated observed angle of each target with respect to the camera; (f) A step of determining, from a second image of a plurality of targets by a camera in a second rotational position, a second set of observed angles calculated from the second image using a camera model, the second set of calculated observed angles including the second calculated observed angles of each target with respect to the camera; (g) A step of determining a function based on the first set and / or the second set of calculated observed angles and a set of difference angles, the difference angles representing the difference between the first calculated observed angle and the second calculated observed angle for each target; (h) A step of determining a camera model error from the function; (i) A step of comparing the camera model error with a predetermined limit; (j) A step of determining that the camera model is acceptable if the camera model error is less than the limit; (k) If the camera model error is not less than the limit, calculating an updated camera model based on the camera model error and repeating steps d), f), g), h), i) and j) or k) based on the updated camera model.

[0103] In this specification, instructions can cause one or more processors of a computer to execute any one or more of the steps.

[0104] Instructions of a computer program product can be further configured to cause one or more processors to control the movement of the camera between different rotational positions described above in this specification.

[0105] In summary, a method, a system, and a computer program product for calibrating a camera including a diffractive aperture as an objective lens are provided. In the calibration method, relevant physical parameters of the camera model can be determined with a relatively simple and inaccurate setup.

[0106] According to the general concept of the present disclosure represented by the first aspect described above in this specification, by arranging a camera and a plurality of targets in one plane, the camera can be calibrated in a direction along the plane by recording two images of the plurality of targets, and during the recording of the two images, the camera rotates within the plane with respect to the axis passing through the diffraction aperture of the camera. The rotation is small, typically about 1 degree or a few degrees. For each of the two recorded images, the calculated observation angle of each target registered on the image sensor is determined using a camera model. From the difference in the calculated observation angles between the first image and the second image for each target, calibration parameters are determined. In a precisely calibrated camera model, the same difference in the calculated observation angles should occur for all targets. If this is not the case, calibration parameters are determined and fed back to the camera model, and the calculation is repeated until the calibration of the camera is considered acceptable.

[0107] The general concept enables calibration in a direction along the plane and along a first axis of the image sensor at the focal length of the camera. Calibration along a second axis of the image sensor perpendicular to the first axis can be performed according to any one of three different alternative forms of the method described above in this specification.

[0108] This method enables accurate calibration of survey equipment based on a diffraction aperture and facilitates structural monitoring / survey with much higher accuracy than equipment using refractive lenses.

[0109] It has been observed that the method according to the present disclosure facilitates reaching the relative observation angle error of a camera having a pinhole or other type of diffraction aperture close to 1 milliradian or 1 millidegree. This is about 50 times better than the achievable error of survey equipment having a glass lens, which is mainly caused by lens deformation.

[0110] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, the embodiments of the present disclosure are not limited to specific embodiments and should be construed to include all modifications, changes, equivalent devices and methods, and / or alternative embodiments of the present disclosure.

[0111] As used herein, the terms "have", "can have", "include", and "can include" indicate the presence of corresponding features (e.g., elements such as numerical values, functions, operations, components, etc.) and do not exclude the presence of additional features.

[0112] As used herein, the terms "A or B", "at least one of A or / and B", or "one or more of A or / and B" include all possible combinations of the items listed therewith. For example, "A or B", "at least one of A and B", or "at least one of A or B" means (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0113] Terms such as "first" and "second" as used herein can modify various elements regardless of the order and / or importance of the corresponding elements and do not limit the corresponding elements. These terms can be used for the purpose of distinguishing one element from another. For example, without departing from the scope of the present invention, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element.

[0114] When an element (e.g., a first element) is "coupled (operatively or communicatively) or connected" to another element (e.g., a second element), it will be understood that the element may be directly coupled to the other element, or there may be an intervening element (e.g., a third element) between the element and the other element. Conversely, when an element (e.g., a first element) is "directly coupled or directly connected" to another element (e.g., a second element), it will be understood that there is no intervening element (e.g., a third element) between the element and the other element.

[0115] As used herein, the expression "configured (or set)" can be used interchangeably with "suitable", "capable", "designed", "adapted", or "able" depending on the context. The term "configured to" does not necessarily mean "specially designed to" at the hardware level. Instead, the expression "a device configured to" can mean that the device "can" do something in a particular situation with other devices or components.

[0116] The terms used in describing various embodiments of the present disclosure are for the purpose of describing particular embodiments and are not intended to limit the present disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. All terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the relevant art, unless specifically defined otherwise. Terms defined in commonly used dictionaries should be interpreted as having the same or similar meaning as their meaning in the context of the relevant art, and should not be interpreted as having an ideal or exaggerated meaning, unless clearly defined herein. Depending on the context, terms defined in the present disclosure should not be interpreted as excluding embodiments of the present disclosure.

[0117] For the purpose of determining the extent of protection conferred by the claims of this document, any element corresponding to an element specified in the claims shall be duly considered.

[0118] The present invention will be described in more detail below with reference to the accompanying drawings.

Brief Description of the Drawings

[0119]

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 4C

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

[0120] Modes for Carrying Out the Invention Generally, the present disclosure relates to the calibration of a system for surveying an object or tracking the movement of an object by tracking one or more targets or survey reflectors attached to the object.

[0121] The illustrated embodiments are described with respect to an optical entry system formed by a diffractive aperture or a non-refractive element in the form of a pinhole in a camera objective lens, i.e., a camera having a camera objective lens, but it should be understood that the non-refractive element may alternatively be any other non-refractive element such as the non-refractive element described in International Publication No. 2021 / 167452. Similarly, the embodiments are described using a prism as a survey reflector, but it should be understood that different reflecting elements, such as another type of prism or a hollow mirror, may also be used.

[0122] FIG. 1A shows an exemplary setup of a system in which an object 3 is monitored. The system includes a sensor device such as a camera 7. The system also includes a plurality of targets 1 in the form of survey reflectors herein, which are attached to the object 3 at multiple locations. The object 3 is shown as including one or more buildings to which the targets 1 are fixed. However, the object 3 may alternatively be any other structure such as a tower, a tunnel (FIG. 1B), or a bridge (FIG. 1C), or it may be a vehicle (such as a land boat) or a natural object such as a large rock.

[0123] The object 3 is monitored by monitoring or measuring the positions of the survey reflectors 1. By monitoring their positions over time, the movement of all or part of the object 3 can be detected. Preferably, the amount, degree, and / or direction of the movement can also be determined. Thereby, the state of the object 3, such as its stability or integrity, or mechanical properties, can be monitored.

[0124] One camera 7 is shown. However, the system may include two or more cameras 7.

[0125] In the view of FIG. 1A, the camera 7 is configured to generate a diverging light beam 5 and transmit it to a plurality of survey reflectors 1. The survey reflector 1 reflects a part of the diverging light beam 5 that impinges thereon, thereby forming a reflected beam 6 that is reflected back to the camera 7. The light beam 5 is substantially conical and has a solid angle Ω1 that covers the field of view of the camera 7. Thereby, a plurality of survey reflectors 1 can be monitored substantially simultaneously. Alternatively, as is known to those skilled in the art, the target may be an active target such as a light emitting diode, an LED, etc. that each emits the light beam 6 recorded by the camera. In such an embodiment, the camera need not be configured to generate and transmit the diverging light beam 5. It should be understood that the calibration method according to the present disclosure can be similarly applied to both the camera 7 including a light source that emits the light beam 5 and the camera 7 without a light source.

[0126] FIG. 1B shows an embodiment in the tunnel 3. A track having sleepers 12 passes through the tunnel 3. Survey reflectors or targets 1 are provided on both the tunnel wall and the sleepers 12. The camera 7 is configured to see all the survey reflectors or targets 1 within its field of view.

[0127] FIG. 1C shows an embodiment on the bridge 3. A plurality of survey reflectors or targets 1 are provided on the bridge 3. The camera 7 is arranged to see all the survey reflectors 1.

[0128] [[ID=~12]]FIG. 2 provides a schematic diagram of the measurement principle of a system 20 for monitoring a plurality of positions on an object 3 to which the calibration method described herein can be applied. For ease of explanation, FIG. 2 shows a system 20 that monitors one target or survey reflector 21. However, as shown in FIGS. 1A - 1C for example, the camera 27 can be configured to monitor a plurality of survey reflectors. In the illustrated embodiment, the survey reflector 21 is formed by a prism. Alternatively, other types of reflectors, such as a hollow mirror, can also be used.

[0129] System 20 includes a camera 27 and a processing unit 29 that can be included in or disposed within the camera 27. Alternatively, it may be disposed away from the camera 27. For calibration purposes, in addition to the processor unit within the camera, one or more processors can be provided outside the camera.

[0130] In the illustrated embodiment, the camera 27 includes a light source 22 that emits a divergent beam 25. The light source 22 generally includes a light emitting diode LED. The beam 25 has a first solid angle Ω1 that can be large enough to cover substantially the entire field of view of the camera 27. Thereby, all survey reflectors 21 located within the field of view of the camera are irradiated with the beam 25 without moving, rotating, or scanning the camera or the light beam (except possibly when a building is being monitored as shown in FIG. 1A and one or more survey reflectors are blocked by obstacles such as pedestrians or vehicles, or when a tunnel is being monitored as shown in FIG. 1B and one or more survey reflectors are blocked by obstacles such as trains).

[0131] The survey reflector 21 reflects a part of the received beam 25 and forms a reflected beam 26 that is reflected back towards the camera 27.

[0132] Alternatively, as described above in this specification, instead of the survey reflector 21, an active target or beacon that usually includes an LED that emits the beam 26 itself can be used. In such an embodiment, the camera does not need to include the light source 22. As will be understood by those skilled in the art, the light beam emitted by such an active target is received and processed by the camera 27 in the same manner as described in this specification with respect to the beam 25.

[0133] The apparatus 20 further includes an image sensor 24 arranged to receive light, i.e., a part 261 of the reflected beam 26 incident on the camera 27. As a result of the reception of the reflected light 261, the image sensor 24 generates data, preferably in the form of a two-dimensional image, in the preferred embodiment.

[0134] The beam 25 may be amplitude - modulated, thereby exhibiting a defined variation over time of its amplitude. Alternatively and / or additionally, other types of encoding may be applied to the first beam. When using an active beacon or target, the light emitted thereby may likewise be encoded. By applying appropriate filtering techniques during the image processing of the image data, the influence of the environment on the measurements, such as interference by ambient light, can be reduced.

[0135] The details and positions of the light source 22 and any possible additional light sources, the beam 25, and the various types of encoding applied to any possible additional beams, and its processing during image processing are described in detail in International Publication No. WO 2021 / 167452.

[0136] A body 28 is disposed between the image sensor 24 and the first light source 22, or at least its light - emitting surface. In the illustrated embodiment, the body is substantially planar and forms part of the camera housing. The body 28 is opaque to light and, in the illustrated embodiment in the form of a diffractive aperture or pinhole 23 forming the camera's objective lens, includes an optical entrance system.

[0137] The description herein focuses on the optical entry system formed by the pinhole, but other types of diffractive elements, particularly as described in International Publication No. WO 2019 / 143250 and International Publication No. WO 2021 / 167452, may likewise be suitable.

[0138] The processing unit 29 is generally configured to determine the position or orientation angle of each survey reflector from the data by image - processing the data provided by the image sensor, and to detect the movement of one or more of the plurality of survey reflectors based on a comparison of the determined position or orientation angle of each survey reflector with a previously determined position or orientation angle.

[0139] Figure 3 shows an example of the camera 7. The exemplary camera 7 has a non-refractive optical system 101, an image sensor 120, a clock 123, memories 15, one or more position and / or orientation measurement components 16, an output unit 17, an input unit (or user interface) 19, an electronic networking module 109, and a processing unit 9 connected to one or more light sources 102. The non-refractive optical system 101 is shown as being connected to the image sensor 120. This latter "connection" need not be a physical connection. Here, "connection" is intended to refer to a situation where the non-refractive optical system 101 is configured to receive ambient light, such that the received ambient light is received by the image sensor 120. As can be understood from the embodiments described above in this specification, not all of the functional elements shown in FIG. 3 need to be present.

[0140] All connections for the purpose of data transmission may be physical connections (wires), but alternatively they may be wireless and based on the transmission of electromagnetic / optical radiation.

[0141] The non-refractive optical system 101 may be any of the types of diffractive apertures described above herein, for example, one or more pinholes. The diameter of the pinhole may range between 50 and 400 μm. Alternatively, as described above, the non-refractive optical system may be replaced by a lens which is preferably a thin lens that enables temperature modulation with low computational effort.

[0142] The processing unit 9 may be any suitable processing unit known in the art.

[0143] The image sensor 120 preferably includes a set of photosensitive elements (pixels) arranged in a two-dimensional matrix that forms the image plane of a camera, such as a CCD sensor or a CMOS sensor. The image sensor 120 is configured to receive the light beam 6 incident through the diffraction aperture 101. Each light beam 6 is focused on a subset of the photosensitive elements. Each such subset corresponds to the solid angle of one incident light beam 6, i.e., both the angle of incidence in the horizontal plane with respect to the Earth and the angle of incidence in the vertical plane. Of course, it is also possible to measure the angle of incidence with respect to another object other than the Earth, such as a geostationary satellite. As long as both the camera 7 and the survey reflector 1 remain in fixed positions, these subsets are stationary for each survey reflector 1.

[0144] In another embodiment, as described in International Publication No. WO 2019 / 143250, a line sensor can be used in combination with an optical slit as an objective lens instead of a pinhole.

[0145] Optionally, a temperature control system 103 can be provided to reduce the thermal effects on the measurement data. The heat capacity of the non-refractive optical system 101 is relatively low compared to a camera 7 that uses a lens system instead of the non-refractive optical system 101. By implementing a temperature control system in the form of a thermostat 103, the thermal stability can be improved. FIG. 3 shows an embodiment having a Peltier element 103 that is reversible with respect to the non-refractive optical system 101 (i.e., configured for both cooling and heating). The Peltier element 103 is connected to the processing unit 9 so that the non-refractive optical element 101 is maintained at a predetermined temperature and its temperature is controlled. Alternatively, the thermal stability can be enhanced by the design of the camera housing, particularly by the materials used therefor, and / or by using a model that measures the temperature at various positions therein and takes into account the thermal effects during the processing of the data from the image sensor.

[0146] In the following, some general aspects of the systems and methods of operation described above in this specification are summarized.

[0147] The image sensors 24, 120 convert the received light 6 into an image. The image is here a set of electronic signals called pixel signals. Each pixel signal is generated by one photosensitive element and has a value corresponding to the light intensity of the light received by the photosensitive element. Thus, the pixel signals can also be related to the object 3 to which the survey reflector 1 is attached and its surroundings.

[0148] The image sensor is arranged such that light entering the camera through the non-refractive element forms a diffraction pattern on the image sensor. The diffraction pattern depends on the characteristics of the non-refractive element and appears as dark or bright regions on the image sensor depending on the distance and angle to the non-refractive element of each pixel of the image sensor. By integrating a plurality of data frames, each containing a large number of pixels, typically at least 100, high-resolution measurement results can be achieved.

[0149] The processing unit 9 is configured to receive pixel signals from the image sensor 120 and store them in the memory 15. The pixel signals may preferably be stored by the processing unit 9 as a single image together with a time stamp and / or a position stamp indicating the position of the camera 7. However, preferably, the pixel signals are stored by the processing unit 9 as a series of pictures forming a video, and each picture is provided with a time stamp and / or a position stamp indicating the position of the camera 7.

[0150] The clock 123 provides a clock signal to the processing unit 9, as is known to those skilled in the art. The clock signal is used for the normal processing of the processing unit 9. The processing unit 9 may determine the time stamp based on these clock signals. However, the camera 7 may also include a GNSS unit for receiving time signals from satellites or may receive time signals from another suitable source.

[0151] Memory 15 can include different types of sub-memories such as ROM (read-only memory) / flash type memories that store appropriate program instructions and data for executing processing unit 9. The memory can also include an appropriate RAM (random access memory) type memory for storing temporary data such as data received from image sensor 120. Memory 15 may also include a cache type memory. Some or all of the sub-memories may be physically located separately from other components. Processing unit 9 may also be configured to transmit all pixel signals to a remote unit via electronic networking module 20 for external storage and processing. A local copy of these pixel signals may be stored in local memory 15 within camera 7, but this is not necessary.

[0152] Memory 15 stores initial position data indicating the initial position of camera 7. Such initial position data can be established by using a theodolite and then stored by the user. Such initial position data can also result from measurements made by camera 7 itself. For example, camera 7 can collect successive photographs from a known "flashing" light source installed on a high-altitude aviation obstacle marker having a known position. Such obstacle markers are arranged at a defined vertical distance on a high structure, thereby enabling triangulation. Memory 15 also stores a camera ID that identifies camera 7 and is used by processing unit 9 in external communication with other devices to identify itself to those other external devices.

[0153] The position and / or orientation measurement component 16 can include one or more accelerometers and / or gyro / gyroscopes, as known to those skilled in the art. They can also include the GNSS unit described above. Such accelerometers and / or gyro / gyroscopes measure the movement of the camera itself and derive the updated position and orientation of the camera from such measurements. The updated camera position and / or orientation is then stored in the memory 15 by the processing unit 9. By doing so, when measuring the position of one or more survey reflectors 1, changes in the position and / or orientation of the camera can be taken into account. The accuracy can be on the order of 1 / 1000. The test showed a peak-to-peak of 2 millidegrees. Additionally, a three-axis accelerometer package can also measure the direction of the earth's gravity at rest. A 3D gyro package with sufficient performance can measure the direction of the earth's rotation axis (even at rest).

[0154] The output unit 17 may include one or more sub-output units such as a display and a speaker.

[0155] The input unit 19 may include one or more secondary input units such as a keyboard and a microphone. The display and the keyboard may be made as two separate touchscreens. However, they may also be implemented as a single touchscreen.

[0156] The electronic networking module 20 can include one or more of LTE (Long-Term Evolution), Ethernet, WiFi, Bluetooth, power line communication, low-power wide area networks (e.g., Lora (trademark) and Sigfox (trademark)), and NFC (Near Field Communication) modules. Technologies known from IoT (Internet of Things), as well as any proprietary communication protocols, can be used.

[0157] At least one light source 102 includes at least one light source, such as a light emitting diode (LED) source, configured to generate light. The processing unit 9 is configured to control each LED light source such that each LED light source generates a light beam.

[0158] Cameras 7, 27 are typically arranged in a fixed position so as to be stationary. The static position is then known and stored in a memory 15 accessible by the processing unit 9 of the camera.

[0159] When all survey reflectors or targets 1, 21 are installed, they have an initial position / orientation angle that can be stored in the memory 15 of the camera.

[0160] Thus, when the system is activated, the camera knows all the initial position / orientation angles of the survey reflectors / targets corresponding to the initial position and orientation of the object 3 to which the survey reflectors are attached.

[0161] The processing unit 9 is configured to calculate the initial solid angle of incidence of each reflected light beam 6. That is, the received reflected light beam is imaged onto one or more photosensitive elements of the image sensor 120 via a non-refractive optical system. The processing unit 9 determines which of these photosensitive elements it is, and then establishes the solid angle of incidence of the corresponding light pulse. Techniques for doing so are known to those skilled in the art and do not require further detailed description here.

[0162] When the object 3 is stable, i.e., not moving, the positions of all the survey reflectors 1 are also stable. This fixes the solid angle of incidence of each reflected light beam on the image sensor of the camera. However, as soon as the object 3 or a part thereof moves, this solid angle of incidence of the reflected light beam 6 changes. The processing unit 9 is configured to calculate this change in the solid angle for each light beam 6.

[0163] The images recorded by the camera are processed by, for example, the methods described in International Publication No. 2019 / 143250 and International Publication No. 2021 / 167452, and / or by any method known to those skilled in the art, to determine the positions, orientations and / or observation angles of a plurality of targets observed by the camera.

[0164] FIG. 4A shows a schematic view as seen from the side of the diffraction aperture 23 of the image sensor 24 and the camera 27, together with the x, y, and z axes of the coordinate system that can be used to define the camera model and calculate the parameter offset values for calibrating the camera model according to the present disclosure. In a preferred embodiment, the nominal position of the center of the diffraction aperture is considered as the origin of the coordinate system. The x-axis and the y-axis are substantially parallel to the surface of the image sensor 24. The z-axis represents the optical axis of the camera, and the distance between the surface of the image sensor 24 on the optical axis and the center of the diffraction aperture 23 at the origin, i.e., along the z-axis, represents the focal length.

[0165] FIG. 4B shows the situation of FIG. 4A as seen from the front of the camera, showing the front surface 28 of the housing provided with the diffraction aperture 23.

[0166] According to an embodiment, the x-axis may be arranged along the horizontal plane and the y-axis may be arranged along a vertical plane substantially perpendicular to the horizontal plane.

[0167] In FIG. 4C, the diffraction aperture 23 and the photon-sensitive surface of the image sensor 24 at the nominal position, as well as the offsets of the diffraction aperture along each of the three coordinate axes, are schematically shown. As schematically shown in FIG. 4C, the position of the diffraction aperture 23 may deviate up to three dimensions from the nominal position, i.e., the expected or designed position.

[0168] The deviations in the x, y, and z dimensions are called the x offset, the y offset, and the focus error, respectively. For simplicity, the explanation of the effect of shifting the parameters of the camera model is limited to a target that can be approximately seen by a horizontal or vertical line (forming a cross) where both lines pass through the optical axis.

[0169] A survey system equipped with a well-calibrated camera exhibits the following behavior.

[0170] When the camera rotates around the opening in the horizontal plane, i.e., around the y-axis as shown in Figure 4B (a "yaw" motion), all of the reported horizontal angles, i.e., the calculated observation angles in the horizontal direction of the horizontally distributed targets within the aforementioned limited target set, change by an amount called the differential angle, which is equal to the rotation angle of the camera.

[0171] When the camera rotates around the opening in a vertical plane parallel to the optical axis, i.e., around the x-axis as shown in Figure 4B (a "pitch" movement), all of the reported vertical angles of the vertically distributed targets within the aforementioned limited target set change by an amount called the differential angle, which is equal to the rotation angle of the camera.

[0172] The above deviation affects the measurement data. Possible deviations in the position of the image sensor in the direction opposite to the deviation of the opening have the same effect.

[0173] Partially independent effects of deviation:

[0174] When only the x-offset is non-zero and the camera is rotated around the Y-axis of the opening, the calculated horizontal angles of the horizontally distributed targets within the aforementioned limited target set change by an amount obtained by adding a so-called differential error (which may be negative) to the rotation angle of the camera. This error is proportional to the horizontal angle between the direction of the target and the optical axis.

[0175] When only the y-offset is non-zero and the camera is rotated around the X-axis of the opening, the calculated vertical angles of the vertically distributed targets within the aforementioned limited target set change by an amount obtained by adding a differential error to the rotation angle of the camera. This error is proportional to the vertical angle between the direction of the target and the optical axis.

[0176] If only the focus error is non-zero and the camera is rotated about the Y-axis of the aperture, the calculated horizontal angles of the horizontally distributed targets within the aforementioned limited target set change by an amount equal to the differential error added to the rotation angle of the camera. The differential error is positive or negative depending on the sign of the focus error. The absolute value of the differential error is maximum for the target on the optical axis. The absolute value of the differential error decreases with the absolute value of the angle between the horizontal angle of the target direction and the optical axis.

[0177] If only the focus error is non-zero and the camera is rotated about the X-axis of the aperture, the calculated vertical angles of the vertically distributed targets within the aforementioned limited target set change by an amount equal to the differential error added to the rotation angle of the camera. The differential error is positive or negative depending on the sign of the focus error. The absolute value of the differential error is maximum for the target on the optical axis. The absolute value of the differential error decreases with the absolute value of the angle between the vertical angle of the target direction and the optical axis.

[0178] The differential angles, and thus their potential errors, can be induced by rotating the camera (slightly), e.g., by 1 degree, in the horizontal plane, or in a vertical plane parallel to the optical axis (or a combination of both).

[0179] When the x, y, and z offsets are determined by iteration of a mathematical formula or algorithm and then compensated for in the processing of the measured image data to determine the observation angle, orientation, and / or position of the target, the residual differential error caused by the aperture offset can be ignored.

[0180] For one or more aperture offsets, targets that are within the field of view of the camera but not in the aforementioned limited directions (i.e., not aligned with either the x-axis or the y-axis) also exhibit differential angular errors, but the relationship between the aperture offset and the differential angular error is more complex.

[0181] Therefore, in order to facilitate the calculation of camera model parameters, it is preferable to use only restricted directions with respect to the target, i.e., to use targets arranged along one or both of the x-axis and the y-axis. Further, as can be understood from the described effects of the aperture offset, the measurement of the horizontal and vertical differential angle errors is performed using a plurality of dispersed targets within the field of view of the camera, for example, even on only one plane of the floor, under the condition that the camera is rotated 90 degrees around its optical axis (not necessarily in that order) between the horizontal angle and the vertical angle.

[0182] FIG. 5 schematically shows a system 100 or setup for calibration measurement according to an embodiment of the present disclosure. The system includes a camera 7, which can be the camera described with reference to FIGS. 2 and 3, a plurality of targets 1 configured to be observed by the camera 7, and an external processing unit 190 including one or more external processors arranged, for example, in a laptop or desktop computer. The external processing unit 190 is connectable or connected to the camera 7, i.e., the electronic networking module 109, for receiving the recorded images and optionally other data from the camera 7 for further processing, such as one or more of the steps of calculating the observation angle, the differential angle, the camera model error, and / or the calibration parameter values as described with respect to various embodiments of the present disclosure. In particular, one or more processors 190 can be configured, i.e., programmed, to execute one or more of steps 7030 to 7100 described with reference to FIG. 7, or one or more of the steps shown in FIGS. 9 or 10.

[0183] The camera 7 is arranged on a mounting device such as a rotary stage. The mounting device may be any of the mounting devices described in the above summary section of this specification. FIG. 5 shows the camera 7 rotating from the first rotational position P1 to the second rotational position P2. By recording images of a plurality of targets 1 with the camera at each of these positions, the aperture offset and calibration parameter values can be calculated.

[0184] As described above in this specification, the target 1 can be either a reflective target or an active light source. The distance d between the target 1 and the camera 7 can be set based on various parameters such as the camera, image processing, and the algorithms used for calculating the target position and observation angle, as well as practical considerations such as spatial constraints when performing calibration in a laboratory setting or on-site.

[0185] In FIG. 5, the targets are shown as being arranged in one plane, for example a horizontal plane, on which the camera is also arranged. In FIG. 5, the targets are further shown as being arranged along a semi-circle, and the distances to all the cameras are substantially the same. However, it is not limited to this, and the targets 1 can also be arranged along a straight line. In other embodiments or alternative forms of the method, the plurality of targets may be provided in a two-dimensional array or matrix, for example as shown in FIG. 8.

[0186] As shown in FIG. 5, the targets are preferably distributed substantially uniformly between a first lateral or peripheral position L1 and a second lateral or peripheral position L2, and one central target is arranged substantially along the optical axis of the camera. This may be preferable as described in the summary section, but other arrangements are also possible.

[0187] Generally, the distance d may be several meters according to the distances used in actual real-world survey settings. In the example described in this specification, the distance was about 3 meters.

[0188] FIG. 6 schematically shows a scene observed by the camera 7 during calibration measurement using the system 100 of FIG. 5 when moving from the first rotational position P1 to the second rotational position P2. The solid circles indicate the observation positions of the plurality of targets 1 by the camera at position P1, and the dotted circles indicate the observation positions of the plurality of targets 1 by the camera at position P2. At each of positions P1 and P2, the position of each target is represented by its calculated observation angle HA i and can be represented. In FIG. 6, the calculated observation angle HA i at P2 is shown together with the differential angle ΔHA i indicating the change in the calculated observation angle for each target between P1 and P2. Calibration of the camera, i.e., calculation of the calibration parameter values, can be performed in a manner as shown in FIG. 7 from the initial camera model and the images recorded at P1 and P2 respectively.

[0189] FIG. 7 shows a flowchart illustrating a method for calibrating the camera 7 according to an embodiment of the present disclosure. Although the method is described herein with respect to a camera of a surveillance or survey system, the method can be similarly applied to cameras directed to other types of computer vision systems.

[0190] Steps 7010 to 7100 of the method of FIG. 7 may advantageously be executed on one or more processors such as the internal processor 109 of the camera and / or one or more external processors 190.

[0191] Before starting the flowchart shown in FIG. 7, a plurality of targets are dispersed within the field of view of the camera. For example, the targets and the camera may be arranged as shown in FIG. 5.

[0192] In step 7010, an initial camera model is provided. This can be a camera model based on the design of the camera, or a camera model obtained during a previous calibration of the camera. The camera model can include a mathematical model of the position of the center of the aperture 23 of the camera, i.e., the pixels of the image sensor relative to the objective lens. The model can be based on the coordinate system described with reference to FIGS. 4A - 4C. The camera model can further take into account additional parameters and characteristics of the camera.

[0193] In step 7020, the camera 7 is placed at a first rotational position P1 with respect to the target 1, e.g., the position P1 in FIG. 5, and a first image of the plurality of targets is recorded.

[0194] In step 7030, using the camera model and algorithm provided in step 7010, a first set of observed angles calculated from the first image is determined, which is known in the art as described above in this specification. The first set of calculated observed angles includes the first calculated observed angles of each target with respect to the camera while the camera is in the first rotational position.

[0195] In step 7040, the camera is moved to a second rotational position, e.g., the position P2 shown in FIG. 5, with respect to the plurality of targets, and a second image of the plurality of targets is recorded by the camera in the second rotational position.

[0196] In step 7050, similar to step 7030, a second set of observed angles calculated from the second image is determined using the camera model provided in step 7010. The second set of calculated observed angles includes the second calculated observed angles of each target with respect to the camera while the camera is in the second rotational position.

[0197] In step 7060, a function or plot is determined based on the set of first and / or second calculated observation angles and the set of difference angles, where the difference angle represents the difference between the first and second calculated observation angles for each target. An example of such a plot is shown in the step between steps 7060 and 7070.

[0198] In step 7070, the camera model error is determined using the function or plot. The camera model error can be determined as an offset value for one or more of the parameters defining the camera model, for example, as shown in FIG. 4C.

[0199] The parameter offset in one or more of the x, y, and z coordinates can be determined as follows. The linear (first-order) component indicates the X offset error (when the movement from the first rotational position to the second rotational position is in the horizontal plane) and / or the Y offset error (when the movement from the first rotational position to the second rotational position is a rotation in the vertical plane). The quadratic (second-order) component indicates the Z (focus) offset error. The flat function / plot indicates that there is no X (Y) or Z error. In this case, all of the calculated observation angles and the actual observation angles are equal.

[0200] The camera model error, or the parameter offset value, can be quantified, that is, the numerical value of the error can be calculated from the first-order and / or second-order components.

[0201] In step 7080, the camera model error is compared with a preset limit that is considered to provide measurement data of sufficient accuracy.

[0202] If the camera model error is not within a preset limit, the camera model error is fed back to the camera model, and the camera model updated in step 7090 is provided. Thereafter, the algorithm or flowchart is re-iterated for steps 7030, 7050, 7060, 7070 and 7080. This re-iteration is performed until the resulting camera model error is lower than the current limit.

[0203] If the camera model error is smaller than the current limit, in step 7100, it is determined that the camera model is acceptable, i.e., should be calibrated to sufficient accuracy.

[0204] As described, the movement of the camera from the first rotational position in 7020 to the second rotational position in 7040 may be performed in one plane, generally in the horizontal plane. The method described in FIG. 7 then results in calibration in that plane and in the focus direction, i.e., the x and z directions.

[0205] In certain applications or scenarios, calibration along x and z may be sufficient. For example, in many survey applications, it has been observed that horizontal calibration is the most important.

[0206] The method described herein with respect to FIG. 7 can be performed either in a laboratory setup or in a field survey setup.

[0207] According to other embodiments, the plurality of targets may be arranged in a two-dimensional arrangement, e.g., an arrangement as shown in FIG. 8 including two one-dimensional arrangements along the x-axis and y-axis respectively, i.e., arranged substantially perpendicularly and intersecting each other. In such embodiments, the movement of the camera from the first rotational position to the second rotational position can include both rotation about the x-axis and rotation about the y-axis, enabling determination of the calculated observation angles with respect to both the x-axis and the y-axis, thereby enabling calibration in all three dimensions using the first and second recorded images.

[0208] Such a method is shown in the flowchart of FIG. 9. Steps 9010, 9020, 9030, 9040, 9050, 9080, 9090, and 9100 are the same as steps 7010, 7020, 7030, 7040, 7050, 7080, 7090, and 7100 of FIG. 7 respectively, and thus will not be described in detail here.

[0209] In step 9060, the first and second sets of observation angles each include both a horizontal angle component and a vertical angle component. The first function or plot is determined based on the first set and / or the second set of the calculated horizontal components of the observation angles and a set of horizontal differential angles, where the horizontal differential angle represents the difference between the first calculated horizontal component and the second calculated horizontal component of the observation angles of each target. Examples of such two plots are shown in step 9065 between steps 9060 and 9070.

[0210] In step 9070, the first and second functions or plots are used to determine the camera model error. The camera model error can be determined as an offset value for one or more of the parameters defining the camera model, for example as shown in FIG. 4C.

[0211] The parameter offset in one or more of the x, y, and z coordinates can be determined as follows. The linear (first-order) component of the first function or plot indicates the X offset error (corresponding to the component of the movement in the horizontal plane from the first rotational position to the second rotation). The linear (first-order) component of the second function or plot indicates the Y offset error (corresponding to the component of the movement in the vertical plane from the first rotational position to the second rotational position). The quadratic (second-order) components of the first and second functions or plots indicate the Z (focus) offset error. Flat first and second functions / plots indicate no X, Y, and Z errors. In this case, all of the calculated observation angles and the actual observation angles are equal.

[0212] In other embodiments where the camera is calibrated in the y direction, after step 7100 where the camera model is considered calibrated in the x direction, the method for calibration can continue as shown in FIG. 10 or FIG. 11. In this case, the camera model that led to the reaching of step 7100 can be used as the input, i.e., the initial camera model, in one of the flowcharts of FIG. 10 or FIG. 11.

[0213] In the method further shown in the flowchart of FIG. 10, the camera model is calibrated along the y-axis following the calibration along the x-axis as shown in FIG. 7. When reaching step 7100 of FIG. 7, the camera is rotated substantially 90 degrees about its optical axis and the calibration continues as shown in the flowchart of FIG. 10. This flowchart essentially directly corresponds to the flowchart of FIG. 7, and this method is repeated in the same way with the camera rotated 90 degrees. This enables calibrating the camera along both the x-axis and the y-axis using, for example, a one-dimensional target array as shown in FIG. 5.

[0214] In step 1010, an initial camera model is provided. This can be the camera model obtained from the method of the flowchart of FIG. 7.

[0215] In step 1020, the camera 7 is placed at a third rotational position P3 with respect to the target 1, and a third image of the plurality of targets is recorded.

[0216] In step 1030, using the camera model provided in step 1010, a third set of observed angles calculated from the third image is determined. The third set of calculated observed angles includes the third calculated observed angles of each target with respect to the camera with the camera in the third rotational position.

[0217] In step 1040, the camera is moved to a fourth rotational position similar to the rotation from the first rotational position P1 to the second rotational position P2 as shown in FIG. 5, for example, position P4. Fourth images of the plurality of targets are recorded by the camera at the fourth rotational position.

[0218] In step 1050, using the camera model, a fourth set of observed angles calculated from the fourth images is determined.

[0219] In step 1060, based on the third and / or fourth set of calculated observed angles and the set of difference angles, where the difference angle represents the difference between the third and fourth calculated observed angles for each target, a function or plot is determined.

[0220] In step 1070, using the function or plot, a camera model error is determined in a manner similar to method step 7070 described with reference to FIG. 7.

[0221] In step 1080, the camera model error is compared with a preset limit representing an acceptable camera model error.

[0222] If the camera model error is not within the preset limit, the camera model error is fed back to the camera model and an updated camera model provided in step 9090 is provided. Thereafter, the algorithm or flowchart is re-iterated for steps 1030, 1050, 1060, 1070 and 1080. This re-iteration is performed until the resulting camera model error is lower than the current limit.

[0223] If the camera model error is smaller than the current limit, in step 1100, it is determined that the camera model is acceptable, i.e., should be calibrated to sufficient accuracy.

[0224] In another method according to an embodiment further shown in the flowchart of FIG. 11, the camera model is calibrated along the y-axis according to the calibration along the x-axis as shown in FIG. 7. When reaching step 7100 in FIG. 7, the camera is rotated with respect to the horizontal plane, i.e., rotated around the x-axis, and the calibration continues as shown in step 1140 of the flowchart of FIG. 11. In this method, a two-dimensional arrangement of the targets as shown in FIG. 8, for example, is used.

[0225] In step 1110, an initial camera model is provided. This can be the camera model obtained from the method of the flowchart of FIG. 7.

[0226] In step 1120, using the images previously recorded at the second rotation position, in step 1130, a third set of calculated observation angles is determined, and the third set of calculated observation angles includes the observation angles of each target with respect to the y-axis. This is shown in FIG. 11 as a separate step executed after the calibration along the x-axis shown in FIG. 7, but the calculation of the third set of calculated observation angles may be executed simultaneously with the determination step 7050 of the second set of calculated observation angles in FIG. 7.

[0227] In step 1140, the camera is moved from the second rotation position to the third rotation position by rotation around the x-axis, i.e., rotation with respect to the horizontal plane. This can also be expressed as the camera being tilted slightly downward or upward. The amount of rotation is preferably the same as when moving from the first rotation position to the second rotation position. The third images of the plurality of targets are recorded by the camera at the fourth rotation position.

[0228] In step 1150, using the camera model, a fourth set of calculated observation angles is determined from the fourth image.

[0229] In step 1160, a function or plot is determined based on the third and / or fourth set of calculated observation angles and the set of difference angles, where the difference angles represent the differences between the third and fourth calculated observation angles for each target.

[0230] In step 1170, using the function or plot, a camera model error is determined in a manner similar to method step 7070 described with reference to FIG. 7.

[0231] In step 1180, the camera model error is compared with a preset limit representing an acceptable camera model error.

[0232] If the camera model error is not within the preset limit, the camera model error is fed back to the camera model, and an updated camera model is provided in step 1190. Thereafter, the algorithm or flowchart is re-iterated for steps 1130, 1150, 1160, 1170, and 1180. This re-iteration is performed until the resulting camera model error is lower than the current limit.

[0233] If the camera model error is smaller than the current limit, in step 11100, it is determined that the camera model is acceptable, i.e., should be calibrated to sufficient accuracy.

[0234] Thus, according to this method, the camera can be calibrated in all three dimensions based on three recorded images using the two-dimensional distribution of the targets.

[0235] In the method described above in this specification, the calibration is described as the calibration of the camera model along the horizontal axis or x-axis and is performed prior to the calibration along the vertical axis or y-axis. However, it should be understood that the order is not essential, and instead, the calibration may be performed first along the vertical axis and subsequently along the horizontal axis.

[0236] Figures 12 to 17 show plots of the differential errors of the targets simulated at different stages of calibrating a 3D x, y, and z camera model according to the method described in this specification. This provides an illustration of the results of the calibration steps described in this specification. The calibration is simulated for both the x offset and the y offset.

[0237] The simulation was performed using the following settings and assumptions.

[0238] The images were slightly noisy (intended to mimic images from a real camera).

[0239] All offsets (x, y, z = focal length) were set to 100 micrometers.

[0240] The rotation of the camera to generate the differential angle was 1 gon.

[0241] For horizontal measurement and calibration, five targets were placed on a horizontal line, approximately 15 gons apart from each other, with one target placed in the center.

[0242] For vertical measurement and calibration, five targets were placed at a horizontal angle of approximately 0 gons (in other words, in the center of the field of view) such that they were on a vertical line, approximately 10 gons apart from each other, with one in the center.

[0243] To provide a realistic simulation of the calibration, the targets were placed in non - perfect regular positions.

[0244] The results can be explained as follows.

[0245] Figure 12 shows plots of the differential errors of each of targets 1 to 5 against each calculated observation angle. Here, before calibration, the differential horizontal angle error is calculated from the measured observation horizontal angle.

[0246] Figure 13 shows a plot of the differential error versus the calculated observation angle after correction of the x offset to make the dashed trend line horizontal. The offset between the center of the diffraction aperture and the center of the image sensor (as thus defined) is actually compensated along one axis, for example the x axis. This axis is parallel to the sensor plane and substantially parallel to the plane in which the target and the camera are located.

[0247] Figure 14 shows a plot after also correcting the focus error to flatten the virtual curvature passing through the dots.

[0248] Figure 15 shows a plot of the differential angle in the vertical direction versus the observation angle after the horizontal calibration of Figure 13. Here, the differential vertical angle error was calculated from the measured observed vertical angle.

[0249] Figure 16 shows a plot in which the y offset has also been compensated to make the trend line horizontal.

[0250] Figure 17 shows a plot in which the differential horizontal angle error was calculated again from the measured observed horizontal angle for verification. As can be seen from the comparison with the plot of Figure 11, the plot is here substantially flat and has no first- or second-order components.

[0251] In this simulated case, if this gave a better trade-off between the horizontal differential error and the vertical differential error, the focal length could have been adjusted a second time based on the measurements in the vertical plane. However, as can be seen from the figure, in 17 this was not necessary in the simulated case since an acceptable focus offset calibration had already been achieved from the horizontal calibration.

[0252] Accordingly, in summary, the present disclosure provides a method and system that enable calibration of a diffractive aperture-based camera for survey and other computer vision purposes. The present disclosure provides different alternatives for achieving calibration of a camera model along three different dimensions that can be performed using a simple arrangement of targets. It is not necessary to know the exact position of the target nor the amount of rotation of the camera.

[0253] It will be apparent to those skilled in the art that the scope of the present invention is not limited to the foregoing examples and that several modifications and variations are possible without departing from the scope of the invention as defined in the appended claims. Although the invention has been illustrated and described in detail in the drawings and the description, such illustration and description should be considered as illustrative or exemplary only and not restrictive. The invention is not limited to the disclosed embodiments and includes any combination of the disclosed embodiments that may be advantageous.

[0254] Variations to the disclosed embodiments can be understood and achieved by those skilled in the art in practicing the claimed invention from a study of the drawings, the description, and the appended claims. The features of the above-described embodiments and aspects can be combined as long as their combination does not result in an obvious technical contradiction.

Claims

1. A method for calibrating a camera in a computer vision system, wherein the camera includes an image sensor and a diffraction aperture configured to project light onto the image sensor, and the method is (a) The step of distributing a plurality of targets within the field of view of the camera, (b) The step of providing a camera model that represents the mathematical model of the camera, (c) The steps of positioning the camera at a first rotational position relative to the target and recording a first image of the plurality of targets with the camera at the first rotational position, (d) A step of determining a first set of observation angles calculated from the first image using the camera model, wherein the first set of calculated observation angles includes a first calculated observation angle of each target with respect to the camera. (e) Rotating the camera to a second rotational position relative to the target, and recording a second image of the plurality of targets with the camera at the second rotational position, (f) A step of determining a second set of observation angles calculated from the second image using the camera model, wherein the second set of calculated observation angles includes a second calculated observation angle of each target with respect to the camera. (g) A step of determining a function or plot based on the first set and / or second set of calculated observation angles and a set of difference angles, wherein the difference angle represents the difference between the first calculated observation angle and the second calculated observation angle for each target, (h) A step of determining the camera model error from the function or plot, (i) A step of comparing the camera model error with a predetermined limit, (j) If the camera model error is smaller than the limit, the step of determining that the camera model is acceptable, A method comprising: (k) if the camera model error is not less than the limit, calculating an updated camera model based on the camera model error, and repeating steps d), f), g), h), i) and j) or k) based on the updated camera model.

2. If the camera model is determined to be acceptable, the camera is rotated substantially 90 degrees around its optical axis. The step of repeating steps c) through k), The method according to claim 1, further comprising:

3. The method according to claim 2, wherein at least a portion of the plurality of targets are arranged in substantially one-dimensional rows as viewed from the camera, and the one-dimensional rows and the camera are arranged substantially in a single plane.

4. The method according to claim 3, wherein step (f) of rotating the camera from the first rotation position to the second rotation position is performed substantially in the plane.

5. The method according to claim 1, wherein the plurality of targets are arranged in a two-dimensional arrangement as seen from the camera, and the step (f) of rotating the camera from the first rotation position to the second rotation position includes rotation in a horizontal plane and rotation in a vertical plane.

6. The aforementioned multiple targets are arranged in a two-dimensional configuration as viewed from the camera, Step (f) of rotating the camera from the first rotation position to the second rotation position is performed substantially in a horizontal plane, around a first axis that extends through the plane of the opening and substantially perpendicular to the horizontal plane, and the first and second calculated observation angles represent angles in the horizontal plane. The method described above is (l) A step of determining a third set of observation angles calculated from the second image using the camera model, the third set of calculated observation angles including a third calculated observation angle of each target with respect to the camera, wherein the third calculated observation angle represents an angle in a direction substantially perpendicular to the horizontal plane, If the camera model is determined to be acceptable in step (j), the method (m) A step of rotating the camera about a second axis passing through its opening, the second axis being substantially perpendicular to the first axis and extending in the horizontal plane, thereby rotating the camera to a third rotational position. (n) The step of recording a third image of the plurality of targets with the camera in the third rotation position, (o) A step of determining a fourth set of observation angles calculated from the third image using the camera model, wherein the fourth set of calculated observation angles includes a fourth calculated observation angle of each target with respect to the camera. (p) A step of determining a second function or plot based on the third and / or fourth set of calculated observation angles and a second set of difference angles, wherein the second difference angle represents the difference between the third calculated observation angle and the fourth calculated observation angle for each target, (q) The step of determining a second camera model error from the second function or plot, (r) The step of comparing the second camera model error with a second predetermined limit, (s) If the second camera model error is smaller than the second limit, the step of determining that the camera model is acceptable, The method according to claim 1, further comprising the step of (t) if the second camera model error is not less than the second limit, calculating an updated camera model based on the second camera model error, and repeating steps (l), (o) through (r) and (s) or (t) based on the updated camera model.

7. The method according to claim 1, wherein step (a) includes arranging the plurality of targets such that the targets are placed in each lateral periphery of the camera's field of view and any remaining targets among the plurality of targets are substantially uniformly distributed among the targets placed in the lateral periphery.

8. (h1) A step of determining calibration parameters as corrections to be applied to parameters representing the mathematical model of the camera, wherein the calibration parameters are determined such that the difference angles are substantially equal for each target. The method according to claim 1, further comprising:

9. The aforementioned camera model has the following parameters: The method according to claim 1, comprising the three-dimensional position of the pixels of the image sensor with respect to the diffraction aperture.

10. The method according to claim 1, wherein the method is performed while actually taking measurements using the computer vision system.

11. The steps include: bringing the camera to a second temperature; and repeating the steps described in any one of claims 1 to 10 at the second temperature. The method according to any one of claims 1 to 10, further comprising:

12. A system for calibrating a camera in a computer vision system, wherein the camera includes an image sensor and a diffraction aperture configured to project incident light onto the image sensor, and the system is Multiple targets observed by the aforementioned camera, A mounting device for mounting the camera such that the plurality of targets are within the camera's field of view when the camera is mounted, the mounting device enables the camera to be rotated between a first rotation position and a second rotation position with respect to the plurality of targets. One or more processors, (b) Provide a camera model that represents a mathematical model of the camera, (c) Record a first image of the plurality of targets with the camera in the first rotation position, (d) Determine a first set of observation angles calculated from the first image using the camera model, wherein the first set of calculated observation angles includes a first calculated observation angle of each target relative to the camera. (e) Record a second image of the plurality of targets with the camera in the second rotation position, (f) Determine a second set of observation angles calculated from the second image using the camera model, wherein the second set of calculated observation angles includes a second calculated observation angle of each target relative to the camera. (g) Determine a function or plot based on the first set and / or second set of calculated observation angles and the set of difference angles, wherein the difference angle represents the difference between the first calculated observation angle and the second calculated observation angle for each target, (h) Determine the camera model error from the above function, (i) Compare the camera model error with a predetermined limit, (j) If the camera model error is smaller than the limit, the camera model is determined to be acceptable. A system comprising (k) one or more processors configured to calculate an updated camera model based on the camera model error if the camera model error is not less than the limit, and to repeat steps d), f), g), h), i) and j) or k) based on the updated camera model.

13. The system according to claim 12, wherein the mounting device includes a microprocessor-controlled rotating device.

14. The system according to claim 12 or 13, wherein the mounting device is configured to allow the camera to rotate around a vertical axis and / or a horizontal axis.

15. A computer program product that includes instructions, wherein when the instructions are executed by a computer, the computer will: (b) Providing a camera model that represents a mathematical model of a camera including an image sensor and a diffraction aperture configured to project incident light onto the image sensor, (d) A step of determining a first set of observation angles calculated from first images of the plurality of targets, recorded by the camera at a first rotational position relative to the plurality of targets, using the camera model, wherein the first set of calculated observation angles includes a first calculated observation angle of each target relative to the camera. (f) A step of determining a second set of observation angles calculated from the second images of the plurality of targets using the camera model, with the camera in the second rotation position, wherein the second set of calculated observation angles includes a second calculated observation angle of each target with respect to the camera. (g) A step of determining a function based on the first set and / or second set of calculated observation angles and a set of difference angles, wherein the difference angles represent the difference between the first calculated observation angle and the second calculated observation angle for each target, (h) A step of determining the camera model error from the above function, (i) A step of comparing the camera model error with a predetermined limit, (j) If the camera model error is smaller than the limit, the step of determining that the camera model is acceptable, (k) A computer program product that, if the camera model error is not smaller than the limit, calculates an updated camera model based on the camera model error, and repeats steps d), f), g), h), i) and j) or k) based on the updated camera model.

Images