Method for calibrating an image acquisition device
The calibration process for image acquisition systems addresses the challenges of existing methods by using image differences to calculate device distance from a reference plane, achieving improved precision and resource efficiency while decoupling distance from orientation.
Patent Information
- Application Number
- FR2023001431
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-02-16
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2043-02-16
AI Technical Summary
Existing calibration methods for image acquisition devices, such as the Levenberg-Marquardt algorithm, face challenges in precision due to strong correlations between position and orientation determinations, and are resource-intensive when applied to multiple parameters.
A calibration process for image acquisition systems that involves acquiring at least two images of a calibration target with characteristic points, determining the differences in positions of these points between the two images, and selecting points that satisfy a predetermined condition to calculate the distance from the image acquisition device to a reference plane, independent of the device's orientation.
This process achieves increased precision in determining the distance of the image acquisition device from a reference plane with reduced resource requirements, and provides independent distance calibration from orientation, enhancing overall precision.
Smart Images
Figure 00000011_0000 
Figure 00000011_0001 
Figure 00000012_0000
Abstract
Description
Title of the invention: Method for calibrating an image acquisition device Technical field
[0001] The present disclosure relates to a method for calibrating an image acquisition device, an associated calibration device, and an associated computer program product. Prior art
[0002] Many technologies are currently being developed to assist drivers in driving vehicles, for example by providing them with additional information about their surroundings using various on-board sensors. These technologies are also necessary in the perspective of fully autonomous vehicles.
[0003] To allow correct processing of the information from these sensors, the latter must be correctly calibrated.
[0004] This is particularly the case for the image acquisition devices of these vehicles; their placement and orientation can vary from one vehicle to another and it is necessary to calibrate them correctly.
[0005] Several calibration methods are known in the prior art for determining the position and orientation of an image acquisition device.
[0006] Among these, the use of the Levenberg-Marquardt algorithm in calibration applications is well known.
[0007] However, to determine the position and orientation relative to a reference frame of an image acquisition device, this method has several drawbacks. Indeed, we can first find a strong correlation in the determination of the position with that of the orientation, thus bringing imprecision to the results. In addition, this optimization algorithm can also be costly in terms of time and computing resources, when it is applied to a plurality of parameters. Summary
[0008] The present disclosure aims to overcome the drawbacks of the prior art by proposing a method and a device for calibrating an image acquisition device, simple in their design and in their operating mode, reliable and rapid.
[0009] A method of calibrating an image acquisition device is provided, comprising determining the distance of the image acquisition device from a reference plane, the method comprising: - the acquisition, by the image acquisition device, of at least two images of a calibration target comprising a plurality of characteristic points, the calibration target being positioned so that: • during the acquisition of each image, several characteristic points have different distances from the reference plane, and • the calibration target is positioned at a constant distance from the reference plane and at two different distances from the image acquisition device respectively during the acquisition of the two images, - for a plurality of characteristic points of the target, determine a difference between the positions of the same characteristic point on the two images, and - select the characteristic points of the target whose difference between the positions on the two images satisfies a predetermined condition, and - from the distances from the reference plane of the selected characteristic points, and the differences between the positions of these points on the two images, determine the distance of the image acquisition device from the reference plane.
[0010] In embodiments, the method comprises selecting a determined number of characteristic points of the target having the smallest differences between the positions on the two images among the set of characteristic points.
[0011] In embodiments, the method comprises selecting characteristic points of the target whose differences between the positions on the two images are less than a predetermined threshold.
[0012] In embodiments, the method comprises estimating, by linear regression, the parameters of a straight line connecting the distance of a point to the reference plane and the difference between the positions of this point on the two images, and determining the distance of the image acquisition device relative to the reference plane, said distance being the root of the straight line.
[0013] In some embodiments, the method further comprises, once the distance of the image acquisition device is determined relative to the reference plane, determining an orientation of said device.
[0014] In variants of these embodiments, the determination of an orientation of the device is implemented by an optimization method applied to the positions of the characteristic points on at least one of the two images.
[0015] According to another aspect, there is provided a computer program product, comprising code instructions for implementing a method comprising: - Receiving two images on which a calibration target is visible, the calibration target comprising a plurality of characteristic points, - for a plurality of characteristic points of the target, the determination of a
[0016]
[0017]
[0018]
[0019]
[0020] difference between the positions of the same characteristic point on the two images, and - the selection of the characteristic points of the target whose difference between the positions on the two images satisfies a predetermined condition, and - from the distances of the selected characteristic points to the reference plane and the differences between the positions of these points on the two images, the determination of a distance of the image acquisition device relative to a reference plane, when it is executed by a processor. Finally, according to a last aspect, there is proposed a device for calibrating an image acquisition device, comprising a calibration target comprising a plurality of characteristic points, and a computer, configured to implement a method comprising: - receiving two images acquired by the image acquisition device, on which a calibration target is visible, the calibration target comprising a plurality of characteristic points, - for a plurality of characteristic points of the target, the determination of a difference between the positions of the same characteristic point on the two images, and - the selection of the characteristic points of the target whose difference between the positions on the two images satisfies a predetermined condition, and - from the distances of the selected characteristic points to the reference plane and the differences between the positions of these points on the two images, the determination of a distance of the image acquisition device relative to a reference plane. The proposed method makes it possible to determine the distance of the image acquisition device to a reference plane with increased accuracy and reduced resource requirements compared to the prior art. Moreover, unlike the Levenberg-Marquardt algorithm, the distance calibration is independent of the orientation of the image acquisition device, thus ensuring a more accurate result. Brief description of the drawings Other features, details and advantages will become apparent upon reading the detailed description below, and upon analyzing the attached drawings, in which: Fig.l [Fig.l] shows a flowchart of a method for calibrating an image acquisition device according to one embodiment. Fig. 2
[0021] [Fig.2] shows a device for calibrating an image acquisition device according to an embodiment. Fig. 3
[0022] [Fig.3] schematically shows the principle of the calibration process of a device image acquisition according to one embodiment. Fig. 4
[0023] [Fig.4] shows a front view of a target with a plurality of characteristic points characteristics according to an embodiment. Fig. 5
[0024] [Fig.5] shows a table reporting the accuracy performance of the method according to the invention.
[0025] Description of the embodiments
[0026] Reference is now made to [Fig. 1], which schematically represents the main steps of a method for calibrating an image acquisition device according to a particular embodiment.
[0027] With reference to [Fig.2], the method is implemented by a calibration device comprising the image acquisition device 201, which may be for example a camera, or a video camera, and a processing unit 203, comprising at least one computer and a memory storing the code instructions 206 leading to the execution of the steps of the calibration method.
[0028] The calibration method is implemented to determine the distance of the image acquisition device relative to a reference plane. In a particular embodiment, the reference plane is the horizontal, and the determined distance is the height of the image acquisition device relative to the horizontal. Alternatively, the reference plane may be a vertical plane, and the distance relative to the reference plane is a distance along the direction of the normal of the plane considered.
[0029] In the example of application of the method in [Fig.2], the image acquisition device is a camera on board a land vehicle, and the calibration method aims to determine the height of the camera relative to the ground.
[0030] The different steps of the method for calibrating the acquisition device 201 are implemented by executing code instructions of a software program 206 on a remote processing unit 203, connected to the image acquisition device 201 by a link 204 allowing communication.
[0031] According to variants, this link 204 can be wired, for example with a USB protocol, or wireless, for example with a Bluetooth or Wi-Fi protocol. According to certain variants, this link 204 can be an ad hoc communication system.
[0032] In another variant, the calculator 203 is directly integrated into the image acquisition device 201, and the link 204 is then internal to said device 201.
[0033] In the embodiments, the calculator 203 comprises at least one processor with a computing core enabling it to execute, at least in part, code instructions of the software program 206 leading to the performance of the steps of the calibration method. In a variant, said calculator 203 comprises a graphics processor.
[0034] The method comprises the acquisition 100 by the image acquisition device 201 of at least two images of a calibration target 202 placed at a constant distance from a reference plane but at different distances from the device. With reference to [Fig. 4], a non-limiting example of a calibration target is shown which comprises a set of characteristic points 400. The characteristic points are points whose position on the target is known and identifiable (for example vertices of polygons or centers of circles). Preferably for the implementation of the method, the characteristic points used are all aligned along the same axis which, during the acquisition of the images, is positioned perpendicular to the reference plane, so that the characteristic points are at different distances from the reference plane. The acquisition of the images of the target therefore makes it possible to obtain two images on which the reference points are visible..
[0035] In a variant, step 100 consists of acquiring a single image of at least two identical targets so that said targets can be visible simultaneously by the image acquisition device, the two targets being at the same constant distance from the reference plane but at different distances from the image acquisition device.
[0036] From this or these images acquired in step 100, the method can determine the height of the image acquisition device according to the following principle, illustrated in a variant by the schematic view in [Fig.3].
[0037] Let a characteristic point P of the target 202 be located at a distance hp from the reference plane. This point is noted Pi when the target is at a distance di from the image acquisition device 201 at C, and P2 when the target is at a distance d2. It is noted that for the present illustrated variant, the target is moved away such that C is coplanar with the translation plane of the target (plane formed by the characteristic points of the target before and after the translation).
[0038] Thus a first image acquisition of the target 202 located at a distance di makes it possible to obtain a point Pf, projection onto the image of the point Pp
[0039] A second image of the target 202 at a distance d2 makes it possible to obtain in the same way a point P2', projection onto the image of the point P2.
[0040] The gap e, distance between the projections Pf and P2', is zero when the points Pi and P2 have the height hp of the image acquisition device 201. We then have hp = hc and we note the particular positions of these points Gi and G2 having the same projection G'.
[0041] In a non-illustrated variant where C is not coplanar with the translation plane of the target, the difference e relates to the distance between two new projections of the points P / and P2' on the axis passing through the center of the image and which forms with C a plane perpendicular to the image plane and to the reference plane. Thus, Gi and G2 have a first distinct projection on the image plane, and a second coincident projection on said axis.
[0042] For positions of Pi and P2 respectively close to points Gi and G2, the gap e is linearly related to the distance hp. We therefore have e — a*hp + b with a and b the parameters of the line which connects the gap e to the distance hp.
[0043] Also, for points Pi and P2 coinciding respectively with points Gi and G2, we have hp = hc , and e — a*hc + b = 0 . The distance hc is then called the root solution of the line. And we deduce hc — -b / a.
[0044] The method of the present disclosure has the principle of thus estimating the positions of the points Gi and G2, and consequently the distance hc of the image acquisition device 201 from different characteristic points Pi and P2 respectively close to the points Gi and G2.
[0045] Thus, step 110 comprises determining the differences e from one photo to another, between the positions of a plurality of characteristic points of the target 202.
[0046] For example, in the case where the method aims to determine the height of the image acquisition device 201 with two images, step 110 aims to determine for a plurality of characteristic points of the target 202, the distance e separating the projections Pf and P2' of each characteristic point.
[0047] The method then comprises a step 120 of selecting a plurality of characteristic points whose deviation calculated in the previous step satisfies a predetermined condition. Said selection condition makes it possible to obtain a plurality of points whose distance hp to the reference plane is sufficiently close to that hc of the image acquisition device 201 to be considered linearly linked to their respective deviation e.
[0048] In one embodiment of step 120, the points are selected whose respective deviations e in absolute value are less than a predetermined threshold.
[0049] Alternatively to step 120 the points whose respective deviations in absolute value are less than a predetermined percentile can be selected.
[0050] As a variant, step 120 comprises the selection of a determined number of points, for example between two and five points, for example between two and three points, whose deviations e separating the projections of the points are minimum among the set of points. for which these deviations were calculated.
[0051] By way of example, in an embodiment where the height of a camera 201 is to be determined, in step 120, the three points whose respective deviations e in absolute value are the smallest are selected, i.e. the three points whose heights hp are closest to hc, that of said camera 201.
[0052] The method comprises determining 130 the distance hc of the image acquisition device 201 to the reference plane from the deviations calculated in step 110 for the points selected in step 120, and the distance of these points relative to the reference plane during the acquisition of the images.
[0053] To do this, the equation of a straight line is determined connecting, for a given characteristic point, the difference between the projection of the point on the two images with the distance of the characteristic point relative to the reference plane. This determination can be carried out, in the case where two characteristic points have been selected in step 120, by linear interpolation, or, when three or more points have been selected in step 120, by linear regression.
[0054] Once the equation of the straight line has been determined, the distance hc from the image acquisition device 201 to the reference plane is determined by calculating the root of the straight line. This estimate is advantageously obtained more quickly and more precisely than what the state of the art allows.
[0055] In embodiments, the determination of the distance of the image acquisition device from the reference plane may be followed by a step 140 of determining the orientation of the image acquisition device, for example by applying a minimization technique, for example the Levenberg-Marquardt algorithm.
[0056] The table illustrated in [Fig.5] reports the performance of the accuracy of the present method for determining the height of a camera relative to the ground, compared to the use of the Levenberg-Marquardt algorithm.
[0057] It is clear, through the results presented in the table of [Fig.5], that the estimated height of the device 201 by the present method is closer to the actual height of said device 201 than that estimated with the techniques of the prior art, and this for different configurations of orientation of the image acquisition device 201 and height of target 202.
[0058] It should be noted that non-linear minimization algorithms are also generally more costly in terms of time than their linear counterparts. This is particularly the case for the present method.
Claims
Claims
1. A method for calibrating an image acquisition device (201), comprising determining the distance of the image acquisition device (201) from a reference plane, the method comprising: - the acquisition (100), by the image acquisition device (201), of at least two images of a calibration target (202) comprising a plurality of characteristic points (400), the calibration target (202) being positioned, such that: * during the acquisition (100) of each image, several characteristic points (400) have different distances from the reference plane, and * the calibration target (202) is positioned at a constant distance from the reference plane and at two different distances from the image acquisition device (201) respectively during the acquisitions of the two images, - for a plurality of characteristic points (400) of the target (202),determining a difference (e) between the positions of the same characteristic point (400) on the two images, and - selecting the characteristic points (400) of the target (202) whose difference (e) between the positions on the two images satisfies a predetermined condition, and - from the distances from the reference plane of the selected characteristic points (400), and the differences (e) between the positions of these points on the two images, determining the distance (hc) of the image acquisition device (201) from the reference plane.,
2. Method according to claim 1, comprising the selection (120) of a determined number of characteristic points (400) of the target (202) having the deviations (e) between the positions on the two images, the smallest among the set of characteristic points (400).
3. Method according to claim 1, comprising the selection of the characteristic points (400) of the target (202) whose differences (e) between the positions on the two images are less than a predetermined threshold.
4. Method according to any one of the preceding claims, comprising estimating, by linear regression, the parameters of a straight line connecting the distance of a point to the reference plane and the difference between the positions of this point on the two images, and determining (130) of the distance (hc) of the image acquisition device (201) relative to the reference plane, said distance (hc) being the root of the straight line.
5. Method according to any one of the preceding claims, further comprising, once the distance (hc) of the image acquisition device (201) has been determined relative to the reference plane, determining (140) an orientation of said device (201).
6. Method according to the preceding claim, in which the determination (140) of an orientation of the device (201) is implemented by an optimization method applied to the positions of the characteristic points (400) on at least one of the two images.
7. Computer program product, comprising code instructions for implementing a method comprising: - receiving two images on which a calibration target (202) is visible, the calibration target (202) comprising a plurality of characteristic points (400), - for a plurality of characteristic points (400) of the target (202), determining (110) a difference (e) between the positions of the same characteristic point (400) on the two images, and - selecting (120) the characteristic points (400) of the target whose difference (e) between the positions on the two images satisfies a predetermined condition, and - from the distances of the selected characteristic points (400) to the reference plane and the differences (e) between the positions of these points on the two images, determining (130) a distance (hc) of the image acquisition device (201) relative to a reference plan, when executed by a processor.
8. A device for calibrating an image acquisition device, comprising a calibration target (202) comprising a plurality of characteristic points (400), and a computer (203), configured to implement a method comprising: - receiving (100) two images acquired by the image acquisition device (201), on which a calibration target (202) is visible, the calibration target comprising a plurality of characteristic points (400), - for a plurality of characteristic points (400) of the target (202), determining (110) a difference (e) between the positions of the same characteristic point (400) on the two images, and - selecting (120) the characteristic points of the target whose difference (e) between the positions on the two images satisfies a predetermined condition, and - from the distances of the selected characteristic points (400) to the reference plane and the differences between the positions of these points on the two images, the determination (130) of a distance (hc) of the image acquisition device (201) relative to a reference plane.