Arrangement and method for calibrating a vehicle image sensor

The movable calibration device with automated feedback simplifies vehicle image sensor calibration by reducing manual adjustments, achieving efficient and time-saving alignment of the calibration pattern.

WO2026238782A1PCT designated stage Publication Date: 2026-11-19HELLA GUTMANN SOLUTIONS GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2026/063873
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-13
Filing Date
2026-05-12
Publication Date
2026-11-19

Smart Images

  • Figure EP2026063873_19112026_PF_FP_ABST
    Figure EP2026063873_19112026_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to an arrangement (10) and a method for calibrating a vehicle image sensor. The arrangement (10) comprises a calibration apparatus (14) having: - a display device (12), arranged outside the vehicle, for displaying the calibration pattern; - at least one computing device (20), wherein the calibration apparatus (14) can be variably aligned relative to the vehicle, and wherein the computing device (20) is designed to determine at least one permissible alignment state of the calibration apparatus (14) such that, when said alignment state is assumed, the calibration pattern can be displayed such that it assumes a target state. In particular, the computing device (20) can determine whether, for a given alignment state, a target state of the calibration pattern can be achieved by shifting the calibration pattern or otherwise suitably setting its position and / or orientation and / or size.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Hella Gutmann Solutions GmbH

[0002] P153053PC00

[0003] Arrangement and method for calibrating a vehicle image sensor. The invention relates to an arrangement for imaging a calibration pattern for calibrating a vehicle image sensor. Furthermore, the invention relates to a method for aligning a calibration device for calibrating a vehicle image sensor.

[0004] Such a generic method and device are known in principle from EP 3716211 A1. There, a display device projects a calibration pattern, which is then captured by an image sensor in the form of a camera for camera calibration. Generally, in such a camera calibration, a display device for showing a calibration pattern is positioned in front of the vehicle at a distance and height specified by the vehicle manufacturer, such that the camera captures the calibration pattern and, based on this, an internal coordinate system of the camera system is calibrated.

[0005] Such calibrations are particularly necessary when vehicle-integrated image sensors are used by driver assistance systems, such as Adaptive Cruise Control (ACC), Emergency Brake Assist, Lane Keeping Assist, Obstacle Detection, Traffic Sign Recognition, and autonomous driving systems (Level 1-5).

[0006] While the display of calibration patterns using a display device, a method known from the prior art, has proven generally effective, there is still room for improvement. In particular, it has become apparent that the setup effort required before calibration can be performed remains significant and demands considerable time and expertise. Currently, a process typically requires iterative adjustments and repeated measurements until the display device is positioned in front of the vehicle in such a way that it displays the calibration pattern as prescribed – that is, until it assumes the target relative position to the vehicle.

[0007] The invention therefore aims to reduce the setup effort required for calibrating a vehicle image sensor.

[0008] This problem is solved by the subject matter of the attached independent claims. Advantageous embodiments are specified in the dependent claims, in this description, and in the figures.

[0009] Accordingly, an arrangement for displaying a calibration pattern for calibrating a vehicle image sensor is proposed, wherein the arrangement comprises a calibration device, with:

[0010] - a display device located outside the vehicle for displaying the calibration pattern, - at least one computer device,

[0011] wherein the calibration device is variably alignable relative to the vehicle, wherein the computer device is configured to determine at least one permissible alignment state of the calibration device, in particular relative to the vehicle, in which the calibration pattern can be displayed in such a way that it assumes a target state.

[0012] The display device may include a projector, a screen and / or an electronic display surface, in particular an LED or LCD display or another flat screen.

[0013] The display unit can be freely movable in space, particularly in all spatial degrees of freedom. It can be actively moved relative to the vehicle, especially if it is stationary. The relative movement of the calibration device relative to the vehicle can include at least an active movement of the display unit relative to the vehicle, and optionally also of the computer unit. The computer unit is external to the vehicle and therefore not a fixed component of the vehicle.

[0014] For example, the display unit can be mounted on a support frame. The support frame can optionally be mounted on a chassis to allow it to be moved along the surface. The surface can be, in particular, a workshop floor and / or generally a flat surface. The support frame can extend upright and, in particular, vertically relative to the surface. The support frame can be configured to hold the display unit at a fixed or optionally variable distance, in particular a height distance, relative to the surface. The support frame can generally be configured to position the display unit in front of a vehicle, and in particular at the height of the windshield and / or in front of a vehicle's rear, opposite the installation location of a rear-view camera, if present.

[0015] The alignment of the calibration device can be performed largely or entirely manually, for example, by moving the display unit and, in particular, the mounting frame coupled to it. The computer system can obtain or determine position information of the vehicle and / or the calibration device and / or specify a relative position between the vehicle and the calibration device. In particular, the computer system can determine information about the relative position of the vehicle to the calibration device or obtain it from a prior art measuring system. For example, the relative position can be determined based on the detection of optical targets, which may be temporarily attached to the vehicle.

[0016] Additionally or alternatively, the computer system can receive or store information regarding at least one target relative position of the calibration pattern relative to the vehicle, for example, from a database provided by the vehicle manufacturer. This information can be used to determine an acceptable alignment state. This state can be achieved, in particular, when the current relative position of the vehicle and the calibration device corresponds to a target relative position.

[0017] The solution disclosed here reduces the requirements for manual positioning of the display device. Initially, a rough positioning of the display device in front of the vehicle may suffice. The computer system can then automatically determine a permissible alignment state, at which point the calibration pattern can be displayed in a target state. Specifically, it can first determine whether a permissible alignment state has already been achieved with the initial alignment and, optionally, whether and / or which adjustments to the alignment are necessary to reach the permissible alignment state. This automated check and optional feedback to the operator can simplify or at least partially eliminate time-consuming manual checks and iterative adjustments to the alignment.

[0018] Further training stipulates that the target state for displaying the calibration pattern includes the completeness of the calibration pattern and / or a target relative position of the calibration pattern relative to the vehicle and / or a target size of the calibration pattern. This ensures that the calibration pattern is neither cut off nor unsuitably positioned. The computer system can thus automatically check whether the calibration pattern is suitable for display during calibration.

[0019] Further training stipulates that the computer system is configured to repeatedly determine, for example during manual changes to the alignment of the calibration device relative to the vehicle, whether a permissible alignment state is being achieved.

[0020] This allows the operator to iteratively move the calibration device until a suitable position is reached. The device preferably provides visual, audible, and / or other feedback. This feedback can include, for example, a display of directional information and / or distance values ​​up to a suitable relative position and / or other information regarding necessary adjustments to the orientation of the display device. While the operator can still manually and, in particular, iteratively change the orientation, they are relieved of the task of continuously verifying whether the orientation change was successful, i.e., whether a permissible orientation state for performing the calibration process has been reached.

[0021] Accordingly, further training stipulates that the system be configured to provide feedback to the operator indicating that a permissible alignment state has been reached. This assists the operator in positioning the device and reduces the effort required for manual adjustment. The feedback can be provided, for example, visually or via acoustic signals. For instance, an additional display device, particularly an operator terminal, can be provided for performing and / or controlling the calibration process.

[0022] Further training provides for the calibration pattern to be variably positioned, oriented, and / or displayed at a variable size within the display area of ​​the display device. This allows the display of the calibration pattern to be flexibly adjusted within the display area to achieve a target state. The computer system can be configured to consider at least one of these variable degrees of freedom when verifying the existence of a target state. This can include checking whether a change in at least one of these degrees of freedom enables the achievement of a target state. Optionally, such a change, including the subsequent display, can be initiated or carried out by the computer system. The operator can then receive positive feedback regarding the achievement of a target state.

[0023] In summary, the computer system can be configured for automatic scaling and / or automatic displacement of the calibration pattern and / or for checking whether such measures, particularly for a given relative positioning, enable the achievement of a target state of the type disclosed herein. Accordingly, a further development provides that the computer system is configured to determine whether, for a given alignment state, a target state of the calibration pattern can be achieved by appropriately defining its position and / or orientation and / or size within the display area.

[0024] The invention also relates to a method for aligning a calibration device for calibrating a vehicle image sensor, wherein the calibration device is variably alignable relative to the vehicle and comprises a display device for displaying the calibration pattern, wherein the method has:

[0025] Computer-aided determination of at least one permissible alignment state of the display device, in which the calibration pattern can be displayed in such a way that it assumes a target state.

[0026] Further training stipulates that the target state includes completeness of the calibration pattern and / or a target relative position of the calibration pattern relative to the vehicle and / or a target size of the calibration pattern.

[0027] Further training stipulates that the procedure also includes, during changes in the alignment of the calibration device relative to the vehicle: repeatedly determining whether a permissible alignment state is being assumed.

[0028] Further training stipulates that the procedure also includes: issuing feedback to an operator that a permissible alignment state has been reached.

[0029] Further training stipulates that the calibration pattern within a display area of ​​the display device can be variably positioned and / or variably oriented and / or displayed with variable size.

[0030] Further training stipulates that the procedure includes: determining whether a target state for a given alignment state can be achieved by appropriately defining the position and / or orientation and / or size of the calibration pattern within the display area.

[0031] All explanations, variations, and further developments disclosed herein in the context of the order may also apply in the context of the procedure, and vice versa. This applies in particular to identical features in the order and the procedure.

[0032] Exemplary embodiments of the invention are explained below with reference to the schematic figures. The same reference numerals can be used across all figures for similar features.

[0033] Figure 1: shows an arrangement according to an embodiment of the invention, Figure 2: shows a flowchart of a method according to an embodiment of the invention.

[0034] Figure 1 shows an arrangement 10 according to an exemplary embodiment, configured for carrying out a method according to an embodiment of the invention. The arrangement 10 comprises a calibration device 14 with a display device 12 arranged outside the vehicle for displaying a calibration pattern. The display device 12 is exemplary configured as a monitor or screen and preferably comprises an LCD screen (Liquid Crystal Display) or another flat screen, for example an LED (Light Emitting Diode) or OLED (Organic Light Emitting Diode) display area of ​​conventional design. The display device 12 is exemplary arranged on a support frame 18, which is optionally mounted on a chassis 16 in order to be moved along a surface, in particular a flat workshop floor.The mounting frame 18 supports the display device 12 against the surface and can extend upright and, in particular, vertically relative to it. The mounting frame 18 is designed to support the display device 12 at a fixed or variable distance from the surface, preferably at a height approximately corresponding to the front of a vehicle or a windshield.

[0035] Furthermore, the calibration device 14 is equipped with a computer unit 20. This unit can be stationary and / or movable independently of the display unit 12. In particular, it can be spatially separated from the display unit 12, for example, within an operator terminal or workshop PC (not shown). As indicated by a dashed double arrow, a data transmission capability to the display unit 12 preferably exists even in this case, in particular to optionally adjust or generally control the display of a calibration pattern there. Alternatively, the computer unit 20 can be integrated into the movable unit shown, consisting of the display unit 12, the mounting frame 18, and the chassis 16, in particular while maintaining the described data transmission capability.

[0036] The computer unit 20 comprises a processor configured to execute program instructions in the manner of conventional computers. In particular, the computer unit 20 is configured to process position information and to determine a permissible alignment state of the calibration device 14 relative to the vehicle (not shown), as explained in more detail in the context of Figure 2.

[0037] During calibration, a vehicle (not shown) is positioned in front of the display device 12 so that the front of the display device 12, facing the viewer of Figure 1, faces the front of the vehicle or, in the case of a rear-view camera, the rear of the vehicle. For example, cameras integrated in or near the vehicle's windshield can capture the display device 12 and, in particular, known calibration patterns displayed thereon, in order to perform calibration measurements according to known methods. The calibration device may include further components not mentioned here, according to the prior art.

[0038] With reference to Figure 2, a process sequence according to an embodiment of the invention is explained below, wherein the process is carried out by means of an arrangement 10 according to an embodiment of the invention, in particular with the arrangement 10 from Figure 1.

[0039] This involves a calibration device 14 comprising a display unit 12 mounted on a movable and height-adjustable support frame, as shown, for example, in Figure 1. The display unit 12 includes a display area of ​​known size and known position within the calibration device 14. It is further assumed that the calibration device 14 is positioned so that it is oriented relative to the vehicle in a predetermined manner (e.g., perpendicular and parallel to a vehicle axis). This can be ensured in a manner known per se by mechanical adjustment and / or by means of spirit levels. It is further assumed that the maintenance of the correct relative orientation is guaranteed even after a subsequent translational displacement of the calibration device and is checked accordingly, for example, by an operator and readjusted if necessary.

[0040] Step S1 of the method shown in Fig. 2 comprises the following: The calibration device 14 is positioned in front of the vehicle. The position of the vehicle relative to the calibration device 14, and thus to its display unit 12, is determined using known solutions of the prior art. For example, optical targets attached to the vehicle, at least temporarily, can be detected.

[0041] Step S2 of the method shown in Fig. 2 comprises the following: The calibration device 14 receives or already possesses information regarding a target relative position of the calibration pattern relative to the vehicle. This information is, or includes, for example, position data specified by the vehicle manufacturer. Optionally, the target relative position can be defined as a function of a wheel arch height or another height-related dimension. The wheel arch height or other height dimension can therefore be considered as a modifiable and / or variable parameter, in particular to take into account the current ride height of the vehicle. The wheel arch height can, for example, be entered manually and / or measured.

[0042] Step S3 of the procedure shown in Fig. 2 comprises the following: The calibration device 14 determines whether, with the current relative position of the calibration device 14 or display unit 12 and the vehicle, the calibration pattern – possibly after necessary modification (e.g., height adjustment, size adjustment, or displacement) – can be completely displayed within the display area. This is based on the known vehicle position and the target relative position of the calibration pattern. More precisely, it is determined whether the calibration pattern can be completely displayed by the display area when the target relative position is reached.

[0043] This can be achieved, for example, by proceeding as follows: It is checked whether the calibration pattern—particularly by moving it within the display area—can be positioned so that it occupies the specified target relative position to the vehicle and is fully visible. More precisely, it is checked whether the specified target relative position of the calibration pattern is achievable relative to the vehicle when the calibration pattern is displayed on the display area in its current position, especially by moving its display position. As an additional condition, the calibration pattern should be fully visible, i.e., it should not virtually extend beyond the edge of the display area when it is displayed and, in particular, moved to achieve the target relative position.In this context, the known position of the calibration device 14 relative to the vehicle is used, and the extent, in particular its location and dimensions, of its display area is calculated in a reference coordinate system, such as a vehicle coordinate system. This calculated extent is compared with a target position and, optionally, a target extent of the calibration pattern in this reference coordinate system, the target position being derived from the target relative position to the vehicle. If the target position of the calibration pattern, and in particular its extent, lies within the calculated display area extent, such that it can be displayed completely, a corresponding positive feedback signal is optionally issued. The calibration procedure can then begin.

[0044] If this is not the case, negative feedback is issued, which preferably prompts the operator to change the position of the display device 12 relative to the vehicle. The feedback specifically concerns actively moving the display device 12 relative to the vehicle. The feedback can contain information about the type of change required, for example, a direction indication, such as an arrow, indicating the direction in which the display device should be moved. Alternatively or additionally, the feedback can contain information about the extent of the required change, for example, the required displacement. The latter can be specified, for example, as a numerical value and / or represented graphically, e.g., by means of a color selection or a symbol size, in particular arrow size. The process then continues with step S4.

[0045] Step S4 of the procedure shown in Fig. 2 comprises the following: If the operator then (i.e., following the second case from step S3) changes the relative position of the calibration device 14, including its display device 12, to the vehicle, the determination according to step S3 is repeated, in particular continuously and / or iteratively. As soon as a relative position is reached at which the calibration pattern can be fully displayed at a freely definable position within the display area while reaching the target relative position, the operator receives corresponding positive feedback. The operator can then immediately stop moving the display device 12, and the calibration pattern is subsequently displayed in the target relative position.As long as this is not the case, the negative feedback can be updated after each review cycle; for example, information regarding a required displacement direction and / or a required displacement extent can be continuously updated.

[0046] In the foregoing, it was assumed by way of example that the relative orientation of the calibration device 14, including its display device 12, to the vehicle corresponds to a target orientation. If this is not the case, this relative orientation can be considered as an additional parameter of the procedure described above. In particular, the position information considered can be extended to include orientation information, and thus the complete spatial position can be determined; that is, in addition to the translational positional degrees of freedom, the orientational and rotational degrees of freedom can also be considered.

[0047] In step S1, this could mean, for example, determining not only the relative position between calibration device 14 and the vehicle, but also its relative orientation. In step S2, a complete target relative position could be taken into account. In step S3, the determination of the display area extent in the reference and, in particular, vehicle coordinate system could be extended to include the orientation of the display device 12, thus determining the complete spatial position and extent of the display area in the vehicle coordinate system. Finally, step S4 could involve an iterative, continuous adjustment not only of the position, but also of the orientation, until the calibration device 14 is aligned in such a way that both a target position and a target orientation are fulfilled as components of the target state for displaying the calibration pattern.This ensures that the calibration pattern is displayed optimally and the calibration process can begin.

[0048] According to an inventive variant of the method described above, the following can be provided additionally or alternatively: Following steps S1 or S2, permissible relative positions of the calibration device 14 or display device 12 and the vehicle are determined in advance, at which the calibration pattern can be displayed while achieving the target state. For example, a spatial range encompassing permissible relative positions can be determined.

[0049] The relative position of the display device 12, or rather of its display surface, to the vehicle is then continuously determined. In particular, if the determined relative position lies within the aforementioned range, a permissible relative position has been reached. A corresponding positive feedback signal is then issued, and the calibration procedure can begin. This variant can also be extended to include orientation information according to a further development. The procedure according to the invention can, for example, result in the alignment of the display device 12 needing to be changed only slightly, and in particular to a minimal extent. As soon as a permissible alignment has been reached for the first time, the calibration pattern can be displayed in the target state.

[0050] For example, when adjusting the height from any given position, only the minimum required stroke can be performed. The calibration pattern can be displayed as soon as this is possible for the first time when the target state is reached, i.e., as soon as a height adjustment has been performed and allows it.

[0051] The same applies to other alignment freedom lines. It is not necessary to align the display device 12 with high precision relative to the vehicle image sensor from the outset and then subsequently adjust the position and / or orientation of the calibration pattern within the display area. Instead, the operator only needs to perform a rough alignment from the beginning, after which the arrangement 10, and in particular its computer unit 20, automatically checks the position and / or orientation of the calibration pattern and, if necessary, determines it, or at least guides the operator to a suitable position and / or orientation of the display device 12 by providing appropriate feedback.

Claims

Patent claims 1. Arrangement (10) for displaying a calibration pattern for calibrating a vehicle image sensor, wherein the arrangement (10) comprises a calibration device (14) comprising: a display device (12) arranged outside the vehicle for displaying the calibration pattern, at least one computer facility (20), wherein the calibration device (14) can be variably aligned relative to the vehicle, wherein the computer device (20) is configured to determine at least one permissible alignment state of the calibration device (14) in which the calibration pattern can be displayed in such a way that it assumes a target state.

2. Arrangement (10) according to claim 1, where the target state includes completeness of the calibration pattern and / or a target relative position of the calibration pattern relative to the vehicle and / or a target size of the calibration pattern.

3. Arrangement (10) according to claim 1 or 2, wherein the computer device (20) is configured to repeatedly determine, during changes in the alignment of the calibration device (14) relative to the vehicle, whether a permissible alignment state is assumed.

4. Arrangement (10) according to one of the preceding claims, wherein the arrangement (10) is configured to provide feedback to an operator that a permissible alignment state has been reached.

5. Arrangement (10) according to one of the preceding claims, wherein the feedback comprises a display of direction information and / or distance values ​​up to a suitable relative position of calibration device (12) and vehicle and / or other information concerning required changes in the orientation of the display device (12).

6. Arrangement (10) according to one of the preceding claims, wherein the calibration pattern can be variably positioned and / or variably oriented and / or displayed with variable size within a display area of ​​the display device (12).

7. Arrangement (10) according to claim 6, wherein the computer device (20) is configured to determine whether, for a given alignment state, a target state of the calibration pattern can be achieved by appropriately specifying the position and / or orientation and / or size of it within the display area.

8. Arrangement (10) according to one of the preceding claims, wherein the computer device (20) is designed external to the vehicle.

9. Arrangement (10) according to one of the preceding claims, wherein the computer device (20) is configured to obtain or determine position information specifying a relative position of the vehicle and the calibration device (14).

10. Method for aligning a calibration device (14) for calibrating a vehicle image sensor, wherein the calibration device (14) is variably alignable relative to the vehicle and comprises a display device (12) for displaying the calibration pattern, wherein the method has: computer-aided determination of at least one permissible alignment state of the display device (12), in which the calibration pattern can be displayed in such a way that it assumes a target state.

11. Method according to claim 10, where the target state includes completeness of the calibration pattern and / or a target relative position of the calibration pattern relative to the vehicle and / or a target size of the calibration pattern.

12. Method according to claim 10 or 11, wherein the procedure further comprises, during changes in the alignment of the calibration device (14) relative to the vehicle: repeatedly determining whether an acceptable alignment state is being assumed.

13. Method according to any one of claims 10 to 12, the procedure further includes: issuing feedback to an operator that a permissible alignment state has been reached.

14. Method according to claim 13, wherein the feedback includes a display of direction information and / or distance values ​​up to a suitable relative position of calibration device (12) and vehicle and / or other information concerning required changes to the orientation of the display device (12).

15. Method according to any one of claims 10 to 14, wherein the calibration pattern can be variably positioned and / or variably oriented and / or displayed with variable size within a display area of ​​the display device (12).

16. Method according to claim 15, the procedure includes: determining whether a target state for a given alignment state can be achieved by appropriately defining the position and / or orientation and / or size of the calibration pattern within the display area.

17. Method according to any one of claims 10 to 16, wherein the computer device (20) is designed external to the vehicle.

8. Method according to one of claims 10 to 17, wherein the computer device (20) is configured to obtain or determine a relative position of the vehicle and the calibration device (14) by providing position information.