Device and method for imaging a calibration pattern for calibrating an image sensor of a vehicle

Abstract

The invention relates to a method and a device for imaging a calibration pattern for calibrating an image sensor of a vehicle, wherein the device comprises: - a display device (12) mounted outside the vehicle for imaging the calibration pattern, and - a holding frame (14) for supporting the display device (12) with respect to an underlying surface, the display device (12) being rotatable relative to the holding frame (14).

Description

[0001] Hella Gutmann Solutions GmbH

[0002] P149541 PC00

[0003] Device and method for mapping a calibration pattern for calibrating a vehicle image sensor. The invention relates to a device and a method for mapping a

[0004] Calibration pattern for calibrating a vehicle image sensor, in particular a camera. Such a generic method and device are known in principle from EP 3 716 211 A1. There, a calibration pattern is displayed by a display device and an image sensor in the form of a camera receives this for the purpose of camera calibration.

[0005] Generally, during such a camera calibration, the calibration pattern is placed in front of the vehicle at a distance and height specified by the vehicle manufacturer, so that the camera captures the calibration pattern and an internal coordinate system of the camera system is calibrated based on it.

[0006] Such calibration is particularly necessary when vehicle-integrated image sensors are used by driver assistance systems, such as Adaptive Cruise Control (ACC), emergency brake assist, lane keeping assist, obstacle detection, traffic sign recognition, and even autonomous vehicles (Level 1-5).

[0007] While the display of calibration patterns using a display device, a method known from the prior art, has proven generally effective, there is still room for improvement. In particular, it has become apparent that existing solutions cannot always satisfactorily display the entire range of calibration patterns specified by the various vehicle manufacturers.

[0008] One task, therefore, is to further improve the calibration of a vehicle's image sensor, particularly with regard to ease of use and range of applications.

[0009] This problem is solved by the subject matter of the attached independent claims. Advantageous embodiments are specified in the dependent claims, in this description, and in the figures.

[0010] Accordingly, a device for displaying a calibration pattern for calibrating a vehicle image sensor is proposed, wherein the device comprises: a display device arranged outside the vehicle for displaying the calibration pattern, a support frame for supporting the display device against a surface, such as a workshop floor, wherein the display device is rotatable relative to the support frame.

[0011] The rotatable display unit allows the display area to be rotated appropriately depending on the spatial orientation of a currently defined calibration pattern. This enables both calibration patterns with pronounced vertical and horizontal dimensions to be displayed precisely and in sufficient size.

[0012] A further advantage stems from the fact that the display unit typically has a rectangular shape and, when rotated vertically, typically occupies little space in a horizontal direction. Depending on the available space, the entire display unit can therefore be rotated to save space when not in use.

[0013] The display device can be or comprise a flat screen. It can include an LED display area. The display device can be a screen, which can be understood to mean, in particular, a self-illuminating and / or backlit device for actively generating and / or displaying visual content. In particular, the screen can comprise one or more integrated light sources and / or a multitude of individually controllable light points or pixels. For example, it can be an LCD, OLED, LED, or TFT screen. In contrast to a passive projection surface or screen, which merely reflects light from an external light source (e.g., a projector) and which cannot be considered screens, the screen disclosed here, according to embodiments, can independently generate visible image content by controlling its integrated pixels.The mounting frame can optionally be mounted on a chassis to allow it to be moved along the surface. The surface can be, in particular, a workshop floor and / or generally a flat surface. The mounting frame can extend upright and, in particular, vertically relative to the surface. The mounting frame can be configured to support the display device at a fixed or optionally variable distance relative to the surface. The mounting frame can generally be configured to position the display device in front of a vehicle, and in particular at the height of the windshield.

[0014] According to one embodiment, the display device is rotatable about an axis of rotation which runs at an angle to a display surface of the display device and in particular at an orthogonal angle to this display surface.

[0015] According to a further embodiment, the device also includes a rotation position detection device configured to detect at least one rotational position of the display device. The detected rotational position can be output as a signal and / or data record to a control unit of the device. It can, for example, be stored in a digital calibration protocol. Such a calibration protocol can comprise or form a data record or a general digital information collection linked to the calibration process and / or the underlying vehicle, for example, via a database. In this way, the execution of the calibration process can be tracked at a later time, and any potential errors can be identified automatically or manually.

[0016] Additionally or alternatively, the detected rotational position can be used to verify that the display unit actually adopts the orientation specified for a given calibration pattern. If this is the case, the calibration pattern can be displayed. If not, the device can be configured, for example, to issue warning messages and / or prevent the calibration process from starting, such as by not displaying the calibration pattern.

[0017] According to one variant, at least one rotational position corresponds to one of:

[0018] - a vertical orientation of the display device, which can correspond, for example, to a so-called portrait mode;

[0019] - a horizontal orientation of the display device, which can correspond, for example, to a so-called landscape mode.

[0020] The display device, and in particular a display area thereof, can have a first dimension and a comparatively shorter second dimension, wherein the orientation of the display device can be defined by the orientation of the first dimension. The first and second dimensions can each represent side lengths of the display device and, in particular, of the display area. The display device, and in particular its display area, can generally be rectangular with different side lengths.

[0021] The vertical and horizontal positions can refer to a spatial coordinate system. The device can be configured so that only these positions, or more generally only two defined rotational positions, can be permanently and, for example, mechanically secured, in particular by means of a locking device as described below. For example, the mechanical locking and / or securing device can include a releasable detent connection or another mechanical, at least temporary, fixation, especially of the display device, relative to the mounting frame. Additionally or alternatively, these defined rotational positions can be end positions of an available rotational range of the display device, which in turn can be mechanically fixed, for example, by means of stop screws or similar devices.

[0022] One embodiment provides that the rotation position detection device comprises at least one of the following: a rotation position limit switch; a rotation position sensor configured to detect one of the display devices even between its rotation position end positions. The rotation position limit switch can be configured to detect the assumption of exactly one rotation position, whereas the rotation position sensor can be configured for continuous detection of the rotational position or angular position of the display device, particularly over its entire rotational range.

[0023] According to another variant, the device further comprises at least one end stop for limiting the rotation of the display about the axis of rotation. The end stop can, in particular, be a mechanical obstacle to block the rotation, for example, a stop screw or another projection.

[0024] In one embodiment, the display of the calibration pattern can be varied depending on the rotational position of the display device. When the display device is rotated, its resolution and / or internal monitor coordinate system can be automatically changed, for example, by evaluating the rotational position detected by a rotational position detection device. This can be accompanied by automatic rotation and / or rendering of the displayed content, including the calibration pattern.

[0025] According to one embodiment, the display device is connected to the holding device via a swivel joint, the swivel joint being configured to hold the display device in various rotational positions. The swivel joint can correspond to or be encompassed by a swivel connection as described below.

[0026] The device may optionally include at least one locking device for securing the display unit in a set rotational position. In particular, the display unit can be fixed in at least two rotational positions using a locking device, for example, in a horizontal and a vertical orientation. This can help to improve positioning accuracy and thus the accuracy of the calibration.

[0027] A further development provides that the display device is height-adjustable and that the locking device or other braking device, designed to suppress rotation of the display device, can be released depending on the height position of the display device. In particular, the height position can be selected such that the display device remains at a distance from the surface even when oriented vertically. This improves operational safety, as collisions between the display device and the surface can be prevented.

[0028] The device may further include an actuator for adjusting the height of the display unit, wherein the permissible height adjustment by the actuator can be limited depending on the orientation of the display unit. For example, this adjustment can be limited to a minimum when the display unit is in a vertical orientation; that is, an arbitrarily small height cannot be set, in order to avoid collisions with the surface. The permissible range of movement or height adjustment of the actuator can be suitably predefined and / or controlled by a control unit of the device, for example, according to the rotational position detected by a rotation position detection device disclosed herein.

[0029] A further development provides that the device is configured to maintain a constant orientation of the display unit relative to a plane orthogonal to the axis of rotation during rotation of the display unit. For this purpose, appropriately dimensioned and / or tilt-resistant rotary bearings can be used, for example, which do not permit any line of freedom except for the rotational line of freedom.

[0030] The invention also relates to a method for mapping a calibration pattern for calibrating an image sensor of a vehicle, wherein the method comprises:

[0031] Detecting the rotational position of a display device arranged outside the vehicle, which is configured to display the calibration pattern, for example by means of a rotational position detection device disclosed herein; and wherein at least one of the following measures is carried out depending on the detected rotational position: starting the display of the calibration pattern, e.g. when a predetermined rotational position is assumed;

[0032] Changing the orientation of the depicted calibration pattern, e.g., when the rotational position is changed;

[0033] Issuing an operator warning, e.g., if a specified rotational position is not assumed;

[0034] Changing a permissible height adjustment of the display device, e.g. when a predetermined and especially vertical rotational position is assumed.

[0035] All designs, variants, and further developments in the context of the device can also apply in the context of the process, and vice versa. This applies in particular to identical features in the device and the process.

[0036] Exemplary embodiments of the invention are explained below with reference to the schematic figures. The same reference numerals can be used across all figures for similar features.

[0037] Figure 1 shows a device according to an embodiment of the invention, by means of which a method according to the invention can be carried out, in a perspective view.

[0038] Figure 2 shows the device from Figure 1 in a side view.

[0039] Figure 3 shows an exploded view of a swivel joint of the device from Figure 1.

[0040] Figure 4 shows a perspective partial view of the swivel joint from Figure 3.

[0041] Figure 5 shows a detailed view of a locking device of the apparatus from Figure 1.

[0042] Figure 6 shows details concerning the end stops of the rotary movement of the device from Figure 1.

[0043] Figure 7 shows details of a rotation position detection device of the device from Figure 1. Figure 1 shows a device 10 according to an exemplary embodiment, which is configured to carry out a method according to an embodiment of the invention. The device 10 comprises a display device 12 in the form of a monitor or screen. By way of example, the display device 12 comprises an LED (Light Emitting Diode) display area of ​​conventional design. Not shown separately is a control device comprised of the display device 12, for example, comprising at least one processor configured to execute program instructions in the manner of conventional computers. In particular, the control device can be configured to change a resolution and / or rotation of a monitor coordinate system when the rotation position of the display device 12 has been changed in the manner described here.

[0044] The device 10 further comprises a support frame 14 with an optional chassis 16. The support frame 14 supports the display device 12 against a surface, typically a workshop floor.

[0045] In addition, the device 10 has a telescopic column which can be extended and retracted vertically by means of a non-visible actuator 11 for the purpose of adjusting the height of the display device 12.

[0046] Figure 1 shows that the display device 12, and more precisely its display surface, is rectangular. A first, and comparatively longer, dimension D1, corresponding to the first side of the display surface, is arranged horizontally. A second, and comparatively shorter, dimension D2, corresponding to the second side of the display surface, is arranged vertically. With reference to the first dimension D1, Figure 1 thus shows a horizontal orientation of the display device 12.

[0047] During calibration, a vehicle (not shown) is positioned in front of the display device 12 so that the front of the display device 12, facing the viewer of Figure 1, is oriented towards the front of the vehicle. Cameras integrated, for example, in or near a windshield, can then image the display device 12 and, more precisely, the calibration patterns depicted on it, in order to perform calibration measurements according to known methods. Figure 2 shows a side view of the device from Figure 1 with the display device 12 still in a horizontal orientation. A pivot joint, here referred to as a pivot-joint connection 18, is also visible between the display device 12 and the mounting frame 14. By means of the pivot-joint connection 18, the display device 12 can be rotated about a spatially horizontal axis of rotation D relative to the mounting frame 14, which may be movable but cannot be rotated.

[0048] This rotation is possible at least insofar as, contrary to the horizontal orientation in Figure 1, the first dimension D1 can be arranged spatially vertically and the second dimension D2 spatially horizontally. In other words, the display device 12 can be rotated by at least 90° about the axis of rotation D counterclockwise and / or clockwise relative to the orientation in Figure 1. Again referring to the longer dimension D1, this results in a vertical orientation of the display direction 12.

[0049] If the device 10 is not currently needed in the workshop, the display unit 12 can be rotated depending on the available space and thus stored in a space-saving manner.

[0050] If, however, calibration is required, the display device 12 can be rotated appropriately depending on the orientation of the calibration pattern to be displayed. For example, if the calibration pattern is spatially stretched horizontally, the horizontal orientation shown in Figure 1 can be selected. If, on the other hand, the calibration pattern is spatially stretched vertically, the horizontal orientation of the display device 12, rotated by 90° relative to Figure 1, can be selected. The orientation to be used can be specified to the operator by means of the display direction 12 or by means of another computer device, for example, an external diagnostic device of a known design.

[0051] The diagnostic device is typically used to control and calibrate the calibration process according to the specifications of a vehicle type or individual vehicle currently being calibrated. The diagnostic device can read a manufacturer-defined calibration pattern, for example, from an internal database or retrieve it from an external server. Depending on the dimensions of this calibration pattern to be displayed and / or on an alignment instruction already stored in the database, the diagnostic device and / or an internal control unit of the device 10 can determine and output the required alignment of the display unit 12.

[0052] In addition to ensuring the precise display of calibration patterns of varying horizontal and / or vertical extent, the device 10 can also improve ease of use as follows: In a manner known per se, the device 10 must be precisely positioned relative to a vehicle containing an image sensor to be calibrated before performing the calibration operation. This specifically concerns the distance relative to the vehicle, the height, and the relative angles to the vehicle in a spatial coordinate system. The solution disclosed here ensures that this orientation is maintained even after changing the orientation of the display direction 12. For example, the mounting frame 14 remains stationary on the surface.Furthermore, the swivel joint 18, which is explained in more detail below, is designed in such a way that, in addition to the rotational movement of the display device 12 about the axis of rotation D, it does not allow any tilting of the same or movements about other degrees of freedom.

[0053] The details of the pivot joint 18 are explained below with reference to Figure 3. Figure 3 shows this pivot joint 18 in an exploded view, in which the individual parts are distributed in or at least near the pivot joint 18 along the axis of rotation D. A mounting plate 20 is shown, to which the display device 12 (not shown) is attached. On its rear side, facing away from the display device 12, the mounting plate 20 is screwed to a pivot bearing 24, and more precisely to a first ring 25, by means of a plurality of screws 22 (see Figure 4). This pivot bearing 24 can be understood as a pivot joint that connects the display device 12 and the mounting frame 14. The pivot bearing 24 is screwed to a holder 26, which is attached to the mounting frame 14, by means of a second ring 27 (see Figure 4) by means of a plurality of screws 28.The first and second rings 25, 27 are rotatable relative to each other about the axis of rotation D. The holding frame 12 is not rotatable about the axis of rotation D, so that, according to the rotation of the display device 12, the first ring 25 is consequently rotated relative to the stationary second ring 27.

[0054] As can be seen in Figure 4, a bushing holder 30 is coupled to the first ring 27 for rotation about the axis of rotation D. The bushing holder 30 has radially extending arms 32 arranged at an angular distance from each other with respect to the axis of rotation D. Three arms 32 are shown as an example, although two arms 32 would also suffice in principle. Bushings 34 are pressed into the radially outer ends of two of the arms 32.

[0055] A spring-loaded detent pin 36, which serves as an example of a locking device, is mounted on the holder 26. The detent pin 36 is designed to protrude towards the bushing holder 30. If, by rotating the bushing holder 30, one of the bushings 34 is positioned opposite the detent pin 36, the pin engages in the bushing 34, making further rotation of the bushing holder 30 difficult and only possible with a significant increase in force. In the latter case, the detent pin 36 is forced out of the bushing 34 by contact forces with the inner walls of the bushing 34. The detent movements of the detent pin 34 are supported in the example shown by its optional conical shape.

[0056] Figure 5 shows a detailed view of the locking bolt 36 engaged with one of the bushings 34.

[0057] For many display devices 12, a single horizontal and a single vertical orientation, also referred to as landscape and portrait modes, are predefined by the manufacturer. This means that the display device cannot be rotated 180° from its horizontal orientation, for example, to assume another permissible horizontal orientation, as only one permissible horizontal orientation is specified by the manufacturer. In other words, it is specified which side of the display device 12 should be positioned vertically at the top when in horizontal orientation, so that only one permissible horizontal orientation exists. The same applies to the vertical orientation, for example, regarding which side should be the left or right side.To switch between the desired orientations of the display devices 12 illustrated in Figures 1 and 2, a rotation angle of 90° around the axis of rotation D is sufficient. However, if several permissible horizontal and / or vertical orientations are possible, a script 34 not yet present in the third arm 32 from Figure 4 could also be inserted there. Likewise, the third arm 32 can alternatively be equipped with a socket 34 instead of the left arm 32 in Figure 4 if the horizontal and / or vertical orientation specified by the manufacturer for a particular display device 12 requires this.

[0058] In the case of Figure 4, the upper bushing 34 is engaged with the locking pin 36. The display device 12 assumes a first position relative to its horizontal and vertical orientation. From this position, the engagement can be released by applying a torque around the axis of rotation D, and the bushing holder 30 can be rotated clockwise by 90°. The bushing 34 on the left in Figure 4 then engages with the locking pin 36, whereupon the display device 12 assumes its correspondingly different position relative to its vertical and horizontal orientation. A reverse rotation movement returns it to its original orientation.

[0059] Figure 4 further shows the optional presence of two bores 40 in the holder 26, into which stop screws 42 can be inserted. Again, depending on the horizontal and vertical orientation specified for a particular display device 12, a stop screw 42 can be inserted into one of the bores 40 to appropriately limit the rotational movement of the bushing holder 30. In principle, however, one bore 40 and one stop screw 42 are sufficient.

[0060] In the example shown in Figure 6, the stop screw 42 prevents further rotation of the bushing holder 30 to the left, but also prevents rotation of the left arm 32 in Figure 6 beyond the position of the stop screw 42. Thus, the possible rotation range of the bushing holder 30, and therefore of the display device 12, is clearly defined. The detent connection of the detent pin 36 and bushings 34 also clearly defines the end positions of the display device 12, allowing it to be held precisely in these positions. Further details of a rotation position detection device 44, at least partially shown in Figure 3, are explained with reference to Figure 7. This device comprises two reed sensors 46 mounted in the holder 26, although other proximity sensors could also be used.Furthermore, the rotation position detection device 44 includes, by way of example, four magnets 48 – three would in principle be sufficient, and in particular the rightmost one could be omitted – which are mounted in the bushing holder 30 and rotatable together with it. The magnets 48 are distributed in a defined manner in order to activate the reed sensors 46 depending on the rotation position of the bushing holder 30 and especially when it assumes its end positions, which correspond to one of the horizontal and vertical orientations of the display device 12.

[0061] In the configuration exemplified in Figure 7, the following applies: When the display device 12 assumes a first horizontal and vertical orientation, both reed sensors 46 are triggered by one of the magnets 48 each. This corresponds to the position shown in Figure 7, in which each of the reed sensors 46 is directly opposite a magnet 48.

[0062] When the display device 12 is in a different horizontal or vertical orientation, only one of the reed sensors 46 is triggered. Due to the positioning of the stop screw 42 as explained in Figure 6, this applies to the left reed sensor 46 in Figure 7. In this way, the orientation of the display device 12 can be checked and verified using the sensors.

[0063] This information can be used, for example, to issue warning messages if the display device 12 is rotated from a horizontal orientation, or if the support frame 14, if height-adjustable, does not have sufficient height to allow a vertical orientation of the display device 12.

[0064] For example, when assuming the vertical orientation of the locking bolts 36 (i.e. the locking device), it could not be manually and / or manually actuated and in particular released if the display device 12 or the holding frame 14 has fallen below a certain height.

[0065] The locking bolt 36 (i.e., the locking device) can only be released when the display unit 12 or the mounting frame 14 has reached a sufficient height to allow for vertical alignment. This reduces the risk of the display unit 12 colliding with the surface or the mounting frame 14. Additionally or alternatively, a braking device (not shown) can be activated in this case, preventing the display unit 12 from rotating into the vertical position.

Claims

1. Device (10) for displaying a calibration pattern for calibrating an image sensor of a vehicle, the device comprising: a display device (12) arranged outside the vehicle in the form of a screen and for displaying the calibration pattern, a support frame (14) for supporting the display device (12) against a surface, wherein the display device (12) is rotatable relative to the support frame (14).

2. Device (10) according to claim 1, wherein the display device (12) is rotatable about an axis of rotation (D) which extends at an angle to a display surface of the display device (12) and in particular at an orthogonal angle to this display surface.

3. Device (10) according to claim 1 or 2, further comprising a rotation position detection device (44) which is configured to detect at least one rotation position of the display device (12).

4. Device (10) according to claim 3, wherein the at least one rotational position corresponds to one of: - a vertical alignment of the display device (12); - a horizontal alignment of the display device (12).

5. Device (10) according to one of claims 3 or 4, wherein the rotation position detection device (44) comprises at least one of: a rotation position limit switch; a rotation position sensor (46) configured to detect a rotation position of the display device (12) also between its rotation position end positions.

6. Device (10) according to one of claims 3 to 5, wherein the device (10) is configured to use the detected rotational position for verification in such a way that an orientation of the display device specified for a given calibration pattern is actually adopted.

7. Device (10) according to claim 6, wherein the device (10) is configured to issue warning messages and / or prevent the calibration process from being carried out if the specified orientation of the display device is not adopted.

8. Device (10) according to one of the preceding claims, further comprising: at least one end stop for limiting the rotation of the display device (12) about the axis of rotation (D).

9. Device (10) according to one of the preceding claims, wherein the imaging of the calibration pattern is variable depending on a rotational position of the display device (12).

10. Device (10) according to one of the preceding claims, wherein the display device (12) is connected to the holding device (14) via a swivel joint (18), wherein the swivel joint (18) is configured to hold the display device (12) in different rotational positions.

11. Device (10) according to one of the preceding claims, further comprising: at least one locking device (36) for locking the display device (12) in a set rotational position.

12. Device (10) according to claim 11, wherein the display device (12) is height-adjustable and the locking device (36) is releasable depending on a height position of the display device (12).

13. Device (10) according to claim 12, wherein the height position is chosen such that the display device (12) remains at a distance from the surface even when assuming a vertical orientation.

14. Device (10) according to one of the preceding claims, further comprising: an actuator (11) for adjusting the height of the display device (12), wherein a permissible height adjustment by the actuator can be limited depending on an orientation of the display device (12).

15. Device (10) according to one of the preceding claims, wherein the device (10) is configured to maintain a constant orientation of the display device (12) with respect to a plane orthogonal to the axis of rotation (D) when the display device (12) is rotated.

16. Method for imaging a calibration pattern for calibrating an image sensor of a vehicle, wherein the imaging is carried out with a device (10) according to one of the preceding claims, the method comprising: Detecting the rotational position of the display device (12), and wherein at least one of the following actions is performed depending on the detected rotational position: a) starting the display of the calibration pattern; b) changing an orientation of the displayed calibration pattern; c) issuing an operator instruction; d) changing a permissible height adjustment of the display device (12).

17. Method according to variant a) of claim 16, wherein the detected rotational position is used for verification in such a way that an orientation of the display device specified for a given calibration pattern is actually adopted.

18. Method according to claim 17, wherein if the predetermined orientation of the display device is not is ingested, warning messages are issued and / or the calibration process is prevented.

Images