Method of calibration of an optical sensor system and optical sensor system

Abstract

The disclosure relates to a method of calibration of an optical sensor system (10) for a vehicle (30), the method comprising transmitting a first optical signal (L1), deflecting the first optical signal (L1) using a first mirror (16.1), detecting the first optical signal (L1) using an optical detection sensor (20), transmitting a second optical signal (L2), deflecting the second optical signal (L2) using a second mirror (16.2), detecting the second optical signal (L2) using the optical detection sensor (20), determining calibration data using the respective detected optical signal (L1, L2), wherein the calibration data is determined depending on the respective mirror (16.1, 16.2) used to deflect the respective optical signal (L1, L2), saving the calibration data. The disclosure further relates to an optical sensor system (10) and a vehicle (30) comprising the optical sensor system (10).

Description

[0001] METHOD OF CALIBRATION OF AN OPTICAL SENSOR SYSTEM AND OPTICAL SENSOR SYSTEM

[0002] Field

[0003] The present disclosure relates to a method of calibration of an optical sensor system for a vehicle. The disclosure further relates to an optical sensor system for a vehicle and a vehicle comprising the optical sensor system.

[0004] Background

[0005] Modern vehicles like cars, vans, trucks, motorcycles, etc. may comprise sensor systems, whose data are used for driver information and / or are used by driver assistance systems.

[0006] Sensor systems are constantly being developed for various functions, e. g. for the acquisition of environmental information in the near and far range of vehicles, such as passenger cars or commercial vehicles. Based on the acquired data, a model of the vehicle environment may be generated and a reaction to changes in this vehicle environment is possible. Sensor systems may also serve as sensors for driver assistance systems, in particular assistance systems for autonomous or semi-autonomous vehicle control. They may for example be used to detect obstacles and / or other road users in the front, rear or blind spot areas of a vehicle. Sensor systems may be based on different sensor principles, such as radar, ultrasound, optics.

[0007] An important optical sensor principle for environment detection, e. g. of vehicles, is the lidar technology (lidar: light detection and ranging). A lidar system comprises an optical transmission device and an optical reception device. The transmission device emits an optical signal, which can be continuous or pulsed. In addition, the optical signal may be modulated. For example, electromagnetic waves in the form of laser beams in the ultraviolet, visual or infrared range may be used as optical signals in a lidar system. The light is received by the optical reception device after reflection from an object in a sensing area of the lidar system. The optical signal can for example be evaluated according to a time-of-flight method and the spatial position and distance of the object on which the reflection occurred can be determined. In addition, it may be possible to determine a relative velocity. Reflection or reflected light is understood to mean any light that is reflected back and should also include, in particular, light that is reflected back by scattering or absorption emission.

[0008] Scanning lidar systems emit light beams that move in a scanning direction. Point scanners illuminate areas of the environment point by point. Line scanners illuminate areas of the environment line by line.

[0009] An optical detection sensor comprised in an optical reception device may comprise several receiving elements, so-called pixels. A pixel of the optical detection sensor generates an electrical signal in response to the detection of an optical signal. The pixels may be set up to receive light from different receiving angles.

[0010] In US11693102B2 a lidar system is described. The lidar system includes a lidar transmitter with a first field of view and a lidar receiver with a second field of view. A controller is configured to shift at least one of the first field of view or the second field of view based on a misalignment in order to optimize an overlap of the first field of view and the second field of view.

[0011] Summary

[0012] A method of calibration of an optical sensor system for a vehicle comprises:

[0013] • transmitting a first optical signal, deflecting the first optical signal using a first mirror, detecting the first optical signal using an optical detection sensor,

[0014] • transmitting a second optical signal, deflecting the second optical signal using a second mirror, detecting the second optical signal using the optical detection sensor,

[0015] • determining calibration data using the respective detected first and second optical signal, wherein the calibration data is determined depending on the respective first or second mirror used to deflect the respective first and second optical signal,

[0016] • saving the calibration data.

[0017] The calibration data comprises first and second calibration data. The first calibration data is determined using the detected first optical signal depending on the first mirror used to deflect the first optical signal. The second calibration data is determined using the detected second optical signal depending on the second mirror used to deflect the second optical signal.

[0018] The described method allows to generate calibration data for the optical sensor system that is different for different mirrors. The different calibration data can account for e. g. differences in mounting tolerances, production tolerances, and / or lifetime movement tolerances, which may be different for each mirror.

[0019] By detecting the respective optical signal with the optical detection sensor, the respective calibration data may be determined. The calibration data may in particular relate to a received intensity of the received light. By determining the intensity of the received optical signal depending on the mirror comprised in the associated optical path of the respective optical signal, the alignment of transmitted and received optical signal may be determined for each mirror separately. The calibration data may in particular relate to an alignment of the transmission and reception of the optical signal used by the optical sensor system. If different mirrors are used by the optical sensor system to deflect the optical signal towards a sensing area, this alignment and related calibration data may depend on the mirror that is used to deflect the optical signal. The method allows for taking this into account when calibrating the optical sensor system.

[0020] The intensity of the optical signal relates to the amplitude of the optical wave. It directly relates to the perceived brightness of the light.

[0021] The optical detection sensor is comprised in an optical reception device of the optical sensor system. The optical sensor system may for example be a lidar system. The optical detection sensor comprises a surface with photosensitive elements which transform received light, i. e. photons, to an electrical quantity like current or voltage. For receiving the optical signal, the optical detection sensor may comprise at least one photodiode, avalanche photodiode, photodiode line or at least one charge-coupled semiconductor component, e. g. a charge-coupled device (CCD). The optical detection sensor may also comprise an active pixel sensor, in particular CMOS sensor, or the like.

[0022] The optical sensor system, in particular the lidar system, may further comprise an optical transmission device configured to transmit the optical signal. The optical signal is transmitted into the sensing area of the optical sensor system. The optical signal reflected from the sensing area may at least partly be received by the optical reception device of the optical sensor system. The sent and received optical signal may then be evaluated to obtain information about the location and / or the object where the reflection occurred. A distance to the point of reflection may be obtained as well as further information about the surface and / or the object on which the reflection occurred. The evaluation of the sent and received light may be performed by a control unit of the optical sensor system. The optical sensor system may be employed on vehicles like passenger cars or commercial vehicles to monitor the surroundings of the vehicle.

[0023] The calibration relates to the inter-operation of the optical reception device and the optical transmission device. The described method of calibration may for example correct a misalignment of transmission device and reception device of the optical sensor system, in particular lidar system. The misalignment may be due to aging or environmental external influences and may in particular depend on the mirror that is placed in the optical path of the optical signal.

[0024] In an embodiment of the method, the saved calibration data is associated with the respective mirror. The first calibration data is saved associated with the first mirror and the second calibration data is saved associated with the second mirror. This allows to generate the calibration data separately for the different mirrors and to calibrate the different mirrors separately. By saving the respective calibration data associated with the respective mirror, the calibration data may later be used to correct a misalignment involving the respective mirror and / or to compensate for it.

[0025] The first and second mirror may be comprised in a plurality of mirrors of a mirror device of the optical sensor system. The mirror device may be able to rotate and to deflect the optical signal such that the optical signal performs a scanning movement within the sensing area of the optical sensor system. The rotating movement may comprise a full rotation movement or a swiveling movement.

[0026] The mirror device may in particular be configured to steer the sent light towards the desired portions of the sensing area and / or to receive the light from the desired portions of the sensing area and steer it towards the reception device. The mirror device may also be controlled by the control unit of the optical detection system. The optical detection system may be configured to perform the stepwise scan of the field of view. The scan may be performed in a horizontal and / or vertical direction. By applying the calibration data, any misalignment between mirror device and reception device and transmission device may be corrected depending on each respective mirror in the optical path of the optical signal.

[0027] There may be cases where more than one mirror is comprised in an optical path of the optical signal. For such cases, the described method and detection system may also be used. The calibration data may then be specific to all the mirrors comprised in the optical path.

[0028] The term mirror used in this disclosure refers to any device capable of changing the direction of light. A mirror may thus comprise a reflective and / or an optical phased array and / or a MEMS mirror for example.

[0029] In embodiments of the method, the method further comprises transmitting and detecting a third and fourth optical signal, the third and fourth optical signal being deflected using a third and fourth mirror respectively, wherein the calibration data is determined using the respective detected optical signal, wherein the calibration data is determined depending on the respective mirror used to deflect the respective optical signal.

[0030] In this embodiment, the calibration data comprises first, second, third and fourth calibration data. The third calibration data is determined using the detected third optical signal depending on the third mirror used to deflect the third optical signal. The fourth calibration data is determined using the detected fourth optical signal depending on the fourth mirror used to deflect the fourth optical signal.

[0031] The saved calibration data is associated with the respective mirror. The third calibration data is saved associated with the third mirror and the fourth calibration data is saved associated with the fourth mirror. By saving the respective calibration data associated with the respective mirror, the calibration data may later be used to correct a misalignment involving the respective mirror and / or to compensate for it.

[0032] The third and fourth mirror may be comprised in the plurality of mirrors of the mirror device of the optical sensor system. The mirror device may for example comprise a polygon mirror. The mirror device may comprise more than four mirrors.

[0033] In an embodiment of the method, each of the respective optical signals is repeatedly transmitted and received, wherein a successive angular position of the respective mirror is controlled such that the respective optical signal is deflected such that it is successively detected in a plurality of detection areas of the optical detection sensor. Respective calibration data for the detection areas may then be determined. The respective calibration data may then be saved associated with the detection areas.

[0034] With the described control of the angular position of the mirror, a scanning lidar system may be realized. By successively changing the angular mirror position, the optical signal is successively deflected with a changed angle such the optical signal scans the sensing area of the lidar system. The optical signal is then successively received by the detection areas of the optical detection sensor. In other words, the different detection areas are associated with different areas of the sensing area and different angular positions of the respective mirror.

[0035] The detection areas are portions of a surface of the optical detection sensor, wherein a detection area may be associated with one or more pieces of calibration data. The partition of the surface into the detection areas allows to obtain the calibration data with a certain spatial granularity with respect to the surface of the optical detection sensor.

[0036] By successively detecting the optical signal in the plurality of detection areas of the optical detection sensor, the respective calibration data for the detection areas may be determined. The calibration data may in particular relate to a received intensity of the received light. By determining the intensity of the received optical signal for the plurality of detection areas, the alignment of reception device and transmission device relating to the respective mirror may be judged. By saving the respective calibration data, e. g. received intensity, associated with the detection areas and the respective mirror, the data may later be used to correct a misalignment and / or to compensate for it.

[0037] In an embodiment of the method, the detection areas partially overlap with each other. This allows to obtain even more refined calibration data, which may also- at least partially - be put into relation with other detection areas.

[0038] In an embodiment of the method, the respective calibration data for a plurality of regions of a plurality of the detection areas is determined. The respective calibration data associated with the respective region of the respective detection area is then saved. The saved calibration data may for example relate to the received intensity of light in a respective region of a respective detection area and to the mirror in the optical path of the optical signal. The regions are portions of the detection areas, wherein a region may be associated with one or more pieces of calibration data. The segmentation of the detection areas into regions allows to obtain the calibration data with a certain spatial granularity with respect to the detection area. The respective calibration data associated with the respective region of the respective detection area are then saved to allow for a further use.

[0039] In an embodiment of the method, the regions within a detection area are disjoint. This allows for an efficient implementation of the method, covering the respective detection area with few regions.

[0040] In an embodiment of the method, the detection areas are of substantially the same shape and / or the regions are of substantially the same shape. In these embodiments, at least some of the detection areas each substantially have the same shape and / or at least some regions each substantially have the same shape. The shape of the regions may be chosen such that the detection areas may be covered by disjoint regions that have the same shape.

[0041] The optical detection sensor may comprise pixels for capturing the optical signal. The optical detection sensor may comprise an array with a two-dimensional field of pixels. Pixels of the two-dimensional field can be activated to receive the optical signal, whereby the activation of the pixels may be possible individually and / or in groups. Each detection area and / or each region may comprise a plurality of pixels.

[0042] In an embodiment of the method, at least one superpixel comprising a plurality of pixels may be used for capturing the optical signal. In a superpixel, several pixels are used together for capturing the optical signal and the output electrical signal may be evaluated together.

[0043] The pixels or the superpixels of the detection sensor may form an array with a two-dimensional field of pixels or superpixels. At least one detection area may comprise a plurality of columns of pixels or of superpixels. At least one detection area may comprise a two-dimensional array of pixels or superpixels.

[0044] The pixels of the detection sensor may form an array with a two-dimensional field of pixels or superpixels. At least one detection area may comprise a two-dimensional array of pixels or superpixels. At least one detection area may comprise a plurality of rows of pixels or superpixels, wherein the respective regions comprise a row or a plurality of rows of pixels or a row or a plurality of rows of superpixels. The at least one detection area may for example comprise a two-dimensional array of pixels and the region may comprise a one-dimensional row of pixels. The at least one detection area may comprise a two-dimensional array of superpixels, and the region may comprise a one-dimensional row of superpixels.

[0045] The determination of the calibration data may comprise determining the maximum illumination of the pixels or the superpixels within the respective detection area or within the respective region. The illumination of pixels or superpixels relates to the amount of light that is hitting the surface of the pixel or the superpixel. The illumination relates to the intensity of the optical signal received by the respective pixel or superpixel.

[0046] In this embodiment, the pixel or superpixel with the maximum illumination is determined, i. e. the pixel or superpixel which is hit by the most light. The stored calibration data may depend on this data and / or comprise this data. This allows to compare the maximum illumination (or intensity) of the different pixels or superpixels within the respective detection area and / or the respective region. As a following step, the pixel or superpixel with the maximum illumination within the respective detection area and / or region may be determined.

[0047] The determination of the calibration data may comprise data depending on the gradient of the maximum illumination in the respective region. In this embodiment it is possible to determine where within the respective region how fast and in which direction the illumination is dropping.

[0048] In some embodiments, the calibration data comprises data depending on the maximum illumination in the respective region associated with the respective mirror and / or the gradient of the maximum illumination in the respective region associated with the respective mirror. The first calibration data comprises data depending on the maximum illumination in the respective region associated with the first mirror and / or the gradient of the maximum illumination in the respective region associated with the first mirror. The second calibration data comprises data depending on the maximum illumination in the respective region associated with the second mirror and / or the gradient of the maximum illumination in the respective region associated with the second mirror. Respective relationships apply to the third and fourth calibration data.

[0049] The method of calibration may be executed during production of the optical sensor system and / or during an initialization phase after switching on the optical sensor system and / or during operation of the optical sensor system. During operation the optical sensor system may be mounted on a vehicle and outputting its sensor data to control units of the vehicle, which may use this data to control actors and / or other sensors of the vehicle.

[0050] In embodiments of the method, the method further comprises applying the calibration data. When applying the calibration data, a misalignment of the optical transmission device and / or optical reception device may be corrected. The correction is possible specific for the respective mirror in the optical path of the optical signal. The correction may thus be specific to the actual mirror used and thus be more accurate and reliable.

[0051] The method allows to identify the zone or portion of the optical detection sensor, where the main area of reception of the optical signal lies and to calibrate the optical detection sensor accordingly. The precision may be improved by taking the mirror that is used into account and correcting the misalignment separately for each mirror. This allows to properly identify the alignment position of the optical detection sensor and the pixels or superpixels to be activated for the reception of the optical signal. This also allows to take misalignment of the optical transmission device and optical reception device into account. These misalignments may be due to aging over the lifetime of the optical sensor system, ageing of the optical detection sensor, optical misalignment due to external impacts, e. g. shocks from the road, people, and / or misalignments due to internal components.

[0052] The optical sensor system for a vehicle comprises the optical transmission device, the optical reception device, a first and a second mirror and a control unit.

[0053] The optical transmission device is configured to transmit the first and the second optical signal. The first and the second mirror are arranged to deflect the first and second optical signal respectively. The optical reception device comprises the optical detection sensor, the optical detection sensor being configured to detect the first and the second optical signal. The control unit comprises a processor and is configured to determine calibration data using the respective detected optical signal. The calibration data is determined depending on the respective mirror used to deflect the respective optical signal. The control unit further comprises a data storage configured to store the calibration data.

[0054] The processor of the control unit is configured to perform the steps of the described method of calibration. In embodiments of the optical sensor system, the optical sensor system comprises a mirror device. The mirror device comprises a plurality of mirrors. The plurality of mirrors comprises the first and the second mirror. The mirror device may for example comprise a polygon mirror. The polygon mirror may for example be a rotatable polygon mirror comprising the first, the second, the third and the fourth mirror.

[0055] In an embodiment of the optical sensor system, the optical transmission device is configured to repeatedly transmit each of the respective optical signals. The optical reception device is configured to repeatedly receive each of the respective optical signals. The control unit is configured to control a successive angular position of the respective mirror such that the respective optical signal is deflected such that it is successively detected in a plurality of detection areas of the optical detection sensor.

[0056] A plurality of pixels of the optical detection sensor may be combined into superpixels. In a superpixel the plurality of pixels comprised in the superpixel is used for capturing the optical signal. The pixels of the detection sensor form an array, wherein the detection areas comprise a plurality of rows of superpixels, wherein the respective regions comprise a row or a plurality of rows of superpixels each.

[0057] In embodiments, the optical sensor system may be comprised in a vehicle.

[0058] The described method and optical sensor system may be used in connection with optical sensor systems using modulated light, in particular modulated laser light. It is also suitable for FMCW lidar systems.

[0059] Brief description of the figures

[0060] Embodiments will now be described with reference to the attached drawing figures by way of example only. Like reference numerals are used to refer to like elements throughout. The illustrated structures and devices are not necessarily drawn to scale.

[0061] Fig. 1 schematically illustrates a vehicle with an optical sensor system.

[0062] Fig. 2 schematically illustrates the optical path of a first optical signal.

[0063] Fig. 3 schematically illustrates the optical path of a second optical signal.

[0064] Fig. 4 schematically illustrates an optical reception device comprising an optical detection sensor. Fig. 5 schematically illustrates a superpixel.

[0065] Fig. 6 schematically illustrates a detection area on the optical detection sensor.

[0066] Fig. 7 schematically illustrates an embodiment of the calibration method.

[0067] Fig. 8 schematically illustrates an embodiment of the calibration method.

[0068] Fig. 9 schematically illustrates illumination values for a first mirror.

[0069] Fig. 10 schematically illustrates illumination values for a second mirror.

[0070] Detailed Description

[0071] Fig. 1 schematically illustrates a vehicle 30, for example a passenger car. The optical sensor system 10, e. g. a lidar system, is arranged in a front area of the vehicle 30. The optical sensor system 10 comprises the optical transmission device 12, the optical reception device 14, mirror device 16 and a control unit 18.

[0072] The mirror device 16 may be arranged such that it deflects the light L sent by the optical transmission device 12 into the sensing area 32 and deflects the light L incident from the sensing area 32 to the optical reception device 14. The mirror device 16 may be controlled such that the optical signal L performs a scanning movement 34 over the sensing area 32.

[0073] The optical reception device 14 comprises an optical detection sensor 20. The optical detection sensor 20 is configured to capture the optical signal L and to convert it into an electrical quantity, which may be further processed within the optical reception device 14 and / or the control unit 18.

[0074] In the control unit 18, the transmitted and received optical signals L may be evaluated e. g. using time-of-flight measurements. The evaluation may serve to detect objects O in the sensing area 32. The control unit 18 may also monitor and control the transmitting process in the transmission device 12, the receiving process in the reception device 14 and / or the operation of the mirror device 16. The control unit 18 may comprise a processor which is configured to perform the steps of a method of calibration described throughout this disclosure.

[0075] The optical sensor system 10 may for example be placed or integrated at the front of the vehicle 30. The sensing area 32 is then located in front of the vehicle 30. Thus, in the example shown, an area in front of the vehicle 30 in the direction of travel can be monitored by the optical sensor system 10. There are also optical sensor systems 10 possible for other parts of the vehicle 30, e. g. for surroundview functions such as at the sides and / or rear of the vehicle 30. It is also possible to arrange several sensor systems 10 on the vehicle 30, in particular also in corner areas of the vehicle 30.

[0076] The optical sensor system 10 can be used to detect stationary or moving objects O in the sensing area 32. Objects O may be vehicles, persons, animals, plants, obstacles, roadway unevenness, in particular potholes or stones, roadway boundaries, traffic signs, open spaces, in particular parking spaces, precipitation, or the like.

[0077] The optical sensor system 10 comprising a lidar system may provide an accurate and dense detection point cloud of the sensing area 32 that can show the contour of objects O, making it a very valuable sensor for e. g. autonomous or semi-autonomous driving. Mounted in front of the vehicle 30, such a lidar system may e. g. detect lane markings, underrideable elevated objects like tunnels or overhead bridges and the like.

[0078] Fig. 2 schematically illustrates the optical sensor system 10 comprising a lidar system. The mirror device 16 comprises a polygon mirror with four mirrors 16.1,

[0079] 16.2, 16.3, 16.4. The mirrors 16.1, 16.2, 16.3, 16.4 are rotatably mounted and are able to rotate around a rotating axis which is parallel to the mirrors 16.1, 16.2,

[0080] 16.3, 16.4. The optical transmission device 12 and the optical reception device 14 are arranged on the same side of the mirror device 16.

[0081] The optical detection sensor 20 may comprise an array of pixels Px with photosensitive surface. The array of pixels Px is configured to receive light and convert it into an electrical quantity. For this purpose, the respective pixel Px may, for example, have photosensitive elements, e. g. photosensitive semiconductor elements.

[0082] The first optical signal LI is transmitted by the optical transmission device 12 and deflected by the first mirror 16.1 toward the sensing area 32. The first optical signal LI that has been reflected at a reflection point in the sensing area 32 is then deflected by the first mirror 16.1 towards the optical reception device 14 and captured by the optical detection sensor 20.

[0083] First calibration data is then determined using the detected first optical signal LI.

[0084] The first calibration data is determined for the first mirror 16.1 which has been used to deflect the first optical signal LI. The first calibration data relating to the first mirror 16.1 is then saved.

[0085] The method of determining the first calibration data comprises:

[0086] • Successively detecting the first optical signal LI in a plurality of detection areas 22 of the optical detection sensor 20. The detection areas 22 may partially overlap and may in particular overlap for the greater part of their area to allow for a precise calibration.

[0087] • Determining respective first calibration data for the detection areas 22. The determination of the first calibration data may comprise determining the maximum illumination within the detection area 22.

[0088] • Saving the respective first calibration data associated with the detection areas 22 and the first mirror 16.1. The association of the first calibration data, the first mirror 16.1 and the detection area 22 allows to retrieve respective first calibration data which has been determined specifically for the associated detection area 22 and the associated first mirror 16.1.

[0089] In some embodiments, the angle of the first mirror 16.1 of the mirror device 16 is controlled by the control unit 18 such that the first optical signal LI is successively received by the respective detection areas 22.

[0090] In the embodiment shown in Fig. 2, the optical path of the first optical signal LI comprises two deflections by the first mirror 16.1. There are other embodiments possible, where an optical path of a respective optical signal comprises e. g. one deflection of a respective mirror or a respective deflection by e. g. two different mirrors.

[0091] Fig. 3 schematically illustrates the optical sensor system 10 of Fig.2. The mirror device 16 comprises the polygon mirror with the four mirrors 16.1, 16.2, 16.3, 16.4. The angular position of the mirror device 16 is different from Fig. 2. The second mirror is now in the optical path of a second optical signal L2.

[0092] The second optical signal L2 is transmitted by the optical transmission device 12 and deflected by the second mirror 16.2 toward the sensing area 32. The second optical signal L2 that has been reflected at a reflection point in the sensing area 32 is then deflected by the second mirror 16.2 towards the optical reception device 14 and captured by the optical detection sensor 20. Second calibration data is then determined using the detected second optical signal L2. The second calibration data is determined for the second mirror 16.2 which has been used to deflect the second optical signal L2. The second calibration data relating to the second mirror 16.2 is then saved.

[0093] The method of determining the second calibration data comprises:

[0094] • Successively detecting the second optical signal L2 in a plurality of detection areas 22 of the optical detection sensor 20. The detection areas 22 may partially overlap and may in particular overlap for the greater part of their area to allow for a precise calibration.

[0095] • Determining respective second calibration data for the detection areas 22. The determination of the second calibration data may comprise determining the maximum illumination within the detection area 22.

[0096] • Saving the respective second calibration data associated with the detection areas 22 and the second mirror 16.2. The association of the second calibration data, the second mirror 16.2 and the detection area 22 allows to retrieve respective first calibration data which has been determined specifically for the associated detection area 22 and the associated second mirror 16.2.

[0097] In some embodiments, the angle of the second mirror 16.2 of the mirror device 16 is controlled by the control unit 18 such that the second optical signal L2 is successively received by the respective detection areas 22.

[0098] Similar steps may be performed for the third mirror 16.3 and the fourth mirror 16.4 to obtain third and fourth calibration data respectively.

[0099] In the embodiment shown in Fig. 3, the optical path of the second optical signal comprises two deflections by the second mirror 16.2. There are other embodiments possible, where an optical path of a respective optical signal comprises e. g. one deflection of a respective mirror or a respective deflection by e. g. two different mirrors.

[0100] The first optical signal LI of Fig. 2 and the second optical signal L2 of Fig. 3 are comprised in the optical signal L. The optional third and fourth optical signal may also be comprised in the optical signal L.

[0101] The method used to obtain the calibration data, which comprises the first and second calibration data and optionally the third and fourth calibration data, may further comprise determining the respective calibration data for a plurality of disjoint regions 24 of a plurality of the detection areas 22 and saving the respective calibration data associated with the respective region 24 of the respective detection area 22.

[0102] The method used to obtain the calibration data, which comprises the first and second calibration data and optionally the third and fourth calibration data, will be described in more detail with reference to Figs. 4 to 8.

[0103] Fig. 4 schematically illustrates the optical reception device 14 comprising the optical detection sensor 20.

[0104] The optical detection sensor 20 comprises an array of pixels Px with photosensitive surface. The array of pixels Px is configured to receive light, e. g. the optical signal L, and convert it into an electrical quantity. For this purpose, the respective pixel Px may, for example, have photosensitive elements, e. g. photosensitive semiconductor elements. A respective pixel Px or a respective group of pixels Px of the optical detection sensor 20 can be designed in particular for receiving light from a specific direction. This directional information can also be referred to as angular information, since it indicates the angular direction in space from which the reflection of the received light occurred. From e. g. direction and distance of the place where the reflection took place, a model of the environment of the optical sensor system 10 can be composed.

[0105] A superpixel SP comprises a plurality of pixels Px, e. g. an array of pixels Px. For the superpixel SP, the electrical quantity that is generated by the pixels Px in the superpixel SP in response to the received light is evaluated together. In the embodiment shown in Fig. 4, each superpixel SP comprises two pixels Px. The optical detection sensor 20 comprises an array of superpixels SP.

[0106] Fig. 5 illustrates an embodiment of the superpixel SP which comprises an array of pixels Px. The array of pixels Px of the superpixel SP may have a size of NlxMl with N1 being the number of rows and Ml being the number of columns. The superpixel SP shown in the embodiment of Fig. 5 comprises an array of 3x9 = 27 pixels Px with Nl = 3 and Ml=9. Other sizes of superpixels SP are also possible.

[0107] Fig. 6 schematically illustrates the detection area 22 on the optical detection sensor 20. The optical detection sensor 20 comprises an array of superpixels SP. The detection area 22 comprises an array of superpixels SP. The array of superpixels SP of the detection area 22 is a subset of the whole of the array of superpixels SP that is comprised in the optical detection sensor 20. The arrow in Fig. 6 indicates a direction in which the detection area 22 is successively moved over the optical detection sensor 20. The successive detection areas 22 may be moved to largely overlap. The amount of the movement may be chosen depending on the circumstances and the situation. For example, the detection area 22 may be moved by one column of superpixels SP at a time. Such a movement of the detection area 22 may for example be chosen for a calibration during production of the optical detection system 10 or during an initialization phase after switching on the optical sensor system 10 when starting an operation phase. In other embodiments, the detection area 22 may be moved by three columns of superpixels SP at a time. Such a movement of the detection area 22 may for example be chosen for calibration during operation of the optical sensor system 10.

[0108] The determination of the calibration data comprises determining the illumination of the superpixels SP within the respective detection area 22. For each superpixel SP within the detection area 22, the illumination is determined. The stored calibration data depends on the determined maximum illumination within the respective detection area 22. For each respective detection area 22, the associated calibration data may be stored.

[0109] The calibration data is determined for each mirror within the optical path of the optical signal L separately. The stored calibration data thus additionally depends on the mirror within the optical path of the optical signal L.

[0110] From the maximum illumination values a gradient may also be calculated. The gradient gives an indication of the slope including the direction of the slope for the respective maximum illumination. The calibration data may depend on the maximum illumination and / or the slope.

[0111] In Fig. 7 an embodiment of the method of calibration is further illustrated. Shown is the array of superpixels SP of the optical detection sensor 20. The detection areas 22 comprise 9 columns of superpixels SP each. The detection area 22 is subdivided into regions 24. In the example shown in Fig. 7, the respective regions 24 are the rows of superpixels SP of the respective detection area 22.

[0112] The overall number of columns in Fig. 7 may for example be between 50 and 100, there may in particular be 84 columns. The determination of the calibration data comprises determining the illumination of the superpixels SP within the respective region 24, which are the rows in the depicted embodiment.

[0113] In the example shown in the first line of Fig. 7, the calibration data depends on the maximum illumination of the respective superpixels SP of a respective row of the detection area 22. The respective maximum illumination value for each row is stored associated with the superpixel SP of the respective column Cl, C2, C3, ..., CM of the respective region 24. This concept is visualized with a respective arrow pointing at the columns Cl, C2, C3, ..., CM in the right half of Fig. 7.

[0114] The illumination values for the regions 24 may be covered N2 times. In the embodiment shown in Fig. 7, capturing the illumination values for the regions 24 may be repeated 75 times in order to cover e. g. 84 columns at least once. For different sizes of optical detections sensors 20, detection areas 22 and different sizes of superpixels SP, the number of repetitions N2 may be different from 75 and N2 may assume other values. The respective maximum illumination value for the last detection area 22 is stored associated with the column CM, as shown in Fig. 7.

[0115] Taking all the maximum illumination values stored associated with the columns Cl, C2, C3, ..., CM the slope including gradient and direction of the illumination of the optical detection sensor 20 may be determined as well.

[0116] The method may for example be performed during the production of the optical sensor system 10, e. g. at an end-of-line calibration. The successive detection areas 22 may then be set to cover all rows and to follow each other in each column. This would result in a very precise initial calibration for the mirrors 16.1, 16.2, 16.3, 16.3, for which the calibration is performed.

[0117] The method may for example be started during the initialization of the optical sensor system 10 or when the optical sensor system 10 is switched on. The successive detection areas 22 may be set to be around N3 rows apart. N3 may for example be 3 in certain embodiments, where the successive detection is performed every 3 rows. N3 may assume other numbers than 3, for example 2 or 4 or more. Such dynamic misalignment function during operation phase of the optical sensor system 10 may confirm if the optical sensor system 10 is still well aligned and in case it is not, where the offsets need to be changed. In other embodiments, the successive detection may be performed every N4 columns, with N4 assuming a value of e. g. 2 or 3 or more. During the operation phase, e. g. when the vehicle 30 is being driven, the successive detection may for example be performed every N3 rows to estimate the possibly minor misalignment due the aging or other environmental disturbances during the operation phase. N3 may for example be 3 in certain embodiments, where the successive detection is performed every 3 rows. N3 may assume other numbers than 3, for example 2 or 4 or more. In other embodiments, the successive detection may be performed every N4 columns, with N4 assuming a value of e. g. 2 or 3 or more.

[0118] For each timing, at which the method is performed, e. g. production, initialization after switching on and / or driving situation, the numbers N3, N4 may be chosen to be the same for each mirror 16.1, 16.2, 16.3, 16.4 for which the method is performed. The numbers N3, N4 may also be chosen to be different for certain mirrors 16.1, 16.2, 16.3, 16.4 for which the method is performed.

[0119] Fig. 8 schematically illustrates how the method may be implemented on the control unit 18, the optical reception device 14 and the optical detection sensor 20.

[0120] A user 80 may initiate the method of calibration in 801, e. g. during production or when starting the optical sensor system 10 for operation or in a service mode. The user may also be a control unit 18 controlling the described method.

[0121] In 802, configuration data is received and set by the optical detection sensor 20. In 803 the configuration data is received and set by the control unit 18. The configuration data may include the size of the superpixel SP, e. g. 3x9, the start value for the column and / or the start value for the slope including gradient and direction.

[0122] The loop 800 is performed repeatedly to successively capture the detection areas 22, as described with respect to Fig. 7. The loop 800 is performed for at least two of the mirrors 16.1, 16.2, 16.3, 16.4 separately.

[0123] In 804, the evaluation of the output of the optical detection sensor 20 is set to the size of the detection area 22 by the optical reception device 14. In 805, the first detection area 22 is captured and in 806 the captured illumination is stored associated with the respective superpixel SP of the first column Cl, C2, C3, ..., CM, as described with respect to Fig. 7. In 807, the next column is set.

[0124] In the method, at each identification loop 800, the maximum illumination, which may also be called intensity, within a row or several rows is determined with respect to the previous column of the row, so that all maximum intensity values in each row can be identified. From the row for which the highest value for the maximum value is identified and the row where the lowest value for the maximum value is identified, calibration data, e. g. offset values, for the optical detection sensor 20 and slope including gradient and tilt direction can be determined, by estimation. The row for which the highest value for the maximum value is identified and the row where the lowest value for the maximum value is identified, corresponds in the example shown to the highest value in a respective column and the lowest value in a respective column Cl, C2, C3, ..., CM.

[0125] At the end of the loop 800, all maximum illumination values in each row may be identified. From the individual maximum illumination values as stored associated with the columns Cl, C2, C3, ..., CM as described with respect to Fig. 7, the slope of the illumination can be estimated.

[0126] In 808, the captured data of the loop 800 is output to the user 80 for further processing. For example, the calibration data including the offset values for the optical detection sensor 20, may be obtained by further processing the obtained values. As the loop 800 is performed for at least two different mirrors 16.1, 16.2, 16.3, 16.4 separately, the calibration data is obtained for at least two different mirrors 16.1, 16.2, 16.3, 16.4 separately.

[0127] The described calibration method may address misalignment of the optical transmission device 12 and / or the optical reception device 14. It may also address misalignment within the optical transmission device 12 and / or within the optical reception device 14. This may comprise identifying the optimal laser receiving position of a lidar system. The misalignment may be due to optical detection sensor 20 aging and external shocks from tee Environment and road to internal optical components of the optical sensor system 10. Each of these may be determined specific for the mirror 16.1, 16.2, 16.3, 16.4 used, which may improve the results obtained.

[0128] The described method may be applied to different types of optical sensor systems 10, including lidar systems, including frequency modulated lidar system, e. g. FMCW lidar.

[0129] Fig. 9 schematically illustrates an example of an outcome of the method of calibration as described for example in connection with Fig. 8 and performed with the setup of Fig. 2 for the first mirror 16.1. The outcome illustrated in Fig. 9 obtained for the first mirror 16.1 schematically illustrates received intensity values or illumination values of the first optical signal LI for different regions 24 of the detection areas 22 of the optical detection sensor 20.

[0130] The number given on the horizontal axis denotes the left column of the detection area 22, for which the shown intensity values were determined according to the method as described with respect to Figs. 7 and 8.

[0131] In the vertical direction, eight regions 24 are shown. The vertical direction of the optical detection sensor 20 has thus been divided into eight regions 24. In the example shown each region 24 comprises 45 rows of superpixels SP. The row count is given on the right side of the chart. The optical detection sensor shown in Fig. 9 thus comprises 360 rows of superpixels.

[0132] The boxes denote the column that is selected for each region 24 that gives the best reception quality. For the different regions from bottom to top, this corresponds to the columns 39, 40, 41, 42, 43, 44, 45, 46 for the first mirror 16.1. These regions 24 are different here in Fig. 9 for the first mirror 16.1 than for the second mirror 16.2 (as shown in Fig. 10).

[0133] The determined column position per region 24 may be stored associated with information that it relates to the first mirror 16.1. The first calibration data may for example be derived from the column position, e. g. as offset values, and also be stored associated with the first mirror 16.1. The calibration data may then be applied during operation of the optical sensor system 10. The application of the calibration data may influence which pixels Px or superpixels SP are activated for reception.

[0134] The curve marked with the circles shows the received intensity per detection area 22. A threshold value of 75% of its maximum value is marked in the chart. This threshold may be used for purposes of performing additional test, e. g. end-of- line-test, on the optical detection sensor 20.

[0135] Fig. 10 schematically illustrates another example of an outcome of the method of calibration as described for example in connection with Fig. 8 and performed with the setup of Fig. 3 for the second mirror 16.2. The outcome illustrated in Fig. 10 obtained for the second mirror 16.2 schematically illustrates received intensity values or illumination values of the second optical signal L2 for different regions 24 of the detection areas 22 of the optical detection sensor 20. The number given on the horizontal axis denotes the left column of the detection area 22, for which the shown intensity values were determined according to the method as described with respect to Figs. 7 and 8.

[0136] In the vertical direction eight regions 24 are shown. The vertical direction of the optical detection sensor 20 has thus been divided into eight regions 24. In the example shown each region 24 comprises 45 rows of superpixels SP. The row count is given on the right side of the chart. The optical detection sensor shown in Fig. 10 is the same as shown in Fig. 9. It comprises 360 rows of superpixels.

[0137] The boxes denote the column that is selected for each region 24 that gives the best reception quality. For the different regions from bottom to top, this corresponds to the columns 41, 41, 42, 43, 44, 45, 46, 37 for the second mirror 16.2. These regions 24 are different here in Fig. 10 for the second mirror 16.2 than for the first mirror 16.1.

[0138] The curve marked with the circles shows the received intensity per detection area 22. A threshold value of 75% of its maximum value is marked in the chart. This threshold may be used for purposes of performing additional test, e. g. end-of- line-test, on the optical detection sensor 20.

[0139] The determined column position per region 24 may be stored associated with information that it relates to the second mirror 16.2. The second calibration data may for example be derived from the column position, e. g. as offset values, and also be stored associated with the second mirror 16.2. The calibration data may then be applied during operation of the optical sensor system 10. The application of the calibration data may influence which pixels Px or superpixels SP are activated for reception.

[0140] The optical reception device 14 may be calibrated according to the calibration data obtained using the method of calibration. The calibration may include adjusting the settings of the optical reception device 14 to take the distribution of received light intensities into account and to improve the reception quality even further by taking the specifics of the mirror 16.1, 16.2, 16.3, 16.4 into account. REFERENCE SIGNS

[0141] 10 optical sensor system

[0142] 12 optical transmission device

[0143] 14 optical reception device

[0144] 16 mirror device

[0145] 18 control unit

[0146] 20 optical detection sensor

[0147] 22 detection area

[0148] 24 region

[0149] 30 vehicle

[0150] 32 sensing area

[0151] 34 scan direction

[0152] 80 user

[0153] 800 loop

[0154] 801-808 method steps

[0155] Cl, C2, C3, CM columns

[0156] L, LI, L2 optical signal

[0157] 0 object

[0158] Px pixel

[0159] SP superpixel

Claims

1. CLAIMS1. Method of calibration of an optical sensor system (10) for a vehicle (30), the method comprising transmitting a first optical signal (LI), deflecting the first optical signal (LI) using a first mirror (16.1), detecting the first optical signal (LI) using an optical detection sensor (20), transmitting a second optical signal (L2), deflecting the second optical signal (L2) using a second mirror (16.2), detecting the second optical signal (L2) using the optical detection sensor (20), determining calibration data using the respective detected optical signal (LI, L2), wherein the calibration data is determined depending on the respective mirror (16.1, 16.2) used to deflect the respective optical signal (LI, L2), saving the calibration data.

2. Method according to claim 1, wherein the saved calibration data is associated with the respective mirror (16.1, 16.2).

3. Method according to claim 1 or 2, wherein the optical sensor system (10) comprises an optical transmission device and an optical reception device, wherein the optical detection sensor is comprised in the optical reception device, wherein the calibration relates to the inter-operation of the optical reception device (14) and the optical transmission device (12).

4. Method according to one of the preceding claims, wherein the method comprises transmitting and detecting a third and fourth optical signal, the third and fourth optical signal being deflected using a third and fourth mirror (16.3, 16.4) respectively, wherein the calibration data is determined using the respective detected optical signal, wherein the calibration data is determined depending on the respective mirror (16.3, 16.4) used to deflect the respective optical signal.

5. Method according to one of the preceding claims, wherein each of the respective optical signals (LI, L2) is repeatedly transmitted and received, wherein a successive angular position of the respective mirror (16.1, 16.2, 16.3, 16.4) is controlled such that the respective optical signal (LI, L2) is deflected such that it is successively detected in a plurality of detection areas (22) of the optical detection sensor (20).

6. Method according to claim 5, further comprising determining respective calibration data for the detection areas (22) and saving the respective calibration data associated with the detection areas (22).

7. Method according to claim 5 or 6, further comprising determining the respective calibration data for a plurality of regions (24) of a plurality of the detection areas (22) and saving the respective calibration data associated with the respective region (24) of the respective detection area (22).

8. Method according to claim 7, wherein the detection areas (22) partially overlap with each other and / or wherein the regions (24) within a detection area (22) are disjoint.

9. Method according to one of the preceding claims, wherein the optical detection sensor (20) comprises pixels (Px) for capturing the optical signal (L).

10. Method according to claim 9, wherein at least one superpixel (SP) comprising a plurality of pixels (Px) is used for capturing the optical signal (L).

11. Method according to claim 9 or 10, wherein the pixels (Px) or the superpixels (SP) of the detection sensor form an array, wherein at least one detection area (22) comprises a plurality of columns of pixels (Px) or of superpixels (SP).

12. Method according to one of claims 9 to 11, wherein the pixels (Px) of the detection sensor (20) form an array, wherein at least one detection area (22) comprises a plurality of rows of pixels (Px) or superpixels (SP), wherein the respective regions (24) comprise a row or a plurality of rows of pixels (Px) or a row or a plurality of rows of superpixels (SP).

13. Method according to one of claims 9 to 12, wherein the determination of the calibration data comprises determining the maximum illumination of the pixels (Px) or the superpixels (SP) within the respective detection area (22) or within the respective region (24).

14. Method according to claim 13, wherein the calibration data comprises data depending on the maximum illumination in the respective region (24) and / or the gradient of the maximum illumination in the respective region (24).

15. Method according to one of the preceding claims, wherein the method is executed during production of the optical sensor system (10) and / or during an initialization phase after switching on the optical sensor system (10) and / or during operation of the optical sensor system (10).

16. Method according to one of the preceding claims, further comprising applying the calibration data.

17. Optical sensor system (10) for a vehicle (30), the optical sensor system (10) comprising an optical transmission device (12) configured to transmit a first and a second optical signal (LI, L2), a first and a second mirror (16.1, 16.2) arranged to deflect the first and second optical signal respectively (LI, L2), an optical reception device (14) comprising an optical detection sensor (20), the optical detection sensor (20) being configured to detect the first and the second optical signal (LI, L2), a control unit (18) comprising a processor, the control unit (18) being configured to determine calibration data using the respective detected optical signal (LI, L2), wherein the calibration data is determined depending on the respective mirror (16.1, 16.2) used to deflect the respective optical signal (LI, L2), the control unit (18) further comprising a data storage configured to store the calibration data.

18. Optical sensor system according to claim 17, comprising a mirror device (16), wherein the mirror device (16) comprises a plurality of mirrors (16.1, 16.2, 16.3, 16.4), the plurality of mirrors (16.1, 16.2, 16.3, 16.4) comprising the first and the second mirror (16.1, 16.2).

19. Optical sensor system according to claim 18, wherein the mirror device(16) comprises a polygon mirror.

20. Optical sensor system according to one of claims 17 to 19, wherein the optical transmission device (12) is configured to repeatedly transmit each of the respective optical signals (LI, L2), wherein the optical re- ception device (14) is configured to repeatedly receive each of the respective optical signals (LI, L2), and wherein the control unit (18) is configured to control a successive angular position of the respective mirror (16.1, 16.2) such that the respective optical signal (LI, L2) is deflected such that it is successively detected in a plurality of detection ar- eas (22) of the optical detection sensor (20).

21. Vehicle comprising the optical sensor system (10) according to one of claims 17 to 20.

Images