Sensor device for a motor vehicle

Abstract

A sensor device for a motor vehicle, comprising an event-based image sensor (1) having a plurality of sensor elements, a digital micromirror device (3) having a plurality of mirror elements, each of which can be moved independently of one another from a first position to a second position and from the second position to the first position, and an optical system (4) configured to influence light (5) emanating from an area (6) outside the vehicle and impinging on the optical system (4) such that it at least partially impinges on the plurality of mirror elements of the digital micromirror device (3), wherein the event-based image sensor (1) is arranged such that light (5) impinging on the mirror elements of the digital micromirror device (3) is reflected onto the image sensor (1) in the first position of the mirror elements and is not reflected onto the image sensor (1) in the second position.and wherein the sensor device comprises a control unit (2) which is designed to control at least one of the mirror elements, in particular several or each of the mirror elements, such that it is transferred from the first to the second position and subsequently from the second to the first position in order to generate brightness changes on at least one of the sensor elements, in particular on several or each of the sensor elements.

Description

[0001] The present invention relates to a sensor device for a motor vehicle according to the preamble of claim 1 and to a method for operating such a sensor device according to the preamble of claim 11.

[0002] From US 2024 0015402 A1 an event-based image sensor with a mirror arrangement for tracking moving and / or accelerating objects is known, wherein the control of the mirrors is carried out based on the image data of the event-based image sensor.

[0003] From DE 10 2018 000 485 A1, a vehicle camera with a mirror system in the beam path of the camera is known, whereby the mirror system serves to deflect images of an extended field of view onto the image sensor.

[0004] From DE 199 50 681 A1, a camera system for a vehicle is known with a plurality of micromirrors in the beam path, by means of which individual parts of the image can be darkened by deflecting incident light from the image sensor.

[0005] A sensor device and a method of the aforementioned type are known from DE 10 2020 114 916 A1. The systems and methods described therein serve to reduce flicker artifacts in image light sources.Exemplary methods known from this document may include: receiving images corresponding to a scene captured by a camera; determining a light source in the scene using at least one artificial intelligence (K1)-based algorithm; determining data representing flickering of the light source using an event-based image sensor or an event-based camera, respectively, wherein the flickering has a frequency; determining, based on the data, that a time duration of the overlap between an on state of the light source and an exposure time of the camera falls below a threshold; and delaying the exposure time of the camera by a delay time to increase the time duration of the overlap.

[0006] An event-based image sensor reacts to brightness changes within its detection range. As soon as there are no brightness changes within its detection range, the image sensor's sensor elements can no longer fully capture the scene being observed. Since only slight changes in light occur within a static scene, this represents a weakness of this sensor concept.

[0007] The problem underlying the present invention is to create a sensor device of the type mentioned above in which the detection by the event-based image sensor is improved despite slight changes in the light in the scene being observed. Furthermore, a method for operating such a sensor device is to be specified.

[0008] This is achieved according to the invention by a sensor device of the type mentioned at the outset with the characterizing features of claim 1 and by a method of the type mentioned at the outset with the characterizing features of claim 11. The subclaims relate to preferred embodiments of the invention.

[0009] According to claim 1, it is provided that the sensor device comprises a digital micromirror device with a plurality of mirror elements, which can be moved independently of one another from a first position to a second position and from the second position to the first position, that the sensor device further comprises an optical system which is configured to influence light emanating from an area outside the vehicle and impinging on the optical system in such a way that it impinges at least partially on the plurality of mirror elements of the digital micromirror device, wherein the event-based image sensor is arranged such that light impinging on the mirror elements of the digital micromirror device is reflected onto the image sensor in the first position of the mirror elements and is not reflected onto the image sensor in the second position, and wherein the sensor device comprises a control unit which is configured toto control at least one of the mirror elements, in particular several or each of the mirror elements, such that it is moved from the first to the second position and subsequently from the second to the first position in order to generate brightness changes on at least one of the sensor elements, in particular on several or each of the sensor elements. By specifically controlling the mirror elements, artificial light changes are generated within a static scene, which enable the image sensor to capture scenes with slight light changes.

[0010] The sensor device may comprise an absorber structure arranged such that light incident on the mirror elements of the digital micromirror device is reflected onto the absorber structure in the second position of the mirror elements. Reflecting the portion of the light reflected in the second position into the absorber structure prevents this portion from being detected by the image sensor.

[0011] It is possible for the sensor elements of the event-based image sensor to be arranged in a two-dimensional array, wherein they are arranged next to one another in two mutually perpendicular directions. Furthermore, it is possible for the mirror elements of the digital micromirror device to be arranged in a two-dimensional array, wherein they are arranged next to one another in two mutually perpendicular directions.

[0012] It can be provided that a mirror element or a group of mirror elements of the digital micromirror device is assigned to each of the sensor elements of the event-based image sensor in such a way that light incident on the mirror element or on the group of mirror elements is reflected onto the assigned sensor element in the first position. Alternatively, it can be provided that one of the mirror elements of the digital micromirror device is assigned to each of the sensor elements or a group of sensor elements of the event-based image sensor in such a way that light incident on the mirror element is reflected onto the assigned sensor element or the assigned group of sensor elements in the first position. Both measures can ensure that targeted changes in brightness can be brought about for individual sensor elements or for groups of sensor elements.

[0013] It is possible for the optics to comprise a lens or be designed as a lens, wherein the optics are configured to allow light emanating from an area outside the vehicle to pass through the lens in such a way that it at least partially impinges on the plurality of mirror elements of the digital micromirror device. The light emanating from the scene to be observed is focused by the lens and projected onto the mirror elements of the digital micromirror device.

[0014] Alternatively, the optics may comprise a lens array or be designed as a lens array, wherein the optics are configured so that light emanating from an area outside the vehicle passes through the lens array such that it at least partially impinges on the plurality of mirror elements of the digital micromirror device. The lens array may, for example, be lenses arranged in a matrix. By incorporating an optical lens matrix in microscopic form, different viewing angles of the scene result. This leads to a significant increase in the amount of information about the scene captured. This increases the robustness of the perception of the surroundings. Furthermore, the different viewing angles can be used, for example, for depth estimation.Furthermore, it is possible to compensate for contamination in front of the microlens matrix to a certain extent using the different viewing angles.

[0015] It can be provided that the event-based image sensor is configured so that each of the sensor elements can detect a change in the brightness of the light incident on the sensor element independently of the other sensor elements, wherein data corresponding to the detected change in brightness can be read from the image sensor. It can further be provided that the control unit or a readout unit different from the control unit is configured to read data from the event-based image sensor and to make the readout data available for creating an image, in particular for creating an image of the area outside the vehicle from which the light incident on the optics emanates.

[0016] According to claim 11, it is provided that at least one of the mirror elements, in particular several or each of the mirror elements, is controlled such that it is moved from the first to the second position and then from the second to the first position in order to generate changes in brightness on at least one of the sensor elements, in particular on several or each of the sensor elements. Data can be read from the event-based image sensor and the mirror elements can be controlled depending on the read data. In particular, sensor elements onto which light strikes from areas of the scene in which no changes in brightness occur can therefore be specifically subjected to changes in brightness by appropriately controlling the associated mirror elements in order to enable detection of these areas of the scene.

[0017] It can be provided that a mirror element associated with a sensor element that has not detected a change in brightness within a predetermined time interval is moved from the first to the second position and then from the second to the first position in order to generate a change in brightness on the sensor element. The time interval can be suitably selected. It is possible to take the speed of the motor vehicle into account when controlling the sensor elements to generate changes in brightness.

[0018] It can be provided that, when very high-intensity light is detected, at least one of the mirror elements, in particular several or each of the mirror elements, is controlled such that it is moved several times in succession from the first to the second position and then from the second to the first position in order to reduce the intensity of the detected light. This measure can increase the dynamic range of the event-based image sensor. Furthermore, a high-frequency change between the first and second positions allows dimming of very high light intensities. This can also be used as a measure to expand the brightness dynamics of the event-based image sensor.

[0019] The invention is explained in more detail below with reference to the accompanying drawings. These show: Fig. 1 a schematic view of an embodiment of a sensor device according to the invention.

[0020] The Fig. The sensor device depicted in Figure 1 comprises an event-based image sensor 1 with a plurality of sensor elements, which are arranged adjacent to one another in a two-dimensional array in two mutually perpendicular directions. Each of the sensor elements can detect a change in the brightness of the light incident on the sensor element independently of the other sensor elements. Data corresponding to the detected change in brightness can be read from the image sensor 1.

[0021] The sensor device further comprises a control unit 2, which can read the data from the image sensor 1 and make the read data available for creating an image. It is also possible for a readout unit (not shown) different from the control unit 2 to be provided for reading the data from the event-based image sensor.

[0022] The sensor device further comprises a digital micromirror device 3, only schematically indicated, which is designed in particular as a commercially available DMD chip. In a manner known per se, the digital micromirror device 3 has a plurality of non-imaged mirror elements arranged side by side in a two-dimensional array in two mutually perpendicular directions. Each of the mirror elements can be controlled by the control unit 2 independently of the other mirror elements such that it is moved from a first position to a second position and from the second position to the first position.

[0023] The sensor device further comprises an optical system 4 that influences light 5 emanating from an area 6 outside the vehicle and impinging on the optical system 4, such that it at least partially impinges on the mirror elements of the digital micromirror device 3. In the illustrated embodiment, the optical system 4 is a lens, or rather, the optical system 4 comprises a lens. The light emanating from the area 6 to be observed or the scene to be observed is focused by the lens and projected onto the mirror elements of the digital micromirror device 3.

[0024] Alternatively, the optics may comprise a lens array or be designed as a lens array. The lenses of the lens array can be arranged next to one another in a two-dimensional array in two mutually perpendicular directions. Light 5 emanating from the area 6 outside the vehicle can also pass through the lens array in such a way that it at least partially impinges on the plurality of mirror elements of the digital micromirror device 3. The introduction of an optical lens matrix in microscopic form results in different viewing angles of the scene.

[0025] The image sensor is arranged such that light 5 incident on the mirror elements of the digital micromirror device 3 is reflected onto the image sensor 1 in the first position of the mirror elements and is not reflected onto the image sensor in the second position. In particular, the sensor device comprises a non-illustrated absorber structure arranged such that light 5 incident on the mirror elements of the digital micromirror device 3 is reflected onto the absorber structure in the second position of the mirror elements.

[0026] Between the digital micromirror device 3 and the image sensor 1, a Fig. 1, a further optic 7, schematically indicated, may be arranged. Alternatively, the further optic 7 may also be omitted.

[0027] It can be provided that exactly one mirror element of the digital micromirror device 3 is assigned to each of the sensor elements of the image sensor 1 such that light 5 incident on the mirror element is reflected onto the assigned sensor element in the first position. Alternatively, it can be provided that a group of mirror elements of the digital micromirror device 3 is assigned to each of the sensor elements of the image sensor 1, or that one of the mirror elements of the digital micromirror device 3 is assigned to each group of sensor elements of the image sensor 1.

[0028] If no brightness changes occur in a specific area 6 to be observed outside the vehicle, for example because the scenery is static, this results in individual sensor elements of the image sensor 1 assigned to this area not generating a signal because they cannot detect any change in brightness. This can be determined by the control unit 2 reading the image sensor 1. In order to still generate image data for this area 6, the control unit 2 controls those mirror elements assigned to the sensor elements without an output signal so that they are moved from the first to the second position and then from the second to the first position in order to generate brightness changes on these sensor elements.

[0029] The speed of the vehicle can be taken into account in this control.

[0030] If light 5 with very high intensity from a specific area 6 to be observed outside the vehicle strikes the digital micromirror device 3 through the optics 4 and from there strikes the mirror elements of the image sensor 1, this can also be determined by the control unit 2 reading the image sensor 1. In order to reduce the intensity of the light 5 striking the mirror elements, the control unit 2 controls those mirror elements assigned to the sensor elements exposed to a high light intensity such that they are moved several times in succession from the first to the second position and then from the second to the first position. By means of this, in particular high-frequency, change between the first and second positions, light intensities can be dimmed. List of reference symbols 1 event-based image sensor 2 control unit 3 digital micromirror device 4 Optics for light from an area outside the vehicle 5 Light from an area outside the vehicle 6 Area outside the vehicle 7 Optics between the digital micromirror device and the image sensor

Claims

[1] Sensor device for a motor vehicle, comprising an event-based image sensor (1) with a plurality of sensor elements, characterized bythat the sensor device comprises a digital micromirror device (3) with a plurality of mirror elements, which can be moved independently of one another from a first position to a second position and from the second position to the first position, that the sensor device further comprises an optical system (4) which is designed to influence light (5) emanating from an area (6) outside the vehicle and impinging on the optical system (4) in such a way that it at least partially impinges on the plurality of mirror elements of the digital micromirror device (3), wherein the event-based image sensor (1) is arranged such that light (5) impinging on the mirror elements of the digital micromirror device (3) is reflected onto the image sensor (1) in the first position of the mirror elements and is not reflected onto the image sensor (1) in the second position, and wherein the sensor device comprises a control unit (2),which is designed to control at least one of the mirror elements, in particular several or each of the mirror elements, such that it is transferred from the first to the second position and then from the second to the first position in order to generate brightness changes on at least one of the sensor elements, in particular on several or each of the sensor elements. [2] Sensor device according to claim 1, characterized by that the sensor device comprises an absorber structure which is arranged such that light (5) incident on the mirror elements of the digital micromirror device (3) is reflected onto the absorber structure in the second position of the mirror elements. [3] Sensor device according to one of claims 1 or 2, characterized bythat the sensor elements of the event-based image sensor (1) are arranged in a two-dimensional array, wherein they are arranged next to one another in particular in two mutually perpendicular directions. [4] Sensor device according to one of claims 1 to 3, characterized by that the mirror elements of the digital micromirror device (3) are arranged in a two-dimensional array, wherein they are arranged next to one another in particular in two mutually perpendicular directions. [5] Sensor device according to one of claims 1 to 4, characterized by that a mirror element or a group of mirror elements of the digital micromirror device (3) is assigned to each of the sensor elements of the event-based image sensor (1) in such a way that light (5) incident on the mirror element or on the group of mirror elements is reflected in the first position onto the assigned sensor element. [6] Sensor device according to one of claims 1 to 4, characterized by that one of the mirror elements of the digital micromirror device (3) is assigned to each of the sensor elements or a group of sensor elements of the event-based image sensor (1) in such a way that light (5) incident on the mirror element is reflected in the first position onto the assigned sensor element or onto the assigned group of sensor elements. [7] Sensor device according to one of claims 1 to 6, characterized by in that the optics (4) comprise a lens or are designed as a lens, wherein the optics (4) are configured so that light (5) emanating from an area (6) outside the vehicle passes through the lens in such a way that it at least partially impinges on the plurality of mirror elements of the digital micromirror device (3). [8] Sensor device according to one of claims 1 to 7, characterized bythat the optics (4) comprises a lens array or is designed as a lens array, wherein the optics (4) is designed so that light (5) emanating from an area (6) outside the vehicle passes through the lens array in such a way that it at least partially impinges on the plurality of mirror elements of the digital micromirror device (3). [9] Sensor device according to one of claims 1 to 8, characterized by that the event-based image sensor (1) is designed so that each of the sensor elements can detect a change in the brightness of the light (5) incident on the sensor element independently of the other sensor elements, wherein data corresponding to the detected change in brightness can be read out from the image sensor (1). [10] Sensor device according to one of claims 1 to 9, characterized bythat the control unit (2) or a readout unit different from the control unit (2) is configured to read out data from the event-based image sensor (1) and to make the readout data available for creating an image, in particular for creating an image of the area (6) arranged outside the vehicle from which the light (5) incident on the optics (4) emanates. [11] Method for operating a sensor device according to one of claims 1 to 10, characterized by that at least one of the mirror elements, in particular several or each of the mirror elements, is controlled such that it is transferred from the first to the second position and then from the second to the first position in order to generate brightness changes on at least one of the sensor elements, in particular on several or each of the sensor elements. [12] Method according to claim 11, characterized bythat data is read from the event-based image sensor (1) and the mirror elements are controlled depending on the read data. [13] Method according to claim 12, characterized by that a mirror element which is associated with a sensor element which has not detected a change in brightness within a predetermined time interval is moved from the first to the second position and then from the second to the first position in order to generate a change in brightness on the sensor element. [14] Method according to one of claims 11 to 13, characterized by that the speed of the motor vehicle is taken into account when controlling the sensor elements to generate changes in brightness. [15] Method according to one of claims 11 to 14, characterized bythat when light (5) of very high intensity is detected, at least one of the mirror elements, in particular several or each of the mirror elements, is controlled in such a way that it is transferred several times in succession from the first to the second position and then from the second to the first position in order to bring about a reduction in the intensity of the detected light (5).

Images