CAMERA SYSTEM FOR A VEHICLE

Abstract

The invention relates to a camera system for a vehicle, comprising: (i) a camera with an image sensor for capturing an image, which has image signals from a spectral infrared range, of a partial area of ​​the interior of the vehicle; (ii) a lighting device configured to illuminate the partial area of ​​the interior with radiation from the spectral infrared range; (iii) a control device configured to control the camera and the lighting device such that the image sensor captures a plurality of temporally successive images of the partial area of ​​the interior, and the lighting device illuminates the partial area of ​​the interior for at least a period during which the plurality of images are captured; (iv) an evaluation device configured to generate an overall image using at least two images from the plurality of the captured images.

Description

[0001] The present invention relates to a camera system for a vehicle, a method, in particular a computer-implemented method, a vehicle, and a computer program.

[0002] Modern motor vehicles are equipped with cameras that can capture the interior or a portion of the interior of the vehicle. This allows the interior to be monitored. In particular, vehicle occupants, their position, and their movements can be recorded. Particular attention is paid to the position and / or movement, as well as the driver's state, particularly their level of attention. This information can be used, for example, to detect the development of dangerous situations and to mitigate the resulting risk through warnings or countermeasures.

[0003] To enable reliable detection of vehicle occupants in the interior, cameras capable of capturing or recording images in the infrared spectral range, so-called infrared cameras, can be used. These allow for accurate differentiation between living beings and vehicle components. To ensure reliable detection in all visibility conditions, especially during the day and at night, infrared lighting can also be used to at least partially illuminate the interior.

[0004] Sunlight entering the vehicle, which also contains infrared or near-infrared light, can influence the captured image, particularly leading to overexposure in certain areas of the image. If sunlight shines through the windshield, the area of ​​the front seats may be overexposed in the captured image, while the area of ​​the rear seats may be underexposed.

[0005] The present invention is based on the object of enabling an improved image from the interior of the motor vehicle.

[0006] This object is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.

[0007] A first aspect of the solution relates to a camera system for a vehicle, comprising: (i) a camera with an image sensor for capturing an image, which has image signals from a spectral infrared range, of a partial region of an interior of the vehicle; (ii) an illumination device configured to illuminate the partial region of the interior with radiation from the spectral infrared range; (iii) a control device configured to control the camera and the illumination device such that a plurality of temporally successive images of the partial region of the interior are captured by the image sensor, and that the partial region of the interior is illuminated by the illumination device at least for a period of time in which the plurality of images are captured; (iv) an evaluation device configured to generate an overall image using at least two images of the plurality of captured images.

[0008] The terms "comprises," "includes," "includes," "has," "has," "with," or any other variation thereof, as used herein, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or has a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or that are inherent in such a method or apparatus.

[0009] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive "or" rather than an exclusive "or." For example, a condition A or B is satisfied by one of the following conditions: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), and both A and B are true (or present).

[0010] The terms "a" or "an" as used herein are defined as "one or more." The terms "another" and "another," and any other variations thereof, are defined as "at least one other."

[0011] The term “plurality” as used here shall mean “two or more”.

[0012] The term “configured” or “set up” to fulfil a specific function (and respective variations thereof) as used here is to be understood to mean that the corresponding device is already in a design or setting in which it can carry out the function or is at least adjustable - i.e. configurable - so that it can carry out the function after being set accordingly. The configuration can be carried out, for example, by appropriately setting parameters of a process sequence or of switches or similar for activating or deactivating functionalities or settings. In particular, the device can have a plurality of predetermined configurations or operating modes, so that the configuration can be carried out by selecting one of these configurations or operating modes.

[0013] The term "image sensor" as used here refers to an image sensor, in particular a camera sensor, capable of sensing a scene. In particular, the image sensor may comprise an electro-optical sensor configured to detect or measure electromagnetic signals, also referred to below as image signals, and convert them into electrical signals. These sensors may, in particular, be a charge-coupled device (CCD), a radar sensor, a lidar sensor, or another sensor.In addition, such a sensor can also be designed to transmit electromagnetic signals to an object and detect the reflected electromagnetic signals from that object, allowing information about the object to be obtained from the transmitted and reflected electromagnetic signals in a subsequent analysis. Such sensors are used in so-called TOF (time-of-flight) cameras.

[0014] The term "spectral range" as used here refers in particular to a range of the electromagnetic spectrum of electromagnetic radiation. A "spectral visible range" is understood in particular to be a wavelength range from approximately 400 nm to 760 nm, which is visible to the human eye. A spectral infrared range is understood in particular to be a wavelength range from approximately 780 nm to 50 µm. A spectral near infrared range, also known as the NIR range, is understood in particular to be a wavelength range from approximately 780 nm to 3 µm.

[0015] The camera system according to the first aspect can achieve a higher image quality of the overall image than the respective image quality of the individual images captured. In particular, by superimposing at least two of the captured images, an increased contrast and / or an improved dynamic range in the overall image can be achieved, and overexposure or underexposure of different areas of the captured image can be compensated. This can be advantageous if the lighting conditions in the interior are not the same everywhere. Likewise, more than two of the majority of the images or even all of the majority of the images can be used to generate the overall image. Using two images can save computing power compared to more than two images. As the number of images used to generate the overall image increases, the contrast can be further improved.

[0016] Preferred embodiments of the camera system are described below, which can be combined with each other as well as with the other aspects described, unless this is expressly excluded or is technically impossible.

[0017] In some embodiments, the control device is further configured such that the image sensor captures the plurality of images with mutually different apertures of a camera lens and / or exposure times. Such a plurality of images, in which the aperture and / or exposure time of the images is changed from image to image, is also known as an exposure series. This allows an overall image with a high brightness range to be generated. In particular, image processing such as that used for generating HDR (high dynamic range) images can be used for generation.

[0018] In some embodiments, the illumination device comprises a light-emitting diode (LED). LEDs are small and require little energy. Furthermore, LEDs enable a relatively broad spectrum in the infrared range.

[0019] In some embodiments, the control device and / or the evaluation device are integrated into the camera. This makes it possible to provide a compact camera system that can be operated partially or completely independently. Partially independent operation would occur if the control device or the evaluation device were integrated into the camera. Completely independent operation would occur if the control device and the evaluation device were integrated into the camera.

[0020] In some embodiments, the lighting device is arranged such that one or more rear seats of the vehicle can be illuminated by the lighting device. This allows, in particular, a vehicle occupant in a rear seat to be illuminated. This can be advantageous because, due to the light entering through the windshield, in many situations the front seats are better or brighter illuminated than the rear seats. By illuminating the rear seats, more uniform illumination can be achieved in the interior, thus resulting in an overall more homogeneously illuminated image, which can prevent overexposure or underexposure of any part of the image.

[0021] In some embodiments, the image sensor is configured to capture image signals exclusively from the spectral, particularly near-infrared, range. This eliminates the need for a color filter, particularly a red-green-blue (RGB) filter. Such color filters, often designed as a so-called Bayer pattern, are arranged in front of the image sensor in the direction of light incidence. Eliminating this filter enables an improved signal for image signals from the spectral infrared range, since these can be influenced by the color filter.

[0022] In some embodiments, the image sensor is configured to capture image signals from the near-infrared spectral range and a visible spectral range. This enables better differentiation of vehicle occupants and vehicle parts.

[0023] A second aspect of the solution relates to a method, in particular a computer-implemented method, for generating an image of a partial area of ​​an interior of a vehicle, comprising the following steps: (i) capturing a plurality of temporally successive images, each of which has image signals from a spectral infrared range; (ii) illuminating the partial area of ​​the interior with radiation from the spectral, in particular near-infrared, range for at least a period of time in which the plurality of images are captured; (iii) generating an overall image using at least two images of the plurality of captured images.

[0024] A third aspect of the solution relates to a vehicle, in particular a motor vehicle, comprising a camera system according to the first aspect.

[0025] A fourth aspect of the solution relates to a computer program with instructions that cause the camera system according to the first aspect to execute the method according to the second aspect.

[0026] The computer program can in particular be stored on a non-volatile data carrier. This is preferably a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program as such is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can be present as a file on a data processing unit, in particular on a server, and can be downloaded via a data connection, for example the Internet or a dedicated data connection, such as a proprietary or local network. In addition, the computer program can have a plurality of interacting individual program modules. The modules can in particular be configured or at least be usable in such a way that they can be used in the sense of distributed computing (DC).“Distributed computing” is carried out on different devices (computers or processor units) that are geographically separated from each other and connected via a data network.

[0027] The features and advantages explained with regard to the first aspect of the solution also apply accordingly to the other aspects described.

[0028] Further advantages, features and possible applications emerge from the following description of preferred embodiments in conjunction with the figures.

[0029] This shows Fig. 1 schematically shows a camera system according to an embodiment; and Fig. 2 schematically shows a motor vehicle according to an embodiment.

[0030] Throughout the figures, the same reference numerals are used for the same or corresponding elements.

[0031] In Fig. 1 schematically shows a camera system 100 according to one embodiment. The camera system 100 comprises a camera 110, in particular a digital camera, a control device 130, and an evaluation device 140. The evaluation device 140 can also be integrated into the control device 130. The camera system 100 further comprises a light-emitting diode (LED) 120 configured to emit signals from the near-infrared (NIR) spectral range. Accordingly, this light-emitting diode is referred to below as NIR-LED 120.

[0032] The camera 110 has an image sensor 150, which can capture image signals for an image from the near-infrared spectral range. It is also conceivable that the image sensor 150 is configured to capture image signals from the near-infrared spectral range and from a visible spectral range. This would have the advantage of allowing vehicle occupants to be more easily distinguished from vehicle parts.

[0033] The control device 130 is configured to control the camera 110 and the NIR LED 120 such that a plurality of temporally successive images of a partial area of ​​the interior 230 are captured by the camera 110, and such that the partial area of ​​the interior 230 is illuminated by the NIR LED 120 at least for the period in which the plurality of images are captured.

[0034] By capturing the partial area of ​​the interior, the captured image can show both a vehicle occupant on a driver's seat 220 and one or more vehicle occupants on one of the rear seats 220 of the motor vehicle 200.

[0035] The control device 130 is further configured so that the image sensor 150 can capture the plurality of images with mutually different apertures of a lens of the camera 110 and / or exposure times. Such a plurality of images is also known as an exposure series.

[0036] The evaluation device 140 is configured to generate an overall image from the majority of captured images. In this case, an overall image can be generated from just two captured images by superimposing them, which overall image has a higher brightness range or contrast compared to the individual images. Furthermore, image processing, as is known for generating HDR (high dynamic range) images, can be used for generation. These so-called HDR images are typically generated from exposure series, but as is known for images from the visible spectral range, and not, as in the present case, for images from the near-infrared spectral range. This near-infrared spectral range is particularly relevant for interior monitoring and the vehicle occupants present therein.

[0037] In Fig. Figure 2 schematically shows a motor vehicle 200 according to one embodiment. The camera system 100 is arranged in the interior 230 of the motor vehicle 200.

[0038] A camera 120 of the camera system 100 is arranged in the area of ​​the interior mirror (not shown here) and directed toward the interior 230 of the motor vehicle 200. Likewise, the camera 110 can be arranged in a front area of ​​the vehicle roof in the direction of travel, or at another position in the interior.

[0039] The majority of images captured by the camera 110 may each include a driver in a driver seat 220 and a vehicle occupant in a rear seat 220. The camera system 100 may generate an overall image from the plurality, as shown in Fig. 1 described.

[0040] The described generation of the overall image can be advantageous particularly in situations when sunlight, which also contains infrared or near-infrared light, shines through a windshield of the motor vehicle 200. Due to the sunlight, the front area in the direction of travel, including the driver's seat, can be significantly brighter than the area with the rear seats. This can influence a single image to be captured, so that the area of ​​the image depicting the front area is overexposed. This overexposure could be avoided, for example, by taking two separate images, with the exposure of the camera 110 being optimized for the front area in one of the images and the exposure for the rear area in the other image. However, if continuous monitoring is to take place, two images would have to be taken continuously and evaluated.This would entail a considerable computational effort.

[0041] By capturing a plurality of images of the desired section of the interior 230, wherein the images are subsequently used to generate an overall image, in particular through processing that is used for HDR images, the need to capture the described second images can be avoided. This can reduce the required computing power and power consumption of the camera system 100. Due to the required reduced computing power, the freed-up computing power can be used, for example, for a faster frame rate for capturing the majority of images.

[0042] While at least one exemplary embodiment has been described above, it should be appreciated that a wide variety of variations exist. It should also be understood that the described exemplary embodiments are merely non-limiting examples and are not intended to limit the scope, applicability, or configuration of the devices and methods described herein. Rather, the foregoing description will provide one skilled in the art with guidance for implementing at least one exemplary embodiment, it being understood that various changes in the operation and arrangement of the elements described in an exemplary embodiment may be made without departing from the subject matter as defined in the appended claims, as well as their legal equivalents. LIST OF REFERENCE SYMBOLS 100 camera system 110 Camera 120 NIR LEDs 130 Control device 140 Evaluation device 150 image sensor 200 motor vehicles 210 Driver's seat 220 rear seat 230 interior

Claims

[1] Camera system (100) for a vehicle (200), comprising: A camera (110) with an image sensor (150) for capturing an image, which has image signals from a spectral infrared range, of a partial area of an interior (230) of the vehicle (200); a lighting device (120) configured to illuminate the partial area of the interior (230) with radiation from the spectral infrared range; a control device (130) configured to control the camera (110) and the lighting device (120) such that a plurality of temporally successive images of the partial area of the interior (230) are captured by the image sensor (150), and that the partial area of the interior (230) is illuminated by the lighting device (120) at least for a period of time in which the plurality of images are captured; an evaluation device (140) which is configured to generate an overall image using at least two images of the plurality of captured images. [2] Camera system (100) according to claim 1, wherein the control device (130) is further configured such that the plurality of images are captured by the image sensor (150) with mutually different apertures of a lens of the camera (110) and / or exposure times. [3] Camera system (100) according to one of the preceding claims, wherein the illumination device comprises a light-emitting diode (120). [4] Camera system (100) according to one of the preceding claims, wherein the control device (130) and / or the evaluation device (140) are integrated in the camera (110). [5] Camera system (100) according to one of the preceding claims, wherein the lighting device is arranged such that one or more rear seats of the vehicle can be illuminated by the lighting device (120). [6] Camera system (100) according to one of the preceding claims, wherein the image sensor (150) is configured to be able to detect image signals exclusively from the spectral infrared range. [7] Camera system (100) according to one of claims 1 to 5, wherein the image sensor (150) is configured to be able to capture image signals from the spectral near infrared range and a spectral visible range. [8] Method for generating an image of a partial area of an interior (230) of a vehicle (200) comprising the following steps: capturing a plurality of temporally successive images, each comprising image signals from a spectral infrared range; Illuminating the partial area of the interior (230) with radiation from the spectral infrared range for at least a period of time in which the plurality of images are captured; Generating an overall image using at least two images of the plurality of acquired images. [9] Vehicle (200) comprising a camera system (100) according to one of claims 1 to 7. [10] Computer program with instructions that cause the camera system (100) according to one of claims 1 to 7 to carry out the method according to claim 8.

Images