Method and output system for outputting measurement data from at least one thermal imaging camera in a vehicle

Abstract

A method (20) for outputting measurement data from at least one thermal imaging camera (11) is disclosed, wherein measurement data are acquired by the at least one thermal imaging camera (11), wherein the acquired measurement data are processed such that cold areas (16) of the measurement data are at least darkened, wherein the processed measurement data are generated by at least one image sensor (12) and output onto at least one projection surface (13), wherein the processed measurement data are divided along a vertical direction H and output onto lateral areas (18) of the projection surface (13). Furthermore, an output system (10) and a vehicle (100) are disclosed.

Description

[0001] The invention relates to a method for outputting measurement data from at least one thermal imaging camera, wherein measurement data are acquired by the at least one thermal imaging camera, an output system and a vehicle with the output system.

[0002] Head-up displays (HUDs) are already familiar in the automotive sector and are used to visualize information within the driver's field of vision. However, HUDs currently in use are limited in their ability to display information such as turn-by-turn directions from navigation systems, speed limits, and warnings.

[0003] German patent DE 10 2023 100 088 A1 describes a method for detecting objects in front of a vehicle in poor visibility or at night. Following object detection, a customized night vision image, combined with object movement information, is displayed on a viewfinder / windshield.

[0004] US 2010 / 0265345A1 discloses a method for operating a vehicle's night vision system. Depending on infrared detection of an object located in front of the vehicle, an object representation is displayed in the driver's field of vision.

[0005] DE 102 61 290 A1 discloses an image correction device for a motor vehicle and a night driving field of vision support, wherein an image captured with a near-infrared camera is displayed on a head-up display device.

[0006] US 2017 / 0237946A1 discloses a cabin monitoring system for observing a rear seat area of ​​a vehicle interior, wherein a lighting device on a vehicle ceiling is equipped with an infrared camera.

[0007] The object of the present invention is to provide a method by which a technically simple visualization of objects detected by night vision is made possible.

[0008] According to the invention, this problem is solved by a method with the features of claim 1, an output system with the features of claim 7, and a vehicle with the features of claim 8. Advantageous embodiments and further developments are described in the dependent claims.

[0009] The method according to the invention serves to output measurement data from at least one thermal imaging camera. In one step of the method, measurement data is acquired by the at least one thermal imaging camera. Subsequently, the acquired measurement data is processed in such a way that cold areas of the measurement data are at least darkened. This can be achieved, for example, by adjusting the contrast of the measurement data, which is designed as night vision images.

[0010] In a further step of the process, the processed measurement data is optically generated by at least one image sensor and displayed on at least one projection surface. The image sensor can, for example, be a TFT display and generate images that are projected onto the projection surface. The image sensor can project the processed measurement data, in the form of images or image sequences, directly or via an optical arrangement onto the projection surface. Such an optical arrangement can include lenses, mirrors, filters, and the like, and optimize the image generated by the image sensor for projection onto the projection surface.

[0011] The processed measurement data is divided along a vertical direction and displayed on lateral areas of the projection surface. This allows a central area of ​​the projection surface to be reserved for additional functions, such as displaying speed limits, warnings, information, navigation instructions, and the like.

[0012] According to a further aspect of the invention, an output system is provided which is configured to execute the method according to the invention. The output system is specifically designed as an optical output system and comprises at least one thermal imaging camera, at least one image sensor, and at least one projection surface. Furthermore, at least one control unit is provided. At least the thermal imaging camera and the image sensor are connected to at least one of the at least one control unit via a data conductor.

[0013] According to another aspect of the invention, a vehicle is provided which has the output system according to the invention. Thus, the vehicle can also be equipped to carry out the method according to the invention.

[0014] This method enables a particularly effective and efficient output of measurement data from thermal imaging cameras. The output can be in the form of a head-up display (HUD) and thus positioned within the driver's field of vision. Splitting the processed measurement data, for example, into split images, allows the central area of ​​the HUD's projection surface to be used for primary functions, as described above. The output of processed measurement data from the at least one thermal imaging camera can be defined as a secondary function, providing assistance in poor visibility conditions or at night. The method according to the invention can therefore be implemented within the framework of a driver assistance system.

[0015] The measurement data from at least one thermal imaging camera exhibits a brightness spectrum that ranges between warm and cold areas or objects. The darkening of the cold areas allows the measurement data to be processed in such a way that these areas are not visible when displayed on the projection surface, as dark areas in the HUD become transparent. Thus, warm areas or objects can be easily filtered out and visualized on the projection surface. A driver can therefore only see the image generated on the projection surface and not the night vision images from the at least one thermal imaging camera.

[0016] The output system enables the realization of an augmented reality head-up display (AR-HUD), which can support the driver at night with a section of the image to the left and right in the projection area.

[0017] In one embodiment, the cold areas of the acquired measurement data are darkened until they appear black. This allows cold areas in the images or measurement data from the thermal imaging camera to be blacked out using an algorithm, preventing them from being displayed in the HUD. The processed measurement data, in the form of night vision images, can then be overlaid on the regular HUD image on the projection surface. Physically, a completely black image cannot be displayed in the HUD, as only bright image content is visible there. Black image content, such as the cold areas in the thermal imaging camera's measurement data, remains transparent.

[0018] According to another embodiment, warm areas of the acquired measurement data are brightened. If the thermal imaging camera detects a living object, this object is recognizable as warm areas in the measurement data. Such warm areas in the measurement data can advantageously be brightened to be particularly clearly visible on the projection surface. Thus, night vision sections, for example in the HUD, can be displayed on the projection surface or the HUD with minimal technical post-processing and independently of other visualizations.

[0019] According to a further embodiment, object detection is performed based on the received measurement data from at least one thermal imaging camera and / or based on received measurement data from at least one sensor. Objects identified through object detection are then visually highlighted, abstracted, and / or framed by the image transmitter. This improves the visibility of detected objects on the projection surface and enables proactive and safer driving at night or in poor visibility conditions. For example, wildlife crossings, pedestrians, or other road users can thus be more easily detected at night.

[0020] In a further embodiment, the distance and / or relative speed of at least one object detected by the object recognition system to the thermal imaging camera or the vehicle is measured and displayed by the image transmitter. This allows for the output of further information about the detected objects, which can help a driver assess the situation.

[0021] According to a further embodiment, the processed measurement data is generated by the at least one image sensor and output onto a projection surface designed as a vehicle windshield or onto a projection surface arranged in front of the windshield in the direction of travel and also designed as a projection screen. This allows the method to be carried out particularly advantageously in a vehicle environment. In particular, the projection surface can be transparent and at least partially reflective in order to display the content or measurement data output by the image sensor to a driver or passenger of the vehicle. Depending on the design, the projection surface can be configured as a section of a projection screen or windshield.

[0022] The invention is schematically illustrated with reference to embodiments in the drawings and is further described with reference to the drawings. It shows: Fig. 1 a side view of a vehicle according to an embodiment of the invention. Fig. 2 a top view of a projection surface of an output system according to an embodiment of the invention to illustrate a visualization of processed measurement data. Fig. 3 a schematic flowchart to illustrate a method according to an embodiment of the invention.

[0023] Elements and components with identical or similar constructive or functional characteristics are provided with the same reference symbols across all figures.

[0024] In Fig. Figure 1 shows a side view of a vehicle 100 according to an embodiment of the invention. The vehicle 100 has an output system 10 which is configured to display a Fig. 3. Visualized procedure 20 to be executed.

[0025] In the illustrated embodiment, the output system 10 is designed as an optical output system 10 and comprises a thermal imaging camera 11, an image sensor 12, and a projection surface 13. By way of example, a radar sensor 17 is also provided as part of the output system 10 to support object recognition based on measurement data from the thermal imaging camera 11, depending on the configuration.

[0026] Furthermore, the output system 10 includes a control unit 14. The thermal imaging camera 11 and the image sensor 12 are connected to the control unit 14 via data transmission. For the sake of simplicity, other components, such as additional sensors, parallel control units, optical modules for beam guidance and shaping between the image sensor 12 and the projection surface 13, and the like, are not shown.

[0027] The thermal imaging camera 11 can sensorially detect the vehicle's surroundings U and generate measurement data. Due to the operating principle of the thermal imaging camera 11, warm objects or areas in the vehicle's surroundings U appear brighter than cold objects or areas. Fig. 2 are examples of warm areas 15 and cold areas 16 visualized.

[0028] The image transmitter 12 has a TFT display and can generate images or image sequences based on received control signals from the control unit 14, which are projected onto the projection surface 13. The projection surface 13 is exemplified as a projection disc made of glass and / or plastic. Alternatively, an unnumbered windshield of the vehicle 100 can serve as the projection surface 13.

[0029] Fig. 2 shows a top view of the in Fig. The projection surface 13 of the output system 10 is described in section 1 and serves to illustrate a visualization of processed measurement data. The processed measurement data is divided along a vertical direction H and displayed on lateral areas 18 of the projection surface 13. This allows a central area 19 of the projection surface 13 to remain free for other functions, such as displaying speed limits, warnings, information, navigation instructions, and the like.

[0030] In the lateral areas 18 of the projection surface 13, processed measurement data from the thermal imaging camera 11 is displayed to assist the driver during follow-up driving. For example, objects O identified by object recognition are highlighted to visualize them for the driver. Relative distances between the vehicle 100 and objects O are also schematically represented in the lateral areas 18 of the projection surface 13. Such distances can be determined by the vehicle 100's object recognition. Furthermore, measurement data from the radar sensor 17 can be used for distance measurement and / or object recognition.

[0031] Fig. Figure 3 shows a schematic flowchart to illustrate a method 20 according to an embodiment of the invention. The method 20 serves to output measurement data from at least one thermal imaging camera 11. In the illustrated embodiment, the method 20 is described with reference to the Fig. 1 and Fig. 2 described. In step 21 of the procedure 20, measurement data are acquired by at least one thermal imaging camera 11.

[0032] Subsequently, the acquired measurement data are processed in such a way that cold areas 16 of the measurement data are at least darkened or blackened. This can be done, for example, by adjusting the contrast of the measurement data from the thermal imaging camera 11, which is designed as night vision images. The processing can be carried out by the control unit 14.

[0033] In a further step 23 of the process 20, the processed measurement data are optically generated by at least one image sensor 12 and output onto at least one projection surface 13. The output of the processed measurement data can be direct, i.e., without further optical elements, or indirect via at least one optical element, such as lenses, filters, holographs, and the like. REFERENCE MARK LIST: 100 vehicles 10 Output system 11 Thermal imaging camera 12 image sensors 13 projection surfaces 14 Control unit 15 warm area 16 cold area 17 additional sensors / radar sensors 18 lateral area of ​​the projection surface 19 central area of ​​the projection surface 20 procedures 21 Recording measurement data 22 Processing measurement data 23 Output of processed measurement data F Direction of travel H Altitude O identified / detected object Vehicle environment

Claims

Method (20) for outputting measurement data from at least one thermal imaging camera (11), wherein measurement data are acquired by the at least one thermal imaging camera (11), wherein the acquired measurement data are processed in such a way that cold areas (16) of the measurement data are at least darkened, wherein the processed measurement data are generated by at least one image transmitter (12) and output onto at least one projection surface (13), wherein the processed measurement data are divided along a vertical direction H and output onto lateral areas (18) of the projection surface (13). Method according to claim 1, wherein the cold areas (16) of the recorded measurement data are darkened until they are blackened. Method according to claim 1 or 2, wherein warm areas (15) of the recorded measurement data are brightened. Method according to one of claims 1 to 3, wherein object recognition is carried out based on the received measurement data from the at least one thermal imaging camera (11) and / or based on received measurement data from at least one sensor (17), wherein objects (O) identified by the object recognition are optically highlighted and / or abstracted and / or framed by the image transmitter (12). Method according to claim 4, wherein a distance and / or a relative speed of at least one object (O) identified by object recognition to the thermal imaging camera (11) or vehicle (100) is measured and output by the image transmitter (12). Method according to one of claims 1 to 5, wherein the processed measurement data is generated by the at least one image sensor (12) and output onto a projection surface (13) designed as a windshield of a vehicle (100) or onto a projection surface (13) arranged in the direction of travel (F) in front of the windshield and designed as a projection screen. Output system (10) which is configured to perform a method (20) according to one of the preceding claims, comprising at least one thermal imaging camera (11), at least one image sensor (12) and at least one projection surface (13), wherein at least the thermal imaging camera (11) and the image sensor (12) are connected to at least one control unit (14) via data transmission. Vehicle (100) comprising an output system (10) according to claim 7.

Images