DEVICE FOR VALIDABLE OUTPUT OF IMAGES

Description

[0001] The present invention relates to a device for the verifiable output of images from one or more cameras on a screen. The present invention further relates to a vehicle with such a device for the verifiable output of images from one or more cameras on a screen.

[0002] For example, military vehicles such as tanks use image display systems that include cameras mounted on the vehicle's exterior and at least one screen located inside the vehicle to display the images captured by the cameras. A multiplexer is connected between the cameras and the screen, which provides the images from a selected camera to the screen.

[0003] It is possible that a camera mounted on the outer shell of the vehicle delivers a still image, i.e., is frozen, and the user inside the vehicle has no way of noticing this.

[0004] A particularly high risk arises if, for example, the camera is linked to a weapon station and the image displayed on the screen is used for shooting by sight.

[0005] It is noteworthy that the image delay time for displaying a video image, for example on an LCD or TFT display, varies depending on the technology and environmental conditions, especially temperature. While a video image is displayed very quickly and smoothly at room temperature, at sub-zero temperatures the display becomes significantly slower, even to the point of blurring the video image in moving scenes. Other examples of error patterns or display errors besides a static image include a delayed image and incorrect viewing, i.e., displaying the wrong camera.

[0006] The publication EP 3 319 315 A1 discloses that several video cameras deliver images as analog or digital signals. These images are stored in a memory. The stored images are combined into a single image, which is converted into an analog or digital signal and fed to a monitor. To prevent outdated images from being displayed, all video images are time-stamped, and this time stamp is stored in the memory along with the video images. When the stored images are subsequently combined and converted into an analog or digital signal, the timestamps of the video images are read and compared with the current time. Only if the timestamp is no more than a predetermined time interval ago is the signal fed to the monitor.Furthermore, it is useful if the signals are provided with an identification code so that it can be verified whether the signal supplied to the monitor originates from the correct video camera.

[0007] Furthermore, German patent application DE 10 2014 213456 A1 discloses a system for transmitting a signal containing optical information from a receiving device to an output device and for monitoring the transmission of the signal. For this purpose, the system comprises a receiving device, a signal transmission device, and an output device. For monitoring the signal transmission, the system includes a signal monitoring device, which in turn includes a reference signal generation device for generating a reference signal within the signal generated by the receiving device. The signal monitoring device further comprises a detection device for detecting at least a portion of a signal transmitted to the output device and / or at least a portion of the signal output by the output device.Furthermore, a monitoring device of the signal monitoring device is designed to monitor whether the signal has been detected by the detection device.

[0008] Furthermore, publication WO 2017 / 140456 A1 discloses a device and a method for monitoring a spatial area, in particular in the vicinity of or within a vehicle, preferably a rail vehicle. The device comprises a detection device for capturing the spatial area, configured to generate a series of images, wherein the images in the image series are representative of the spatial area at each point in time during capture; a display device for displaying the generated image series; and a dynamic means configured to generate an optically perceptible change over time, wherein the detection device is configured to interact with the dynamic means such that each image contains a state of change over time.

[0009] Further state of the art is represented by publication US 2016 / 098957 A1.

[0010] Against this background, one object of the present invention is to provide a solution for the validatable output of images.

[0011] The invention is defined in the independent claim. Preferred embodiments are defined in the dependent claims.

[0012] According to a first aspect, a device for the validatable output of images is proposed, which has: a screen for outputting images based on image data, a number N1 of cameras, with N1 ≥ 1, each of the cameras being configured to provide image data based on a capture of camera images, an integration unit for outputting enhanced image data based on an integration of a signature visible in the images output by the screen into the image data provided by one of the cameras, a numberN2 of photosensors arranged in front of the screen, with N2 ≥ 1, wherein the N2 photosensors are configured to detect the integrated signature in the images output by the screen, and a validation unit coupled to the N2 photosensors, which is configured to validate the detected signature and trigger a security function depending on the validated signature.

[0013] Depending on the validated signature, detected by at least one of the photosensors and validated by the validation unit, the present device, a part of the device, e.g. the screen, and / or a device coupled to the device, for example a driver's view or a weapon station, can be placed in a safe state by means of the safety function.

[0014] For example, if the delay time of the device, in particular the difference between the detection of the signature by the photosensor and the imprinting or integration of the signature into the provided image data, is greater than a predetermined threshold, for example due to a very low ambient temperature, a safety function can be triggered by the validation unit.

[0015] Optical signature detection via the number of photosensors allows errors in the display to be detected by the screen itself.

[0016] The signature can also be described as a stimulus or embedded image information, embedded in the images to be displayed.

[0017] Monitoring the additional embedded image information can be used to design and implement a corresponding safety function for all three safety-critical aspects mentioned above: frozen image, delayed image, and false view. In particular, the integrated signature or embedded image information can be used to detect the error patterns of delayed images, frozen images, and false views. If such an error pattern is detected, a safety function can be triggered.

[0018] Examples of such a safety function include issuing a message to a user (visual and / or audible and / or haptic), turning off the screen and / or blocking a weapon station or weapon coupled to the device.

[0019] The device can also be referred to as a vision system. The camera in question can be, for example, a video camera, a daylight camera, a thermal imaging camera, an infrared camera, or a still camera. In particular, the cameras can comprise different camera types. The photosensor can also be referred to as a photodetector, an image sensor, or an optical detector.

[0020] The screen could, for example, be a flat screen located in the interior of a military vehicle. The screen can also be referred to as a monitor or display. The display could be an LCD display, a TFT display, or an optical display, for example, consisting of one or more LEDs.

[0021] In particular, an image processing unit or control unit is provided, which is configured to control the integration unit and the screen. The image processing unit includes, for example, electronics such as a microprocessor, a CPLD (Complex Programmable Logic Device), or an FPGA.

[0022] According to the first aspect, the validation unit is designed to determine a time difference between a second time of detection of the signature by the N2 photosensors and a first time of integration of the signature into the provided image data and to trigger the safety function if the determined time difference is greater than a predetermined threshold.

[0023] Different thresholds can also be used to differentiate between various error patterns, such as a delayed image or a frozen image. Depending on the validation of the detected signature, and especially depending on the different thresholds used, the validation unit can also trigger different security functions.

[0024] According to another embodiment, the validation unit is configured to detect or differentiate different errors or error patterns depending on the determined time difference, in particular by using different threshold values. The different error patterns include, for example, "delayed image", "frozen image", and "incorrect view".

[0025] For example, if a maximum permissible delay time is reached, a safety function can be activated that signals to the user that the limit has been exceeded in the event of a "delayed image" error and reliably puts the device into a safe state, at least until the current delay time has fallen below the maximum permissible delay time again.

[0026] Furthermore, a trigger specifying the initial time of signature integration can be transmitted directly from the integration unit to the validation unit. The signature itself can also be used as a trigger. If, after receiving the trigger, the validation unit does not detect any expected change on the screen via the photosensors, a "frozen image" error is assumed, and the system can be put into a safe state.

[0027] According to the first aspect, the signature is designed as an encoded signature, which includes a timestamp specific for the integration of the signature into the provided image data to indicate the first time point.

[0028] The timestamp thus indicates the first time the signature was integrated into the provided image data.

[0029] According to the first aspect, the signature is designed as a coded signature, which includes a timestamp specific for the integration of the signature into the provided image data to indicate the first time point and a source indication specific for the camera providing the image data.

[0030] The source information, as part of the encoded signature and thus as part of the extended image data, is suitable for identifying the camera providing the image data. This source information can also be referred to as identification information, ID, or source ID. The source information is preferably generated based on a hardware ID of the camera.

[0031] According to another embodiment, the respective photosensor is configured to monitor a predetermined output area of ​​the screen with an MxN pixel area for signature detection.

[0032] Here, M denotes the number of rows and N the number of columns of the output area or the portion of the screen output area on which the signature is displayed. The output area is typically located at the edge or a corner of the screen, so that the central area of ​​the screen remains available for user output.

[0033] According to another embodiment, the pixels of the MxN pixel area of ​​the output area have a defined brightness, in particular a defined and identical brightness, to form the visible signature.

[0034] According to another embodiment, the pixels of the MxN pixel area of ​​the output area form a predetermined pattern to create the visible signature.

[0035] The predetermined pattern can also be referred to as a pattern. Various pieces of information can be encoded in the pattern, which are evaluated by the validation unit and used for validation or for selecting the appropriate safety function.

[0036] The pattern can also be implemented as a video pattern. For example, when using four different cameras, a camera-specific video pattern can be assigned to each camera and displayed and detected at specific locations, such as the four corners of the screen. This allows the error case of "incorrect view" to be detected using the four photosensors and the downstream validation unit.

[0037] According to another embodiment, four photosensors are provided, with N2 = 4, wherein the respective output area monitored by one of the four photosensors is located in one of the four corners of the screen.

[0038] The output areas are located in the four corners of the screen, and accordingly the four photosensors are arranged in front of these four corners of the screen, so that the central area of ​​the screen remains available for displaying the images from the cameras and thus for the user.

[0039] By using four photosensors, each positioned in a corner of the screen, four different positions on the screen can be observed and evaluated. For example, when using four different cameras, camera-specific embedded image information can be displayed in the top left, top right, bottom left, and bottom right corners. Thus, as described above, the photosensors and the downstream validation unit can also detect the error case of "incorrect view." If, for example, the validation reveals that a particular screen is supposed to display images from camera A but is actually displaying images from camera B, the appropriate action can be to disable the weapon that can be coupled to the device.

[0040] According to another embodiment, a plurality of photosensors are provided, with N2 ≥ 4. The output areas monitored by the photosensors are located in the vertical and / or horizontal edge areas of the screen.

[0041] According to the first aspect, a number N3 of temperature sensors are provided for measuring a respective temperature, with N3 = N2. One of the temperature sensors is positioned in front of the screen and next to one of the photosensors.

[0042] According to another embodiment, the respective temperature sensor is configured to measure the temperature of the MxN pixel area of ​​the screen's output area.

[0043] According to another embodiment, the validation unit is configured to validate the detected signature depending on the temperatures measured by the N3 temperature sensors.

[0044] The temperature sensors are specifically designed to measure the temperature of the respective active area, i.e., the respective MxN pixel area of ​​the output region. This results in a correlation between the delay time and the display pixel temperature. This correlation can be used to validate the delay time. Furthermore, the same temperature sensors can advantageously be used to control a display heater, which regulates the screen to a target temperature at which the delay time remains below the specified threshold or limit, thus ensuring continuous operation of the screen.

[0045] According to another embodiment, the safety function includes issuing a message to a user, switching off the screen and / or blocking a weapon coupled to the device.

[0046] According to another embodiment, the device comprises a transmission channel connected between the N1 cameras and the screen for transmitting the image data from the N1 cameras to the screen, wherein the integration unit is arranged in the transmission channel in front of the screen.

[0047] The transmission channel between the cameras and the screen includes, in particular, a multiplexer downstream of the cameras, an image processing unit downstream of the multiplexer, and preferably also the integration unit and the validation unit. Alternatively, the integration unit and the validation unit can also be directly associated with the screen or be part of it.

[0048] The interfaces used in the transmission channel include, for example, DVI and / or Camera Link and / or a wireless interface, such as WLAN.

[0049] According to another embodiment, the device includes a control device which is coupled to the validation unit.

[0050] In particular, the control device is designed to lock the weapon that can be coupled to the device if the validation indicates a display of a still image, a display of an image from the wrong camera ("false view"), or a display of a severely delayed image.

[0051] According to another embodiment, the control device is configured to control a safe failure indicator, which indicates a failure of one or more of the cameras, depending on the validation.

[0052] This gives the user reliable information about whether a specific camera has failed, and preferably which camera. This significantly increases the security of the entire system.

[0053] The respective unit, for example, the integration unit or the validation unit, can be implemented in hardware and / or software. In a hardware implementation, the respective unit can be a device or part of a device, for example, a computer, a microprocessor, or an FPGA. In a software implementation, the respective unit can be a computer program product, a function, a routine, part of program code, or an executable object.

[0054] Furthermore, a vehicle is proposed with a device for the validatable output of images as described above, wherein the N1 cameras are arranged on an outer skin of the vehicle and the screen is located in a room of the vehicle.

[0055] The vehicle is preferably a military vehicle. The vehicle can be a tank, a truck, a special vehicle such as a crane, a loader or an excavator, a ship or a flying system such as a drone or a helicopter.

[0056] Other possible implementations of the invention also include combinations of features or embodiments described previously or subsequently with regard to the exemplary embodiments, even if not explicitly mentioned. In such cases, the person skilled in the art will also add individual aspects as improvements or additions to the respective basic form of the invention.

[0057] Further advantageous embodiments and aspects of the invention are the subject of the dependent claims and the exemplary embodiments of the invention described below. The invention will now be explained in more detail with reference to preferred embodiments and the accompanying figure. Fig. 1 shows a schematic block diagram of an embodiment of a device for the validatable output of images; and Fig. 2 shows a schematic block diagram of an embodiment of a screen for a device for the validatable output of images.

[0058] In the figures, identical or functionally equivalent segments have been given the same reference symbols, unless otherwise indicated.

[0059] In Fig. 1 A schematic block diagram of a first embodiment of a device 10 for the validatable output of images is shown. For this purpose, the Fig. 2 a schematic block diagram of an exemplary embodiment of a screen 20 for the device 10 of the Fig. 1 .

[0060] Device 10 of the Fig. 1The system comprises a number N1 of cameras 31, 32, a screen 20, and a transmission channel 40 connected between the cameras 31, 32 and the screen 20. Without limiting generality, the exemplary embodiment of Fig. 1 two cameras (N1 = 2). Furthermore, the Fig. 1 a single screen 20. Without limiting the generality, the present is also applicable to a plurality of screens.

[0061] The screen 20 is suitable for outputting images based on image data. The screen 20 is, for example, a display, preferably an LCD display or a TFT display. The respective camera 31, 32 is configured to output image data BD1, BD2 based on a camera image capture. The image data BD1, BD2 are transmitted to the transmission channel 40. For this purpose, the transmission channel 40 includes a multiplexer 41. In addition to the multiplexer 41, the transmission channel 40, according to the exemplary embodiment of the Fig. 1 an image processing unit 42, an integration unit 43 and a validation unit 44. Without limiting the generality, the transmission channel 40 can include further units.

[0062] The multiplexer 41 provides one of the received inputs as an output, in this case either the image data BD1 from camera 31 or the image data B2 from camera 32. In the example shown in the Fig. 1 The multiplexer 41 provides the image data BD1 from camera 31 to the image processing unit 42. In this example, the image processing unit comprises the integration unit 43. The integration unit 43 is configured to output extended image data eBD1 based on the integration of a signature S visible in the images displayed by screen 20 into the image data BD1 output by camera 31.

[0063] Furthermore, the device 10 comprises a number N2 of photosensors 51, 52 arranged in front of the screen 20. In the example of the Fig. 1 Two photosensors 51 and 52 are shown. Without loss of generality, a larger number of photosensors can be used. For example, the Fig. 2 an embodiment with four photosensors 51-54.

[0064] The photosensors 51, 52 are configured to detect the integrated signature S in the images displayed by the screen 20. Accordingly, the signature S can also be described as embedded additional image information.

[0065] The respective photosensor 51-54 is specifically designed to detect a predetermined output area 21-24 (see Fig. 2The screen 20 is monitored with an MxN pixel area for detecting the signature S. The pixels of the MxN pixel area of ​​the output area 21-24 have, in particular, a defined brightness, preferably a defined and identical brightness, to form the visible signature S. Alternatively, the pixels of the MxN pixel area can also form a predetermined pattern, for example, a video pattern, which is preferably camera-specific.

[0066] Furthermore, the Fig. 1 The photosensors 51 and 52 are connected to the validation unit 44 so that the signals P1 and P2 of the photosensors 51 and 52 can be transmitted to the validation unit 44. The signals P1 and P2 of the photosensors 51 and 52 contain information that is indicative of the detected signature S.

[0067] The validation unit 44 is configured to validate the detected signature S, in particular using the signals P1, P2 received by the photosensors 51, 52, and to trigger a safety function depending on the validated signature S. For this purpose, the validation unit 44 can also transmit a validation result V generated as a result to the image processing unit 42, which can then execute the triggered safety function.

[0068] In particular, the validation unit 44 is designed to determine a time difference between a time of detection of the signature S by the photosensors 51, 52 and a first time of integration of the signature S into the provided image data BD1, BD2 and to trigger the safety function if the determined time difference is greater than a predetermined threshold.

[0069] This can be done, as in Fig. 1As shown, the integration unit 43 transmits the signature S to the validation unit 44, particularly at the first time point. Alternatively, a simple trigger signal can be used, which is transmitted from the integration unit 43 to the validation unit 44 at the first time point.

[0070] Furthermore, the signature S can also be designed as an encoded signature, which includes a timestamp specific to the integration of the signature S into the provided image data BD1 to indicate the first time point.

[0071] Furthermore, the encoded signature can also include a source specification specific to camera 31 providing the image data BD1.

[0072] The safety function, which can be triggered or activated by the validation unit 44, may include, for example, issuing a message to a user, turning off the screen 20 and / or locking a weapon or weapon station coupled to the device 10.

[0073] As already explained above, the Fig. 2 an embodiment of a screen 20 for the device 10 of the Fig. 1 with four output areas 21-24, wherein the respective output area 21-24 is located in one of the four corners of the screen 20, and four photosensors 51-54 arranged accordingly.

[0074] Thus, photosensor 51 is located in output area 21 (top left of screen 20) and photosensor 52 is located in output area 22 (top right of screen 20). Furthermore, photosensor 53 is located in output area 23 (bottom left of screen 20) and photosensor 54 is located in output area 24 (bottom right of screen 20).

[0075] In the exemplary embodiment of the Fig. 2 Four temperature sensors 61-64 are also provided. One of the temperature sensors 61-64 is positioned in front of the screen 20, next to one of the photosensors 51-54. Each temperature sensor 61-64 is configured to measure the temperature of the respective output area 21-24 of the screen 20. In this case, the validation unit 44 can also use the temperatures measured by the temperature sensors 61-64 for validation.

[0076] Although the present invention has been described using exemplary embodiments, it can be modified in many ways. REFERENCE MARK LIST

[0077] 10 Device 20 Screen 21 Output area 22 Output area 23 Output area 24 Output area 31 Camera 32 Camera 40 Transmission channel 41 Multiplexer 42 Image processing unit 43 Integration unit 44 Validation unit 51 Photosensor 52 Photosensor 53 Photosensor 54 Photosensor 61 Temperature sensor 62 Temperature sensor 63 Temperature sensor 64 Temperature sensor BD1 Image data BD2 Image data eBD1 Extended image data P1 Photosensor signal P2 Photosensor signal S Signature V Validation result

Claims

1. A device (10) for validatable output of images, comprising: a screen (20) for outputing images based on extended image data, a number N1 of cameras (31, 32), where N1 ≥ 1, wherein each of the cameras (31, 32) is configured to provide image data (BD1, BD2) based on a recording of camera images, an integration unit (43) for outputting extended image data (eBD1, eBD2) based on an integration of a signature (S) visible in the images that are output by the screen (20) into the image data (BD1, BD2) provided by one of the cameras (31, 32), a number N2 of photosensors (51-54), where N2 ≥ 1, arranged in front of the screen (20), wherein the N2 photosensors (51-54) are configured to detect the integrated signature (S) in the images that are output by the screen (20), and a validation unit (44) coupled to the N2 photosensors (51-54), which is configured to validate the detected signature (S) and to trigger a safety function depending on the validated signature, wherein the validation unit (44) is configured to determine a time difference between a second point in time of the detection of the signature (S) by the N2 photosensors (51-54) and a first point in time of the integration of the signature (S) into the provided image data (BD1, BD2) and to trigger the safety function if the determined time difference is greater than a predetermined threshold value, wherein the signature (S) is formed as a coded signature, which comprises a time stamp specific for the integration of the signature (S) into the provided image data (BD1, BD2) for indicating the first point in time and a source indication specific for the camera (31, 32) providing the image data (BD1, BD2), wherein a number N3 of temperature sensors (61-64) is provided for measuring a respective temperature, where N3 = N2, wherein a respective one of the temperature sensors (61-64) is arranged in front of the screen (20) and next to a respective one of the photosensors (51-54), wherein the validation unit (44) is configured to validate the detected signature (S) depending on the temperatures measured by the N3 temperature sensors (61-64), and / or wherein the temperature sensors (61-64) are configured to control a heating of the screen (20), which regulates the screen (20) to a setpoint temperature.

2. The device according to claim 1, characterized in that the respective photosensor (51-54) is configured to monitor a predetermined output area (21-24) of the screen (20) with an MxN pixel area for detecting the signature (S).

3. The device according to claim 2, characterized in that the pixels of the MxN pixel area of the output area (21-24) have a defined brightness for forming the visible signature (S).

4. The device according to claim 2, characterized in that the pixels of the MxN pixel area of the output area (21-24) form a predetermined pattern for forming the visible signature (S).

5. The device according to any one of claims 2 to 4, characterized in that four photosensors (51-54) are provided, where N2 = 4, wherein the respective output area (21-24) monitored by one of the four photosensors (51-54) is located in one of the four corners of the screen (20).

6. The device according to any one of claims 2 to 4, characterized in that a plurality of photosensors (51-54) is provided, where N2 ≥ 4, wherein the output areas (21-24) monitored by the photosensors (51-54) are located in the vertical and / or horizontal edge areas of the screen (20).

7. The device according to claim 1, characterized in that the respective temperature sensor (61-64) is configured to measure a temperature of the MxN pixel area of the output area (21-24) of the screen (20).

8. The device according to any one of claims 1 to 7, characterized in that the safety function comprises outputting a message to a user, switching off the screen (20) and / or blocking a weapon coupled to the device (10).

9. The device according to any one of claims 1 to 8, characterized by a transmission channel (40) connected between the N1 cameras (31, 32) and the screen (20) for transmitting the image data (BD1, BD2) from the N1 cameras to the screen (20), wherein the integration unit (43) is arranged in front of the screen (20) in the transmission channel (40).

10. A vehicle comprising a device (10) for validatable output of images according to any one of claims 1 to 9, wherein the N1 of cameras (31, 32) are arranged on an outer skin of the vehicle and the screen (20) is arranged in a space of the vehicle.