Display device, image recording device and method for measuring video latency in an image recording device

Abstract

Display device comprising a computing unit (5), a display (4) and a communication interface (6) for connecting the display device (2) to a camera (3), wherein the computing unit (5) is configured to control the display (4) to display various images and to evaluate video data transmitted from the camera (3) connected to the communication interface (6) to the communication interface (6), wherein the display (4) of the display device (2) is depicted in the video data and the computing unit (5) is configured, in a measuring operating mode, to control the display (4) to successively display two images with a different brightness and to determine a time duration between a brightness change between the displayed images and a corresponding brightness change in the video data of the camera (3).

Description

The invention relates to a display device comprising a computing unit, a display and a communication interface for connecting the display device to a camera, wherein the computing unit is configured to control the display for displaying different images and for evaluating video data transmitted from the camera connected to the communication interface to the communication interface. The invention further relates to an image recording device and to a method for measuring a latency in an image recording device.Arrangements which can both record images by means of a camera and display these recorded images via a screen are used in many fields. In particular in motor vehicles, various such arrangements are used for capturing and displaying an environment of the motor vehicle, for example in the form of a rear view camera, in the form of virtual external mirrors or in the form of a camera-based environment capture for displaying a top view of the motor vehicle. In this case, the camera and the screen used for displaying the images captured by the camera are spatially separated as a result of the principle, since the camera is arranged on the outside of the motor vehicle in order to capture the surroundings of the motor vehicle, and the screen is generally arranged in an interior of the motor vehicle, such that the latter can be captured by a driver.For transmitting the images recorded by the camera to the display, a connection for data transmission, for example a network, can be used, wherein, however, a delay, referred to as latency or video latency, can occur between the events recorded by the camera in the environment of the motor vehicle for its corresponding display on the screen. The latency depends on a large number of different parameters, for example on a selected transmission method, on the type of network and on parameters inside the network, such as a compression method used, an available bandwidth, buffer management or the like. A latency resulting from these influencing factors can have a disadvantageous effect on the control of the motor vehicle if the latency, that is to say the time delay between the actual event and the display of the event on the screen, is too great. For this reason, it is desirable that latency in such a camera-based display system can be determined and evaluated. Various approaches are known for this purpose from the prior art.From DE 10 2017 010 887 A1 a method is known for measuring a transmission time between the generation of an image signal from a camera sensor of a camera and the display of the image signal on a screen. The image signal is transmitted between the camera and the screen, a light source being arranged directly in front of the camera sensor for generating an image signal, and the light source being switched on at a switch-on time for a switch-on duration. A display time at which the image signal is displayed on the screen is detected by a light sensor, wherein the transmission time is determined as the difference between the display time and the switch-on time.US 2019 / 0 098 293 A1 discloses a method and a device for determining the display latency of a display device. In this case, a device designed to record a video is positioned in such a way that it captures the display device and a computing unit connected to the display device. The screen latency is determined by comparing a light flashing on the computing device and an activation of the screen carried out by the computing device with a corresponding frequency, wherein the screen latency is determined from the frequency offset between the light on the computing device and the screen.CN 106 534 843 A describes a method in which a clock is captured via a camera and a video of the captured clock is displayed on a screen which is connected to the camera. By creating an image on which both the real clock and the display of the clock can be seen on the screen, the time difference displayed on the clocks can be read and a latency of the display system determined.The printing step JP 2013-247489 A describes a method for measuring a delay time in a video system. A camera is connected to a device which can continuously generate periodic images. The periodic images generated by the device are transmitted via a network. The data recorded by the camera of a first half of a display on which the images generated by the device are displayed again pass via a connection for display and are displayed there on a second half of the display, so that a time offset corresponding to the delay of the network results between the image sequences displayed in the different halves of the display.An apparatus for measuring a reaction time is known from the publication EP 2 857 973 A1. In this case, a response time of a video network is ascertained by ascertaining the time offset between the transmission of a test image and the capturing of the test image on a display.A game device is known from the printing step US 2012 / 0172 118 A1. A single test image is used, wherein a time offset is determined between the transmission of the test image to a display and the acquisition of the representation of the test image.The object of the invention is to specify a display device which makes it possible to specify a latency of an image recording device comprising the display device.To achieve this object, in a display device of the type mentioned at the beginning, the invention provides for the display of the display device to be imaged in the video data and for the arithmetic unit to be configured, in a measurement operating mode, to actuate the display for the successive display of two images having a different brightness and to determine a time duration between a brightness change between the displayed images and a corresponding brightness change in the video data of the camera.An external camera can be connected to the display device via the communication interface of the display device. In this case, the camera is in particular oriented such that it completely or at least partially captures at least the display of the display device, such that the display is represented in the video data which are transmitted from the camera to the display device. Ideally, the camera is positioned such that the image region captured by it is filled in particular completely by the display of the display device or at least a part of the display of the display device. The display of the display device can be, in particular, a screen or a display.The video data received from the camera via the communication interface is evaluated by the computing unit connected to the communication interface. The video data comprise a sequence of video images recorded with the camera, with in particular such short time intervals between the video images that they can be perceived as a liquid movement. Furthermore, the computing unit is configured to control the display of the display device for the purpose of displaying images. In a normal operating mode of the display device, for example, these can be the video images which are described by the video data and which the camera of an image recording device comprising the camera and the display device transmits to the display device. In the normal operating mode, the video data is therefore displayed by the camera on the display of the display device.However, in the measurement operation mode, the video images described by the video data of the camera are not displayed on the display of the display device, but the arithmetic unit controls the display of the display device to successively display two images having different brightness. For example, this may be a completely black display with a corresponding low brightness followed by a completely white display with a corresponding high brightness. Furthermore, in the measuring operating mode, the computing unit evaluates the video data transmitted from the camera to the computing unit with regard to a brightness change, which corresponds to the brightness change by the display of the images on the display of the display device.The time offset or duration between a representation of the successive images having the different brightness on the display of the display device and the receipt of the video data having the corresponding brightness change from the camera represents a latency of the display device, which indicates the delay between the capturing of the reality via the camera and its representation on the display of the display device. This latency can also be referred to as video latency or as end-to-end latency or as glass-to-glass latency of an image recording device comprising the display device and the camera.The configuration of the computing unit of the display device for carrying out the measurement operating mode or for determining the time duration describing the latency has the advantage that the latency of the image recording device comprising the display device can be determined without the aid of further measurement technology. A graphics unit of the display device can advantageously be used as the computing unit, which is also used for displaying video images in a normal operating mode. The use of expensive and sometimes cost-intensive measuring instruments, such as an oscilloscope, as well as a complex construction using illumination means, photodiodes or the like can be advantageously avoided. Advantageously, the image recording devices comprising the display device can thus be tested with little effort with respect to their respective latency, in particular before they are used in a motor vehicle.According to the invention, it can be provided that the computing unit is configured to start the determination of the time duration if the brightness determined from the video data falls below or exceeds a first threshold value, and to end the determination of the time duration if the brightness determined from the video data exceeds or falls below a second threshold value. The consideration of a threshold value for starting the determination of the time duration or for ending the determination of the time duration can take into account fluctuations in the brightness of the video data and / or in an environment of the display device that may additionally be captured by the camera. Furthermore, the use of the first threshold value and the second threshold value can take account of the fact that individual video images with intermediate states between a display representing an image with a high brightness and an image with a low brightness can be present in the video data.In a preferred embodiment of the invention, it can be provided that the display device has a control interface, wherein the control interface is connected to the arithmetic unit and the arithmetic unit is designed to start the measurement operating mode after receiving a start command at the control interface. This has the advantage that the display device, or the computing unit of the display device, can be connected to an external computing device, for example a computer or the like, wherein the measurement operating mode of the computing unit of the display device can be controlled automatically and externally. This advantageously makes it possible to control a plurality of measurement processes in an automated manner one after the other.According to the invention, it can be provided that the computing unit is configured to transmit an item of information describing the determined time duration to the control interface after the time duration is determined. From the control interface, this information can be read out, for example, by an external computing device connected to the control interface and stored, for example, in the external computing device, so that a plurality of measurements can be carried out in succession in an automated manner and their results or the determined time periods can subsequently be evaluated. As a result, for example, the determination of the time duration in the image recording device comprising the display device can be improved by averaging and / or statistics can be created about the distribution of different measured time durations and the respective minimum values and maximum values.According to the invention, it can be provided that the computing unit is configured to activate the display for displaying the time duration and / or an item of information describing the determined time duration on the display after the time duration has been determined. This advantageously allows a representation of the time duration determined by the computing unit to be possible even without an external computing device, since a determined time duration can already be displayed by the display device itself and read from it. This furthermore advantageously reduces the outlay for determining latency in an image recording device comprising the display device. The time duration can be represented, for example, as a sequence of letters and / or numbers on the display of the display device.According to the invention, it can be provided that the communication interface is an electrical connection interface and / or a network interface, in particular an Ethernet interface or a low-voltage differential signaling interface. After the capturing, the video data can be encoded by the camera in a video format, for example an MPEG format, in particular H.264 or H.265, and then, in accordance with the configuration of the communication interface or of a network connected to the communication interface, can be transmitted via the latter, for example by means of real-time transport protocol (RTP).According to the invention, it can be provided that the computing unit is designed for processing compressed video data and / or that the computing unit comprises at least one video decoder unit and / or at least one deserialization unit. Advantageously, a computing unit can be used to carry out the measuring operating mode, which is also already present in a normal operating mode of the display device, or of an image recording device comprising the display device, for the presentation and / or decoding of, in particular, compressed video data.For an image recording device according to the invention, it is provided that it comprises a display device and a camera connected to the communication interface of the display device.All the advantages and details described above with respect to the display device according to the invention also apply analogously to the image recording device according to the invention.For a method according to the invention for measuring a video latency in an image recording device according to the invention, it is provided that the camera is arranged in such a way that it captures the display of the display device, wherein the display is controlled by the arithmetic unit for the successive display of two images having a different brightness and the video data transmitted from the camera to the communication interface are evaluated by the arithmetic unit with respect to a brightness of the video data, wherein a time duration between a brightness change between the displayed images and a corresponding brightness change in the video data of the camera is determined by the arithmetic unit.According to the invention, it can be provided that the determination of the time duration is started by the computing unit when the brightness determined from the video data falls below or exceeds a first threshold value, and the determination of the time duration is ended when the brightness determined from the video data exceeds or falls below a second threshold value.In a preferred embodiment of the invention, it can be provided that the display device has a control interface, wherein the control interface is connected to the computing unit, wherein the measurement operating mode of the computing unit is started after a start command is received at the control interface. The start command can be transmitted to the display device, for example, by an external computing device which is connected to the control interface of the display device, and trigger the start of the measurement operating mode in the computing unit.In a preferred embodiment of the invention, it can be provided that after the time duration has been determined, information describing the determined time duration is transmitted to the control interface. This advantageously enables, when the display device is connected to an external computing device, a plurality of measurement processes to be carried out by starting the measurement operating mode a plurality of times, wherein in each case the time duration determined in the measurement operating mode is transmitted to the external computing device and stored there, for example, so that a determination of a video latency from the time duration can be carried out, for example, by averaging a plurality of measured time durations.In a preferred embodiment of the invention, it can be provided that after the duration of time has been determined, the display is actuated to display the duration of time and / or an item of information describing the duration of time determined on the display. The time duration can be represented directly there, for example, as a sequence of letters or numbers.The advantages and embodiments described above with respect to the display device according to the invention also apply correspondingly to the method according to the invention.Further advantages and details of the invention are evident from the exemplary embodiments described below and on the basis of the drawings. These are schematic representations and show: FIG. 1 shows an image recording device according to the invention having an exemplary embodiment of a display device according to the invention, FIG. 2 shows a diagram in which a brightness ascertained from the video data is shown over time, and FIG. 3 shows a flow diagram of an exemplary embodiment of a method according to the invention.FIG. 1 shows an exemplary embodiment of an image recording device 1. The image recording device 1 comprises a display device 2 and a camera 3. The display device 2 comprises a computing unit 5, a display 4 designed as a screen or display and a communication interface 6, which is designed in particular as an electrical connection interface and / or a network interface, in particular an Ethernet interface or a low-voltage differential signaling interface.The computing unit 5 comprises a video decoder unit and / or a deserialization unit, which are designed as a common video decoder and deserialization unit 7. The arithmetic unit 5 is connected to the communication interface 6 and to the display device 4 and is designed, in a normal operating mode of the image recording device 1, to display video data transmitted from the camera 3 to the communication interface 5 on the display 4 of the display device 2.The video data received via the communication interface 6 can be processed, for example, with the aid of the video decoder and deserialization unit 7, in particular as a function of a network 8 connected to the communication interface 6 for data transmission and / or a compression method used for compressing the video data transmitted by the camera 3, so that the display 4 can be controlled by the arithmetic unit 5 to display a video described by the video data or a sequence of video images described by the video data.The camera 3 comprises a camera sensor 9 having a detection region 10 in which the surroundings of the camera 3 can be detected by the image recording device 9. The camera 3 can furthermore comprise a communication interface 11 by means of which the camera 3 can be connected to the communication interface 5 of the display device 2 via the network 8. The camera 3 furthermore comprises a serializing unit and a video decoder unit, which are designed as a common serializing and video decoder unit 12, with which the video images captured by the camera can be decoded according to a video format and processed for transmission via the network 8. The video data can be encoded by the camera 3, for example after the capturing, in a video format, for example an MPEG format, in particular H.264 or H.265, and then, in accordance with the configuration of the communication interface 6 or of the network 8 connected to the communication interface 6, can be transmitted via the latter, for example by means of real-time transport protocol (RTP), to the display device 2.The arithmetic unit 5 is configured to control the display 4 of the display device 2 to display at least two images having different brightness. This may be, for example, an image in which the display 4 of the display device 2 is completely black, after which a further image having a high brightness, for example a completely white display 4 of the display device 2, is displayed. This brightness change in the display 4 is recorded by the camera 9, which is oriented to carry out the measurement operating mode in such a way that it captures the display 4 of the display device 2 with its capture range 10, and is transmitted as video data via the communication link 8 to the communication interface 5 of the display device 2.The arithmetic unit 5 is configured to evaluate the video data with regard to a brightness change, which corresponds to the brightness change of the display 4, and to determine a time duration that has elapsed between the display of the successive images having the different brightness 2 on the display 4 and the receipt of the corresponding brightness change in the video data from the camera 3. This time duration represents an end-to-end latency or a glass-to-glass latency of the image recording device 1.The camera 3 can be positioned in particular such that the video data generated by it reacts sufficiently strongly to a brightness change on the display 4, in particular such that the capture range 10 of the display 4 is filled completely or at least to a large extent by the display device 4 or a part of the display 4 of the display device 2. In this case, the video images included in the video data also change their brightness corresponding to a dark brightness and a high brightness upon detecting the image change on the display 4.The display device 2 further comprises a control interface 13, wherein the display device is connected to an external computing device 14 via the control interface 13. A start command can be transmitted from the computing device 14 to the control interface 13 or the computing unit 5, wherein the computing unit 5 starts the measurement mode after receiving the start command and determines a time duration between the brightness change in the two successive images displayed on the display 4 and the corresponding brightness change in the video data transmitted by the camera 3. The determined time duration can then be transmitted from the computing unit 5 via the control interface 13 to the external computing device 14. In particular, a plurality of start commands can be transmitted one after the other by the external computing device 14, wherein the respectively received time periods can be stored in the computing device and can be used, for example, to determine an average value or statistics of the time period or the latency of the image recording device 1.In the diagram 15 in FIG. 2, the brightness described in the video data is represented by the curve 16, the brightness H being represented on the ordinate and the time t on the abscissa. It is possible, for example, for a control of the display device 4 to be controlled for displaying a dark image at a time t 1 or before the time t 1. At the time t 2, at which the brightness determined from the video data falls below a first brightness threshold value H 1, which corresponds to a detected dark display 4, the brightness of the video data is now controlled in a measuring operating mode of the display device 4 to display an image with a high brightness, which follows the previously displayed image with the low brightness, and the determination of a time duration is started. If it is determined from the brightness of the video data evaluated by the arithmetic unit 5 that the brightness at the time t 3 exceeds a second threshold value H 2 the determination of the time duration at the time t 3. is stopped. The difference between the time t 3 and the time t 2 represents the time duration with which the brightness change between the images displayed on the display 4 is also described in the video data from the camera 3. This value represents a latency of the image capturing device 1.FIG. 3 shows a flow diagram of an exemplary embodiment of a method according to the invention. In step S 1, a measuring operation of the display device 2 is started, which can be triggered, for example, by a start command being sent to the control interface 13 via the control interface 13 of the display device 2 from the external computing device 14 connected to the control interface 13.In step S 2, a first image having a first brightness value is displayed, wherein, for example, as described with reference to FIG. 2, an image having a dark brightness is first displayed. In step S 3, if the brightness determined in the video data falls below the first threshold value H 1 a subsequent image with a high brightness is displayed on the display 4 and the determination of the time duration is started.After the brightness determined from the video data describes a corresponding brightness change to the display 4, for example when the brightness described by the video data reaches the second threshold value H 2 the determination of the time duration is ended in step S 4. In a subsequent step S 5, the ascertained time duration can be output on the display 4.Additionally or alternatively, an item of information describing the ascertained time duration can also be displayed. Additionally or alternatively, it is also possible for the determined time duration to be transmitted via the control interface 13 to the external computing device 14, so that, for example, a plurality of time durations for determining an averaged latency of the image recording device 1 can be automatically obtained by the computing device 17 via a subsequent performance of a plurality of measurements in a measurement operating mode of the control unit 5.

Claims

Display device comprising a computing unit (5), a display (4) and a communication interface (6) for connecting the display device (2) to a camera (3), wherein the computing unit (5) is configured to control the display (4) to display different images and to evaluate video data transmitted from the camera (3) connected to the communication interface (6) to the communication interface (6), wherein the display (4) of the display device (2) is imaged in the video data and the computing unit (5) is configured to control the display (4) to successively display two images having a different brightness in a measurement operating mode and to determine a time duration between a brightness change between the displayed images and a corresponding brightness change in the video data of the camera (3).Display device according to Claim 1, characterized in that the arithmetic unit (5) is configured to start the successive display of the two images and the ascertainment of the time duration if the brightness determined from the video data falls below or exceeds a first threshold value, and to end the ascertainment of the time duration if the brightness determined from the video data exceeds or falls below a second threshold value.Display device according to Claim 1 or 2, characterized in that display device (2) has a control interface (13), the control interface (13) being connected to the arithmetic unit (5) and the arithmetic unit (5) being designed to start the measurement operating mode after a start command is received at the control interface (13).Display device according to Claim 3, characterized in that the arithmetic unit (5) is configured to transmit an item of information describing the ascertained time duration to the control interface (13) after ascertaining the time duration.Display device according to one of the preceding claims, characterized in that the arithmetic unit (5) is configured to activate the display (4) after the duration has been determined in order to display the duration and / or an item of information describing the duration determined on the display (4).Display device according to one of the preceding claims, characterized in that the communication interface (6) is an electrical connection interface and / or a network interface, in particular an Ethernet interface or a low-voltage differential signaling interface.Display device according to one of the preceding claims, characterized in that the arithmetic unit (5) is designed to process compressed video data and / or in that the arithmetic unit (5) comprises at least one video decoder unit and / or at least one deserialization unit.Image recording device comprising a camera (3) and a display device (2), wherein the display device (2) comprises a computing unit (5), a display (4) and a communication interface (6) for connecting the display device (2) to the camera (3), wherein the camera (3) is connected to the communication interface (6) of the display device (2), wherein the computing unit (5) is configured to control the display (4) for displaying different images and for evaluating video data transmitted from the camera (3) connected to the communication interface (6) to the communication interface (6), wherein the display (4) of the display device (2) is imaged in the video data and the computing unit (5) is configured to display different images, in a measuring operating mode, the display (4) is controlled to successively display two images having a different brightness and to determine a time duration between a brightness change between the displayed images and a corresponding brightness change in the video data of the camera (3).Method for measuring a video latency in an image recording device comprising a camera (3) and a display device (2) having a computing unit (5), a display (4) and a communication interface (6), wherein the camera (3) is connected to the communication interface (6) of the display device (2), wherein the camera (3) is arranged in such a way that it captures the display (4) of the display device (2), wherein the display (4) is actuated by the computing unit (5) for the successive display of two images having a different brightness and the video data transmitted from the camera (3) to the communication interface (6) are evaluated by the computing unit (5) with respect to a brightness of the images described by the video data, wherein a time duration between a brightness change between the displayed images and a corresponding brightness change in the video data of the camera (3) is determined by the computing unit (5).Method according to Claim 9, characterized in that the determination of the time duration is started by the arithmetic unit (5) if the brightness determined from the video data falls below or exceeds a first threshold value, and the determination of the time duration is ended if the brightness determined from the video data exceeds or falls below a second threshold value.Method according to Claim 9 or 10, characterized in that the display device (2) has a control interface (13), the control interface (13) being connected to the arithmetic unit (5), the measurement operating mode of the arithmetic unit (5) being started after a start command is received at the control interface (13).Method according to one of Claims 9 to 11, characterized in that, after the time duration has been determined, information describing the determined time duration is transmitted to the control interface (13).Method according to one of Claims 9 to 12, characterized in that, after the period of time has been determined, the display (4) is actuated in order to display the period of time and / or an item of information describing the period of time determined on the display (4).

Images