Apparatus and method for checking a playback of a video sequence of a mirror replacement camera

Abstract

Device for checking a reproduction of a video sequence of a mirror replacement camera for a motor vehicle, the device (1) comprising: - a movable camera device (10) which is designed to capture an image and provide image data; - an image data processing device (20) which is coupled to the camera device (10) and which is designed to generate the video sequence based on the captured image data; - an output device (30) which is designed to display the generated video sequence; and - a diagnostic device (40) which is designed to move the camera device (10) in order to check the displayed video sequence, wherein the diagnostic device (40) is designed to carry out the checking of the displayed video sequence if eye contact of an occupant with the output device (30) has been detected.

Description

Technical area

[0001] The present invention relates to systems for exterior and interior mirror replacement.

[0002] In particular, the present invention relates to an apparatus and a method for checking a reproduction of a video sequence of a mirror replacement camera for a motor vehicle. Technical background

[0003] Exterior and interior mirrors are increasingly being replaced by camera-monitor systems. Such mirror replacement systems comprise a camera and a monitor and replace, for example, exterior and interior mirrors in motor vehicles.

[0004] In addition to advantages in aerodynamics and the resulting reduced fuel consumption, there are also technical improvements that are only possible through the use of cameras.

[0005] This allows images from multiple cameras to be merged or combined to create an all-round view system, eliminating disruptive distortions or image influences and preventing the driver from being blinded.

[0006] To replace the conventional mirrors and thus achieve a nearly 100% functional system, advanced camera and display functions must be developed to make the system fail-safe. Known problems that don't affect the conventional mirror, but are all the more likely to affect the mirror-replacement system, include image freezing during display due to errors in the image processing system.

[0007] When driving along a road, the driver should be able to recognize fairly quickly whether such a fault is present, but if the background changes only slowly or not at all, such as when a car is parking or leaving a parking space, a static image may also be a fault in the mirror replacement system.

[0008] For example, the driver looks at the mirror monitor and sees that there is no danger and drives out of the gap. However, the image may have been stationary at that moment.

[0009] DE 10 2015 111 773 A1 describes a method for displaying a camera video with an indicator on a display, wherein the indicator on the display is static in response to the camera video being frozen and the indicator is dynamic when the camera video is live.

[0010] DE 10 2009 023 435 A1 describes a method for detecting a playback state of a camera system, wherein a moving control graphic is coupled into the image flow of the camera system.

[0011] Document US 6801249 B1 describes a device for outputting digital video images, comprising an image capture unit for generating the images and a unit for forming at least one signal in the field of view of the image capture unit, such that the signal appears in at least some of the images generated by the image capture unit. The signal is shaped to vary over time, making it possible to detect any freezing of the image by the device.

[0012] Document US 2013 / 188032 A1 relates to a method for modulating the operation of a device, comprising providing an attentive user interface for obtaining information about a user's attention state and modulating the operation of a device based on the obtained information, wherein the modulated operation is initiated by the device. The information about the user's attention state is the user's eye contact with the device, which is detected by the attentive user interface.

[0013] A method for operating a driver assistance system of a motor vehicle is known from US 9393917 B2. A first image sequence of a first rearward surrounding area of ​​the motor vehicle is provided by a first camera system of an electronic rearview mirror, and the first image sequence is displayed on a display device of the electronic rearview mirror, which is arranged in an interior of the motor vehicle. Furthermore, a second image sequence of a second surrounding area of ​​the motor vehicle, which at least partially overlaps with the first surrounding area, is provided by a second camera system of an environment detection device. The electronic rearview mirror is then checked for the presence of a malfunction by a computing device of the driver assistance system, and if the malfunction is present, the second image sequence is displayed on the display device of the electronic rearview mirror.

[0014] US 2016 / 119586 A1 discloses a rear view camera system that provides a view to the rear when the driver's view directly out of the rear window is obstructed due to the load. Using one or more sensors, the rear view camera system detects whether the driver's line of sight to the rear of the vehicle is blocked. If the view is obstructed, the rear view camera records the traffic behind the vehicle live and displays the image in front of the driver.

[0015] Document WO 2015 / 062602 A1 relates to a driver assistance system for a motor vehicle, comprising a camera, a control and evaluation unit and a display, wherein the position of the camera relative to the motor vehicle can be adjusted by means of an adjustment mechanism.

[0016] Document DE 600 25 864 T2 describes a multifunctional vehicle camera system comprising a camera device with several cameras mounted on a vehicle, with images captured by the cameras corresponding to the left and right door mirror fields of view. The cameras are each designed to be movable, and the image data captured by the cameras is processed using an image processing device and displayed to the vehicle driver via monitors.

[0017] Document JP 2004 328 246 A relates to the provision of an on-board video display device which selectively displays image data from an imaging means mounted on a vehicle and which allows easy adjustment of additional settings of the imaging means.

[0018] US 2016 / 140403 A1 discloses providing multidirectional assistance when maneuvering a vehicle. The vehicle includes a side camera that can be moved between multiple viewing positions. It is determined whether the current viewing position of the side camera matches the current vehicle direction of movement. If it is determined that the current viewing position of the side camera does not match the current vehicle direction of movement, the viewing position of the side camera is automatically adjusted so that the viewing position of the side camera matches the current vehicle direction of movement. Summary of the invention

[0019] It is an object of the present invention to provide an improved camera function for a driver assistance system.

[0020] This problem is solved by the subject matter of the independent patent claims. Further developments and embodiments can be found in the dependent patent claims, the description, and the figures of the drawings.

[0021] A first aspect of the present invention relates to a device for checking a reproduction of a video sequence of a mirror replacement camera for a motor vehicle.

[0022] The device comprises a movable camera device configured to capture an image and provide image data; furthermore, an image data processing device coupled to the camera device and configured to generate the video sequence based on the captured image data; furthermore, an output device configured to display the generated video sequence; and furthermore, a diagnostic device configured to move the camera device to check the displayed video sequence.

[0023] The present invention proposes, for example, a technical system and a method for detecting the freezing of video images in camera monitor systems for a motor vehicle.

[0024] The present invention proposes, for example, an automatic and self-repeating test.

[0025] The present invention proposes, for example, to identify still or frozen images of the mirror replacement system by means of camera or image movements and by image comparison.

[0026] In addition, the procedure provides an opportunity to alert the driver to hazards.

[0027] The present invention is intended for mirror replacement systems where an increase in functional safety is particularly necessary.

[0028] The proposed procedures and methods make it possible to reduce the probability of failure of the mirror replacement system.

[0029] If errors occur due to freezing images, the system can now heal itself or diagnose an error, and the driver can be given confidence in the technology and the new system.

[0030] This leads to an improvement in the acceptance of digital systems for mirror replacement applications.

[0031] The present invention contributes to increasing the safety of imaging camera systems, which are becoming increasingly important in automobiles. The present invention is transferable to other imaging systems in vehicles, such as radar or ultrasound-based imaging sensor systems.

[0032] The present invention offers solutions to increase the safety of cameras or mirror replacement systems. Furthermore, the present invention can prevent accidents caused by technical systems, indicate to the driver or passengers that there is (or is not) a fault, and create confidence in safe systems.

[0033] Advantageous embodiments of the present invention can be found in the subclaims.

[0034] According to the invention, the diagnostic device is designed to check the displayed video sequence if eye contact of an occupant with the output device has been detected.

[0035] In an advantageous embodiment of the present invention, the diagnostic device is configured to perform the review of the displayed video sequence if a predetermined trigger event has been detected. The trigger event can be a driving situation, a driving condition, a driver action, or a periodic system check.

[0036] In an advantageous embodiment of the present invention, it is provided that the diagnostic device is designed to repeatedly check the displayed video sequence within predetermined time periods.

[0037] In an advantageous embodiment of the present invention, it is provided that the diagnostic device is designed to check the displayed video sequence based on prevailing driving situations and / or vehicle conditions.

[0038] In an advantageous embodiment of the present invention, the movable camera device is configured to move the camera's field of view in order to change the captured image. In other words, the camera device is pivoted, for example, by a movement through a predetermined angular range.

[0039] The described designs and further training courses can be combined as desired.

[0040] Further possible embodiments, further developments and implementations of the present invention also include combinations of features of the present invention described previously or below with regard to the embodiments that are not explicitly mentioned.

[0041] The accompanying drawings are intended to provide a further understanding of the embodiments of the present invention.

[0042] The accompanying drawings illustrate embodiments and, in conjunction with the description, serve to explain concepts of the present invention.

[0043] Other embodiments and many of the aforementioned advantages will become apparent with reference to the figures of the drawings. The elements illustrated in the figures of the drawings are not necessarily drawn to scale. Short description of the characters

[0044] They show: Fig. 1: a schematic representation of the overall system with camera, abbreviated MRS, for “Mirror Replacement System” to explain the present invention; Fig. 2: a schematic representation of a mirror replacement camera for explaining the present invention; Fig. 3: a schematic representation of a timeline to illustrate the temporal sequence of two images taken one after the other according to an embodiment of the present invention; Fig. 4: a schematic representation of an exemplary modification of the camera lens with a different field of view according to another embodiment of the present invention; Fig. 5: a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to a further embodiment of the present invention; Fig. 6: a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to a further embodiment of the present invention; Fig. 7: a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to a further embodiment of the present invention; Fig. 8: a schematic representation of a possible mounting of the mirror replacement camera according to a further embodiment of the present invention; Fig. 9: a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to a further embodiment of the present invention; Fig. 10: a schematic representation of a device for testing a playback of a video sequence of a mirror replacement camera according to a further embodiment of the present invention; and Fig. 11: a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to a further embodiment of the present invention. Detailed description of the implementation examples

[0045] In the figures of the drawings, the same reference symbols designate the same or functionally equivalent elements, parts, components or process steps, unless otherwise stated.

[0046] The motor vehicle or vehicle is, for example, a motor vehicle or a hybrid vehicle, for example a hybrid vehicle with a sailing function, for example a motorcycle, a bus or a truck or a bicycle.

[0047] Device for checking a playback of a video sequence of a mirror replacement camera can be used for other camera and sensor-based systems.

[0048] In principle, the method can also be transferred to a radar or lidar sensor and the device can be used to check the playback of a video sequence.

[0049] The sensor as such does not have to be made movable, since it can also be intended that the vehicle itself is used as a movable object or that a part of the vehicle - door, electric window, tail light, indicator light - is used as a movable object, depending on the extent of the image recorded.

[0050] The Fig. 1 shows a schematic representation of a description of the overall system with camera, a mirror replacement system, abbreviated MRS, for English “Mirror Replacement System” to explain the present invention.

[0051] The Fig. 1 shows a camera or a camera device which is mounted in a vehicle and which has a specific field of view.

[0052] Furthermore, an image data processing device, MRS, and a display in the form of an output device for displaying the image data or the video sequence are shown.

[0053] The Fig. 2 shows a schematic representation of a mirror replacement camera for explaining the present invention.

[0054] The Fig. 2 shows an example view of a mirror camera.

[0055] In most cases, a part or a partial area of ​​the vehicle is visible - or such an area should be visible so that the driver can better orientate himself as to what the camera is pointing at.

[0056] The procedure shown describes a system: i) which captures different frames and compares them over time; and / or ii) which automatically aligns or moves the camera and compares the images of the

[0057] The present invention proposes a mechanism that allows the camera to be moved during operation and on-screen display to check the display and the entire system. Furthermore, multiple cameras can be controlled accordingly, and test movements can be performed with these cameras.

[0058] The camera or lens is moved left or right, for example, to capture a different scene. This brings new objects into focus, even in a static scene, meaning there is no movement from the vehicle or other objects in the captured scene.

[0059] This means that an image is taken that is different from the previous image taken before the movement.

[0060] For example, the camera can be aimed towards the vehicle body, as a background is always expected here.

[0061] This can also be pre-stored and can be determined unambiguously based on the angle of the camera - with the exception of extremely heavy fog and rain.

[0062] The Fig. 3 shows a schematic representation of a timeline to illustrate the temporal sequence of two images taken one after the other according to an embodiment of the present invention.

[0063] The Fig. Figure 3 shows a comparison of two consecutive images. This can also be a method for detecting a freeze or image loss in the image processing system.

[0064] The camera system is intended to store consecutive images over time, and these consecutively recorded images are subsequently compared with each other in the computer system using software-based image processing methods.

[0065] The Fig. 4 shows a schematic representation of an exemplary modification of the camera lens with a different field of view according to another embodiment of the present invention.

[0066] The Fig. 4 shows by way of example that when the camera is moved, a different field of view is created - one that differs from the field of view of the first image.

[0067] The Fig. 5 shows a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to another embodiment of the present invention.

[0068] The Fig. Figure 5 shows a possible difficulty in simply comparing recorded images over time.

[0069] For example, if you want to leave a parking space or parking lot, it may happen that there are few or no moving objects visible in the camera's field of view at the time the picture is taken.

[0070] The camera takes consecutive images as shown in the figure and compares these captured images.

[0071] In this case, the camera does not detect any difference - it may be a freeze of the camera monitor system, but it may also not be, ie there may be no movement of the displayed objects.

[0072] The Fig. 6 shows a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to another embodiment of the present invention.

[0073] The Fig. 7 shows possible trigger events for initiating the method for checking a playback of a video sequence of a mirror replacement camera according to a further embodiment of the present invention.

[0074] The Fig. Figure 7 shows an embodiment of the present invention in which recordings or images of certain parts or partial areas of the vehicle side are to be used to detect movements more quickly and reliably.

[0075] The Fig. 8 shows another possibility if the camera is fixed in a holder, here a mirror.

[0076] According to an embodiment of the present invention, it can be provided to move the camera via the mirror adjustment or to detect the freezing of the image processing and display process in the mirror replacement system.

[0077] The Fig. 9 shows a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera according to another embodiment of the present invention.

[0078] The Fig. Figure 9 shows a flowchart for detecting an error in the image by adjusting the camera.

[0079] The Fig. Figure 9 shows the result of the procedure. Since the camera most likely records your own vehicle, or at least parts of it, you can clearly see a difference here, for example, if a turn signal was activated. Optionally, marker points could also be attached to the vehicle.

[0080] The Fig. 10 shows a schematic representation for checking a playback of a video sequence of a mirror replacement camera for a motor vehicle according to a further embodiment of the present invention.

[0081] The device 100 for checking a playback of a video sequence of a mirror replacement camera for a motor vehicle comprises a camera device 10, an image data processing device 20, an output device 30, and a diagnostic device 40.

[0082] The movable camera device 10 is designed to capture an image and provide image data.

[0083] The image data processing device 20 is coupled to the camera device 10 and is designed to generate the video sequence based on the captured image data.

[0084] The output device 30 is designed to display the generated video sequence.

[0085] The diagnostic device 40 is designed to move the camera device 10 and / or to generate an image movement when generating the video sequence in order to check the displayed video sequence.

[0086] The Fig. 11 shows a schematic representation of a flowchart of a method for checking a playback of a video sequence of a mirror replacement camera for a motor vehicle according to a further embodiment of the present invention.

[0087] The method for testing a playback of a video sequence from a mirror replacement camera for a motor vehicle comprises the following method steps: As a first method step, an image is captured S1 and image data is provided by means of a movable camera device 10. As a second process step, the video sequence S2 is generated based on the acquired image data. As a third process step, the generated video sequence is displayed S3. As a fourth method step, a movement S4 of the camera device 10 and / or a generation of an image movement takes place when generating the video sequence in order to check the displayed video sequence.

[0088] Although the present invention has been described above using preferred embodiments, it is not limited thereto but can be modified in a variety of ways. In particular, the invention can be changed or modified in a variety of ways without deviating from the essence of the invention.

[0089] It should also be noted that “comprehensive” and “comprising” do not exclude other elements or steps, and “a” or “an” does not exclude a plurality.

[0090] Furthermore, it should be noted that features or steps described with reference to one of the above embodiments may also be used in combination with other features or steps of other embodiments described above. Reference symbols in the claims are not to be considered as limiting.

Claims

[1] Device for testing a playback of a video sequence from a mirror replacement camera for a motor vehicle, the device (1) comprising: - a movable camera device (10) designed to capture an image and provide image data; - an image data processing device (20) which is coupled to the camera device (10) and which is designed to generate the video sequence based on the captured image data; - an output device (30) designed to display the generated video sequence; and - a diagnostic device (40) which is designed to move the camera device (10) to check the displayed video sequence, wherein the diagnostic device (40) is designed to perform the check of the displayed video sequence if eye contact of an occupant with the output device (30) has been detected. [2] Device according to claim 1, wherein the diagnostic device (40) is configured to repeatedly check the displayed video sequence within predetermined time intervals. [3] Device according to claim 1 or 2, wherein the diagnostic device (40) is configured to perform the verification of the displayed video sequence based on prevailing driving situations and / or based on prevailing vehicle conditions. [4] Device according to one of the preceding claims, wherein the movable camera device (10) is configured to perform a movement of the camera's field of view in order to change the recorded image. [5] Mirror monitor system comprising a mirror replacement camera and a device according to any of the preceding claims. [6] Method for testing a reproduction of a video sequence from a mirror replacement camera for a motor vehicle, the method comprising the following steps: - Capturing (S1) an image and providing image data using a movable camera device (10); - Generating (S2) the video sequence based on the captured image data; - Display (S3) of the generated video sequence; and - Moving (S4) the camera device (10) to check the displayed video sequence, checking the displayed video sequence if eye contact of an occupant with the output device (30) has been detected.

Images