METHOD FOR OPERATING AN OPERATING DEVICE OF A MOTOR VEHICLE

Description

[0001] The invention relates to a method for operating an operating device of a motor vehicle having the features of the preamble of patent claim 1.

[0002] A method for operating an operating device of a motor vehicle is known from WO 2021 / 010201 A1. Specifically, a motor vehicle with a lighting device and a camera located in the motor vehicle for object detection is described. In addition to the lighting device, the motor vehicle also has a radiation source for irradiating an area to be monitored by the camera. Furthermore, an adjustment unit for the radiation source is provided, which is coordinated with the switch-on cycles of the lighting device of the motor vehicle. In the time intervals in which the lighting device is switched off, the additional light source emits short light pulses. The light pulses are emitted depending on the images recorded by the camera while the lighting device of the motor vehicle was switched on. The additional light pulses are intended to create images of recorded objects (e.g.This allows the camera to better capture objects (e.g., highway signs, objects on the road, white lanes) because the additional light pulses are matched to the color of the objects. Furthermore, close coordination between the camera's exposure time and the brightness of the additional light source compensates for areas of the images captured by the camera that would normally be over- or underexposed without an additional light source.

[0003] DE 10 2017 204 836 A1, which has the features of the preamble of claim 1, describes a method for detecting objects in the surroundings of a vehicle. The method comprises the following steps: capturing images of the surroundings of the vehicle using a camera arranged on the vehicle, evaluating image information from the captured images to determine the position of at least one object relative to the vehicle. In order to improve the quality of the capture, several different lighting situations are realized one after the other when capturing the images by controlling headlights arranged on the vehicle, and thus several images are captured one after the other under the different lighting situations. When evaluating the image information, the lighting situation realized for each image is taken into account. Finally, the position data of the object is output.It is also proposed to use two cameras and to take several images consecutively under the different lighting situations in order to increase the number of images that can be evaluated.

[0004] DE 10 2009 026571 A1 discloses a method for vehicle-supported illumination of traffic environments. Specifically, light is generated by at least one light source, which is irradiated by a vehicle into a traffic environment with an instantaneous intensity suitable for dazzling the human eye. Critical objects in the traffic environment are detected, and when the light is generated, the time-averaged intensity is reduced to avoid dazzling the human eye if a critical object is detected. The light is irradiated into the traffic environment intermittently according to ON and OFF periods. During an OFF period, no light is irradiated into the traffic environment by the light source, while during an ON period, the light is irradiated into essentially the same traffic environment.

[0005] The present invention is based on the object of providing an alternative method for operating an operating device of a motor vehicle, with which the detection of objects in the vicinity of the motor vehicle can be improved. This object is achieved by a method having the features of patent claim 1. Advantageous embodiments and further developments of the invention can be found in the dependent claims.

[0006] The invention is initially based on a method for operating an operating device of a motor vehicle, which has multiple radiation sources for emitting electromagnetic radiation in and / or against a forward direction of travel. Furthermore, at least one camera is provided in or on the motor vehicle for detecting objects in the vicinity of the motor vehicle. The objects to be detected are illuminated by the radiation from said radiation sources.

[0007] It is further proposed that the recording of images by the camera for detecting objects be coordinated with the emission of electromagnetic radiation by the radiation sources in such a way that at least during the recording of an image by the at least one camera only a part of the radiation sources remains switched on.

[0008] In other words, at least one of these radiation sources is switched off while the camera is taking a picture.

[0009] This creates a fundamental prerequisite for the camera to better capture an object. The invention is based on the insight that the presence of multiple radiation sources emitting radiation toward an object impairs clear shadow formation and thus also good contrast formation. However, good contrast formation is very important for the camera to capture an object. By reducing the number of radiation sources when capturing an image, the contrast between the resulting shadows and the object to be detected can be noticeably increased by reducing partial shadows.

[0010] According to a further development of the invention, it is proposed that, while the camera is capturing an image, at least one of the radiation sources is left switched on whose electromagnetic wave spectrum can be detected by the at least one camera. Furthermore, at least one radiation source is switched off whose wave spectrum is the same as that of the radiation source that is left switched on.

[0011] In this way, shadow formation can be reduced regardless of whether the vehicle's radiation sources operate in the visible wavelength spectrum (for example, as light sources of lighting devices) or in the non-visible range of the wavelength spectrum (for example, as infrared radiation sources).

[0012] Furthermore, the radiation sources are components of a left and a right headlight of the motor vehicle. The radiation sources emit electromagnetic radiation in the visible wavelength spectrum, with the left and right headlights each having at least one such radiation source. Furthermore, the radiation sources are controlled with a time offset from one another such that when the radiation source of the left headlight is switched on, the radiation source of the right headlight is switched off, and vice versa. According to the invention, at least two cameras are provided which are synchronized with the switch-on times of the headlights such that one of the cameras is switched on when the radiation source of the left headlight is switched on and is then switched off when the radiation source of the right headlight is switched on and the other of the cameras is switched on, and vice versa.

[0013] Such features of the method make it possible to use the existing front lighting of the motor vehicle to easily reduce the shadow formation on an object to be recognized and thus improve its recognizability.

[0014] In particular, it is very useful to have one camera on the right and another on the left, as seen in the forward direction of travel. The right-hand camera is switched on when the light source of the left headlight is switched on and the light source of the right headlight is switched off. The left-hand camera, on the other hand, is switched on when the light source of the right headlight is switched on and the light source of the left headlight is switched off.

[0015] This design allows the camera's view of the object to be captured and the shadow cast by the object to be optimized for image recognition.

[0016] In this context, it's perfectly sufficient and even advantageous to have just one camera, synchronized with the headlights' activation times so that it always captures an image when only one of the headlights is on. It's important that the camera captures the object being captured from a different angle than the headlights or their light sources. However, having two cameras is more advantageous, because the greater the angle difference between the illumination direction and the recording direction, the larger the resulting shadow.

[0017] Another development of the method according to the invention proposes that the radiation sources be part of the headlights and / or taillights of the motor vehicle and are controlled via so-called pulse-width modulation (PWM). Images are captured by the at least one camera only when a switch-on signal for a radiation source is present during a pulse-width modulation cycle.

[0018] However, this process requires that the PWM cycles of the vehicle's exterior lighting be synchronized with each other in a specific way. Specifically, it is conceivable that the synchronization occurs in such a way that the pulse-width modulation cycle lengths are the same for all radiation sources. Additionally, the switch-on signals of one headlight must be staggered in time from the switch-on signals of the other headlight. If the switch-on signals of both headlights do not overlap in time, for example, whenever the left headlight is switched on, the right headlight is always switched off, and vice versa.However, if the switch-on signals overlap, it must be ensured that during each (synchronized) PWM cycle there is at least one such point in time in which either only one headlight or only the other headlight is switched on.

[0019] A camera, which is also synchronized with the headlights' PWM cycle, always receives a recording signal when a switch-on signal is present for only one of the headlights. This means that an image is only recorded when only one light source, or only one of the headlights, is switched on.

[0020] The same procedure can be followed for the rear lights of the vehicle.

[0021] The method according to the invention can also be further developed in that, alternatively or additionally, the radiation sources and the at least one camera are operated in the infrared wave spectrum of the electromagnetic radiation.

[0022] This allows the method to be extended to a frequently used frequency range for object detection.

[0023] For improved object detection, it is also advantageous to have an object detection algorithm that evaluates which radiation source was switched on at the time a camera took a picture and the relative position of the radiation source to the camera. For example, the object detection algorithm can recognize that if an object is illuminated by a radiation source located on the right side of the vehicle and recorded by a camera located on the left side of the vehicle, the shadow must be formed on the left side of the object.

[0024] This can facilitate the evaluation and plausibility check of the captured image.

[0025] Yet another development of the invention proposes that the radiation sources are components of headlights and that a recording by the at least one camera is carried out alternately for those cases in which only at least one radiation source of the left headlight, only at least one radiation source of the right headlight and radiation waves of both headlights are switched on.

[0026] This allows the amount of different information contained in the captured images to be optimized for an object recognition algorithm. This further increases the chances of correctly detecting an object.

[0027] Additionally or alternatively, according to another embodiment of the invention, it is also conceivable that the radiation sources are components of rear lights and that a recording by the at least one camera (then arranged at the rear) is carried out alternately in those cases in which only at least one radiation source of the left rear light, only at least one radiation source of the right rear light and radiation sources of both rear lights are switched on.

[0028] Analogous to the previously described further development, the information rate for rear-end object detection for an object detection algorithm can also be improved here.

[0029] Preferred embodiments of the invention are illustrated in the figures and explained in more detail in the following description with reference to the figures. This also makes further features and advantages of the invention clear. The same reference numerals, even in different figures, refer to the same, comparable, or functionally identical components. Corresponding or comparable properties and advantages are achieved even if a repeated description or reference to them is not made. The figures are not, or at least not always, to scale. In some figures, proportions or distances may be exaggerated in order to more clearly emphasize features of an embodiment. If the term "and / or" is used in a list of two or more terms or objects, this may mean that any of the listed terms or objects can be used alone.It can also mean that any combination of two or more of the listed terms or objects can be used.

[0030] They show, schematically Fig. 1 shows a motor vehicle prepared for carrying out the method from above, Fig. 2 shows a signal flow plan to explain the components involved in the method, Fig. 3 shows a signal diagram to explain an embodiment of the method, Fig. 4 shows an operating state of a motor vehicle operating according to the method according to the invention and Fig. 5 shows a motor vehicle operating according to a conventional method.

[0031] In the Fig. 1 A motor vehicle K is shown from above, which is prepared for carrying out the method according to the invention. The motor vehicle K has a right headlight 1r and a left headlight 1l. Furthermore, a right taillight 2r and a left taillight 2l are visible.

[0032] The headlights 1l, 1r emit electromagnetic radiation (light) in a forward direction of travel F and thus illuminate the area in front of the motor vehicle K. The rear lights 2l, 2r, on the other hand, emit light opposite to the forward direction of travel F and thus illuminate, at least to a certain extent, the area behind the motor vehicle K.

[0033] Furthermore, a rear light 2m is located in the center of the upper area of ​​a rear window 8. The rear light 2m is also referred to as the "third brake light."

[0034] In order to detect objects in the area in front of the motor vehicle K, a camera 3r arranged on the right and a camera 3l arranged on the left are provided in the interior of the motor vehicle K, namely in the upper area on a windscreen 7.

[0035] The cameras 3r, 3l are preferably designed as CCD cameras. Alternatively or additionally, the cameras 3r, 3l can also be designed as infrared cameras.

[0036] To support infrared image recognition, motor vehicle K has a right infrared radiation source 4r and a left infrared radiation source 4l in the area of ​​its front end. The infrared radiation sources 4r and 4l illuminate the area in front of motor vehicle K with infrared radiation, thus facilitating the capture of images of the surroundings by infrared cameras, even in very poor lighting conditions.

[0037] To capture the surroundings behind the motor vehicle K, a rear-mounted camera 3h is provided, which is also arranged in the upper area of ​​the rear window 8. The rear-mounted camera 3h is also preferably designed as a CCD camera, but can also be designed, analogous to the cameras 3l, 3r, additionally or alternatively as an infrared camera.

[0038] Images captured by cameras 3r, 3l and / or camera 3h can be displayed on a display surface of an instrument cluster 9. Alternatively or additionally, the display of the images on another display device, for example on a touchscreen arranged in the area of ​​a center console (not shown), is also conceivable.

[0039] In Fig. 2 A signal flow diagram of components of the motor vehicle K which are involved in the method according to the invention is now visible.

[0040] A control device 6 can be seen, which has a memory unit 6a, an evaluation and logic unit 6b, and a control unit 6c. Furthermore, the headlights 1r, 1l and the taillights 2r, 2l are shown. The headlights 1l, 1r and the taillights 2r, 2l have radiation sources 10, which are designed as lighting devices. The radiation sources 10 can be, for example, light-emitting diodes. Furthermore, the cameras 3r, 3l, and 3h can be seen. All of the aforementioned components, including the instrument cluster 9, are connected for signaling purposes to a data bus 11, which can be designed, for example, as a CAN bus.

[0041] Signals generated by the cameras 3l, 3r and / or 3h are forwarded to the control device 6. There, the signals are processed into images in the evaluation and logic unit 6b, which also includes an image processing unit, and the image data are fed to a display unit of the instrument cluster 9 via the control unit 6c.

[0042] The memory unit 6a contains, among other things, an object recognition algorithm which the evaluation and logic unit 6b accesses during its evaluation and processing.

[0043] In the present embodiment, the headlights 1l, 1r and also the taillights 2l, 2r, and 2m are controlled using the known PWM (pulse width modulation) method. The brightness of the light emitted by the radiation sources 10 can be varied via the pulse width and pulse frequency of the signal control.

[0044] The PWM process is well known and therefore does not need to be explained further here.

[0045] In Fig. 3 A possible embodiment of the method according to the invention is shown by way of example in a signal diagram. A recording signal Al for the left camera 3l, a recording signal Ar for the right camera 3r, and, at the very bottom, a recording signal A for the rear camera 3h are plotted against time t.

[0046] Furthermore, a PWM signal PWM1l for the left headlight 1l and a PWM signal PWM1r for the right headlight 1r can be seen in between.

[0047] Finally, the PWM cycles Z are numbered. It is clear from the PWM cycles Z that they are synchronized for the two headlights 1l and 1r. In other words, the PWM cycles Z are the same length for both the left headlight 1l and the right headlight 1r. For the sake of completeness, it should be mentioned that the PWM cycles of the rear lights 2l, 2r, and 2m are also synchronized with each other (not shown).

[0048] It is now essential that the control of the headlights 1l, 1r is precisely coordinated with the control of the cameras 3l and 3r.

[0049] As can be seen from the figure, the PWM control of headlights 1l, 1r is initially carried out in such a way that in each PWM cycle Z, there is at least one point in time at which only the right headlight 1r receives a PWM+ switch-on signal or only the left headlight 1l receives a PWM+ switch-on signal. If a PWM- switch-off signal is present, headlights 1l, 1r do not emit any light.

[0050] This could also be realized if the switch-on signals PWM+ of both headlights 1l, 1r would temporarily overlap (indicated by dashed lines).

[0051] The control of the cameras 3l and 3r is now coordinated with the PWM control of the headlights 1l, 1r in such a way that whenever a PWM+ switch-on signal is present at only one of the headlights 1l, 1r (i.e. only one of the headlights 1l, 1r emits light), the control unit 6c of the control device 6 (compare Fig. 2 ) a recording signal Al for the left camera 3l or a recording signal Ar for the right camera 3r is also generated (compare, for example, times t1 and t2).

[0052] When an object is photographed by one of the cameras 3l, 3r, only a part of the spotlights 1l, 1r, i.e. only one spotlight, is switched on.

[0053] In particular, in the present embodiment, the control is carried out in such a way that when a PWM+ switch-on signal is present at the right headlight 1r, a recording signal Al is generated for the left camera 3l. However, if a PWM+ switch-on signal is present at the left headlight 1l, a recording signal Ar is applied to the right camera 3r, which then records an image.

[0054] Since at the moment of a recording signal Al or Ar the amount of radiation sources, here the spotlights 1l, 1r, is reduced, the shadow formation of the objects illuminated by the spotlights 1l, 1r and to be recorded can be intensified and thus a contrast between shadow and the object to be recorded can be increased.

[0055] Analogously, it is conceivable that the PWM+ switch-on signals represent switch-on signals for the rear lights 2l, 2r. Since only a single rear camera 3h is present in the exemplary embodiment, the number of recording signals A used to control the rear camera 3h increases accordingly. However, here too, a recording signal A is only ever generated when only one of the rear lights (2l or 2r) is switched on (see, for example, times t3 and t4).

[0056] The resulting effect should be shown schematically using the Fig. 4 will be explained in more detail. Thus, in this figure, only one camera 3 is present. This camera has a detection area E and is used to detect an object O that appears in front of the motor vehicle K' in a forward direction of travel F.

[0057] The motor vehicle K' has a left headlight 1l and a right headlight 1r, each equipped with at least one radiation device 10. The radiation devices 10 are preferably designed as light-emitting diodes and emit electromagnetic radiation ES in the visible range.

[0058] Motor vehicle K' also operates according to the method according to the invention. Therefore, camera 3 only captures an image when a PWM+ switch-on signal is present at the left headlight 1l (shown in the image) or the right headlight 1r (not shown in the image), causing it to emit light.

[0059] In this way, at the moment of recording by camera 3, only a single shadow S is created. This shadow is very dark and thus stands out very clearly in contrast to the object O to be detected. This leads to a significant improvement in object recognition by the motor vehicle K'.

[0060] A similar, advantageous effect in shadow formation would also be achieved if the radiation sources 10 were replaced or supplemented by radiation sources in the infrared range and the camera 3 were replaced or supplemented by an infrared camera.

[0061] In contrast, in a conventional procedure in a motor vehicle K" (cf. Fig. 5), in which a picture is taken by the camera 3 when both headlights 1l, 1r are switched on, due to the electromagnetic radiation ES of both headlights 1l, 1r, a core shadow KS located behind the object O and penumbra HS located to the side of the object O are created. However, the penumbra HS have a reduced contrast with respect to the object, so that the detection of the object O is made more difficult. List of reference symbols

[0062] 1l, 1rHeadlight left, right 2l, 2rTaillight left, right 2mTaillight center, third brake light 3Camera 3hRear camera 3l, 3rCamera left, right 4l, 4rInfrared radiation source left, right 5Mirror base 6Control unit 6aStorage unit 6bEvaluation and logic unit 6cControl unit 7Windscreen 8Rear window 9Instrument cluster 10Radiation sources, lamps 11Data bus ARecording signal for rear camera AlRecording signal for the left camera ArRecording signal for the right camera EDetection range ESElectromagnetic radiation FForward direction of travel HSPartial shadow KSCorbit shadow K, K', K"Motor vehicle OBject PWM11PWM signal for the left headlight PWM1rPWM signal for the right headlight PWM+Switch-on signal PWM-Switch-off signal SShadow tTime t1-t4Time points ZPWM cycles

Claims

1. Method for operating an operating device of a motor vehicle (K, K'), comprising a plurality of radiation sources (4l, 4r; 10) for emitting electromagnetic radiation (ES) in and / or against a forward direction of travel (F) and comprising at least one camera (3; 3l, 3r, 3h) for detecting objects (O) in the surroundings of the motor vehicle (K, K') which are illuminated by the radiation (ES) from the radiation sources (4l, 4r; 10), the recording of images by the camera (3; 3l, 3r, 3h) for detecting objects (O) being coordinated with the emission of electromagnetic radiation (ES) by the radiation sources (4l, 4r; 10) in such a way that at least during the recording of an image by the at least one camera (3; 3l, 3r, 3h) only some of the radiation sources (4l, 4r; 10) remain switched on, the radiation sources (10) being part of a left and a right headlight (1l, 1r) of the motor vehicle (K, K') and emitting electromagnetic radiation (ES) in the visible wave spectrum, the left and the right headlight (1l, 1r) each having at least one such radiation source (10) and the radiation sources (10) being controlled with a time offset from one another such that when the radiation source (10) of the left headlight (1l) is switched on, the radiation source (10) of the right headlight (1r) is switched off and vice versa, characterized in that at least two cameras (3l, 3r) are present which are synchronized with the switch-on times of the headlights (1l, 1r) such that one (3r) of the cameras (3l, 3r) is switched on when the radiation source (10) of the left headlight (1l) is switched on and is then switched off when the radiation source (10) of the right headlight (1r) is switched on and the other (3l) of the cameras (3l, 3r) is switched on and vice versa.

2. Method according to claim 1, characterized in that during the recording of an image by the camera (3; 3l, 3r, 3h) at least one radiation source (4l, 4r; 10) of the radiation sources (4l, 4r; 10) of which the electromagnetic wave spectrum can be detected by the at least one camera (3; 3l, 3r, 3h) is left switched on, and at least one radiation source (4l, 4r; 10) of which the wave spectrum is the same as that of the radiation source (4l, 4r; 10) left switched on is switched off.

3. Method according to claim 1 or 2, characterized in that viewed in the forward direction of travel (F), one camera (3r) is arranged on the right and another camera (3l) is arranged on the left, the camera (3r) arranged on the right being switched on when the radiation source (10) of the left headlight (1l) is switched on and the radiation source (10) of the right headlight (1r) is switched off, and the camera (3l) arranged on the left being switched on when the radiation source (10) of the right headlight (1r) is switched on and the radiation source (10) of the left headlight (1l) is switched off.

4. Method according to any of the preceding claims, characterized in that the radiation sources (10) are part of headlights (1l, 1r) and / or tail lights (2l, 2r) of the motor vehicle (K) and are controlled via a pulse width modulation (PWM1l, PWM1r), the recording of images by the at least one camera (3l, 3r) only taking place when a switch-on signal (PWM+) for a radiation source (4r, 4l; 10) is present in a cycle (Z) of the pulse width modulation (PWM11, PWM1r).

5. Method according to any of the preceding claims, characterized in that the radiation sources (4r, 4l) and the at least one camera (3l, 3r) are operated in the infrared wave spectrum of the electromagnetic radiation (ES).

6. Method according to any of the preceding claims, characterized in that an object recognition algorithm is present which evaluates which radiation source (4l, 4r; 10) was switched on at the time of a recording by a camera (3l, 3r; 3h, 3) and which relative position the radiation source (4l, 4r; 10) has to the camera (3l, 3r, 3h, 3).

7. Method according to any of the preceding claims, characterized in that the radiation sources (10) are part of headlights (1l, 1r) and a recording by the at least one camera (3l, 3r) takes place alternately for cases in which only at least one radiation source (10) of the left headlight (1l), only at least one radiation source (10) of the right headlight (1r) and radiation sources (10) of both headlights (1l, 1r) are switched on.

8. Method according to any of the preceding claims, characterized in that the radiation sources (10) are part of tail lights (2l, 2r) and a recording by the at least one camera (3h) takes place alternately for cases in which only at least one radiation source (10) of the left tail light (2l), only at least one radiation source (10) of the right tail light (2r) and radiation sources (10) of both tail lights (2l, 2r) are switched on.