Functional error detection for switching state recognition in optical signaling systems

Abstract

The present disclosure relates to a method for checking the functional plausibility of a camera arrangement by a controller, the camera arrangement having at least two cameras for determining switching states of a light signal system. The method comprises determining a first switching state of the light signal system based on measurement data from a first camera, determining a second switching state of the light signal system based on measurement data from at least one second camera, and using the first switching state and the second switching state to check the functional plausibility of the camera arrangement. The plausibility check can, but need not, involve simultaneous detection of the switching states by both cameras; delayed detection is also possible, for example, if the light signal system moves out of the field of view of one camera during this time. The present disclosure also relates to a controller, a vehicle, a computer program, and a machine-readable storage medium.

Description

Technical Field

[0001] The present invention relates to a method for checking the plausibility of the operation of a camera system with at least two cameras by a control unit and a vehicle for determining the switching state of a light signaling system. The invention also relates to a control unit, a computer program, and a machine-readable storage medium. Background Art

[0002] In the automotive sector, driver assistance functions are known that detect the switching states of traffic lights or light signal systems. The individual active signal emitters and their colors are identified by evaluating camera measurement data.

[0003] Particularly in autonomous vehicles, the reliable detection of the switching states of light signal systems is a safety-relevant function. This places particularly high demands on the reliability of components such as recognition cameras and evaluation controllers.

[0004] To increase the reliability of traffic light recognition, sensors are often used redundantly. However, this increases the cost of the camera system. Furthermore, a more power-efficient controller is required to simultaneously evaluate two cameras. This increases the cost of the camera system, the required installation space, and the energy requirements. Summary of the Invention

[0005] An object of the present invention is to provide a method and a controller for improving the reliability of a camera device.

[0006] The solution of the present invention for achieving the above-mentioned object is the subject of the independent claim. The advantageous technical solutions of the present invention are described in the subject of the dependent claims.

[0007] According to one aspect of the present invention, a method for checking the plausibility of the functionality of a camera arrangement by a controller is provided. The camera arrangement has at least two cameras for determining a switching state of a light signal system.

[0008] In one step, a first switching state of the light signal system is determined based on measurement data from a first camera.

[0009] In a further step, a second switching state of the light signal system is determined based on measurement data from at least one second camera.

[0010] Alternatively, the first switching state can also be determined based on measurement data from at least one second camera of the camera arrangement, and the second switching state can be determined based on measurement data from the first camera of the camera arrangement.

[0011] The camera device is then checked for plausibility of operation using the first switching state and the second switching state.

[0012] According to another aspect of the present invention, a controller for evaluating measurement data from at least two cameras of a camera arrangement is provided, the controller being configured to perform the above method.

[0013] According to another aspect of the present invention, a vehicle is provided having at least one camera device, wherein the camera device has at least two cameras for identifying a switching state of a light signal system and is connectable to the controller.

[0014] In addition, according to another aspect of the present invention, a computer program is provided. The computer program includes instructions that, when executed by a controller, cause the controller to perform the above-described method. According to yet another aspect of the present invention, a machine-readable storage medium is provided, on which the above-described computer program is stored.

[0015] According to the BASt definition, a vehicle can preferably be assisted, partially automated, highly automated and / or fully automated or unmanned. To this end, the vehicle can have a vehicle control system that can access environmental sensing systems and actuators to steer, accelerate and brake the vehicle.

[0016] The camera device can be used to identify the switching state of a traffic light or a light signal system. The first camera and at least one further camera can also be used to implement functions other than traffic light recognition, such as general environment recognition.

[0017] In this regard, the switching states of the light signal systems determined based on the measurement data from the respective cameras can be compared with one another. The results can be checked for logical and technical plausibility. For example, if the controller detects a contradiction, it can be assumed that a malfunction has occurred in the camera system and / or in the evaluation of the measurement data.

[0018] The cameras may preferably have different scanning areas, so that the switching state of the light signal system is determined by different cameras at different times and evaluated by the controller. For example, the cameras may have different focal lengths, so that, for example, a first camera has a larger scanning angle and a smaller usable distance than at least one second camera.

[0019] The at least two cameras may belong to different assistance systems of the vehicle and may already be installed in the vehicle.

[0020] A light signal system can have one or more signal emitters that indicate at least one switching state of the light signal system. The signal emitters can be activated continuously or flashingly. At least two cameras can determine properties of the activated signal emitters, such as color, position within the light signal system, absolute position, shape, and light duration, and assign the switching state of the light signal system.

[0021] The method described makes it possible to eliminate the redundant use of identical or very similar cameras with identical or very similar installation positions and properties (field of view and resolution), thereby reducing costs, resources and installation space requirements for traffic light recognition.

[0022] By means of plausibility checks, the method can be designed to be robust against all types of randomly occurring faults and malfunctions.

[0023] According to one embodiment, the first switching state and / or the second switching state of the light signal system is determined by determining the position of the signal transmitter in the light signal transmitter housing, the color of the signal transmitter and / or the shape of the signal transmitter.

[0024] This allows the image evaluation algorithm to determine the housing of the light signal system. Within the housing or extension of the light signal system, the color of the signal emitters and their position within the light signal system provide information about the switching state. For example, the topmost signal emitter in the housing could represent "red" or "stop," while the bottommost signal emitter in the housing could represent "green" or "go."

[0025] For example, if at least one camera detects that the lower signal transmitter is activated and subsequently detects that the uppermost signal transmitter is activated, the two signal transmitters can be checked for plausibility with respect to one another. Due to the common housing, the signal transmitters can be assigned to the light signal system.

[0026] In public transportation, such as trams and buses, the shape of the activated signal emitter can also define the switching state of the light signal system. For example, signal emitters in the shape of a bar, circle, or cross can be detected and assigned a switching state. Furthermore, arrow-shaped signal emitters can be detected and assigned a switching state based on their shape.

[0027] The first switching state and the second switching state may be the same or different. The second switching state is detected by the second camera with a time delay, so that the light signal system may change its switching state during this period, thereby activating a different signal transmitter.

[0028] In this case, the result is evaluated as logical or reasonable if, for example, the results of the camera measurements determine the same position and the same color at different times.

[0029] According to another embodiment, a malfunction of the camera arrangement is detected if, based on the measurement data from the first camera and the second camera, at least one signal emitter is determined to have the same position but different colors. For example, a malfunction of at least one camera can be detected if the colors of the signal emitters assigned or detected within the light signal system or within the light signal system housing differ from camera to camera.

[0030] Furthermore, a malfunction can be detected if two cameras simultaneously detect the same traffic light, but with different colors of the activated light signal emitters. If the traffic light can be measured simultaneously in the field of view of both cameras, the cameras detect the same traffic light. The position of the light signal emitter is irrelevant for detecting a malfunction.

[0031] According to another embodiment, a malfunction of the camera arrangement is detected if, based on the measurement data from the first camera and the second camera, it is determined that at least one signal emitter has a different position but the same color. In this case, inconsistencies can similarly be detected during the determination of the measurement data from the cameras or during the evaluation of the measurement data.

[0032] In these situations, there is a high probability that traffic light recognition or the camera system used will malfunction. To increase road safety, this function can be deactivated and the vehicle can be placed in a safe state or manually controlled via the controller.

[0033] According to another embodiment, the position of at least one signal transmitter relative to the light signal system or relative to the camera device is determined. If the position is set relative to the position or shape of the light signal system, the positioning or position determination of the activated signal transmitter can be achieved in a particularly simple technical manner.

[0034] Alternatively or additionally, the activated signal transmitter can be positioned absolutely within the scanning area or relative to the camera device. This allows the position of the signal transmitter within the light signal system housing to be determined and assessed even in darkness or low visibility. In this case, the corresponding signal transmitter must be assigned to the common light signal system.

[0035] For example, a position may have specific coordinates or may be specified as a relative position, such as "top," "middle," or "bottom."

[0036] According to another embodiment, the controller receives the position of at least one signal emitter of the light signal system and the corresponding color of the respective signal emitter from a database.

[0037] This allows the corresponding possible locations of the signal transmitter to be retrieved from a database, which can be used, for example, to create precise maps for highly automated driving.

[0038] According to another embodiment, the first camera is designed as a long-range camera and the at least one second camera is designed as a short-range camera. In this case, the measurement data from the second camera is recorded at a later time than the measurement data from the first camera.

[0039] In this way, the remote camera can first detect the optical signal system, and the controller can calculate the switching state based on its measurement data.

[0040] If the vehicle moves towards the light signal system, the light signal system will gradually exit the scanning range of the long-range camera and can be detected by the short-range camera, for example, within a distance of 70m between the vehicle and the light signal system.

[0041] This camera arrangement allows cameras to sequentially monitor the light signal system. This reduces the computational workload of the controller when evaluating the measurement data, as redundant camera arrangements are no longer necessary for each measurement area. Furthermore, the use of remote cameras allows for early detection of the switching state of the light signal system, thereby improving road traffic comfort and safety.

[0042] According to a further embodiment, the camera device is moved relative to the light signal system, wherein the position of at least one signal emitter of the light signal system relative to the camera device is tracked based on the measurement data from the first camera.

[0043] When determining the time-varying measurement data, the signal emitters of the light signal system detected in the scanning area of ​​the first camera may have moved. The measurement data can preferably be designed as moving images or video data. If the light signal system is arranged over a traffic situation, the detected signal emitters may move toward the upper area of ​​the image.

[0044] When the camera device moves in a direction approaching a lateral boundary of the scanning area, the light signal system arranged on one side of the road and detected by the first camera will move in the measurement data.

[0045] The movement of the light signal system in the measurement data occurs until the signal transmitter is outside the scanning area and is therefore no longer detected by the first camera.

[0046] The movement of such a light signal system and in particular of at least one detected signal transmitter through the scanning area can be tracked or traced by a controller. The relative arrangement of at least two cameras of the camera arrangement is known in advance, thereby increasing the accuracy of the plausibility check and the reliability of traffic light recognition.

[0047] According to another embodiment, based on a check of the camera device's functional plausibility, the controller generates and transmits at least one signal and / or at least one control command. The at least one signal and / or at least one control command can be generated when the camera device's functional plausibility check is confirmed and therefore functions normally, or when the controller detects a functional error in the camera device. In particular, the at least one signal can be transmitted, for example, to a remote operator or an external server unit. The signal can take the form of an error message or a warning.

[0048] At least one control command can be used for operating or influencing the vehicle control of the vehicle. For this reason, for example, a steering command, a braking command, an acceleration command etc. can be generated by the controller and transmitted to the vehicle control system.

[0049] For example, if a malfunction of the camera system is detected, the vehicle can be placed in a safe state. Alternatively or additionally, the automatic driving mode can be deactivated due to a malfunction of the camera system, so that the vehicle can still be manually operated. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The preferred embodiment of the present invention is described in detail below with reference to a highly simplified schematic diagram. In the diagram:

[0051] Figure 1 A schematic top view of a vehicle with a camera arrangement is shown, wherein the light signal system is arranged in the scanning area of ​​the remote camera; and

[0052] Figure 2 A schematic top view of a vehicle with a camera arrangement is shown, wherein the light signal system is arranged in the scanning area of ​​the short-range camera. DETAILED DESCRIPTION

[0053] Figure 1 A schematic top view of a vehicle 1 is shown with a camera arrangement 2. The light signal system 4 is arranged in a scanning area A1 of a remote camera 6 and can thus be detected by the remote camera 6.

[0054] In particular, the light signal system 4 can be detected by the control unit 8 of the vehicle 1 based on an image evaluation of the remote camera 6. The remote camera 6 is designed as a first camera 6 of the camera arrangement 2.

[0055] The remote camera 6 has a telephoto lens (not shown), whereby the scanning area A1 available or evaluable by the controller 8 has a relatively small scanning angle (eg 20°) and a long distance (eg 200 m).

[0056] Furthermore, camera device 2 includes a second camera 10. Second camera 10 is designed as a short-range camera 10 and has a short focal length. This allows short-range camera 10 to have a scanning area A2 with a larger scanning angle than first camera 6, for example, greater than 50°. The usable range of second camera 10 is, for example, up to 80 meters.

[0057] The first camera 6 and the second camera 10 are placed parallel to or adjacent to each other in the front area of ​​the vehicle 1 and can scan the environment in front of the vehicle 1 .

[0058] The controller 8 is connected in a data-conducting manner to the first camera 6 and the second camera 10 and can thus receive, store and evaluate measurement data.

[0059] The usable distance or usable scanning angle of the cameras 6, 10 can be defined, for example, by the focus plane and the optical system or lens of the cameras 6, 10. For example, the measurement data of the objects in the scanning areas A1, A2 can be clearly scanned with high contrast, and can then be analyzed by the controller with high quality.

[0060] According to this embodiment, the signal emitter 12 of the light signal system 4 is detected by the first camera 6. The detected signal emitter 12 is a red activated or switched on signal emitter. Thereby, the controller 8 can assign the “red” switching state to the light signal system 4.

[0061] In the illustrated case, the yellow and green signal transmitters 14 of the light signal system 4 are deactivated. The signal transmitters 12 , 14 of the light signal system 4 are arranged in a housing 5 of the light signal system 4 .

[0062] The signal transmitters 12 , 14 of the light signal system 4 are designed as LED lighting devices.

[0063] By evaluating the measurement data from the first camera 6 , the signal transmitter 12 can be assigned the red color in the illustrated case.

[0064] Furthermore, by evaluating the measurement data, the position of the signal transmitter 12 can be determined by the controller 8 . This position corresponds to the upper or highest position relative to the light signal system 4 .

[0065] Figure 2 A schematic top view of a vehicle 1 with a camera arrangement 2 is shown, wherein a light signal system 4 is arranged in a scanning area A2 of a short-range camera 10. For example, a long-range camera 6 can no longer perceive the light signal system 4 because the light signal system 4 is located outside the scanning area A1 of the long-range camera 6 at a distance of approximately 70 m from the vehicle 1.

[0066] Figure 1 and Figure 2 In particular, the present invention serves to explain a method according to one embodiment for detecting a malfunction of a camera arrangement 2 having at least two cameras 6 , 10 by means of a controller 8 for determining a switching state of a light signaling system 4 .

[0067] In particular, Figure 2 The situation shown is Figure 1 The situation shown is a few seconds later, for example 5 seconds. This means that the vehicle 1 is moving in the direction of the light signal system 4. Since the vehicle's own movement is known, it is possible to "track" the fixed position light signal system. Figure 1 and Figure 2 Make a clear allocation between the moments in time.

[0068] The light signal system 4 is also in the “red” switching state. Thus, by evaluating the measurement data of the short-range camera 10 , only the active signal transmitters 12 can be determined and assigned the red color and the uppermost position within the light signal system 4 .

[0069] Controller 8 can determine that the position and color of signal transmitter 12 determined based on the measurement data from first camera 6 and the measurement data from second camera 10 correspond to different times. Therefore, the result of the traffic light (i.e., the "red" switching state) is recognized as reasonable, and a functional error can be ruled out.

[0070] If there is a switch from red to green in the scanning area A2 of the second camera 10, the lowest signal transmitter 14 is activated and detected by the second camera 10. In this way, the controller can determine the different positions and different colors of the signal transmitters 12, 14 based on the measurement data of the cameras 6, 10. It can therefore also be assumed that the camera device 2 is functioning correctly.

[0071] If, on the other hand, the colors of the signal emitters 12, 14 of the light signal system 4 match but their positions differ, the control unit 8 can detect a discrepancy in the switching state recognition of the light signal system 4. In this case, a functional error exists and the vehicle 1 function can be deactivated, for example.

[0072] Reference Signs List

[0073] 1 vehicle

[0074] 2 Camera Device

[0075] 4 Optical Signal System

[0076] 5. Housing of optical signal system

[0077] 6 First Camera

[0078] 8 Controller

[0079] 10 Second Camera

[0080] 12 Activated signal transmitters

[0081] 14 Deactivate the signal transmitter

[0082] A1 Scanning area of ​​the first camera

[0083] A2 Second camera scanning area

Claims

1. A method for checking the plausibility of the function of a camera arrangement (2) by means of a controller (8), the camera arrangement (2) having at least two cameras (6, 10) for determining the switching state of a light signal system (4), wherein: - determining a first switching state of the light signal system (4) based on measurement data from a first camera (6); - determining a second switching state of the light signal system (4) based on measurement data from at least one second camera (10); as well as - checking the functional plausibility of the camera device (2) using the first switching state and the second switching state; wherein the measurement data from the second camera is recorded at a later time than the measurement data from the first camera; wherein the first switching state and / or the second switching state of the optical signal system (4) is determined by determining the position of the signal transmitter, the color of the signal transmitter and / or the shape of the signal transmitter within the housing (5) of the optical signal system (4); A functional error of the camera device (2) is detected when it is determined based on the measurement data from the first camera (6) and the second camera (10) that at least one signal emitter (12, 14) has the same position but different colors.

2. The method according to claim 1, wherein A functional error of the camera device (2) is detected when it is determined based on the measurement data from the first camera (6) and the second camera (10) that the at least one signal transmitter (12, 14) has a different position but the same color.

3. The method according to claim 1, wherein The position of the at least one signal transmitter (12, 14) relative to the light signal system (4) or relative to the camera device (2) is determined.

4. The method according to claim 1, wherein The controller (8) receives the position of at least one signal emitter (12, 14) of the light signal system (4) and the corresponding color of the respective signal emitter (12, 14) from a database.

5. The method according to claim 1, wherein The first camera (6) is configured as a long-range camera, and the at least one second camera (10) is configured as a short-range camera.

6. The method according to claim 1, wherein The camera device (2) moves relative to the light signal system (4), wherein the position of at least one signal emitter (12, 14) of the light signal system (4) relative to the camera device (2) is tracked based on measurement data from the first camera (6).

7. The method according to claim 1, wherein Based on the check of the functional plausibility of the camera device (2), at least one signal and / or at least one control command is generated and sent by the controller (8).

8. A controller (8) for evaluating measurement data from at least two cameras (6, 10) of a camera arrangement (2), wherein The controller (8) is configured to perform the method according to the preceding claim 1.

9. A vehicle (1) having at least one camera device (2), wherein: The camera device (2) has at least two cameras (6, 10) for detecting the switching state of the light signal system (4) and can be connected to a controller (8) according to claim 8.

10. A computer program product comprising instructions which, when executed by a controller (8), cause the controller to perform the method according to any one of claims 1 to 7.

11. A machine-readable storage medium having a computer program stored thereon, wherein: The computer program comprises instructions which, when executed by a controller (8), cause the controller to carry out the method according to claim 1.

Images