Method and device for determining the state of a vehicle light

Abstract

The invention relates to a method for determining a state of a vehicle light of a vehicle (115). The method comprises a reading step and an analysis step. In the reading step, an image signal (130) is read, which comprises a camera image of a vehicle region of the vehicle (115) detected. In the analysis step, the image signal (130) is analyzed using an artificially learnable model (135), in particular an artificial neural network. Here, at least two state classes are distinguished, which each represent a state of the vehicle light of the vehicle (115). As an analysis result, a state class is determined, which represents a current state of the vehicle light of the vehicle (115).

Description

TECHNICAL FIELD

[0001] The invention proceeds from a device or a method of the type according to the independent claims. A computer program is also a subject matter of the invention. BACKGROUND

[0002] It is possible to recognize whether the tail lights of a vehicle driving in front are flashing. To this end, the intensity progression of the tail lights can be investigated in terms of frequency using the image signals of the tail lights of the vehicle driving in front, and the likelihood of the tail lights being activated, i.e. flashing, can be determined.

[0003] US 2017039435 A1 describes a method for recognizing a vehicle driving in front as flashing. SUMMARY

[0004] In this context, a method and a device for determining a state of a vehicle light of a vehicle according to the main claim and the training of an artificially learnable model, in particular an artificial neural network, are proposed by means of the solutions presented here. Advantageous refinements and improvements of the device specified in the independent claim can be achieved by the measures listed in the dependent claims.

[0005] By means of the solutions presented here, it is possible to recognize the state of a vehicle light of a vehicle using a rear-facing image of a vehicle driving in front or using a front-facing image of an oncoming vehicle. In addition to flashing, braking can also be recognized. The shape and the combination of the tail lights can be classified, for example (braking, warning flasher, braking and flashing, etc.). The tail lights should therefore be lit at least once for recognition. Flashing, i.e. repeatedly or periodically switching the light on / off, is not mandatorily required. Here, advantageously, individual positioning or individual tracking of the individual vehicle lights is not required. The state of the vehicle light can be recognized independently of the design and the type of light used, whereby, for example, also pulse-type LED vehicle lights can be recognized. In addition, also partially defective vehicle lights can be recognized. Advantageously, flashing vehicle lights can be recognized independently of the flashing frequency.

[0006] A method for determining a state of a vehicle light of a vehicle is proposed. The method has a reading step and an analysis step. In the reading step, an image signal is read, which comprises a camera image of a vehicle region with a detected vehicle. In the analysis step, the image signal is analyzed using an artificially learnable model, in particular an artificial neural network. Here, at least two state classes are distinguished, each representing one state of a vehicle light of the vehicle. As a result of the analysis, a state class is determined which represents the current state of the vehicle light of the vehicle.

[0007] The vehicle can be a land vehicle or a commercial vehicle or other means of transport, which can also be implemented as a self-driving vehicle with automated driving operation. The "vehicle light" can be understood, for example, as a vehicle lighting device, for example a rear light of the vehicle, i.e. a direction indicator, a brake light, a reversing light or a headlight (Scheinwerfer), for example a front headlight or a rear fog light of the vehicle. The image signal can be read via an interface with a sensor device of the vehicle. The image signal can comprise a bounding volume of a vehicle region of the vehicle The vehicle region can be, for example, a rear portion of the vehicle or a front portion of the vehicle. The artificially learnable model can be understood as a model with a learnable structure, which can be used to re-identify patterns in the read (image) signal. The artificially learnable model can also be referred to simply as a learnable model. For example, the (artificially) learnable model can be implemented as an artificial neural network or a single-layer, multi-layer or recurrent network. Alternatively, the step of analysis processing can also be carried out using other sequential machine learning methods, for example a Hidden Markov Model (HMM), a Conditional Random Field (CRF) or an arbitrary recurrent neural network (RNN). These learning methods or models can also be understood as artificially learnable models or training of artificially learnable models. The at least two state classes, each representing a state of a vehicle light of the vehicle, can represent, for example, the operating state of the vehicle light (for example on or off) and, in addition or alternatively, the type of vehicle light, i.e. the recognition of the light type. The current state of the vehicle light can be determined, for example "left flasher on" or "rear light on" or "reversing light off".

[0008] According to an embodiment, at least two further state classes, each representing a state of a further vehicle light of the vehicle, can be distinguished in the step of analysis processing. In this case, a further state class representing the current state of the further vehicle light of the vehicle can be determined as a result of the analysis processing. This is advantageous for determining the current state of a plurality of vehicle lights by means of the method. Thereby, for example, the current state of all vehicle lights (for example all rear lights) visible in the vehicle region can be determined, whereby different light states, which can also include a plurality of vehicle lights, can be distinguished. It is therefore advantageously possible to distinguish between a braking flash or an emergency braking flash by means of the method.

[0009] According to an embodiment, in the step of reading, an image signal can be read, which has at least two image partial signals. The at least two image partial signals can each contain different color information of the image signal. The step of analysis processing can be carried out using the at least two image partial signals. As different color information, for example different image regions, different color channels of the same image region or a color histogram of the image region with a subdivision of the image region can be contained.

[0010] Furthermore, according to an embodiment, in the step of reading at least two image partial signals can be read, which respectively represent a vehicle region in a different spectral range. This is advantageous with regard to the clear contrast of the vehicle lights. Additionally or alternatively, in the step of reading at least two image partial signals can represent different vehicle partial regions of the detected vehicle. As vehicle partial region, for example a color histogram of an image region of the vehicle silhouette or of a vehicle bounding box can be used. This is advantageous with regard to the recognition of spatial information of the image signals.

[0011] According to an embodiment, the step of analyzing can be implemented using a model of a learnable feedback, in particular a feedback neural network as artificial neural network. For this, the model of a learnable feedback can for example have a long short-term memory, so-called "long short-term memory" (LSTM), or have a "gated recurrent unit" (GRU).

[0012] According to an embodiment, in the step of reading a detection signal can also be read. The detection signal can represent a first recognition of the vehicle. In the step of analyzing, the model of a learnable feedback can be brought into a predetermined state in response to the detection signal. The model of a learnable feedback can be brought into a basic state, for example into an initial state after successful training of the model of a learnable artificial. This advantageously prevents values of a previous method for ascertaining the light state of a vehicle from influencing a subsequent method.

[0013] Furthermore, according to an embodiment, in the step of reading the image signal can comprise a camera image with a vehicle region of a further detected vehicle. Here, in the step of analyzing at least two state classes can be distinguished, which respectively represent one state of the lights of the further vehicle. A state class representing a current state of the lights of the further vehicle can then be ascertained as a result of the analysis. Thereby, it is advantageously possible to ascertain the state of the lights of a plurality of vehicles in order to make a situation assessment, for example in a traffic situation with a plurality of traffic participants. This can be advantageous in particular for applications in vehicles with automated driving operation.

[0014] According to an embodiment, if a further vehicle is detected, the step of analyzing can be implemented using a further model of a learnable artificial, in particular a further artificial neural network, in order to ascertain the state of the lights of the further vehicle. For example, one model of a learnable artificial or neural network can be initialized for each detected vehicle represented in the image signal, respectively, in order to ascertain the state of the lights of the detected vehicle.

[0015] According to an embodiment, the analysis result can also be determined in the step of analysis processing using the function argumentum maximi. This is advantageous for determining the confidence of the determined state of the vehicle light.

[0016] According to an embodiment, the method can also comprise a step of outputting an analysis signal. The analysis signal can represent the analysis result. Furthermore, the analysis signal can be output as a visual signal and additionally or alternatively as an acoustic signal. The analysis signal can be provided, for example, via an interface to a display device of the vehicle. In order to display the current state of the vehicle light of the vehicle, for example, a camera image of the detected vehicle or a bounding box of the vehicle can be displayed on the display device of the vehicle, and the state of the vehicle light can be displayed, for example, in color and symbolically, for example by means of a colored arrow on the position of the activated vehicle light of the displayed vehicle, in order to visualize, for example, a flashing process or a braking process identified by means of the determined state of the vehicle light. Advantageously, the analysis result can thus be provided to the driver of the vehicle clearly and time-savingly.

[0017] By means of this solution, a method for learning an artificially learnable model, in particular an artificial neural network, in order to analyze an image signal is also proposed. The method comprises a step of reading and a step of training. In the step of reading, an image signal and a target signal are read. The image signal comprises a camera image of a vehicle region with a detected vehicle. The target signal represents a state class of a vehicle light of the vehicle. Here, the state class corresponds to a state of the vehicle light reflected in the image signal. In the step of training, the artificially learnable model, in particular the artificial neural network, is trained such that the artificially learnable model identifies the state class in response to the image signal. Here, the artificially learnable model can be trained in order to distinguish between at least two state classes each representing one state of the vehicle light of the vehicle.

[0018] The method proposed here can be implemented, for example, in software or hardware or in a mixture of software and hardware, for example in a control device.

[0019] The solution proposed here also implements a device which is configured for carrying out, controlling or implementing the steps of a variant of the method proposed here in a corresponding apparatus. By means of the inventive implementation variant in the form of a device, the task on which the present invention is based can be solved quickly and efficiently.

[0020] To this end, the device can have at least one computing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor or to an actuator for reading sensor signals from the sensor or for outputting data signals or control signals to the actuator and / or at least one communication interface for reading or outputting data, which is embedded in a communication protocol. The computing unit can be, for example, a signal processor, a microcontroller or the like, wherein the storage unit can be a flash memory, an EPROM or a magnetic storage unit. The communication interface can be configured for reading or outputting data wirelessly or wired, wherein the communication interface, which can read or output data wired, can read data from or output data into a corresponding data transmission line, for example, electrically or optically.

[0021] The device can be understood here as an electrical device which processes sensor signals and outputs control signals and / or data signals in accordance therewith. The device can have an interface which can be configured in a hardware manner and / or in a software manner. In a hardware configuration, the interface can be, for example, a part of a so-called system ASIC which comprises the most different functions of the device. It is also possible, however, that the interface is a separate integrated circuit or at least partially consists of discrete components. In a software configuration, the interface can be a software module which, for example, coexists with other software modules on a microcontroller.

[0022] A computer program product or computer program having program code which can be stored on a machine-readable carrier or storage medium, such as a semiconductor memory, a hard disk memory or an optical memory, and which is used, in particular, for implementing, carrying out and / or controlling the steps of a method according to one of the previously described embodiments when the program product or program is executed on a computer or device, is also advantageous. BRIEF DESCRIPTION OF DRAWINGS

[0023] Embodiments of the solution presented here are shown in the attached drawings and explained in detail in the following description.

[0024] Figure 1 A schematic diagram of a vehicle having a device for determining a state of a vehicle light is shown according to one embodiment;

[0025] Figure 2 A flowchart of a method for determining a state of a vehicle light is shown according to one embodiment;

[0026] Figure 3 A flowchart of a method for determining a state of a plurality of vehicle lights is shown according to one embodiment;

[0027] Figure 4 and Figure 5a schematic diagram of a method for determining a state of a vehicle light according to one embodiment;

[0028] Figures 6 to 8 a schematic diagram of a display of a state of a vehicle light of a vehicle according to one embodiment;

[0029] Figure 9 and Figure 10 a schematic diagram of an artificially learnable model according to one embodiment;

[0030] Figure 11 a flow chart of a method for learning an artificially learnable model for analyzing an image signal according to one embodiment. DETAILED DESCRIPTION

[0031] In the following description of advantageous embodiments of the application, identical or similar elements shown in the different figures and acting identically are provided with identical or similar reference signs, wherein repeated description of these elements is omitted.

[0032] Figure 1 A schematic diagram of a first vehicle 100 with a device 105 for determining a state of a vehicle light 110 of a second vehicle 115 is shown according to one embodiment. The device 105 comprises a reading means 120 and an analyzing means 125. The reading means 120 is configured to read an image signal 130. The image signal 130 comprises a camera image with a vehicle region of a detected vehicle, here the second vehicle 115. The analyzing means 125 is configured to analyze the image signal 130 using an artificially learnable model 135. In the further course of the description, for reasons of simplifying the exposition, the application will be described by way of example using an artificial neural network as an artificially learnable model, wherein the person skilled in the art will undoubtedly also recognize from the previous embodiments that other artificially learnable models, such as hidden Markov models, etc., can alternatively or additionally also be used for the artificial neural network. Thus, for the sake of simplicity, the artificial neural network is understood or described as a representative or synonym for the artificially learnable model 135, wherein the structure of the respective model as an artificial neural network or as another design form of an artificially learnable model is secondary for the description and understanding of the application presented here.

[0033] Here, at least two state classes are distinguished, which each represent one state of the vehicle light 110 of the detected vehicle 115. The state class representing the current state of the vehicle light 110 of the detected vehicle 115 is determined as an analysis result.

[0034] According to the embodiment shown here, the evaluation result is provided in the form of an evaluation signal 140. The image signal 130 is exemplarily provided by a surrounding detection device 145 of the vehicle 100. The detected vehicle light 110 of the vehicle 115 is exemplarily embodied as a tail light. By means of the device 105 shown here, it is advantageously possible, for example, to ascertain the state of different light types. It is thereby possible, for example, to identify not only a flashing of a vehicle 115 driving in front, but also a braking process or an emergency braking. This is possible because the device 105 shown here is sufficient for identifying commercially customary flashing lights without an intensity consideration of the vehicle light by means of a subsequent Fourier transformation and without a frequency analysis. Here, it is not necessary to precisely locate the position of the vehicle light 110 and to individually track the individual vehicle lights 110. In the case of the use of the image signal 130, it is possible to ascertain the state of the vehicle light 110 independently of the design and the light type used, for example, a pulsed light-emitting diode tail light. Furthermore, the device 105 is also configured to identify a partially defective vehicle light 110 or a rapidly flashing flashing light as a vehicle light 110 in order to ascertain the current state of the detected vehicle light 110 of the vehicle 115. The device 105 can be used during the day and at night. Furthermore, it is not necessary to buffer previous camera images or image blocks of the image signal 130, for example, in order to be able to calculate a difference image, which is cost-saving, in particular in the case of the use of an application-specific integrated circuit for the device 105.

[0035] Figure 2 According to one embodiment, a flow chart of a method 200 for ascertaining a vehicle light state of a vehicle is shown. The method 200 has at least a reading step 205 and an evaluation step 210. In the reading step 205, an image signal is read, which comprises a camera image having a vehicle region of a detected vehicle. In the evaluation step 210, the image signal is evaluated using an artificial neural network. Here, at least two state classes are distinguished, which each represent one state of a vehicle light of a vehicle. A state class representing a current state of a vehicle light of a vehicle is ascertained as an evaluation result.

[0036] According to one embodiment, the method 200 further comprises an outputting step 215 of an evaluation signal. The evaluation signal represents the evaluation result of the evaluation step 210. The evaluation signal is outputted as a visual signal and, alternatively or additionally, as an audible signal. This is exemplarily referred to below Figures 6 to 8

[0037] Figure 3 ​According to one embodiment a flow chart of a method 200 for determining a state of a vehicle light of a plurality of vehicles 115, 305, 310 is shown. Herein a flow of a classification of vehicle lights of a plurality of vehicles 115, 305, 310 is shown. The method 200 is performed in sub-steps in order to determine a state of a vehicle light of a plurality of vehicles 115, 305, 310. According to one embodiment, in a reading step 205, the image signal comprises a camera image 315 having a vehicle region of a detected vehicle 115 and a vehicle region of a detected further vehicle 305. Herein, the camera image 315 exemplarily comprises a vehicle region of a detected third vehicle 310. In order to analyze the image signal having the detected three vehicles 115, 305, 310, in an analyzing step 210, for each detected vehicle 115, 305, 310, at least two state categories are distinguished which respectively represent one state of a vehicle light of the further vehicle 305, 310. The state category representing the current state of the vehicle light of the further vehicle 305, 310 is determined as an analysis result.

[0038] According to the embodiment shown herein, the analyzing step 210 is performed using a further artificial neural network for each further detected vehicle 305, 310. For this purpose, the step 210 is performed in sub-steps for each detected vehicle 115, 305, 310. Herein, the camera image 315 is exemplarily provided as a bounding box of the vehicle 115, 305, 310. For each detected vehicle 115, 305, 310 and thus for each bounding box of the vehicle 115, 305, 310, an artificial neural network is initialized respectively. In the case of three vehicles 115, 305, 310 shown herein exemplarily, three artificial neural networks are running in parallel, wherein one vehicle is considered by one artificial neural network respectively in order to determine the state of the vehicle light. Herein, the maximum number of detected vehicles can be arbitrarily determined in correspondence to the number of artificial neural networks.

[0039] Further, herein, after determining the state of the vehicle light of the vehicle 115, 305, 310, an optional step 215 of outputting an analysis result signal is performed. Herein, the analysis result of the sub-step of the analyzing step 210 is outputted in three sub-steps respectively.

[0040] Figure 4 According to one embodiment a schematic diagram of a method 200 for determining a state of a vehicle light of a vehicle 115 is shown. It is shown that the state of the vehicle light of the vehicle 115 is determined based on an image region of a camera image 315. Herein, the camera image 315 exemplarily shows a vehicle tail of a detected vehicle 115, for example.

[0041] According to the embodiment shown here, the step of the analysis process is performed in the case of using the feedback neural network 405 as artificial neural network. Here, the artificial neural network comprises at least one long short-term memory layer, a "long short-term memory layer" (LSTM layer), as feedback neural element.

[0042] Furthermore, according to the embodiment shown here, in the step of reading, a detection signal 410 is read. The detection signal 410 represents a first identification of the vehicle 115. Thus, it is checked whether the vehicle 115 is detected for the first time. Then, in the step of the analysis process, the feedback neural network 405 is brought into a predetermined state in response to the detection signal 410. For this purpose, the feedback neural network 405 is reset into a basic state. The feedback neural network 405 is brought into an initial state, for example, after a successful training of the artificial neural network.

[0043] According to the embodiment shown here, in the step of reading, the image signal 130 has at least two image partial signals 415. The at least two image partial signals 415 each contain different color information of the image signal 130. In this case, the step of the analysis process is performed in the case of using the at least two image partial signals 415.

[0044] The at least two image partial signals 415 each represent a vehicle region of the vehicle 115 in a different spectral range. Here, the image signal 130 has exemplarily three image partial signals 415, which each represent a different color channel of the vehicle region. In addition or alternatively, the at least two image partial signals 415 represent different vehicle partial regions of the detected vehicle 115, as shown according to the following Figure 5 .

[0045] In the step of the analysis process, at least two state categories 420 are distinguished which each represent a state of a vehicle light of the vehicle 115. According to one embodiment, at least two further state categories 420 are distinguished which each represent a state of a further vehicle light of the vehicle. In this case, the state categories 420 which each represent the current state of the vehicle light or of the further vehicle light are determined as the result of the analysis process. According to the embodiment shown here, the detected vehicle 115 has, for example, six different vehicle light types: left flasher, right flasher, brake light, reversing light, tail light and fog light. In the step of the analysis process, the six different vehicle light types are each distinguished. In addition, the operating state of the vehicle light (vehicle light on or off) is each distinguished. Thus, twelve state categories 420 are distinguished which each represent a state of a vehicle light of the vehicle 115: left flasher on, left flasher off, right flasher on, right flasher off, brake light on, brake light off, reversing light on, reversing light off, tail light on, tail light off, fog light on, fog light off. The result of the analysis process can comprise, for example, the state categories 420 of all vehicle lights. Advantageously, it is possible with the aid of the method 200 shown here and thus with the aid of the unique method 200 to achieve a classification of the state of all tail lights of the vehicle 115. Thus, it is possible with the aid of the method 200 to distinguish between different tail light states, for example, flashing, braking or emergency braking. The output of the result of the analysis process accordingly comprises the state of all vehicle lights of the vehicle 115, for example, the state of all tail lights, in the detected vehicle region of the vehicle 115.

[0046] According to one embodiment, the classification result of the vehicle light of the feedback neural network 405 is determined by argmax, i.e. in the case of use of the function argumentum maximi, whereby for each determined state of the state categories 420 a confidence is determined which is optionally output together with the result of the analysis process.

[0047] Figure 5 According to one embodiment, a schematic diagram of a method 200 for determining the state of a vehicle light of a vehicle is shown. The flow of the method 200 shown here is similar to the embodiment described above with reference to Figure 4 the embodiment described above, with the camera image 315 of the detected vehicle 115, the detection signal 410, the feedback neural network 405 and the twelve state categories of the vehicle light of the vehicle 115.

[0048] However, according to the embodiment shown here, the at least two image portion signals 415 read together with the image signal 130 represent different vehicle portion regions 505 of the detected vehicle 115 here. For this purpose, the vehicle region of the vehicle 115 detected by means of the camera image 315 is divided into a plurality of image regions. Here, the vehicle region is exemplarily divided into three columns corresponding to the light positions (right, middle, left) of the vehicle 115. Here, the three vehicle portion regions are represented by means of three color histograms 510 in three different colors, respectively. As input for the feedback neural network 405, nine color histograms 510 are read accordingly. The image regions of the vehicle contour or the bounding box of the vehicle 115, the vehicle portion regions 505, are the basis for the color histograms 510, which are divided into a plurality of image portions, here exemplarily three vehicle portion regions 505, for the purpose of spatial information of the vehicle region.

[0049] Figure 6 A schematic diagram showing the display of the state of the vehicle light 605 of the vehicle 115 is shown according to one embodiment. The view of the interior space of the first vehicle is shown from the perspective of the driver of the first vehicle towards the detected vehicle 115. In the interior space of the first vehicle, a display device 610 is arranged on which the camera image 315 of the image signal is exemplarily displayed and the detection of the vehicle tail of the vehicle 115 is displayed as a bounding box.

[0050] According to the embodiment shown here, the analysis processing result of the method for ascertaining the state of the vehicle light 605 of the vehicle 115 is output in the form of an analysis processing signal 615. The analysis processing signal 615 represents the analysis processing result and is output as a visual signal on the display device 610 here. In addition or alternatively, the analysis processing signal 615 can also be output as an audible signal. In the embodiment shown here, the detected vehicle 115 is braking, so that exemplarily three vehicle lights 615 of the vehicle 115 are active. In addition to the flickering of the vehicle light 605 being recognizable on the display device 610, the braking of the vehicle 115 is visually signaled by means of the analysis processing signal 615, here exemplarily by means of a red arrow pointing downwards on the bounding box of the vehicle tail of the vehicle displayed on the display device 610.

[0051] Figure 7 A schematic diagram showing the display of the state of the vehicle light 605 of the vehicle 115 is shown according to one embodiment. The embodiment shown here is similar to the one according to Figure 6The described embodiment, which has another state of the vehicle light 605 of the detected vehicle 115, which here is not braking but flashing on the left. This is here exemplarily shown by displaying the analysis processing signal in the form of the camera image 315 with the bounding box of the vehicle region of the detected vehicle on the display device 610. In order to visualize the flashing process, an orange arrow pointing in the direction of the flashing is here exemplarily shown at the bounding box of the vehicle tail.

[0052] Figure 8 A schematic diagram of a state display of a vehicle light 605 of a vehicle 115 is shown according to an embodiment. The embodiment shown here is similar to the one described according to Figure 6 and Figure 7 The described embodiment, in which the detected vehicle 115 here is flashing on the right. This is here exemplarily shown by displaying the analysis processing signal 615 in the form of the camera image 315 with the bounding box of the vehicle region of the detected vehicle on the display device 610, together with an orange arrow pointing right corresponding to the direction of the flashing, in order to visualize the ascertained state of the vehicle light 605 of the detected vehicle 115.

[0053] Figure 9 A schematic diagram of the manually learnable model 135 is shown according to an embodiment. An exemplary architecture for distinguishing four state categories is shown here. As input for the manually learnable model 135 or neural network, it is here assumed as described according to Figure 4 The Convolutional Neural Network, also called convolutional neural network, is shown here. The artificial neural network as a learnable model 135 here has a convolutional layer 905 with filters 16, kernel size 6x6 and stride 2. The convolutional layer 905 is associated with a "batch normalization" 910, the output of which is processed in another convolutional layer 915, which is associated with another "batch normalization" 920, followed by a max-pooling layer 925 with a size (pool size) of 2x2 and a stride of 2. The max-pooling layer 925 has a Dropout of 0.5 and is associated with a feedback element 930, an LSTM layer 10 with a Dropout of 0.6 and a Dropout of 0.25, followed by two fully connected layers 935 and 940, Dense-Layers 10 and 4, wherein a Dropout of 0.8 occurs between the layers 935 and 940.

[0054] Figure 10A schematic diagram of an artificially learnable model 135 in the form of a neural network is shown according to one embodiment. An exemplary architecture for distinguishing four state categories is shown here. The inputs to the artificial neural network 135, as shown in... Figure 4 As described, it is assumed here that the image patch is an image signal. An artificial neural network 135 based on the Inception concept is shown. Here, the artificial neural network 135 has a first convolutional layer 1005 with a filter 32, a kernel size of 3×3, and a stride of 2. This first convolutional layer is associated with a second convolutional layer 1010 with a filter 32, a kernel size of 2×2, and a stride of 1. The output of the second convolutional layer 1010 is processed in the next layer, which has a third convolutional layer 1015, a fourth convolutional layer 1020, and a fifth convolutional layer 1025. The third convolutional layer 1015 has a filter 64, a kernel size of 1×1, and a stride of 2, while the fourth convolutional layer 1020 has a filter 64, a kernel size of 3×3, and a stride of 2, and the fifth convolutional layer 1025 has a filter 64, a kernel size of 5×5, and a stride of 2. Next is a max-pooling layer 1030 with a pooling size of 2×2, a stride of 2, and Dropout of 0.6. Max pooling layer 1030 is associated with batch normalization 1035. A similar structure with additional layers, namely a sixth convolutional layer 1040, a seventh convolutional layer 1045, and an eighth convolutional layer 1050, is then repeated. This structure is associated with another max pooling layer 1055 having a pool size of 2×2 and a stride of 2. The sixth convolutional layer 1040 has a filter 128, a kernel size of 3×3, and a stride of 2; the seventh convolutional layer 1045 has a filter 128, a kernel size of 1×1, and a stride of 2; and the eighth convolutional layer 1050 has a filter 128, a kernel size of 5×5, and a stride of 2. The max pooling layer 1050 is associated with the feedback element 1060 and the LSTM layer 16 with a dropout of 0.6 and a dropout of 0.4, followed by two fully connected layers 1065 and 1070, dense layer 10 and dense layer 4, wherein a dropout of 0.7 occurs between layers 1065 and 1070.

[0055] Figure 11 A flowchart of a method 1100 for learning a learnable model for analyzing and processing image signals is shown according to one embodiment. Method 1100 includes a reading step 1105 and a training step 1110. In the reading step 1105, an image signal and a target signal are read. The image signal includes a camera image of a vehicle area having detected vehicles. The target signal represents the state category of the vehicle's lights. Here, the state category corresponds to the state of the lights reflected in the image signal. In the training step 1110, the learnable model is trained such that the learnable model identifies the state category in response to the image signal.

[0056] If an embodiment comprises "and / or" between a first feature and a second feature, this can be interpreted either as the embodiment comprising only the first feature, or only the second feature, or both the first feature and the second feature.

Claims

1. A method (200) for determining a state of a vehicle light (605) of a vehicle (115), wherein The method (200) has the following steps: reading (205) an image signal (130) comprising a camera image (315) having a vehicle region of a detected vehicle (115); and analyzing (210) the image signal (130) using an artificially learnable model (135), wherein at least two state classes (420) are distinguished which each represent one state of a vehicle light (605) of the vehicle (115), wherein the artificially learnable model (135) is trained such that it identifies the state classes (420) in response to the image signal (130) and thereby determines a state class (420) representing a current state of the vehicle light (605) of the vehicle (115) as an analysis result, wherein it is not necessary to precisely locate the position of the vehicle light (605) and to individually track individual vehicle lights, wherein: (a) in the reading step (205), the image signal (130) is read, which has at least two image partial signals (415) which each contain different color information of the image signal (130), wherein the analyzing step (210) is performed using the at least two image partial signals (415), wherein, as different color information, different image regions, different color channels of the same image region or a color histogram of an image region with a subdivision of the image region are contained, and / or (b) a feedback-learnable model (405) is used as the artificially learnable model (135), and in the reading step (205), a detection signal (410) is read which represents a first identification of the vehicle (115), wherein the feedback-learnable model (405) is brought into a predetermined state in response to the detection signal (410) in the analyzing step (210).

2. The method (200) of claim 1, wherein In the analyzing step (210), at least two further state classes (420) are distinguished which each represent one state of a further vehicle light (605) of the vehicle (115), wherein a further state class (420) representing a current state of the further vehicle light (605) of the vehicle (115) is determined as an analysis result.

3. The method of claim 1 or 2, wherein, In the reading step (205), the at least two image partial signals (415) are read which each represent a vehicle region in a different spectral range, and / or wherein, in the reading step (205), the at least two image partial signals (415) each represent different vehicle partial regions (505) of the detected vehicle (115).

4. The method (200) of claim 1 or 2, wherein, The analyzing step (210) is performed using a feedback-learnable model (405) as the artificially learnable model (135).

5. The method (200) of claim 1 or 2, wherein, In the step of reading (205), the image signal (130) comprises a camera image (315) having a vehicle region of a detected further vehicle (305, 310), wherein, in the step of analysis processing (210), at least two state classes (420) are distinguished which respectively represent one state of a vehicle light (605) of the further vehicle (305, 310), wherein a state class (420) representing a current state of the vehicle light (605) of the further vehicle (305, 310) is determined as an analysis processing result.

6. The method (200) of claim 5, wherein, The step of analysis processing (210) is carried out using a further artificially learnable model.

7. The method (200) of claim 1 or 2, wherein, In the step of analysis processing (210), the analysis processing result is determined using the function argumentum maximi.

8. The method (200) according to claim 1 or 2, further having the step (215) of outputting an analysis processing signal (615) representative of the analysis processing result, wherein, The analysis processing signal (615) is output as a visual signal and / or an acoustic signal.

9. The method (200) of claim 1, wherein, The artificially learnable model (135) is an artificial neural network.

10. The method (200) of claim 4, wherein, The feedback-capable learning model (405) is a feedback neural network.

11. The method (200) of claim 6, wherein, The further artificially learnable model is a further artificial neural network.

12. A method (1100) for learning an artificially learnable model (135) for analyzing a processing (210) of an image signal (130), wherein The method (1100) has at least the following steps: reading (1105) an image signal (130) and a target signal, the image signal comprising a camera image (315) having a vehicle region of a detected vehicle (115), the target signal representing a state class (420) of a vehicle light (605) of the vehicle (115), the state class corresponding to a state of the vehicle light (605) reflected in the image signal (130), and training (1110) the artificially learnable model (135) such that the artificially learnable model (135) recognizes the state class (420) in response to the image signal (130), wherein: (a) in the step of reading (205), the image signal (130) is read, the image signal having at least two image partial signals (415) which respectively contain different color information of the image signal (130), wherein, as different color information, different image regions, different color channels of the same image region or a color histogram of an image region with a subdivision of the image region are contained, and / or (b) a feedback-capable learning model (405) is used as the artificially learnable model (135), and, in the step of reading (205), a detection signal (410) is read, the detection signal representing a first recognition of the vehicle (115), wherein the feedback-capable learning model (405) is brought into a predetermined state in response to the detection signal (410).

13. The method (1100) of claim 12, wherein, The artificially learnable model (135) is an artificial neural network.

14. The method (1100) of claim 12, wherein, The feedback-capable learning model (405) is a feedback neural network.

15. A device (105) for determining a state of a vehicle light (605) of a vehicle (115), which is configured to implement and / or control the steps of the method (200) according to any one of claims 1 to 11 or the steps of the method (1100) according to any one of claims 12 to 14 in a respective unit (120, 125).

16. A computer program product configured to perform and / or control the method (200) according to any one of claims 1 to 11 or the method (1100) according to any one of claims 12 to 14.

17. A machine-readable storage medium having stored thereon the computer program product according to claim 16.

Images