A method for assigning a traffic light to a lane on a street by an assigning system, a computer program product, a non-transitory computer-readable storage

Abstract

A traffic light 12 is assigned to a lane 14 on a street 16 by using capturing device 18 of a vehicle 20 to capture the traffic light and determining a current traffic light colour C thereof. A current

Description

[0001] The present invention relates to the field of automobiles. More specifically, the present invention relates to a method for assigning a traffic light to a lane on a street by an assigning system according to the pending claim 1. Furthermore, the present invention relates to a corresponding computer program product, a corresponding non-transitory computer-readable storage medium, as well as to a corresponding assigning system. BACKGROUND INFORMATION

[0002] Autonomous vehicles require semantic understanding of the given traffic scene to navigate complex environments safely. At intersections, understanding the assignment of traffic lights to lanes is a prerequisite to determine whether to stop. This assignment information is used in safety-critical applications and currently cannot be derived by an online system with the required reliability.

[0003] The traffic light to lane assignment (TL2LA) is defined by the geometric arrangement of traffic lights relative to the lanes in an intersection and optionally by indication inlays inside the traffic light bulbs, such as arrows. Using this information to automate the map annotation in a scalable way does not reach the required level of correctness. The huge variety of geometric configurations of traffic lights and lanes, and the unreliable detection of traffic light inlays make it very challenging to assign individual traffic lights to the respective lanes precisely. Hence, the assignment is provided a priori, for example by a high-definition (HD) map, which historically involved laborious manual annotation efforts. Additionally, fully capturing the geometric layout and semantic relationships between traffic elements of an urban traffic scene may be difficult even for humans. To reduce costs and enable to distract traffic light to lane assignment in a scalable, country-independent way, automation is required. SUMMARY OF THE INVENTION

[0004] It is an object of the present invention to provide a method, a corresponding computer product, a corresponding non-transitory computer-readable storage medium, as well as a corresponding assigning system, by which traffic light to lane assignment may be performed in an improved manner.

[0005] This object is solved by a method, a corresponding computer program product, corresponding non-transitory computer-readably storage medium, as well as a corresponding assigning system according to the independent claims. Advantageous embodiments are presented in the dependent claims.

[0006] One aspect of the invention relates to a method for assigning a traffic light to a lane on a street by an assigning system. The traffic light of the street is captured by a capturing device of a vehicle. A current traffic light color is determined depending on the captured traffic light by the capturing device. A current position of the vehicle is captured by a position capturing device of the vehicle. A current lane of the vehicle is determined depending on the captured position by the position capturing device. A current velocity of the vehicle is captured by a velocity capturing device of the vehicle and depending on the current velocity, the current position and the current traffic light color assigning the captured traffic light to the current lane by the assigning system is performed.

[0007] Therefore, a method to automatically derive traffic light to lane assignment statistically based on vehicle traffic crossing on a red traffic light is performed. This approach is independent to the geometric arrangement and therefore generalizes to any country and intersection layout. Additionally, safety considerations are covered by this approach since it conservatively assumes a valid traffic light to lane assignment for each missing pair. Furthermore, the approach can be combined with a deriving traffic light to lane assignment based on the geometric arrangement for redundancy or in an integrated model.

[0008] In particular, the present invention replaces manually labeling the traffic light to lane assignment, provides extreme cost savings for data annotation, and only requires recordings of vehicles at traffic light controlled intersections. Furthermore, it generalizes to all countries and intersections. Thereby, enabling urban automated driving in a scalable and cost efficient way is performed. Furthermore, the source of recordings is secondary. However, recordings of vehicles at a traffic light controlled intersection is already available from customer fleet. Other possible sources can be for example used from drones or the infrastructure. Furthermore, it is an independent way to derive traffic light to lane assignment fully automatically without using geometric arrangement. Combining the approach with a geometric-based approach can yield the best results by either using both sources of information in an integrated model or by having redundancy. Furthermore, safety considerations are covered by minimizing false negatives and assuming a valid traffic light to lane assignment for each missing pair. The provided TL2TL according to the invention is statistically safe, in particular with a probability of a = 0.001 to incorrectly rejecting Ho, which means that the traffic light is assigned to the lane, but might sacrifice the availability of the function. This ensures to always obey a red traffic light but may result in stopping at a green traffic light when a false positive traffic light is in state red. Furthermore, the applied binomial test additionally provides a p-value that serves as a score for confidence.

[0009] In particular, the velocity, the position, and the traffic light color are captured over time and time series of these information is generated by the assigning system. The assigning system therefore is configured to use the time series for assigning the traffic light to a lane.

[0010] According to an embodiment, a plurality of current velocities, current positions and current traffic light colors is captured and determined by a plurality of vehicles, and depending on that the assigning of the traffic light to the current lane is performed. In particular time series of the plurality of information is used. These time series may be collected over multiple days to get large statistic and also from multiple cars from a fleet.

[0011] Furthermore, in another embodiment, the current position, the current velocity, and the current traffic light color are transmitted to a central electronic computing device of the assigning system and the assigning is performed by the central electronic computing device.

[0012] In another embodiment a binomial hypothesis test is used for assigning the traffic light to the lane. Alternatives to the binomial hypothesis test may be used, for example a statistical test in general, a rule-based system, a heuristic, a learning-based assignment. The filtering of inputs can be done as an optional pre-processing step.

[0013] In another embodiment the current velocity is analyzed depending on a relative position of the vehicle to a stop line of the traffic light.

[0014] According to another embodiment, the current velocity is analyzed depending on a first position before the traffic light and a first point in time and depending on a second position beyond the traffic light and a second point in time.

[0015] In particular, the present invention is a computer implemented method. Therefore another aspect of the invention relates to a computer program product comprising program code means for performing a method according to the preceding aspect.

[0016] A still further aspect of the invention relates to a non-transitory computer-readable storage medium comprising at least the computer program product according to the preceding aspect.

[0017] Furthermore, the present invention relates to an assigning system for assigning a traffic light to a lane on a street, wherein the assigning system is configured for performing a method according to the proceeding aspect. In particular, the assigning system is configured for performing a method according to the preceding aspect.

[0018] Furthermore, the assigning system comprises at least one electronic computing device, in particular a central electronic computing device.

[0019] A computing unit / electronic computing device may in particular be understood as a data processing device, which comprises processing circuitry. The computing unit can therefore in particular process data to perform computing operations. This may also include operations to perform indexed accesses to a data structure, for example a look-up table, LUT.

[0020] In particular, the computing unit may include one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits, ASIC, one or more field-programmable gate arrays, FPGA, and / or one or more systems on a chip, SoC. The computing unit may also include one or more processors, for example one or more microprocessors, one or more central processing units, CPU, one or more graphics processing units, GPU, and / or one or more signal processors, in particular one or more digital signal processors, DSP. The computing unit may also include a physical or a virtual cluster of computers or other of said units.

[0021] In various embodiments, the computing unit includes one or more hardware and / or software interfaces and / or one or more memory units.

[0022] A memory unit may be implemented as a volatile data memory, for example a dynamic random access memory, DRAM, or a static random access memory, SRAM, or as a non-volatile data memory, for example a read-only memory, ROM, a programmable read-only memory, PROM, an erasable programmable read-only memory, EPROM, an electrically erasable programmable read-only memory, EEPROM, a flash memory or flash EEPROM, a ferroelectric random access memory, FRAM, a magnetoresistive random access memory, MRAM, or a phase-change random access memory, PCRAM.

[0023] Further advantages, features, and details of the invention derive from the following description of a preferred embodiment as well as from the drawing. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figure and / or shown in the figure alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention. BRIEF DESCRIPTION OF THE DRAWING

[0024] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and / or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0025] The drawing show in:

[0026] The only Fig. 1 a schematic top view according to an embodiment of an assigning system performing an embodiment of a method.

[0027] In the figure the same elements or elements having the same function are indicated by the same reference signs. DETAILED DESCRIPTION

[0028] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0029] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawing and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0030] The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by “comprises” or “comprise” does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0031] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawing that forms part hereof, and in which is shown by way of illustration a specific embodiment in which the disclosure may be practiced. This embodiment is described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0032] Fig. 1 shows a schematic top view according to an embodiment of an assigning system 10 for assigning a traffic light 12 to a lane 14 of a street 16. The traffic light 12 is captured by a capturing device 18 of a vehicle 20. In particular, the capturing device 18 may be for example a camera. A current traffic light color c depending on the captured traffic light 12 is determined by the capturing device 18. A current position P of the vehicle 20 is captured by a position capturing device 22 of the vehicle 20. A current lane 14 of the vehicle 20 is determined depending on the captured position P by the position capturing device 22. A current velocity v of the vehicle 20 is captured by a velocity capturing device 24 of the vehicle 20. Depending on the current velocity v, the current position P and the current traffic light color c the captured traffic light 12 is assigned to the current lane 14 by the assigning system 10.

[0033] For example, the assigning system 10 is configured as a central system comprising a central electronic capturing device 26. In particular, the current position P, and the current velocity v, and the current traffic light color c are transmitted to the central electronic computing device 26 of the assigning system 10 and the assigning is performed by the central electronic computing device 26.

[0034] Furthermore, a plurality of current velocities v, current positions P and current traffic light colors c are captured and determined by a plurality of vehicles 20, and depending on that the assigning of the traffic light 12 to the current lane 14 is performed.

[0035] Furthermore, a binomial hypothesis test is used for assigning the traffic light 12 to the lane 14. Furthermore, the current velocity v is analyzed depending on a relative position of the vehicle 20 to a stop line 28 of the traffic light 12. Furthermore, the current velocity v is analyzed depending on a first position before the traffic light 12 and a first point in time and depending on a second position beyond the traffic light 12 and a second point in time.

[0036] In particular, Fig. 1 shows to get from temporal correspondences between motion patterns of the vehicle 20 and the traffic light states to the traffic light 12 the following procedure is needed as shown in Fig. 1. Based on motion patterns of the vehicle 20 a state of the traffic light 12 is captured. In particular, evidence for or against an association between the vehicle 20 and the traffic light 12 is derived. Based on geometric position P of the vehicle 20 and lane geometries assigning the vehicle 20 to its current driving lane 14 is performed. Determining the traffic light to lane assignment (TL2TL) between the traffic light 12 and the lane 14 is performed by aggregating individual evidences.

[0037] Deriving the traffic light to lane assignment from motion data and implementing the described procedure does not require a complex learning-based approach. Instead, a statistical method, in particular a so-called falsification method is proposed.

[0038] The falsification method formulates the problem as a hypothesis test. Conservatively, a valid assignment for every pair of traffic light 12 and lane 14 is assumed, representing the null hypothesis Ho. Removing the assignment is only performed, if a significant amount of vehicles has been recorded passing the intersection on a red traffic light. In particular, the hypothesis Ho describes that the traffic light 12 is assigned to the lane 14, and the hypothesis Hi describes that the traffic light 12 is not assigned to the lane 14.

[0039] The detection of a vehicle 20 passing the intersection is measured relative to the stop line 28 of the intersection. If the velocity v of a vehicle 20 directly in front of the stop line 28 is greater than a specific threshold, a pass is assumed. This is just a simple heuristic to determine whether a vehicle 20 is crossing an intersection. Another method may be to localize the vehicle 20 and a lane Io that is part of an intersection entry of the intersection I at time to and afterwards localize it on a lane h that is part of an intersection exit of the intersection I at time ti.

[0040] A binomial hypothesis test is used to falsify the null hypothesis Ho based on a significance level a. This conservative approach minimizes false negatives and improves the recall while accepting false positives and lower precision. A further advantage is that a prediction is provided for all pairs of traffic lights 12 and lanes 14. It is assumed that a valid traffic light to lane assignment exists if less than 5% of recorded vehicles 20 pass on a red light (binomial distribution with p = 0.05). To minimize the likelihood of false rejections of Ho, a significance level a = 0.001 is chosen. Reference signs 10 12 14 16 18 20 22 24 26 28 c P V assigning system traffic light lane street capturing device vehicle position capturing device velocity capturing device electronic computing device stop line color position velocity

Claims

1. A method for assigning a traffic light (12) to a lane (14) on a street (16) by an assigning system (10), comprising the steps of:- capturing the traffic light (12) of the street (16) by a capturing device (18) of a vehicle (20);- determining a current traffic light color (c) depending on the captured traffic light (12) by the capturing device (18);- capturing a current position (P) of the vehicle (20) by a position capturing device (22) of the vehicle (20);- determining a current lane (14) of the vehicle (20) depending on the captured position (P) by the position capturing device (22);- capturing a current velocity (v) of the vehicle (20) by a velocity capturing device (24) of the vehicle (20); and- depending on the current velocity (v), the current position (P) and the current traffic light color (c) assigning the captured traffic light (12) to the current lane (14) by the assigning system (10).

2. The method according to claim 1, characterized in thata plurality of current velocities (v), current positions (P) and current traffic light colors (c) is captured and determined by a plurality of vehicles (20), and depending on that the assigning of the traffic light (12) to the current lane (14) is performed.

3. The method according to claim 1 or 2, characterized in thatthe current position (P), and the current velocity (v), and the current traffic light color (c) is transmitted to a central electronic computing device (26) of the assigning system (10) and the assigning is performed by the central electronic computing device (26).

4. The method according to any one of claims 1 to 3, characterized in thata binomial hypothesis test is used for assigning the traffic light (12) to the lane (14).

5. The method according to any one of claims 1 to 4, characterized in thatthe current velocity (v) is analyzed depending on a relative position of the vehicle (20) to a stop line (28) of the traffic light (12).

6. The method according to any one of claims 1 to 5, characterized in thatthe current velocity (v) is analyzed depending on a first position before the traffic light (12) and a first point in time and depending on a second position beyond the traffic light (12) and a second point in time.

7. A computer program product comprising program code means for performing a method according to any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium comprising at least the computer program product according to claim 7.

9. An assigning system (10) for assigning a traffic light (12) to a lane (14) on a street (16), wherein the assigning system (10) is configured for performing a method according to any one of claims 1 to 6.

Images