Method, traffic monitoring system and computer program product

Abstract

The invention relates to a method (100) for operating a traffic monitoring system (10) for recording a traffic scene (11) on a roadway (12), at least one vehicle (13) being able to travel along the roadway (12), the method (100) comprising the following steps: – reading in (110) the traffic scene (11) on the roadway (12) as lidar data (14) by means of at least one lidar sensor (15) of at least one optical capture unit (16), – recognizing (120) predefined check points (17) from surroundings data (18) in the captured lidar data (14) by means of a processing unit (19), the check points (17) being positioned at defined locations along and / or on the roadway (12) in the surroundings data (18), – converting (130) the check points (17) to plane data (20) of a plane (21) by means of the processing unit (19), the plane (21) corresponding to the roadway (12), – converting (140) the captured lidar data (14) and the plane data (20), using a transformation algorithm, to bird's eye view data (22) of a bird's eye perspective for recording the traffic scene (11) on the roadway (12) by means of the processing unit (19), – evaluating (150) the bird's eye view data (22) of the recording of the traffic scene (11) on a roadway (12) to form evaluation data by means of an evaluation unit (23) for operating a traffic monitoring system (10).

Description

[0001] Procedure, traffic monitoring system and computer program product

[0002] Description

[0003] The invention relates to a method having the features of independent patent claim 1, a traffic monitoring system having the features of independent patent claim 12 and a computer program product having the features of independent patent claim 15.

[0004] Excessive vehicle speed is a major problem when it comes to enforcing traffic safety laws. Another cause of accidents on motorways and expressways is driving too close to the vehicle in front. Road traffic regulations in European countries (Netherlands, Germany, and Austria) and the USA have different laws and the resulting measurement methods for punishing distance control violations depending on the speed and / or high speeds. The usual measurement methods differ in the sensors used, the angle of the sensors, and the type of installation and application. As a rule, a single-point measurement is not considered conclusive; therefore, any violation should be measured over a longer distance and documented in a way that is admissible in court.

[0005] Furthermore, even in countries with driver liability, court or expert evidence, automated measurement, documentation, and ticket / penalty notice generation for following distance violations in road traffic require proof with a driver photo. There is a legal requirement to prove the following distance violation over a distance of at least 250 m. Point-based measurements using radar / camera are not suitable for this purpose, or only with limitations. However, it has been found that existing systems and procedures for punishing traffic violations can only inaccurately depict the speed and, above all, the distance between two vehicles due to the perspective, so the tolerance range and thus the inaccuracy are very high. Lidar and camera arrangements exist which, from above at an angle of approximately 90° (degrees) above the roadway, already provide a bird's eye view.This eliminates the need to convert image or sensor data. However, the disadvantage is that neither a license plate nor a driver can be detected without the need to operate an additional sensor at a suitable angle. Speed-measuring sensors, such as radar sensors, cannot be positioned at a 90° angle to the road. This would require the costly planning and positioning of two different sensors for different viewing angles.

[0006] It is therefore an object of the present invention to at least partially overcome at least one of the above-described disadvantages of measurement methods for punishing traffic violations. In particular, the object of the invention is to provide a method and a traffic monitoring system for punishing traffic violations that generate perspective images of measured vehicles, allowing for improved assessment of distances between vehicles and more accurate speed measurement.

[0007] For an economical and efficient arrangement of traffic monitoring sensors, it is advantageous to use as few sensors as possible. An angular arrangement between 0° and 70° (degrees) and an algorithmic transformation to a bird's-eye view with a theoretical angle of 90° to the road plane are preferred.

[0008] The above object is achieved by a method having the features of independent patent claim 1, by a traffic monitoring system having the features of independent patent claim 12, and by a computer program product having the features of independent patent claim 15. Further features and details of the invention emerge from the subclaims, the description, and the drawings. Features and details described in connection with the method according to the invention naturally also apply in connection with the traffic monitoring system according to the invention and / or in connection with the computer program product according to the invention, and vice versa, so that with regard to the disclosure of the individual aspects of the invention, reference is or can always be made to each other.

[0009] According to a first aspect of the invention, a method is provided for operating a traffic monitoring system for recording a traffic scene on a roadway, the roadway being passable by at least one vehicle, the method comprising the following steps:

[0010] - Reading the traffic scene on the road as Lidar data by means of at least one Lidar sensor of at least one optical detection unit,

[0011] - Detecting predefined control points from environmental data in the acquired lidar data by means of a processing unit, wherein the control points are positioned at specified locations along and / or on the roadway in the environmental data,

[0012] - Converting the control points to layer data of a plane by means of the processing unit, where the plane corresponds to the roadway,

[0013] - Converting the acquired lidar data and the plane data using a transformation algorithm to Birds-Eye-View data for recording the traffic scene on the roadway by means of the processing unit,

[0014] - Evaluating the birds-eye view data from the recording of the traffic scene on a roadway into evaluation data using an evaluation unit to operate the traffic monitoring system. A traffic monitoring system is a system for monitoring objects of all kinds, and in particular traffic in general or a traffic scene.

[0015] In this case, a traffic scene is understood to be a monitored area of ​​road traffic. The traffic scene can be an intersection area that is monitored, for example, to detect a red light violation at a traffic light in the said intersection area, but also to generally monitor the traffic in this intersection area in order to possibly adapt the traffic light cycle to the traffic. However, a traffic scene can also be a multi-lane road, such as a motorway or a country road or similar, where the main focus is on congestion monitoring or speeding violations or distance violations. The traffic scene can also be understood independently of the road traffic regulations in that objects are generally monitored (e.g. pedestrians, passers-by, event participants - even indoors) and these objects in the (traffic) scene are tracked and the resulting data is processed according to the invention.This is particularly true in the context of civil security.

[0016] The carriageway is the part of the road on which vehicles travel. This includes motor vehicles, such as passenger cars, trucks, land vehicles, scooters, motorcycles, and bicycles.

[0017] The lidar data recorded with the lidar sensor can be 2D and 3D. This enables improved data accuracy for evaluating the respective traffic scene. Images captured with a lidar sensor are also weather-independent, thus providing consistently good and accurate data about what's happening in a traffic scene, regardless of the weather.

[0018] The checkpoints are positioned at specified locations along and / or on the roadway. Along the roadway is defined as being along the side of the roadway, i.e., parallel to a roadway boundary. A roadway boundary separates the roadway from an area where no vehicles are moving. A roadway boundary can be a curb edge to a footpath, a boundary to a motorway median, a guardrail, or a grass verge. However, the checkpoints can also be positioned on the lane markings, which serve to divide the roadway into multiple lanes.

[0019] In descriptive geometry, a bird's-eye view (also known as a bird's-eye view) refers to perspectives that show an object from above or obliquely. The observer thus sees the object as if it were a bird flying high above them.

[0020] In contrast, standard perspective refers to the view from the normal eye level of a standing observer, and worm's-eye perspective refers to the view from a point below normal eye level. However, all three types do not actually refer to the position and orientation of objects relative to the observer, but rather to the respective position of the horizon line (the observer's eye level) relative to the center of the image (see Wikipedia.org).

[0021] When analyzing the Bird's Eye View data, the vehicles in the traffic scene and their relative characteristics are evaluated. This can include data about the respective vehicle, such as vehicle type or speed, or the vehicle owner. It is conceivable that the data could be individual images or sequences. Furthermore, the analysis of the Bird's Eye View data could be visualized as an image or video.

[0022] It is conceivable that the evaluation data from several evaluations of individual traffic scenes are combined into one overall evaluation.

[0023] For example, several traffic monitoring systems with at least one optical detection unit can be installed in a complex intersection area. Each individual traffic monitoring system is assigned to a lane and a specific direction of travel. The traffic monitoring systems can be linked to each other so that they execute the process synchronously. All recorded sensor data is time-stamped. The respective evaluations of the individual lanes can be combined into an evaluation of the entire intersection area.

[0024] The method enables a traffic scene, including the vehicles in it, to be displayed from a bird's-eye view. This bird's-eye view allows for a more accurate representation and measurement of the distances between vehicles or the distance of a vehicle from a roadway boundary, such as a no-stopping or no-parking zone. This allows for a more precise assessment of the situation in the traffic scene and leads to more accurate, more plausible, and thus legally admissible recordings or analyses of a traffic scene, for example, on a roadway.

[0025] Regardless of the tracked objects, a bird's-eye view can be displayed simultaneously and synchronously with image and / or lidar data without transformation into birds-eye view data. This allows any potential transformation errors to be verified through the simultaneous and synchronous (time-stamped) display in the original data format. This can be achieved, for example, by splitting a monitor to display a traffic scene, with the birds-eye view displayed as a video on the left half of the monitor and the display without birds-eye view on the right half of the monitor.

[0026] Within the scope of the invention, it can be advantageous for images of the traffic scene on the roadway to be read in as image data by means of at least one camera unit of the at least one optical detection unit, wherein after conversion to the plane data and before or after conversion to the birds-eye view data, the image data and the lidar data are overlaid to form overlay data. An additional representation of the traffic scene with image data increases accuracy and is particularly helpful for the legally sound representation of a violation in the traffic scene, since not only the lidar data (laser data) but also the visual representation can be provided. The image data can also be provided as 2D or 3D data.

[0027] The camera unit can, for example, contain a stereo camera or a 360-degree camera.

[0028] It is conceivable for the image data and the lidar data to represent different perspectives; for example, the lidar data can be overlaid in a bird's eye view and the image data in the recording perspective. This allows, for example, the driver to be displayed and the distance measured in a single analysis, thus enabling improved identification and classification of the vehicle and thus a plausible presentation option as a video display, for example in court. The image data can also be generated, in particular, by a stereo camera or a 360-degree camera. Furthermore, image data can be supplied from different spectral ranges, both in the visible and invisible range. A particularly preferred variant is an infrared (IR) camera, which delivers special thermal images.

[0029] Within the scope of the invention, it is conceivable that the overlay data are converted into the birds-eye view data of the bird's perspective using the transformation algorithm.

[0030] By overlaying the lidar data and the image data and converting them together into birds-eye view data, a more accurate representation of the traffic scene can be achieved when evaluating the data.

[0031] Within the scope of the invention, a method can be provided that, when the traffic scene on a roadway is read in, an object on the vehicle is detected in the lidar data by the processing unit, wherein the detected object serves to identify the vehicle type and / or the vehicle owner. Additionally or alternatively, a driver is detected in the lidar data and / or image data of the vehicle by the processing unit. The object is read in with the traffic scene on a roadway as object data and / or the driver is read in with the traffic scene on a roadway as driver data.The object data and / or the driver data are converted with the lidar data and the layer data using a transformation algorithm to the birds-eye view data of a bird's-eye view for recording the traffic scene on the roadway by means of the processing unit, whereby the object data and / or the driver data are assigned to the vehicle in the birds-eye view data.

[0032] In this context, an object on a vehicle is understood to be a license plate for owner identification or specific features for identifying the vehicle type, such as a specific sticker for identifying a truck. This can be useful, for example, if there are different speed limits for a vehicle type depending on the day or time of day, so that a violation of a vehicle-specific limit can be traced and punished.

[0033] By converting the object data and / or driver data into birds-eye view data, the detected object and / or driver can be assigned to the respective vehicle and displayed during the traffic scene analysis. This is particularly advantageous when recording the traffic scene sequentially, since, as mentioned, an assignment of identification data—i.e., the object data and / or driver data—takes place.

[0034] It is also conceivable that the object and / or the driver is recognized by the recording unit when the image data is read in.

[0035] It can further be provided in a method that an extrinsic calibration is carried out for aligning the optical detection unit to the roadway, wherein the extrinsic calibration is carried out once or cyclically and / or wherein the extrinsic calibration is carried out automatically, at least partially automatically, or manually, in particular wherein the data from the extrinsic calibration is used for positionally accurate conversion into the plane data of the plane.

[0036] The primary purpose of extrinsic calibration is to ensure that reliable and accurate information can be derived from the lidar data or image data. Extrinsic calibration ensures that the internal and external orientations match, meaning that the evaluated lidar data or image data of the traffic scene match the actual surroundings. During calibration, the traffic scene, and especially the plane data of the plane or roadway, are first aligned with the surrounding data.

[0037] For extrinsic calibration, the traffic scene is imported as lidar data, the control points are identified, and converted into plane data of the plane, i.e., the roadway. The plane data and the lidar data are then converted into birds-eye view data and evaluated. During evaluation, the birds-eye view data is compared with the real environmental data, and care is taken to ensure that the markings, such as lane markings or road edge markings, in the birds-eye view data match the corresponding markings in the real environmental data. If this is not the case, the alignment of the optical unit is adjusted manually or automatically, and the above steps are repeated until the markings in the birds-eye view data align with the markings in the real environmental data.

[0038] This is advantageous, for example, when the traffic scene under observation contains slight curves. The position of the optical detection unit can be easily adjusted, as the course of the plane, or rather the roadway, from the process can be adapted to the course of the road in the real environment. Extrinsic calibration can prevent incorrect evaluation of a traffic scene.

[0039] It can additionally be provided in a method that the at least one optical detection unit of the traffic monitoring system and / or the method is calibrated and / or aligned by means of a 2D or 3D code, in particular a QR code, at a specific position on a stop line on the roadway in the traffic scene on the roadway.

[0040] The code can be positioned so that it can be easily detected. It can be used as a reference point for calibration and, above all, for recalibration of the optical detection unit. This is particularly advantageous if the position of the optical detection unit has changed due to environmental influences. For example, the original position of the code can be stored for recalibration so that any deviation between the target position of the optical detection unit and the stored actual position can be determined. The code can be read in each time the traffic scene on the road is read in, but cyclical reading and evaluation of the code is conceivable in order to detect any change in position in the evaluated Bird's Eye View data.

[0041] A method can also provide for the traffic scene on the roadway to be read by means of the optical detection unit from a height of 1.5 m to 10 m, preferably 2 m to 2.8 m, more preferably 2.3 m to 2.7 m.

[0042] This allows for recording from a fixed position, meaning the traffic scene is captured from a predetermined position. This is particularly advantageous for legally binding measurements or recordings of the traffic scene.

[0043] It is also possible in a method that, depending on the traffic scene to be monitored on the roadway, at least one measuring line is stored in reference data stored in the evaluation unit, wherein the at least one measuring line is or will be aligned orthogonal or substantially orthogonal to a roadway boundary and / or orthogonal or substantially orthogonal to a lane direction of the roadway and / or parallel or substantially parallel to a front and / or rear bumper of a vehicle detected in the lidar data and / or image data. The measuring line is used in the evaluation of the evaluation data of the traffic scene.

[0044] The reference data is adapted to the respective traffic scene and reflects the respective events to be observed or the violations to be penalized. The measuring line(s) are adapted to the reference data. For example, on a highway, the reference data is adapted to distance violations, traffic observations, and speed violations, so that the measuring line runs across the width of the roadway and is designed to measure a vehicle's speed or acceleration.

[0045] For this purpose, several measuring lines can be provided parallel to each other along the entire traffic scene in order to be able to read and evaluate the speed and / or acceleration and / or the distance to another vehicle over a certain distance. A parking violation, on the other hand, can be found in the reference data for monitoring an intersection area. However, several measuring lines can be present in an intersection area. Here, too, several measuring lines can be arranged parallel to each other for measuring speed or acceleration. Alternatively or additionally, the shorter measuring lines can cover an area of ​​a no-parking zone or a

[0046] Delineate a no-parking zone on the roadway to detect parking violations. The measurement lines can also be stop lines, traffic light stop lines, and yellow box lines.

[0047] Alternatively or additionally, a method may provide for the measuring line to be orthogonal or substantially orthogonal to a lane boundary, in particular a lane boundary for delimiting the part intended for oncoming traffic.

[0048] In this context, road markings are understood to mean markings that delimit the roadway or make the edge of the roadway visible and separate the roadway from shoulders or special paths. A road marking within the meaning of the invention can also be a grass verge or a guardrail.

[0049] Within the scope of the invention, it is optionally possible for the method to be carried out at different times, in particular cyclically, and / or upon a triggering event, in particular a crossing of the at least one measuring line by a vehicle detected in the lidar data.

[0050] Cyclically reading the lidar data, or the lidar data and image data, allows for more precise monitoring of the traffic scene. However, reading this data during a trigger event can be advantageous, as it requires less storage capacity. A trigger event is defined here as exceeding a specified value. For example, a defined speed can be stored as a specified value, so that the process is triggered when a vehicle crosses the measuring line at a speed higher than the speed specified in the specified value. It is also conceivable to store an object to identify a vehicle type in order to detect vehicle-type-specific speed limits and their exceedance.

[0051] Furthermore, it is conceivable in a method that several spaced-apart optical detection units are provided for reading the traffic scene on the roadway from different angles.

[0052] Multiple optical detection units enable capture from different angles and perspectives, thus avoiding a "blind spot" when observing a traffic scene. This is suitable, for example, for intersections with poor visibility or poor visibility. This results in a more accurate capture of a traffic scene.

[0053] In this case, a method may provide for the lidar data or the overlay data to be recorded and overlaid at different times and / or from different angles.

[0054] This increases the accuracy of the evaluation of the traffic scene.

[0055] Furthermore, within the scope of the invention, it can be provided in a method that a vehicle speed and / or vehicle acceleration of at least one vehicle on the roadway and / or a stopping violation, in particular a stopping violation in an intersection area or a traffic light area of ​​the traffic scene, and / or a distance violation of a vehicle from a preceding vehicle is evaluated from the birds-eye view data at one point in time or at several points in time to the evaluation data.

[0056] Using the analysis data, the traffic scene can be easily analyzed and visualized, especially with regard to violations, to obtain legally admissible data. The analysis data can be converted into a single image, an image sequence, or a video.

[0057] Furthermore, within the scope of the invention, it can be provided in a method that the evaluation data are transmitted by means of a data interface or a transmitter of the evaluation unit to an external data storage or external further processing unit.

[0058] The external data storage or the external processing unit can, for example, already be located within a public authority, so that the allocation and final evaluation for penalizing violations can be carried out quickly and easily. This shortens the time and increases the efficiency of traffic monitoring. It would also be conceivable for the external processing unit to be directly available to an enforcement agency for penalizing the respective offense, so that the punishment can be carried out directly. For example, the external processing unit could be installed in a police car that is currently monitoring the traffic scene, so that a warning of a speeding violation or a following distance violation is issued along with the relevant identification data, allowing the police car to directly follow the violator and penalize the violation.

[0059] Furthermore, it is possible in one method that, during the superimposition, at least one position and / or at least one coordinate of at least one data point of the lidar data is compared with a pixel of the image data corresponding to the data point. An alarm and / or a rejection of the measured value is triggered if a deviation between the data point and the pixel is detected. This can, in particular, trigger an extrinsic calibration.

[0060] This allows for easy verification of whether the lidar data and the image data belong together. This prevents lidar data and image data from different points in time from overlapping. It is also possible to compare the lidar data and image data of an actual state with the target state from the first extrinsic calibration to determine whether the optical detection unit has changed its position due to environmental influences. If a change is detected, a calibration is triggered.

[0061] Within the scope of the invention, it can be advantageous in a method that radar data is acquired by means of at least one radar sensor of the at least one optical detection unit and / or ultrasound data is acquired by means of at least one ultrasound sensor of the at least one optical detection unit. The radar data and / or the ultrasound data are used to convert them into the plane data and / or to convert them into the birds-eye view data and / or to evaluate the birds-eye view data with the evaluation data and / or to overlay them with the overlay data. By adding at least one further sensor to the optical detection unit, the accuracy of the data read in for the evaluation of the traffic scene is increased. This has a positive effect on the legally sound evaluation of the respective traffic scene.

[0062] The above object is further achieved according to a second aspect of the invention by a traffic monitoring system for carrying out the method described above, with at least one optical detection unit with at least one lidar sensor for reading in the traffic scene on a roadway as lidar data, and with a processing unit for detecting predefined control points from environmental data in the lidar data, for converting the control points to plane data of the plane, wherein the plane corresponds to the roadway, and for converting the detected lidar data and the plane data using a transformation algorithm to birds-eye view data of the bird's perspective, and with an evaluation unit for evaluating the traffic scene on a roadway to form the evaluation data.

[0063] The predefined control points can be specified manually by the user (e.g., a surveyor) or by an automatic system (computer program product) that detects suitable boundary points, road edges, and / or lane lines and aligns them with the control points, thus (pre)defining them. The control points primarily serve to define a plane, whereby the user should generally set as few control points as necessary in the normal perspective (not a bird's-eye view). Subsequent input by adding or moving control points in the bird's-eye view represents a simpler and thus more error-resistant method for the user (or the computer program product).

[0064] In general, traffic monitoring systems can be used for a variety of tasks in road traffic. For example, they have proven effective in analyzing traffic flows, controlling traffic management systems, and, in particular, in recording and tracking traffic violations.

[0065] Traffic monitoring systems are equipped with an optical detection unit for the prosecution of traffic violations, as this makes subsequent prosecution of the violation, possibly even in court, much easier. Traffic violations include, for example, exceeding the speed limit, ignoring red lights at traffic lights, failing to maintain a minimum following distance, etc.

[0066] Within the scope of the invention, it can be provided in a traffic monitoring system that the at least one optical detection unit has at least one camera unit for capturing image data from images.

[0067] The images taken with the camera unit increase accuracy and are particularly suitable for the court-proof depiction of a traffic violation.

[0068] Within the scope of the invention, it can be provided in a traffic monitoring system that several optical detection units are provided for reading the traffic scene on the roadway, wherein the optical detection units are arranged at a distance from one another.

[0069] Multiple optical detection units enable detection from different angles and perspectives, thus avoiding a "blind spot" when observing a traffic scene. This is suitable, for example, for intersections with poor visibility or confusing areas.

[0070] Furthermore, within the scope of the invention, it can be provided for a traffic monitoring system that the evaluation unit has the data interface or transmitter for transmitting the evaluated data to an external data storage or external processing unit. The external data storage or external processing unit can, for example, already be located within a government agency, so that the allocation and final evaluation for penalizing violations can be carried out quickly and easily. This shortens the time and increases the efficiency of traffic monitoring.

[0071] Within the scope of the invention, it can additionally be provided in a traffic monitoring system that the at least one optical detection unit is arranged next to the roadway and / or above the roadway.

[0072] In this case, the arrangement of the optical detection unit next to the roadway means that the optical detection unit is positioned next to the roadway, i.e., behind a road barrier, and records the traffic scene from a lateral perspective. The optical detection unit can be mounted on a mast.

[0073] If the optical detection unit is arranged above the roadway, it is located between a first roadway boundary and a second roadway boundary. The optical detection unit can be attached, for example, to a gate, a tunnel wall, or a tunnel arch.

[0074] Both the next-to-lane and above-lane configurations can be implemented using a one-pole solution. A one-pole solution involves positioning an optical detection unit in such a way that the traffic scene is captured from a fixed point at a single, fixed angle between the optical detection unit and the vehicle, with a view of the driver and license plate. Only a single power connection is required.

[0075] Within the scope of the invention, it is conceivable for the optical detection unit to be arranged at a height of 1.5 m to 10 m, preferably 2 m to 2.8 m, more preferably 2.3 m to 2.7 m. Furthermore, in a traffic monitoring system, it is possible for the at least one optical detection unit to have at least one radar unit for detecting radar data and / or at least one ultrasonic sensor for detecting ultrasonic data. Accordingly, lidar data can be combined or fused with image data or any desired sensor data (e.g., radar data, ultrasonic sensor data).

[0076] By adding at least one additional sensor to the optical detection unit, the accuracy of the data read for the evaluation of the traffic scene is increased. This has a positive effect on the legally sound evaluation of the respective traffic scene.

[0077] The above object is further achieved according to a third aspect of the invention by a computer program product according to the invention, comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method described above by means of a traffic monitoring system described above.

[0078] Advantages that are described in detail with regard to the method for operating a traffic monitoring system for recording a traffic scene on a roadway according to the first aspect of the invention apply equally to a traffic monitoring system according to the second aspect of the invention and to a computer program product according to the third aspect of the invention.

[0079] Further advantages, features, and details of the invention will become apparent from the following description, in which several embodiments of the invention are described in detail with reference to the drawings. The features mentioned in the claims and in the description may be essential to the invention individually or in any combination. The invention is explained in more detail below with reference to the accompanying drawings. In the drawings:

[0080] Fig. 1 is a schematic representation of a method according to the invention,

[0081] Fig. 2 is a schematic representation of a visualized evaluation of the method according to the invention with reference to a roadway before an extrinsic calibration,

[0082] Fig. 3 is a schematic representation of a visualized evaluation of the method according to the invention with reference to a roadway after an extrinsic calibration,

[0083] Fig. 4 is a schematic representation of a view of a traffic scene on a roadway before conversion to a bird’s eye view,

[0084] Fig. 5 is a schematic representation of an evaluation of the method according to the invention with reference to an intersection area, and

[0085] Fig. 6 is a schematic representation of a traffic monitoring system according to the invention.

[0086] Fig. 1 shows a schematic representation of the method 100 for operating a traffic monitoring system 10 for recording a traffic scene 11 on a roadway 12. The roadway 12 is accessible to at least one vehicle 13. The method 100 comprises the following steps:

[0087] - reading 110 of the traffic scene 11 on the roadway 12 as lidar data 14 by means of at least one lidar sensor 15 of at least one optical detection unit 16,

[0088] - Detecting 120 predefined control points 17 from environmental data 18 in the acquired lidar data 14 by means of a processing unit 19, wherein the control points 17 are positioned at specified locations along and / or on the roadway 12 in the environmental data 18, see Figs. 2 and 3,

[0089] - Converting 130 the control points 17 to plane data 20 of a plane 21 by means of the processing unit 19, wherein the plane 21 corresponds to the roadway 12, see Figs. 2 and 3, - Converting 140 the acquired lidar data 14 and the plane data 20 using a transformation algorithm to birds-eye view data 22 of a bird's-eye view for recording the traffic scene 11 on the roadway 12 by means of the processing unit 19,

[0090] - Evaluation 150 of the Birds-Eye-View data 22 of the recording of the traffic scene 11 on a roadway 12 to the evaluation data 35 by means of an evaluation unit 23 for operating a traffic monitoring system 10.

[0091] In Fig. 2 to Fig. 4, 3D lidar data 14 and in Fig. 5, 2D lidar data 14 were read in and further processed by the lidar sensor 15 of the optical detection unit 16.

[0092] Furthermore, the method 100 provides that images of the traffic scene 11 on the roadway 12 are read in as image data 24 by means of at least one camera unit 25 of the at least one optical detection unit 16, this can be seen in Fig. 1 and Fig. 4, 120. After the conversion 130 into the plane data 20 and before the conversion 140 into the birds-eye view data 22, an overlay 160 of the image data 24 and the lidar data 14 to form overlay data 26 is carried out.

[0093] The overlay data 26 are also converted into the birds-eye view data 22 of the bird's-eye view using the transformation algorithm, 140. This creates an accurate representation of the traffic scene 11 in a bird's-eye view.

[0094] In addition, during the reading 110 of the traffic scene 11 on a roadway 12, an object 27 on the vehicle 13 is detected in the lidar data 14 by the processing unit 19. As can be seen 120 in Fig. 5, the detected object 27 is a license plate, i.e., an object 27 for identifying the vehicle owner. It would also be conceivable here to detect an object 27 for identifying the vehicle type. In addition, a driver 28 was detected in the lidar data 14 by the processing unit 19 for at least one of the vehicles 13 in the traffic scene 11, since a red light violation has occurred.

[0095] In this case, the object 27 or the objects 27 and the driver 28 are read in with the traffic scene 11 on a roadway 12 as object data 29 and the driver 28 with the traffic scene 11 on a roadway 12 as driver data 30. Both the object data 29 and the driver data 30 are converted 140 with the lidar data 14 and the plane data 20 using a transformation algorithm to the birds-eye view data 22 of a bird's-eye view for recording the traffic scene 11 on the roadway 12 by means of the processing unit 19. In the evaluation and especially in the visualization of the evaluation, the object data 29 and the driver data 30 are assigned to the vehicle 13 in the birds-eye view data 22, as shown in Fig. 5. Optionally, lidar data can be combined with image data or any sensor data (e.g. radar data, ultrasonic sensor data).

[0096] Fig. 2 shows a representation of the roadway 12 before an extrinsic calibration of the optical detection unit 16. It can be seen 120 that the lane markings 44 stored for evaluation (shown in dashed lines) do not correspond to the lanes 45 of the surrounding lane; in this case, an incorrect evaluation of the traffic scene 11 may occur.

[0097] The calibration and alignment 180 of the optical detection unit 16 to the roadway 12 using an extrinsic calibration 170 is initially performed once and / or optionally automatically. It is conceivable to repeat this calibration if a deviation of the actual position from the target position of the optical detection unit 16 is detected. In this case, the data from the extrinsic calibration is used for precise conversion into the plane data 20 of plane 21. The calibration is only considered complete when the lane markings 44 stored for evaluation have been aligned with the lane markings 45 of the surrounding lane, as shown in Fig. 3.

[0098] For calibration, especially for recalibration of the optical detection unit 16 of the traffic monitoring system 10 or of the method 100 itself, a 2D code 31, in particular a QR code, is provided at a specific position on a stop line 32 on the roadway 12 in the traffic scene 11 on the roadway 12. This step is shown as an optional step in dashed lines in Fig. 1. Such a 2D code 31 can, for example, be attached to a stop line 32, as shown schematically in Fig. 5.

[0099] Depending on the traffic scene 11 to be monitored on the roadway 12, at least one measuring line 33 is stored in reference data 34 which is stored in the evaluation unit 23.

[0100] In this case, the at least one measuring line 33 can be or become aligned orthogonally or substantially orthogonally to a road boundary and / or orthogonally or substantially orthogonally to a lane direction of the roadway 12 and / or parallel or substantially parallel to a front and / or rear bumper of a vehicle 13 detected in the lidar data 14 and / or image data 24, and wherein the measuring line 33 is used in the evaluation of the evaluation data 35 of the traffic scene 11.

[0101] In Fig. 3, two measuring lines 33 are stored in the reference data 34. These run orthogonally or essentially orthogonally to a lane boundary and are used to measure speed or distance in order to detect violations in this regard. In Fig. 5, several measuring lines 33 are stored; these measuring lines 33 each run along the traffic light stop line, and thus partially across the lane 12, but orthogonally to the lane boundary. Another measuring line 33 marks a no-parking zone and runs parallel to a lane boundary. The method 100 is carried out at different times or upon a triggering event, i.e. when a vehicle 13 identified in the lidar data 14 crosses the at least one measuring line 33, such as a red light violation in Fig. 5.

[0102] For reading 110 the traffic scene 11 on the roadway 12, several spaced-apart optical detection units 16 can be provided so that the traffic scene 11 can be displayed from different angles. This allows the lidar data 14 or the overlay data to be recorded and overlaid at different times and / or from different angles.

[0103] The vehicle speed and / or the vehicle acceleration of at least one vehicle 13 on the roadway 12 can be evaluated from the birds-eye view data 22 at one or more times. Additionally or alternatively, a stopping violation is evaluated. For example, in Fig. 5, a stopping violation in a traffic light zone, i.e., a red light violation, and a parking violation are evaluated using the birds-eye view data 22. The traffic scenes 11 of Figs. 2 to 4 are each evaluated to form evaluation data 35.

[0104] These evaluation data 35 are transmitted, 210, to an external data storage 38 or external further processing unit 39 by means of a data interface 36 or a transmitter 37 of the evaluation unit 23.

[0105] In order to always have precise and, above all, legally sound evaluations and evaluation data 35, at least one position and at least one coordinate of at least one data point of the lidar data 14 are compared 220 with a pixel of the image data 24 corresponding to the data point during the overlay 160. An alarm and / or a rejection 190 of the measured value is triggered if a deviation between the data point and the pixel is detected. At the same time, a new extrinsic calibration is triggered when the deviation is detected. In addition to the lidar data 14 and the image data 24, radar data 40 is acquired by means of at least one radar sensor 41 of the at least one optical detection unit 16, and ultrasound data 42 is acquired by means of at least one ultrasound sensor 43 of the at least one optical detection unit 16.The radar data 40 and the ultrasound data 42 are used for converting to the plane data 20 and for converting to the birds-eye view data 22 and for evaluating 150 the birds-eye view data 22 and / or for overlaying 160 to the overlay data 26.

[0106] Fig. 6 shows a traffic monitoring system 10 for carrying out the method 100 according to Figs. 1 to 5. This system has at least one optical detection unit 16 with at least one lidar sensor 15 for reading in 110 the traffic scene 11 on a roadway 12 as lidar data 14, and a processing unit 19 for detecting 120 predefined control points 17 from environmental data 18 in the lidar data 14, for converting the control points 130 to plane data 20 of plane 21, wherein plane 21 corresponds to the roadway 12, and for converting the detected lidar data 14 and the plane data 20 using a transformation algorithm to bird's-eye view data 22, and an evaluation unit 23 for evaluating 150 the traffic scene 11 on a roadway 12.

[0107] The evaluation unit 23 has the data interface 36 or the transmitter 37 for transmitting 210 the evaluated data to an external data storage 38 or external further processing unit 39.

[0108] In addition, the at least one optical detection unit 16 has at least one camera unit 25 for detecting image data 24 from images, at least one radar sensor 41 for detecting radar data 40 and at least one ultrasonic sensor 43 for detecting ultrasonic data 42.

[0109] In the specific case of Fig. 6, two optical detection units 16 are provided, each comprising a lidar sensor 15 and a camera unit 25. One of the two additionally comprises an ultrasonic sensor 43, and the other a radar sensor 41.

[0110] The traffic monitoring system according to Fig. 5 is mounted on a gate between 2.3 m and 2.7 m, or optionally up to 10 m, and can thus be arranged above the roadway 12. It provides several optical detection units for reading 110 the traffic scene 11 on the roadway 12. These optical detection units are arranged at a distance from one another in order to be able to record different angles of the traffic scene 11.

[0111] It is also conceivable that the traffic monitoring system 10 is mounted next to the roadway 12 on a mast at a height of between 2.3 and 2.7 m or optionally up to 10 m.

[0112] List of reference symbols

[0113] 10 Traffic monitoring system

[0114] 11 Traffic scene

[0115] 12 lane

[0116] 13 vehicles

[0117] 14 Lidar data

[0118] 15 Lidar sensor

[0119] 16 optical detection unit

[0120] 17 checkpoints

[0121] 18 Environmental data

[0122] 19 Processing unit

[0123] 20 layer data

[0124] 21 Level

[0125] 22 Birds-Eye View data

[0126] 23 Evaluation unit

[0127] 24 image data

[0128] 25 Camera unit

[0129] 26 Overlay data

[0130] 27 objects

[0131] 28 drivers

[0132] 29 Object data

[0133] 30 driver data

[0134] 31 2D code

[0135] 32 stop line

[0136] 33 measuring line

[0137] 34 Reference data

[0138] 35 evaluation data

[0139] 36 Data interface

[0140] 37 channels

[0141] 38 external data storage

[0142] 39 external processing unit radar data

[0143] radar sensor

[0144] Ultrasound data

[0145] Ultrasonic sensor stored lane markings

[0146] lane

[0147] Proceedings

[0148] Read in

[0149] Recognize

[0150] Converting the control points

[0151] Convert to Birds Eye View data

[0152] Evaluate

[0153] Overlay

[0154] Calibrate

[0155] Align

[0156] Discard

[0157] Trigger

[0158] Transmitted

[0159] Compare

Claims

Patent claims 1. A method (100) for operating a traffic monitoring system (10) for recording a traffic scene (11) on a roadway (12), wherein the roadway (12) is passable by at least one vehicle (13), the method (100) comprising the following steps: - reading (110) the traffic scene (11) on the roadway (12) as lidar data (14) by means of at least one lidar sensor (15) of at least one optical detection unit (16), - detecting (120) predefined control points (17) from environmental data (18) in the acquired lidar data (14) by means of a processing unit (19), wherein the control points (17) are positioned at specified locations along and / or on the roadway (12) in the environmental data (18), - converting (130) the control points (17) into plane data (20) of a plane (21) by means of the processing unit (19), the plane (21) corresponding to the roadway (12), - converting (140) the acquired lidar data (14) and the plane data (20) using a transformation algorithm to birds-eye view data (22) of a bird's-eye view for recording the traffic scene (11) on the roadway (12) by means of the processing unit (19), - evaluating (150) the birds-eye-view data (22) of the recording of the traffic scene (11) on a roadway (12) to form evaluation data by means of an evaluation unit (23) for operating the traffic monitoring system (10).

2. Method (100) according to claim 1, characterized in that images of the traffic scene (11) on the roadway (12) are read in as image data (24) by means of at least one camera unit (25) of the at least one optical detection unit (16), wherein after the Converting (130) into the plane data (20) and before or after converting (140) into the birds-eye view data (22) an overlay (160) of the image data (24) and the lidar data (14) to form overlay data (26) is carried out.

3. Method (100) according to claim 2, characterized in that the overlay data (26) are converted (140) into the birds-eye view data (22) of the bird's perspective using the transformation algorithm.

4. Method (100) according to one of the preceding claims, characterized in that during the reading (110) of the traffic scene (11) on a roadway (12), an object (27) on the vehicle (13) is recognized in the lidar data (14) by means of the processing unit (19), wherein the recognized object (27) serves to identify the vehicle type and / or the vehicle owner, and / or that a driver (28) of the vehicle (13) is recognized in the lidar data (14) and / or image data (24) by means of the processing unit (19), wherein the object (27) with the traffic scene (11) on a roadway (12) is read in as object data (29) and / or the driver (28) with the traffic scene (11) on a roadway (12) is read in as driver data (30),and wherein the object data (29) and / or the driver data (30) are converted (140) with the lidar data (14) and / or image data (24) and the plane data (20) using a transformation algorithm to the birds-eye view data (22) of a bird's-eye view for recording the traffic scene (11) on the roadway (12) by means of the processing unit (19), and wherein the object data (29) and / or the driver data (30) are assigned to the vehicle (13) in the birds-eye view data (22).

5. Method (100) according to one of the preceding claims, characterized in that an extrinsic calibration (170) is carried out for aligning (180) the optical detection unit (16) to the roadway (12), wherein the extrinsic calibration is carried out once or cyclically and / or wherein the extrinsic calibration (170) is carried out automatically, at least partially automatically, or manually, in particular wherein the data from the extrinsic calibration is used for positionally accurate conversion into the plane data (20) of the plane (21).

6. Method (100) according to one of the preceding claims, characterized in that, depending on the traffic scene (11) to be monitored on the roadway (12), at least one measuring line (33) is stored in reference data (34) stored in the evaluation unit (23), wherein the at least one measuring line (33) is or will be aligned orthogonally or substantially orthogonally to a roadway boundary and / or orthogonally or substantially orthogonally to a lane direction of the roadway (12) and / or parallel or substantially parallel to a front and / or rear bumper of a vehicle (13) detected in the lidar data (14) and / or image data (24), and wherein the measuring line (33) is used in the evaluation of the evaluation data (35) of the traffic scene (11).

7. Method (100) according to one of the preceding claims, characterized in that the method (100) is carried out at different times, in particular cyclically, and / or upon a triggering event, in particular a crossing of a vehicle (13) detected in the lidar data (14) over the at least one measuring line (33).

8. Method (100) according to one of the preceding claims, characterized in that from the birds-eye view data (22) at one point in time or at several points in time, a vehicle speed and / or vehicle acceleration of at least one vehicle (13) on the roadway (12), and / or a stopping violation, in particular a stopping violation in an intersection area or a traffic light area of ​​the traffic scene (11), and / or a distance violation of a vehicle (13) from a preceding vehicle (13) is evaluated to the evaluation data (35).

9. Method (100) according to one of the preceding claims, characterized in that the evaluation data (35) are transmitted (210) by means of a data interface (36) or a transmitter (37) of the evaluation unit (23) to an external data storage (38) or external further processing unit (39).

10. Method (100) according to one of the preceding claims, characterized in that during the superimposition (160) at least one position and / or at least one coordinate of at least one data point of the lidar data (14) is compared (220) with a pixel of the image data (24) corresponding to the data point, wherein an alarm and / or a rejection (190) of the measured value is triggered if a deviation between the data point and the pixel is detected, wherein in particular an extrinsic calibration is triggered.

11. Method (100) according to one of the preceding claims, characterized in that radar data (40) are acquired by means of at least one radar sensor (41) of the at least one optical detection unit (16) and / or ultrasonic Data (42) are acquired by means of at least one ultrasonic sensor (43) of the at least one optical acquisition unit (16), wherein the radar data (40) and / or the ultrasonic data (42) are used to convert to the plane data (20) and / or to convert to the birds-eye view data (22) and / or to evaluate (150) the birds-eye view data (22) with the evaluation data (35) and / or to superimpose (160) the overlay data (26).

12. Traffic monitoring system (10) for carrying out the method (100) according to one of the preceding claims, - with at least one optical detection unit (16) with at least one lidar sensor (15) for reading (110) the traffic scene (11) on a roadway (12) as lidar data (14), and - with a processing unit (19) for detecting (120) predefined control points (17) from environmental data (18) in the lidar data (14), for converting the control points (130) to plane data (20) of the plane (21), wherein the plane (21) corresponds to the roadway (12), and for converting the detected lidar data (14) and the plane data (20) using a transformation algorithm to bird's-eye-view data (22) of the bird's perspective, and - with an evaluation unit (23) for evaluating (150) the traffic scene (11) on a roadway (12) to obtain the evaluation data (35).

13. Traffic monitoring system (10) according to claim 12, characterized in that the at least one optical detection unit (16) has at least one camera unit (25) for detecting image data (24) from images.

14. Traffic monitoring system (10) according to claim 12 or 13, characterized in that that several optical detection units are provided for reading (110) the traffic scene (11) on the roadway (12), wherein the optical detection units are arranged at a distance from one another.

15. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method (100) according to one of claims 1 to 11 by means of a traffic monitoring system (10) according to one of claims 12 to 14.

Images