Method and system for measuring the speed of vehicles in road traffic

Abstract

The invention relates to a method (10) for measuring the speed of a vehicle (34, 36) in road traffic, comprising the following method steps: - generating (11) a plurality of captured images (38, 40) of a vehicle (34); - identifying (12) features (42) of the vehicle (34) within the captured images (38, 40); - determining (13) the position Pi of the identified features (42); - checking (14) whether the identified features (42) are situated within or outside a first capture area (B1); - performing (15) a coarse measurement for a speed vector of a vehicle (34), wherein the coarse measurement is based on the position of the features (42) that are situated within the first capture area (B1); - checking (16) whether the speed vector determined during the course measurement represents an exceedance of a specified maximum speed; and - performing (17) a fine measurement for the speed vector if an exceedance of the specified maximum speed was determined during the preceding check.

Description

[0001] Method and system for measuring the speed of vehicles in road traffic

[0002] The present invention relates to a method for measuring the speed of vehicles in road traffic and a corresponding device for speed measurement.

[0003] Various approaches for determining vehicle speeds are known from the state of the art. These are used primarily to increase road safety and reduce the number of traffic accidents in the long term.

[0004] The provision of efficient, precise, and cost-effective speed measurement methods offers the advantage that they can be used in all traffic sections where traffic accidents are frequently observed. The use of comprehensive speed measurement methods directly increases road users' awareness of the need to comply with speed limits and makes it easier for the responsible authorities to punish violations of speed limits.

[0005] To date, both active and passive measurement methods have been used to determine vehicle speed.

[0006] In active methods, the measuring device emits electromagnetic radiation, which is reflected by a vehicle and subsequently detected by the measuring device. By analyzing the reflected radiation, the vehicle's speed can be determined. For example, the signal propagation time of the radiation emitted by the measuring device and reflected by the vehicle can be evaluated. Alternatively, triangulation methods can be used to calculate the position of a vehicle at two different points in time and to determine the vehicle speed from the difference between the calculated positions and the time difference between the individual measurements.A disadvantage of active methods is their relatively high energy consumption, which makes autonomous operation of corresponding measuring devices difficult, as providing the required energy from batteries, accumulators, and / or solar cells becomes significantly more complex. Furthermore, the measuring devices for active speed measurement are often associated with high acquisition costs.

[0007] In addition to active measuring methods, passive measuring methods for speed measurement are also known, in which the measuring device does not emit radiation. Some of the passive measuring methods mentioned simply measure the time a vehicle takes to cover a specified distance. For this reason, such methods are often referred to as distance-time methods. Light barriers, brightness sensors, or pressure sensors can be used, which generate a signal when a vehicle passes. A disadvantage of the distance-time methods described above is that they often require structural measures, which can make the initial acquisition and commissioning costs relatively high. In addition, distance-time methods usually require the use of a separate camera to record the vehicle data and / or person-specific data.A further disadvantage of the methods described above is that subsequent measurement assignment and subsequent verification of the calculated measured values ​​are practically impossible. This is particularly problematic when a driver denies allegedly exceeding the speed limit.

[0008] As an alternative to the distance-time method, it is in principle possible to calculate the speed of a vehicle purely using a camera. This involves using a camera that allows a specific section of road to be recorded. The camera has at least one image sensor, which is typically a CCD sensor or a CMOS sensor. By using specific feature recognition algorithms, it is possible to detect one or more distinctive features of a vehicle (for example, a corner point of a license plate or a windshield) and to determine the position of this feature at the time of an initial recording. The camera is usually calibrated in such a way that it allows an assignment between the individual pixels of the image sensor and the position of the feature within the observed section of road.By re-capturing the vehicle with the image sensor at a second time, the same feature of the vehicle can be detected in the second image, and the position of that feature at the second time can be determined. If the position of a feature is known at the first and second times, and the time interval between the individual images is also known, the speed of a feature or of the vehicle can be calculated.

[0009] It is also possible for a camera system to have two image sensors that capture a vehicle from different perspectives and then enable a three-dimensional position determination for a feature.

[0010] One problem with the camera-based methods discussed above is that errors in feature recognition can significantly impair the measurement result. In particular, it can happen that several features are recognized in one image recording, but these can be assigned to different vehicles. In this case, the speeds of several vehicles are undesirably included in the measurement, leading to a corrupted result. Furthermore, a challenge with the measurement methods known to date is enabling an efficient calculation of the vehicle speed. Based on the above problem, the object of the present invention is to provide a method for measuring the speed of vehicles in road traffic that is particularly precise, robust, and efficient.

[0011] To achieve the above-mentioned object, the present invention proposes a method for measuring the speed of a vehicle in road traffic, wherein the method is carried out using a measuring device comprising at least one image sensor, a memory unit, a computing unit and a communication unit, and wherein the method comprises the following steps:

[0012] Generation of multiple images of a vehicle;

[0013] Detecting features of the vehicle within the image recordings; determining the position of the features detected in the image recordings;

[0014] Check whether the detected features are inside or outside a first recording area;

[0015] Performing a coarse measurement for a velocity vector of a vehicle, the coarse measurement based on the position of the features located within the first acquisition area;

[0016] Check whether the speed vector determined during the rough measurement represents an exceedance of a specified maximum speed; and

[0017] Carrying out a fine measurement of the speed vector if the previous test revealed that the specified maximum speed was exceeded.

[0018] The method according to the invention enables particularly precise and efficient determination of vehicle speed, significantly reducing the risk of measurement errors. A further significant advantage is that the device's calibration can be continuously adjusted while a measurement is in progress, enabling robustness against external influences (e.g., thermal or mechanical effects). Furthermore, the method according to the invention offers the advantage of enabling subsequent assignment of individual vehicles and the associated measurement. This allows for subsequent verification of the measured values ​​obtained. As already explained above, this is particularly advantageous in cases where a driver disputes an alleged speeding offense.

[0019] The method according to the invention makes it possible to initially perform a rough measurement of the speed vector with optimal use of the available computing resources and to only perform a fine measurement, which involves greater computational effort, in those cases where speeding has been detected. In this way, the more precise fine measurement is only performed when necessary. If, for example, the specified maximum speed on a section of road is 100 km / h and the speed determined in the rough measurement is only 80 km / h, a fine measurement of the speed is omitted. If, on the other hand, a speed of 110 km / h is determined in the rough measurement, it is advantageous to perform the fine measurement in order to obtain a particularly precise measurement result and to determine the speeding exactly.

[0020] The measuring method referred to as "rough measurement" in the context of the present invention can also be referred to as a "first measuring method", while the measuring method referred to as "fine measurement" can be referred to as a "second measuring method".

[0021] The images can be generated by a single image sensor. If multiple image sensors are provided, the images can be generated by multiple image sensors. The term "velocity vector" used in the context of the present invention emphasizes that the speed of a vehicle can be detected in multiple dimensions. Depending on the embodiment of the present invention, the velocity vector can be three-dimensional, two-dimensional, or one-dimensional.

[0022] The coarse measurement of a velocity vector describes a first measurement method in which the computational effort is relatively low, whereas the fine measurement uses a second measurement method in which the computational effort is higher but the measurement accuracy is increased compared to the coarse measurement. For example, for the coarse measurement, it may be sufficient to use only two images and evaluate the features contained therein, whereas for the fine measurement, all available images are used for the measurement. Within the scope of the present invention, however, different methods can be used for the coarse measurement and the fine measurement, as long as the fine measurement can achieve a higher measurement accuracy than the coarse measurement.

[0023] According to some embodiments of the present invention, it may be provided that two image sensors are provided and that the above-mentioned coarse measurement comprises the following method steps:

[0024] Creating a first image at a first time using a first image sensor and creating a second image at the first time using a second image sensor;

[0025] Creating a third image at a second time using the first image sensor and creating a fourth image at the second time using the second image sensor; detecting features in the individual image recordings;

[0026] comparing the features in the images recorded by the first image sensor with the features contained in the images recorded by the second image sensor at the same time, whereby those features which are contained only in the images recorded by the first image sensor or in the images recorded by the second image sensor are discarded;

[0027] Comparing the corresponding features in the images recorded by the first image sensor with the features in the images recorded by the second image sensor with regard to their epipolar geometry, whereby those features for which the epipolar conditions are not met are discarded;

[0028] Comparing the features in the images captured by the first image sensor, discarding those features that are not included in both images captured by the first image sensor and those features that are not included in both images captured by the second image sensor; determining features whose position has remained unchanged from the first time to the second time and removing those features whose position has remained unchanged;

[0029] Determining the spatial position of the features using a triangulation method;

[0030] Determination of a velocity vector for each feature that was not previously rejected;

[0031] Determining the velocity for each feature from the velocity vector determined for the corresponding feature;

[0032] Determination of the average vehicle speed by averaging the determined speeds for each feature. The two image sensors can be arranged, for example, side by side (as a left and right image sensor) or one above the other (as an upper and lower image sensor).

[0033] The two image sensors each capture an image at a first point in time and a second point in time. However, the capture of images is not limited to capturing images at exactly two points in time. Rather, the image sensors can also be designed to capture numerous images, so that the information from a large number of images can be used to determine the speed. Therefore, the two points in time mentioned above are to be understood as a minimum, so that the individual image sensors, according to the embodiments of the invention described above, generate at least two images.

[0034] The first image sensor and the second image sensor can together constitute a camera system. The image sensors can be arranged in a housing. Furthermore, the camera system can have recording optics, each of which is arranged between an image sensor and the recording area to be monitored (in which the vehicles are to be recorded).

[0035] According to some embodiments of the invention, specific features are recognized in each of the images generated by the image sensors. One of numerous feature recognition algorithms known from the prior art can be used for this purpose. For example, the SIFT (Scale Invariant Feature Transform), SURF (Speed ​​Up Robust Feature), BRIEF (Binary Robust Independent Elementary Feature), or ORB (Oriented FAST and Rotated BRIEF) methods can be used to recognize features. A Harris Corner Detector can also be used for feature recognition to detect prominent corner points within an image.

[0036] The features can be a prominent point within the image. For example, a corner point within an image that exhibits a particularly high contrast can be detected. Alternatively, the feature can also refer to a line or another geometric shape (e.g., a rectangle or a trapezoid). In practice, several tens, several hundred, several thousand, or even more features can be detected within an image or belong to the same vehicle.Since processing a large number of features increases the computational effort and consequently the required computing time, on the one hand, and reduces inaccuracy due to expected errors in the detection of individual features in the image recordings, the present invention performs a specific selection of the detected features before the vehicle speed is ultimately determined based on the detected features. This significantly increases the efficiency of the measurement method and reduces the method's susceptibility to errors.

[0037] During selection, the features recorded by the first image sensor are compared with the features recorded by the second image sensor. If a feature, for example a prominent point, was detected at a first time t1 in the first image recording and this feature does not appear in the second image recording (also recorded at time t1), this feature is discarded for further processing. After this process step, only those features remain that are actually contained in the images recorded by both image sensors at the same time. In this way, an initial consistency check is performed, with all inconsistent features being disregarded for the speed calculation.

[0038] A known feature in an image can, for example, be described by a vector. Ideally, two corresponding features in two images have identical values. Consequently, two features are considered identical if the feature vectors are identical. In practice, however, the comparison of two features can be achieved not only by determining the same feature based on the identity of two feature vectors, but also by first calculating a distance between two feature vectors. The identity of the two features is then assumed if the distance between the feature vectors is smaller than a predetermined threshold. The Hamming distance or the Euclidean distance, for example, can be used to calculate the distance between two feature vectors.

[0039] Additionally, the features in the images captured by the first image sensor and the second image sensor at the same time can be compared with each other in terms of their epipolar geometry. If the epipolar condition is met for two features, it is assumed that these points are consistent. In this case, the features are retained. If the epipolar condition is not met for two features, this indicates that the features are inconsistent; the inconsistent features are removed and neglected in the subsequent velocity determination. In general, the epipolar condition is met for two points if the following equation (also called the epipolar equation) is satisfied:

[0040] P^FPi = 0 or

[0041] Here, pi denotes a point in the first image, p2 a point in the second image, and F the fundamental matrix. The fundamental matrix is ​​calculated from the geometric relationship between the two cameras (translation and rotation) and the intrinsic camera parameters. It describes a mapping rule between the coordinate systems of both cameras. If the features represent individual points, it can be directly determined by checking the above condition whether the two features correspond. If the features represent geometric shapes, for example, one or more points of the geometric shape of the respective image recordings can be compared with each other with regard to their epipolar geometry.For example, in the case of a line, the epipolar condition test can be applied to the two endpoints of the line to check whether the lines detected in two image acquisitions describe the same feature.

[0042] In practice, the fulfillment of the epipolar condition can be defined such that the product of p2 (transposed), the fundamental matrix F and pi is not exactly equal to 0, but is smaller than a given limit.

[0043] Furthermore, the features determined in two consecutive images captured by an image sensor are checked for consistency. In doing so, those features that are not included in both images captured by the first image sensor or the second image sensor are discarded. As already explained above, an image sensor can be designed to generate more than two images. In this case, it can be provided that those features that are not included in all images generated by an image sensor are discarded.

[0044] To further increase the efficiency of the method according to the invention, the present invention also determines the features whose positions have remained unchanged over time, discarding these "constant" features. This further reduces the total number of features used for speed determination to those features that actually move between the first and second points in time and therefore contain information about the vehicle's speed. This advantageously prevents unnecessary consideration of features that can be attributed, for example, to a building or a road marking.

[0045] After discarding those features that are considered inconsistent, the spatial position of the features is determined using a triangulation method. Triangulation methods are well known in the art and allow the position of an object to be determined using trigonometric relationships.

[0046] Once the position of a feature at the first and second times has been determined, a velocity vector is determined for each feature. The velocity for each of the (consistent) features can then be determined from the time difference between two consecutive images and the displacement vector (also called motion vector) for each (consistent) feature. The time difference between two image recordings can generally be assumed to be known. This can be derived, for example, from the image recording frequency of the image sensors used. If the image sensors are used with an image recording frequency of 20 fps (frames per second), it can be deduced that the time difference between two consecutive image recordings is 50 ms.If two images from a video sequence are used to determine the speed, which are ten individual images apart, it can be concluded that the time difference between the images used is 500 ms. In this respect, a speed vector can be determined as the quotient of a displacement vector, which describes the movement of a feature between two images and is defined by the start position and the end position of the feature, and the time difference between the two images. According to the present invention, the speed vector can therefore be determined for the respective feature by determining the quotient of a displacement vector, which describes the movement of a feature between two images and is defined by the start position and the end position of the respective feature, and the time difference between two images.The velocity can be determined for one feature at a time by determining the magnitude of the velocity vector for the corresponding feature.

[0047] Subsequently, an average vehicle speed can be determined based on the rough measurement by calculating an average value from the individual speeds determined for each feature. For example, an arithmetic mean can be calculated from the individual speed values ​​or the median value can be determined from the individual speed values. For example, only the features or speed values ​​for those features that were previously considered consistent and therefore not discarded can be considered.

[0048] According to one embodiment of the invention, it can be provided that the coarse measurement additionally comprises the following method step: checking the consistency of the velocity vectors using a RANSAC method, wherein those velocity vectors which were assessed as inconsistent are discarded.

[0049] RANSAC (Random Sample Consensus) methods are generally known from the state of the art. The use of a RANSAC method allows the detection of outliers in a measurement data set. Taking into account the mean of the recorded measurement data and a specified tolerance range, those measured values ​​that lie outside the specified tolerance range are identified.

[0050] As an alternative to the RANSAC method, a special variant of the DBScan method can also be used according to the invention. In this case, the determined 3D points and velocity vectors are simultaneously "clustered" in a 6-dimensional space, i.e., consistent groups are formed with respect to a 6-dimensional metric. Each consistent group corresponds to a vehicle moving within the visible field of view.

[0051] A consistent group that is visible across multiple consecutive images, but at least across two consecutive images, is referred to as a "cluster." The velocity measurement value for a cluster is calculated as soon as its geometric center of gravity falls below a defined distance from the measuring device. The velocity values ​​of all points belonging to all consistent groups of the cluster are taken into account.

[0052] In the coarse measurement, the consistent groups can be combined into a cluster by determining the geometric center of gravity for each consistent group and checking whether this center of gravity has moved by the distance corresponding to the velocity vector of the respective consistent group within the corresponding time difference between the frames.

[0053] According to one embodiment of the method according to the invention, during the coarse measurement, a time signature (often referred to as a time stamp, digital time stamp, or in English, a time stamp) is generated when each image is taken, which is attached to or assigned to the image. The additional time signature allows the exact time at which a recording was actually taken to be precisely defined. In this way, the accuracy of the method according to the invention can be increased.

[0054] According to an embodiment of the method according to the invention, it can also be provided that the rough measurement comprises the following method step:

[0055] Checking the consistency of the direction of the individual velocity vectors, whereby the angle between the individual velocity vectors and a predetermined reference line is determined and an angle mean value is determined from the determined angle values ​​and those velocity vectors are regarded as inconsistent for which the difference between the determined angle and the angle mean value is greater than a predetermined limit value, and whereby those velocity vectors which are regarded as inconsistent are discarded.

[0056] The reference line can, in particular, run parallel to the monitored roadway. In general, it can be expected that the detected vehicle characteristics indicate a direction of movement that is essentially parallel to the roadway. If the direction of movement deviates significantly from the reference line (and consequently the determined angle value exceeds a specified limit), this can be interpreted as an indicator that a determined speed vector is inconsistent. Using this approach, the individual speed vectors that are considered inconsistent can be discarded. By subsequently limiting the subsequent speed calculation to the consistent speed vectors, the accuracy of the method can be further increased. At the same time, the susceptibility to errors in the measurement method is reduced.

[0057] Furthermore, according to the present invention, it can be provided that the rough measurement comprises the following method step:

[0058] Checking the consistency of the magnitude of the individual velocity vectors using a RANSAC method, discarding those velocity vectors that are considered inconsistent.

[0059] While the previously described process step checked the consistency of the direction of the velocity vectors, this process step checks the consistency of the magnitude. Outliers determined by the RANSAC method are ignored in the subsequent calculation of the vehicle speed. A threshold value can be defined, with those measured values ​​that exceed the threshold being considered outliers. By checking the consistency of the vector magnitudes and discarding inconsistent values, the efficiency of the current measurement method can be further increased while simultaneously reducing the susceptibility to errors.

[0060] According to a preferred embodiment of the invention, it can be provided that the rough measurement comprises the following method step:

[0061] Comparison of the mutually corresponding features in the image recordings taken by the first image sensor at the first time and at the second time and comparison of the mutually corresponding features in the image recordings taken by the second image sensor at the first time and at the second time with regard to their epipolar geometry, wherein those features for which the epipolar condition is not fulfilled are discarded.

[0062] This allows the number of features used for the final speed determination to be further reduced, with only consistent features being considered for the speed determination. This further increases the efficiency of the method according to the invention and simultaneously improves the precision of the method, since significantly fewer measured values ​​need to be used for the final speed determination and inconsistent measured values ​​are disregarded in the speed determination.

[0063] When checking whether the speed vector determined during the rough measurement represents an exceedance of a specified maximum speed, the magnitude of the speed vector can be compared with the specified maximum speed. If the magnitude of the speed vector is greater than the specified maximum speed, the speed vector determined during the rough measurement represents an exceedance of the speed limit, and a fine measurement is then performed.

[0064] With the fine measurement, however, it can be planned that significantly more data is evaluated, which can lead to a more precise measurement result. For example, it can be planned that the fine measurement takes into account all images that depict a vehicle (fully or at least partially) within the first recording area. For example, for the coarse measurement, only the position of the features in two images can be used, while for the fine measurement, which is only carried out in the event of a speeding violation, the position of the features in 20 or 30 images can be used. For this purpose, the corresponding features for the fine measurement can be determined subsequently using the coarse measurement method. This is possible because the corresponding images are stored in the working memory, even if they are not processed during the coarse measurement.

[0065] In the method according to the invention, a first recording area is defined, and a check is performed to determine whether all of the features detected in an image are located within the defined first recording area. If some features are located outside the first recording area, they are discarded and are not taken into account in the speed measurement or speed calculation. This reduces the risk that features attributable to different vehicles are taken into account in the speed measurement. Consequently, the measurement inaccuracies of the method are reduced, and the method's robustness against measurement errors is increased.

[0066] The first recording area can be defined in different ways. In particular, the first recording area can be defined by two numerical values ​​that define the boundaries of the first recording area. For example, the values ​​zl and z2 can define the boundaries of the first recording area, with the values ​​zl and z2 defining the distances to a reference point along a first axis (z-axis). The first axis can run parallel to the longitudinal direction of the route section captured by a camera, or parallel to a vector that runs orthogonal to the sensor surface of the camera's image sensor, whereby in the latter case the reference point can be arranged on the sensor surface. For example, zl and z2 can be defined as follows: (zl, z2) = {(25 m, 35 m), (20 m, 30 m), (15 m, 25 m)}.

[0067] After all features outside the first acquisition range have been discarded, a velocity vector is determined for each of the features not discarded. The position of the respective feature within the first image captured at the first time point and within the second image captured at the second time point are determined, as well as the time difference between the two images. Furthermore, an average velocity vector is determined from the individual velocity vectors. For this purpose, the arithmetic mean or the median value can be used, for example.

[0068] According to a preferred embodiment of the method according to the invention, it can be provided that the implementation of the fine measurement for the velocity vector comprises the following steps:

[0069] Storing those image recordings F which have features within the first recording area, whereby a total of N image recordings are stored;

[0070] Assuming a constant velocity vector v of the vehicle in the first recording area; for all image recordings Fi with i = 2,..,N :

[0071] (a) Calculate a position Pij for each of the features j detected in the image Fi, from the previously determined position of the feature in the previous image Fi-i and the assumed velocity vector v, according to:

[0072] Pi,j = Pi-i,j + v ■ At, where Pi,j is the position of the feature j detected in the image Fi determined during the fine measurement, Pi-i is the position of the feature j detected in the image Fi-i determined during the coarse measurement, v is the assumed speed vector of the vehicle in the first recording area and At is the time difference between the individual image recordings;

[0073] (b) calculating an average distance value between the positions Pi,j of the individual features j determined during the coarse measurement and the positions Pij of the individual features j determined during the fine measurement;

[0074] (c) Change the velocity vector v and repeat steps (a) and (b) until a predefined termination criterion is reached.

[0075] In this embodiment of the present invention, the velocity measurement is calculated using a larger data set than the coarse measurement. The iterative approach enables a precise calculation of the velocity vector. Steps (a) and (b) are repeated until a specified quality is achieved. As explained below, different termination criteria can be defined within the scope of the present invention.

[0076] As already explained above, the speed vector v can be three-dimensional, two-dimensional or one-dimensional. For example, a speed vector v can be assumed that was determined during the rough measurement. If a speed of 105 km / h was determined during the rough measurement and the permissible maximum speed is 100 km / h, the fine measurement can be carried out with a speed of 105 km / h assumed to be constant. This speed can be changed within a predefined range (e.g. 105 km / h + / - 5 km / h). The change can be made, for example, in steps of 1 km / h. Depending on the defined termination criterion, the iterations can be continued until either all previously specified speed values ​​have been passed through or until a predefined termination condition occurs.

[0077] The time difference Δt can be calculated, in particular, from the frame rate of the camera system used. For example, if the camera system is set up to capture 25 frames per second, the time difference Δt between two consecutive image captures is 40 ms.

[0078] When changing the velocity vector v, in particular the position Pi-i of the corresponding features determined during the coarse measurement can be replaced by the position Pij of the features determined during the fine measurement, provided that the calculated mean distance value is less than the mean distance value calculated on the basis of the position determined during the coarse measurement.

[0079] According to one embodiment of the method according to the invention, the termination criterion can be defined such that the average distance value is not reduced after a change in the velocity vector. As soon as it is recognized that a change in the velocity vector does not lead to any additional improvement in the measurement, the iterative process can be terminated, and the velocity vector for which the smallest distance value was determined can be output as the result of the fine measurement. Further iterations can be considered unnecessary in this case, so they are avoided to avoid increased computational complexity (i.e., the required computing time or effort). Furthermore, various gradient-based optimization methods, such as the so-called "gradient descent" method or the ADAM method, can be used in the implementation of this embodiment.According to a further advantageous embodiment of the method according to the invention, the termination criterion can be defined such that the change in the mean distance value following a change in the speed vector is less than a predetermined limit value. The previously calculated mean distance value is regarded as an indicator of the accuracy of the determined speed. As soon as the distance value no longer shows a significant change from one iteration to the next, the iterative process can be terminated at this point in time. In this way, an efficient determination of the vehicle speed can be ensured without the need for additional iterations, which in any case would probably not lead to a significant increase in measurement accuracy.

[0080] Furthermore, within the scope of the method according to the invention, it can be provided that the termination criterion is defined such that steps (a) and (b) have previously been carried out for a predetermined set of velocity vectors. In this way, it is ensured that all velocity values ​​within a predetermined velocity range are iteratively checked before a final measurement result is determined and output. In principle, it is possible that after a few iterations, the measurement data could give the impression that the optimal result was already achieved after just a few iterations, since, for example, after a few iterations the distance value shows no significant change. However, it is quite possible that the optimal measurement result is only determined in one of the subsequent iterations.This allows for particularly reliable measurement results by running through all speed vectors from a previously defined range. For example, a speed of 130 km / h may have been determined during the rough measurement, while several equidistant speed values ​​in the range from 120 km / h to 140 km / h are checked during the fine measurement to achieve a particularly precise measurement result. The speed values ​​within the specified range can be run through in steps of 0.1 km / h, 0.2 km / h, 0.5 km / h, or 1 km / h, for example, with an average distance value being calculated for each speed value.

[0081] When calculating the mean distance value, different distance values ​​can be used. Preferably, the mean distance value can be calculated from a Euclidean distance between the position Pi determined during the rough measurement and the position Pi,j determined during the fine measurement. The position Pij determined during the rough measurement describes the initially determined position. This can be replaced during the iterative process by the position i determined during the fine measurement, provided this leads to a reduction in the mean distance value. In this case, the distance value can be calculated as follows: where P X tj, pj and denote the x-, y- and z-components of i and P jP^j and Pij denote the x-, y- and z-components of Pi.

[0082] To determine the mean distance value, the sum of the individual distance values ​​deukiid j over the individual images and all features can then be calculated: ' d e uklid,i,ji,j

[0083] Following the calculation of S, the assumed velocity vector v can be changed iteratively, with S being recalculated after each change. The iterative change of the velocity vector v and the calculation of S are repeated until a predetermined termination criterion is reached. Different termination criteria can be used within the scope of the present invention, as already explained above.

[0084] According to a further embodiment of the method according to the invention, the mean distance value can be determined from a reprojection error. The reprojection error can be calculated as an alternative to calculating the Euclidean distance as described above. The reprojection error RF per feature can be calculated as follows: rfi,j = C' 1 ■ Pi,j - W

[0085] Here, C denotes the calibration matrix, which allows the conversion of real 3D coordinates to 2D pixel coordinates. The reprojection error indicates how far apart the pixel values ​​W = C' calculated from Pij are. 1 ■ Pi,j are away from the originally detected pixel values ​​of the relevant feature W = (u, v).

[0086] The total reprojection error is then calculated as the sum of all previously calculated reprojection errors:

[0087] It is then minimized, with v and C serving as variation parameters. This has two additional advantages: First, it improves an inaccurate speed calculation, and the covariance can be used to determine the accuracy of the speed calculation. Second, the calibration matrix can be updated and optimized during operation, and its quality can be tested. This can be done using either a single vehicle or multiple vehicles.

[0088] As a further alternative to varying v (and C), the 3D coordinates of each relevant feature can also be varied to achieve even more precise velocity determination. Additional constraints can be imposed on the 3D coordinates. For example, the range of variation of the x, y coordinates can be restricted, since the error in the z direction can be considered significantly higher.

[0089] Furthermore, it may be required that if the 3D coordinates of a relevant feature are shifted in one image, the same shift must also be performed in other image recordings, since the relative positions of the 3D coordinates cannot change as they belong to a rigid object (a vehicle).

[0090] Furthermore, the method according to the invention can provide for the additional variation of the 3D coordinates to be performed depending on the results of the calculation of the reprojection error. For example, the additional variation of the 3D coordinates can occur if the reprojection error or the covariance is greater than a previously defined limit.

[0091] Furthermore, the steps of the method according to the invention described above can be repeated by varying the algorithm parameters. The previously defined covariance can serve as the termination criterion. Alternatively, the difference in the previously described speed determination can also serve as the termination criterion. Furthermore, to achieve the above-mentioned object, a device for measuring the speed of vehicles in road traffic is proposed, wherein the device comprises the following: an image sensor for generating image recordings; a memory unit for storing the generated image recordings; a computing unit for carrying out several processing steps; and a communication unit for transmitting measurement data or

[0092] measurement results to an external server unit, wherein the computing unit is designed to generate multiple images of a vehicle;

[0093] Detect features of the vehicle within the image recording; determine the position of the detected features; check whether the detected features are located inside or outside a first recording area; carry out a coarse measurement for a speed vector of a vehicle, the coarse measurement being based on the position of the features located in the first recording area; check whether the speed vector determined in the coarse measurement represents an exceedance of a specified maximum speed; and carry out a fine measurement for the speed vector if the previous test found that the specified maximum speed was exceeded.

[0094] Preferably, in the device according to the invention, it can be provided that the computing unit is designed, during the fine measurement for the velocity vector, to store those image recordings F which have features in the first recording area;

[0095] Assuming a constant velocity vector v of the vehicle in the first recording area; for all image recordings Fi with i = 2,..,N :

[0096] (a) to calculate a position Pij for each of the features j detected in the image Fi from the previously determined position of the feature j in the previous image Fi-i and the assumed velocity vector v according to:

[0097] Pi,j = Pi-i,j + V ■ At, where Pi,j is the position of the feature j detected in the image recording Fi determined during the fine measurement, Pi-i is the position of the feature j detected in the image recording Fi-1 determined during the coarse measurement, v is the assumed speed vector of the vehicle in the first recording area and At is the time difference between the individual image recordings;

[0098] (b) to calculate an average distance value between the positions Pi,j of the individual features j determined during the coarse measurement and the positions Pij of the individual features j determined during the fine measurement;

[0099] (c) changing the velocity vector v and repeating steps (a) and (b) until a predefined termination criterion is reached.

[0100] In addition, the device according to the invention can be provided with the termination criterion being defined in such a way that the mean distance value is not reduced after a change in the speed vector.

[0101] Furthermore, it can be provided that the termination criterion is defined such that the change in the average distance value after a change in the velocity vector is less than a predetermined limit value. According to a preferred embodiment of the device according to the invention, it can also be provided that the termination criterion is defined such that the computing unit has previously performed steps (a) and (b) for a predetermined number of velocity vectors.

[0102] In addition, it can be provided that the mean distance value is calculated from a Euclidean distance between the position Pi,j determined during the coarse measurement and the position Pij determined during the fine measurement.

[0103] Finally, the device according to the invention can be provided so that the mean distance value is determined from a reprojection error.

[0104] Furthermore, according to the method according to the invention, the first recording area can be defined by two points that run along a first axis, wherein the first recording area is delimited by two parallel planes that run orthogonal to the first axis and through the two points, and wherein the first axis runs in particular parallel to the image sensor normal or to the longitudinal axis of a travel section. This allows the first recording area to be defined particularly efficiently, thereby reducing the computational effort of the entire method.

[0105] The following invention of the figures is described in more detail below, wherein the figures show the following:

[0106] Fig. 1 shows an embodiment of the method according to the invention, Fig. 2 shows an embodiment of the device according to the invention, Fig. 3 shows a schematic representation of the speed measurement in road traffic,

[0107] Fig. 4 shows a first and a second image taken at different times, Fig. 5 shows different mean distance values ​​calculated for different velocity vectors.

[0108] Fig. 1 shows an embodiment of the method 10 according to the invention. In the method according to the invention, a total of seven method steps 11-17 are carried out. In the first method step 11, several recorded images of the vehicle are generated using an image sensor. In the second method step 12, characteristic features are detected in the individual recorded images. One of the methods mentioned above, in particular the SIFT method or a Harris Corner Detector, can be used for this purpose. In the third method step 13, the position of the detected features is determined, before a check is carried out in the fourth method step 14 to determine whether the detected features are located inside or outside a first recording area. Those features that are outside the first recording area can be discarded.Subsequently, in the fifth method step 15, a rough measurement for a vehicle's speed vector is carried out. The rough measurement is based on the position of the features located within the first recording range. For example, only two images showing a vehicle within the first recording range can be used for the rough measurement. In principle, it may be sufficient to determine the position of a feature within the first image recording and the second image recording during the rough measurement. From the difference between the two positions and the time between the two image recordings, the speed of the vehicle can be determined within the framework of the rough measurement. The computational requirements for the rough measurement are relatively low.Alternatively, several features in two image recordings can be evaluated, whereby the vehicle speed can be calculated as part of the rough measurement from an average of the individual speeds determined for each individual feature. When averaging the individual speeds, for example, an arithmetic average can be performed or a median value can be determined. In the sixth method step 16, a check is carried out to determine whether the speed vector determined as part of the rough measurement represents an exceedance of a specified maximum speed. This check can be used to determine whether a new speed calculation is necessary that is more precise than the rough measurement. In cases where the rough measurement determined that the vehicle speed is lower than the specified maximum speed in a certain section of road, the result of the rough measurement is sufficient.In this case, a fine measurement is not necessary because the exact speed value is not relevant. However, if a maximum speed has been exceeded, a fine measurement is necessary because the exact speed value is particularly important for determining the legal consequences (fine or, if applicable, revocation of the driving license). Therefore, in the seventh method step 17, a fine measurement for the speed vector is performed if the previous test detected that the specified maximum speed was exceeded. The method according to the invention allows, on the one hand, an efficient calculation of the vehicle speed, which can be carried out in real time, and, on the other hand, a particularly precise calculation of the vehicle speed in special circumstances (exceeding the permitted maximum speed).In this way, the available computing capacity is utilized in a particularly efficient manner. Furthermore, efficient computation can significantly reduce power consumption. Consequently, battery-operated measuring devices can be provided that offer significantly longer battery life. Furthermore, the thermal design of the measuring devices can be simplified, as less heat needs to be dissipated overall. Finally, the computationally efficient implementation of the method according to the invention allows the use of a simple and cost-effective computing unit, thereby reducing the overall cost of the measuring device.

[0109] Even if the method steps are described above in a specific order, it will be apparent to a person skilled in the art that this order is to be understood as purely exemplary and that individual method steps can also be carried out in a different order.

[0110] Fig. 2 shows an embodiment of the device 22 according to the invention. The device 22 has an image sensor 24, a memory unit 26, a computing unit 28 and a communication unit 30. The image sensor 24 can in particular be designed as a CCD sensor or a CMOS sensor. In addition, a second image sensor can also be provided, which is not shown in Fig. 2. The memory unit 26 can in particular be designed as a non-volatile data memory. The computing unit 28 can, for example, use a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The communication unit 30 can in particular have a wireless communication module, wherein in particular a GSM, a UMTS, an LTE or a 5G communication module can be provided. The communication unit is configured to communicate with a central unit and to process the recorded measurement data orMeasured values ​​can be transmitted as needed. On the one hand, the measurement results (i.e., the determined vehicle speeds) can be transmitted along with an image showing the corresponding vehicle and personal data (in particular, a license plate number and / or the vehicle owner). On the other hand, the raw data used for the speed calculation can also be transmitted (exclusively or in addition), so that the measurement result can be reconstructed from this raw data at a later date if necessary. This can increase the transparency of speed measurement, which is also expected to increase public acceptance of the process.

[0111] Fig. 3 shows a schematic representation of speed measurement in road traffic. As shown in this figure, the measuring device 22 is aligned so that the image sensor 24 can capture a recording area. Alternatively, the image sensor can also be arranged so that a bird's-eye view image is generated. In the exemplary embodiment shown in Fig. 3, the entire recording area is divided into a first recording area B1, a second recording area B2 and a third recording area B3. The second recording area B2 and the third recording area B3 border on the first recording area B1. The recording areas B1, B2, B3 can be defined by a total of four numerical values ​​(z0, z1, z2, z3) that describe four points along a first axis 32 (also referred to as the z-axis).The three recording areas B1, B2, B3 can be uniquely described by the four numerical values ​​mentioned, wherein the recording areas B1, B2, B3 are each delimited by two parallel planes that run orthogonal to the first axis 32 and through the defined points. For example, (z0, z1, z2, z3) = (40, 35, 25, 10), while the individual values ​​indicate the distance to a reference point along the first axis (in meters). In the exemplary embodiment shown, the image sensor 24 detects a first vehicle 34 located in the first recording area B1 and a second vehicle 36 located in the second recording area B2. According to the invention, all features detected outside the first recording area B1 are neglected when determining the vehicle speed.This advantageously results in only those characteristics associated with the first vehicle 34 being taken into account when calculating the speed. This reduces the risk that characteristics of the second vehicle are included in the speed determination. Consequently, the precision and robustness of the measurement method are increased.

[0112] Fig. 4 shows two images 38, 40 that were generated at two different points in time. The first vehicle 34 is depicted in both images 38, 40. A feature 42 is detected in the images, which in the exemplary embodiment shown is a corner point of a vehicle license plate. The position of the feature 42 is determined using one of the methods known from the prior art. From the position of the feature 42 in the first image 38 and the position of the same feature in the second image 40, as well as the time difference between the two images 38, 40, the speed of the vehicle or the speed for a feature can be determined. In practice, however, it is typically not a single feature that is detected and analyzed, but several hundred features.

[0113] Finally, Fig. 5 shows an average distance value d that was calculated for different velocity vectors Vk. Fig. 5 shows the embodiment in which the iterative steps described above were carried out for a predetermined set of velocity vectors. In this embodiment, ten velocity vectors are provided (k = 1 to 10), for each of which the average distance value d was calculated. In the case shown in Fig. 5, it can be seen that the fifth velocity vector (k = 5) leads to the lowest average distance value. From this, it can be seen that the fifth velocity vector provides the best result for the vehicle's velocity. As an alternative to the method shown in Fig.5, it can be provided that the termination criterion is defined such that the distance value d is calculated only for the first six velocity vectors and that after the calculation of the sixth average distance value d the termination criterion is reached because the distance value d calculated for the sixth velocity vector has no longer decreased compared to the average distance value calculated for the fifth velocity vector.

[0114] LIST OF REFERENCE SYMBOLS

[0115] 10 Inventive method

[0116] 11 first procedural step

[0117] 12 second procedural step

[0118] 13 third procedural step

[0119] 14 fourth procedural step

[0120] 15 fifth procedural step

[0121] 16 sixth procedural step

[0122] 17 seventh procedural step

[0123] 22 Measuring device

[0124] 24 image sensor

[0125] 26 storage unit

[0126] 28 computing unit

[0127] 30 Communication unit

[0128] 32 first axis

[0129] 34 first vehicle

[0130] 36 second vehicle

[0131] 38 first image capture

[0132] 40 second image capture

[0133] 42 feature

[0134] Bl first recording area

[0135] B2 second recording area

[0136] B3 third recording area

Claims

CLAIMS Method (10) for measuring the speed of a vehicle (34) in road traffic, using a measuring device (22) comprising at least one image sensor (24), a memory unit (26), a computing unit (28) and a communication unit (30), wherein the method (10) comprises the following method steps: Generation (11) of several image recordings (38, 40) of a vehicle (34); Recognition (12) of features (42) of the vehicle (34) in the image recordings (38, 40); Determining (13) the position of the features (42) detected in the image recordings (38, 40); Checking (14) whether the detected features (42) are located inside or outside a first recording area (Bl); Performing (15) a coarse measurement for a velocity vector of a vehicle (34), the coarse measurement being based on the position of the features (42) located within the first recording area (B1); Check (16) whether the speed vector determined during the rough measurement represents an exceedance of a specified maximum speed; and Carrying out (17) a fine measurement for the speed vector if the previous test detected an exceedance of the specified maximum speed. Method (10) according to claim 1, characterized in that carrying out (17) the fine measurement for the speed vector comprises the following steps: Storing those image recordings F which have features (42) within the first recording area (Bl), wherein a total of N image recordings are stored; Assuming a constant velocity vector v of the vehicle (34) in the first recording area (Bl); for all image recordings Fi with i = 2,..,N : (a) Calculate a position Pij for each of the features j detected in the image Fi from the previously determined position of the feature in the previous image Fi-i and the assumed velocity vector v according to: Pi,j = Pi-i,j + V ■ At, where Pi,j is the position of the feature j detected in the image recording Fi determined during the fine measurement, Pi-i is the position of the feature j detected in the image recording Fi-i determined during the coarse measurement, v is the assumed speed vector of the vehicle (34) in the first recording area (Bl) and At is the time difference between the individual image recordings; (b) calculating an average distance value between the positions Pi,j of the individual features j determined during the coarse measurement and the positions Pij of the individual features j determined during the fine measurement; (c) changing the velocity vector v and repeating steps (a) and (b) until a predefined termination criterion is reached. Method (10) according to claim 2, characterized in that the termination criterion is defined such that the distance value is not reduced after a change in the velocity vector. Method (10) according to claim 2, characterized in that the termination criterion is defined such that the change in the distance value after a change in the velocity vector is less than a predetermined limit value. Method (10) according to claim 2, characterized in that the termination criterion is defined such that steps (a) and (b) have previously been carried out for a predetermined set of velocity vectors. Method (10) according to one of claims 2 to 5, characterized in that the mean distance value is calculated from a Euclidean distance between the position Pi determined during the coarse measurement and the position Pij determined during the fine measurement. Method (10) according to one of claims 2 to 6, characterized in that the mean distance value is calculated from a Reprojection error is determined. Device (22) for measuring the speed of vehicles (34) in road traffic, comprising: an image sensor (24) for generating image recordings (38, 40); a memory unit (26) for storing the generated image recordings (38, 40); a computing unit (28) for carrying out a plurality of processing steps; and a communication unit (30) for transmitting measurement data or measurement results to an external server unit, wherein the computing unit (28) is designed to generate a plurality of image recordings (38, 40) of a vehicle; to recognize features (42) of the vehicle (34) within the image recordings (38, 40); to determine the position of the recognized features (42); to check whether the recognized features (42) are located inside or outside a first recording area (B1); to carry out a rough measurement for a speed vector of a vehicle (34), wherein the rough measurement is based on the position of the features (42) located in the first recording area (B1); to check whether the speed vector determined during the rough measurement represents an exceedance of a predetermined maximum speed; and to carry out a fine measurement for the speed vector if an exceedance of the predetermined maximum speed was determined during the previous test.Device (22) according to claim 8, characterized in that the computing unit (28) is designed, during the fine measurement for the velocity vector, to store those image recordings F which have features within the first recording area (Bl); to assume a constant velocity vector v of the vehicle in the first recording area (Bl); for all image recordings Fi with i = 2,..,N :. (a) to calculate a position Pij for each of the features j detected in the image Fi from the previously determined position of the feature j in the previous image Fi-i and the assumed velocity vector v according to: Pi,j = Pi-i,j + V ■ At, where Pi,j is the position of the feature j detected in the image recording Fi determined during the fine measurement, Pi-i is the position of the feature j detected in the image recording Fi-i determined during the coarse measurement, v is the assumed speed vector of the vehicle (34) in the first recording area (Bl) and At is the time difference between the individual image recordings (38, 40); (b) to calculate an average distance value between the positions Pi,j of the individual features j determined during the coarse measurement and the positions Pij of the individual features j determined during the fine measurement; (c) changing the velocity vector v and repeating steps (a) and (b) until a predefined termination criterion is reached. Device (22) according to claim 9, characterized in that the termination criterion is defined such that the distance value is not reduced after a change in the velocity vector. Device (22) according to claim 9, characterized in that the termination criterion is defined such that the change in the distance value after a change in the velocity vector is less than a predetermined limit value. Device (22) according to claim 9, characterized in that the termination criterion is defined such that the computing unit (28) has previously performed steps (a) and (b) for a predetermined set of velocity vectors. Device (22) according to one of claims 9 to 12, characterized in that the mean distance value is derived from a Euclidean Distance between the position Pij determined during the coarse measurement and the position Pij determined during the fine measurement is calculated. Device (22) according to one of claims 9 to 13, characterized in that the average distance value is calculated from a Reprojection error is determined.

Images