Model-based road estimation

Abstract

The invention discloses model-based road estimation. The disclosed embodiments provide model-based techniques for estimating a road model having any number of lanes on a road and any entrance lane (i.e., entrance ramp or branch) and exit lane (i.e., exit ramp or exit branch). More specifically, embodiments disclosed herein utilize Bayesian methods for multi-lane tracking in various traffic scenes, such as, for example, highway scenes. A model-based algorithm is employed to estimate a lane center curve of a multi-lane road based on vehicle motion data (e.g., velocity, angular velocity) and perception data (e.g., camera output, lidar output, radar output, etc. The perceptual data includes detected road features (e.g., lane markings, road edges, road obstacles, guardrails, road markings, etc.) or other objects (e.g., other road users).

Description

Technical Field

[0001] The disclosed technology relates to methods, apparatus, and related aspects for estimating a road model for a vehicle traveling on a road. Specifically, but not exclusively, the disclosed technology relates to a model-based algorithm to estimate a center curve of a multi-lane road and an on-ramp to and off-ramp from the multi-lane road. Background Art

[0002] During the past few years, research and development activities related to autonomous vehicles have surged in number, and a variety of different approaches are being explored. More and more modern vehicles have advanced driver assistance systems (ADAS) to improve vehicle safety and more generally road safety. ADAS (which can be represented by adaptive cruise control (ACC) collision avoidance systems, forward collision warnings, etc.) is an electronic system that can help the driver of a vehicle while driving. Today, there is continuous research and development in a variety of technical fields related to both the fields of ADAS and autonomous driving (AD). ADAS and AD will be referred to herein as the generic term autonomous driving system (ADS), which corresponds to all different automation levels, such as the levels defined by SAE J3016 driving automation levels (1 to 5) (particularly levels 3, 4 and 5), for example.

[0003] As computer hardware in vehicles continues to advance, it allows more and more effort to be put into developing automated driving systems, and many active safety systems such as ACC and lane support systems (LSS) rely on having some knowledge of the road around the vehicle. Therefore, improving the accuracy and redundancy of road estimation systems is critical to improving driver safety and comfort. Summary of the invention

[0004] The techniques disclosed herein attempt to mitigate, alleviate, or eliminate one or more of the above-identified deficiencies and shortcomings in the prior art to address various issues related to performance and redundancy in complex environments.

[0005] Various aspects and embodiments of the disclosed invention are defined in the following and appended independent and dependent claims.

[0006] A first aspect of the disclosed technology includes a computer-implemented method for estimating a road geometry of a road on which a vehicle is traveling, wherein the road geometry is described by a filter state tracked by a tracking filter. The filter state includes a plurality of road model hypotheses, wherein each road model hypothesis is associated with a probability, and includes a parameterization of the road describing the geometry of one or more lanes of the road, and wherein each lane of each road model hypothesis is associated with an existence probability. The method includes, for each time step of a plurality of consecutive time steps, predicting the filter state by determining a probability density for each road model hypothesis based on vehicle motion data and a prior filter state determined at a previous time step. In addition, the method includes associating a road feature obtained from sensory data derived from output of one or more onboard sensors with the parameterization of the road for each road model hypothesis that predicts the filter state. The method further includes determining a probability of existence for each lane of each road model hypothesis based on the a priori probability of existence of each corresponding lane determined at the previous time step and the road feature association, and determining a probability for each road model hypothesis of the plurality of road model hypotheses based on the a priori filter state determined at the previous time step and the determined probability of existence for each lane of the road model hypothesis. Furthermore, the method includes selecting a road model hypothesis associated with a highest probability based on the determined probability of each road model hypothesis, and outputting a road model based on a parameterization of the road of the selected road model hypothesis.

[0007] A second aspect of the disclosed technology includes a computer program product, the computer program product including instructions, when the program is executed by a computing device of a vehicle, the instructions cause the computing device to perform a method according to any one of the embodiments of the first aspect disclosed herein. For this aspect of the disclosed technology, similar advantages and preferred features exist as in the other aspects.

[0008] A third aspect of the disclosed technology includes a (non-transitory) computer-readable storage medium comprising instructions that, when executed by a computing device of a vehicle, cause the computing device to perform a method according to any one of the embodiments of the first aspect disclosed herein. For this aspect of the disclosed technology, similar advantages and preferred features exist as in the other aspects.

[0009] As used herein, the term "non-transitory" is intended to describe a computer-readable storage medium (or "memory") that does not include propagating electromagnetic signals, but is not intended to otherwise limit the type of physical computer-readable storage device encompassed by the phrase computer-readable medium or memory. For example, the terms "non-transitory computer-readable medium" or "tangible memory" are intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in non-transitory form on tangible computer-accessible storage media may also be transmitted via transmission media or signals (such as electrical, electromagnetic, or digital signals), which may be transmitted via a communication medium (e.g., a network and / or wireless link). Thus, the term "non-transitory" as used herein is a limitation on the medium itself (i.e., tangible, rather than a signal), as contrasted with a limitation on the persistence of data storage (e.g., RAM compared to ROM).

[0010] A fourth aspect of the disclosed technology includes an apparatus for estimating a road geometry of a road on which a vehicle is traveling, wherein the road geometry is described by filter states tracked by a tracking filter. The filter states include a plurality of road model hypotheses, wherein each road model hypothesis is associated with a probability, and include a parameterization of the road describing the geometry of one or more lanes of the road, and wherein each lane of each road model hypothesis is associated with a probability of existence. The apparatus includes a control circuit configured to predict the filter state for each time step of a plurality of consecutive time steps by determining a probability of each road model hypothesis based on vehicle motion data and a prior filter state determined at a previous time step. The control circuit is further configured to associate road features obtained from perception data derived from output of one or more on-board sensors with the parameterization of the road for each road model hypothesis that predicts the filter state. Moreover, the control circuit is configured to determine the existence probability of each lane of each road model hypothesis based on the prior existence probability of each corresponding lane determined at the previous time step and the road feature association, and determine the probability of each road model hypothesis of multiple road model hypotheses based on the prior filter state determined at the previous time step and the determined existence probability of each lane of the road model hypothesis. The control circuit is also configured to select the road model hypothesis associated with the highest probability based on the determined probability of each road model hypothesis, and output a road model based on the parameterization of the road of the selected road model hypothesis. For this aspect of the disclosed technology, similar advantages and preferred features exist as in other aspects.

[0011] A fifth aspect of the disclosed technology comprises a vehicle comprising an apparatus according to any one of the embodiments disclosed herein. For this aspect of the disclosed technology, similar advantages and preferred features exist as in the other aspects.

[0012] The disclosed aspects and preferred embodiments may be appropriately combined with each other in any manner obvious to a person skilled in the art, such that one or more features or embodiments disclosed with respect to one aspect may also be considered disclosed with respect to another aspect or embodiment of another aspect.

[0013] Further embodiments are defined in the dependent claims. It should be emphasized that the term "comprises / comprising" when used in this specification is used to indicate the presence of stated features, integers, steps or components. It does not exclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0014] These and other features and advantages of the disclosed technology will be further elucidated hereinafter with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above-described aspects, features, and advantages of the disclosed technology will be more fully understood by referring to the following illustrative and non-limiting detailed description of example embodiments of the present disclosure when taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 is a schematic diagram of the Bayesian network representation of the state-space model.

[0017] Figure 2 is a set of schematic diagrams of road parameterizations according to some embodiments.

[0018] Figure 3a is a schematic diagram of an image frame captured by a forward-facing vehicle camera with re-projected lane markings.

[0019] Figure 3b is reprojected to Figure 3a Schematic diagram of a bird's-eye view of the detected 3D lane markers in the camera image.

[0020] Figure 4 is a schematic flow chart representation of a method for estimating road geometry of a road on which a vehicle is traveling, according to some embodiments.

[0021] Figure 5 is a schematic block diagram representation of a vehicle including an apparatus for estimating a road geometry of a road on which the vehicle is traveling, according to some embodiments. DETAILED DESCRIPTION

[0022] The present disclosure will now be described in detail with reference to the accompanying drawings, in which some example embodiments of the disclosed technology are shown. However, the disclosed technology may be implemented in other forms and should not be construed as being limited to the disclosed example embodiments. The disclosed example embodiments are provided to fully convey the scope of the disclosed technology to the technician. It will be appreciated by those skilled in the art that the steps, services, and functions explained herein may be implemented using separate hardware circuits, using software running in conjunction with a programmed microprocessor or general-purpose computer, using one or more application specific integrated circuits (ASICs), using one or more field programmable gate arrays (FPGAs), and / or using one or more digital signal processors (DSPs).

[0023] It will also be understood that when the present disclosure is described in terms of a method, it may also be implemented in an apparatus comprising one or more processors, one or more memories coupled to the one or more processors, wherein computer code is loaded to implement the method. For example, in some embodiments, the one or more memories may store one or more computer programs that, when executed by the one or more processors, cause the apparatus to perform the steps, services, and functions disclosed herein.

[0024] It should also be understood that the terms used herein are only for the purpose of describing specific embodiments, and are not intended to be limited. It should be noted that, as used in the specification and the appended claims, the words "one", "the" and "said" are intended to mean that there are one or more elements, unless the context clearly indicates otherwise. Therefore, for example, in some contexts, mentioning "unit" or "the unit" can refer to more than one unit, etc. In addition, the words "include", "comprise" do not exclude other elements or steps. It should be emphasized that the term "include / comprises" is used to indicate the existence of stated features, integral bodies, steps or parts when used in this specification. It does not exclude the existence or addition of one or more other features, integral bodies, steps, parts or their groups. The term "and / or" should be interpreted as also meaning "both" and each is alternative.

[0025] It will also be understood that, although the terms first, second, etc. can be used to describe various elements or features in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the embodiment, the first signal can be referred to as the second signal, and similarly, the second signal can be referred to as the first signal. Both the first signal and the second signal are signals, but they are not the same signal.

[0026] Overview

[0027] The technology disclosed herein provides a model-based technique for estimating a road model having any number of lanes on the road and any entry lanes (i.e., entry ramps or branches) and exit lanes (i.e., exit ramps or exit branches). In more detail, the embodiments disclosed herein utilize a Bayesian approach to perform multi-lane tracking in various traffic scenarios, such as highway scenarios. The model-based algorithm employed estimates the lane center curve of a multi-lane road based on vehicle motion data (e.g., speed, angular velocity) and perception data (e.g., camera output, lidar output, radar output, etc.), the perception data including detected road features (e.g., lane markings, road edges, road obstacles, guardrails, road signs, etc.) or other objects (e.g., other road users).

[0028] Specifically, according to some embodiments, the technology disclosed herein utilizes road parameterization that supports describing an arbitrary number of lanes, entry ramps, and exit ramps, and the road parameterization is suitable for tracking using a Bayesian filter. Specifically, the embodiments herein parameterize the road using a multi-object tracking filter coupled with multiple hypotheses. Multiple hypotheses are used to process detected data associations from road features, and coupled lanes are used to simulate parallel lanes. In addition, each lane within each hypothesis is given a probability of existence, and some embodiments handle object generation (i.e., lane generation) and extinction.

[0029] In the context of this disclosure, the term "road model" refers to a digital representation or virtual model of the road environment. It is considered to be a vital component used by autonomous vehicles to perceive and understand the surrounding road conditions. Road models typically include various elements, such as the geometry of the road and lane markings, and may also include traffic signs, traffic lights, obstacles, and other relevant features. The road model acts as a reference for the autonomous driving system to make decisions and safely navigate the vehicle. It allows the system to determine the vehicle's position, plan its trajectory, detect and track objects, interpret traffic signs and signals, and predict potential hazards or obstacles on the road. By continuously updating and analyzing the road model in real time, ADS can adapt to changing road conditions, make appropriate driving decisions, and ensure the safety and efficiency of the vehicle's operation.

[0030] In the following, relevant background theories and other details are presented to facilitate understanding of the embodiments disclosed herein.

[0031] There are many interpretations of the concept of probability, and Bayesian probability is one of them. In this framework, all events have a prior probability of occurring, which allows the representation of the probability of events even if no or limited experiments are done to test the ratio between positive and negative outcomes. The key to Bayesian probability is Bayes' theorem.

[0032] p(A|B)p(b)=p(B|A)p(A) (1.1)

[0033] This can be derived from p(A∪B)=p(A|B)p(B).

[0034] For the purpose of state estimation, we assume that our control system is described by a discrete state space model. At each time step k, we usually use the random variable x k to represent the state estimate, and also with the random variable y k The measured values ​​are received in the form of Figure 1 The Bayesian network representation of the state-space model is shown in . This figure visualizes some of the conditional independence between random variables that arise from the state-space representation, specifically

[0035] p(x k |x k-1 ,y 1:k-1 )=p(x k |x k-1 ) (1.2)

[0036] and

[0037] p(y k |x k ,y 1:k-1 )=p(y k |x k ) (1.3)

[0038] The goal of Bayesian state estimation is to express the given measurements from time 1 to k and the prior p(x k-1 |y i:k-1 )’s posterior state density p(x k |y 1:k ). However, to find p(x k |y 1:k ), we first need to find p(x k |y 1:k-1 ). Using equation (1.2) and the law of total probability, we obtain the Chapman-Kolmogorov equation

[0039] p(x k |y 1:k-1 )=∫p(x k |x k-1 )p(x k-1 |y 1:k-1 )dx k-1 (1.4)

[0040] Among them, Figure 1 We can deduce p(x k |x k-1 ,y1:k-1 )=p(x k |x k-1 ).

[0041] In addition, equation (1.3) combined with Bayes' theorem, that is, equation (1.1) gives

[0042]

[0043] Here, p(y k |x k ) is known as a measurement possibility. Since it is usually k We are interested in the density of , so we can treat the denominator of equation (1.5) as a constant normalization factor and discard it. However, it can be later replaced by enforcing p(x k |y 1:k ) should be recreated with a score of 1.

[0044] Thus, the basic building blocks of a general Bayesian state estimator have been formulated. Given a prior density p(x k-1 |y 1:k-1 ), we first perform the so-called prediction step of estimation by applying the Chapman-Kolmogorov equation (1.4) to obtain p(x k |y 1:k-1 ). Based on the known expression for the measurement likelihood, we then perform a so-called update step by applying equation (2.5) to obtain p(y k |x k ). This process is then repeated recursively for time steps k+1, k+2, etc. For the initial time step, an expression for the initial prior p(x0) needs to be provided.

[0045] The Kalman filter is a Bayesian state estimator specifically for linear and Gaussian systems. The linear Gaussian state space model is given by

[0046] x k =A k x k-1 +B k u k +q k (1.6)

[0047] y k =C k x k +D k u k +r k (1.7)

[0048] Among them A k , B k , Ck , D k is a constant matrix, u k is the control signal, and are independent random vector variables. Equations (1.6) and (1.7) are called the motion and measurement models, respectively, and q k and r k They are called process noise and measurement noise respectively.

[0049] It can be shown that for all time steps the state estimates will also be Gaussian distributed, i.e.

[0050]

[0051]

[0052] Thus, the Kalman filter is a conjugate prior filter, meaning that the prior and posterior densities belong to the same family, in this case Gaussian. For linear and Gaussian systems, it can also be shown that the Bayesian state estimation equation is specialized for

[0053] predict:

[0054]

[0055]

[0056] renew:

[0057]

[0058]

[0059]

[0060]

[0061] P k|k =P k|k-1 -K k C k P k|k-1 (1.10g)

[0062] here, represents the estimate of x up to and including the given observation time b ≤ a. Moreover, equation (1.10a) is the predicted state estimate, equation (1.10b) is the predicted covariance estimate, equation (1.10c) is the innovation or measurement residual, equation (1.10d) is the innovation or residual covariance, equation (1.10e) is the Kalman gain, equation (1.10f) is the updated state estimate, and equation (1.10g) is the updated covariance estimate.

[0063] The extended Kalman filter (EKF) is a variation of the Kalman filter suitable for nonlinear state-space models and is given by

[0064] x k =f k (x k-1 ,u k )+q k (1.11)

[0065] y k =h k (x k ,u k )+r k (1.12)

[0066] Process and measurement noise q k and r k Like in a normal Kalman filter. At each time step, the motion and measurement models are linearized around the prior mean. The normal Kalman filter equations are then applied to the linearization. This leads to an approximate solution to the state estimation problem, where all densities are assumed to be Gaussian.

[0067] The complete EKF equation is given as

[0068] predict:

[0069]

[0070]

[0071] renew:

[0072]

[0073]

[0074]

[0075]

[0076] P k|k =P k|k-1 -K k H k P k|k-1 (1.13g)

[0077] in

[0078]

[0079] Similar to before, equation (1.13a) is the predicted state estimate, equation (1.13b) is the predicted covariance estimate, equation (1.13c) is the innovation or measurement residual, equation (1.13d) is the innovation or residual covariance, equation (1.13e) is the Kalman gain, equation (1.13f) is the updated state estimate, and equation (1.13g) is the updated covariance estimate.

[0080] Filters such as the Kalman filter and its variants are suitable for tracking the state of a single object (such as a car or other vehicle) in a clutter-free environment, i.e., every measurement is known to originate from the tracked object. However, many real-world scenarios involve sensors that introduce clutter, or sometimes fail to detect real objects.

[0081] A typical example illustrating clutter and the possibility of not detecting a tracked object is tracking a single aircraft using a radar sensor. Each time the radar is polled, it returns a variable length list of range-angle detection pairs that were made since the last time step. Even given that we know for sure that only one aircraft is within the radar's field of view, the sensor may produce zero, one, or more detections when polled due to interference. This is in contrast to simple sensors such as those included in inertial measurement units (IMUs) and encoders, which always return a fixed-size data structure when polled. Those radar detections that are caused by interference and are not associated with the object we expect to track are called clutter. Given a set of detections from a radar, it is also possible that none of them correspond to a real object. If the radar does not detect anything at the current time step, this will obviously be the case, and may be, for example, the result of the object being at an unfavorable angle to the radar, or something is blocking the line of sight from the radar to the object. For example, in order to use such a sensor as a radar in a tracking system, methods for dealing with clutter and missed detections need to be developed.

[0082] One way to deal with clutter is to simply discard detections that cannot originate from a plane given a priori. This is called gating. In many more complex filters, gating forms an important part of the clutter suppression measure. In situations with low clutter intensities, it may be feasible to use gating alone to deal with clutter, however as the amount of clutter increases, it becomes more important to model the clutter in order to suppress it with greater accuracy. A commonly used model of clutter is the Poisson Point Process (PPP). In this model, the clutter measurements are independent and identically distributed. At each time step, the number n of clutter measurements in the volume V monitored by the sensor is calculated as the expected value In addition to introducing cluttered measurements, imperfect sensors may also fail to detect the tracked object at each time step. This property can be modeled using a Bernoulli process. Using a Bernoulli process model, at time step k, an object is detected with probability ρ and not detected with probability 1-ρ. Given that an object is detected, it is distributed according to some distribution with a probability density function g(y).

[0083] In addition, the Multi-Bernoulli Mixture (MBM) filter is an algorithm for tracking multiple individual objects in the presence of clutter. The MBM filter represents the state as a superposition of hypotheses, where each hypothesis is a union of Bernoulli processes. Each such Bernoulli process can be interpreted as a possible tracked object.

[0084] As described above, the methods and apparatus of the embodiments disclosed herein estimate road geometry by estimating lane centers of lanes of a road. According to some embodiments, the curve of the lane center is parameterized using a clothoid spline, a polynomial, or a lateral offset at a fixed longitudinal position. Although the present description may focus on lane center parameterization using a clothoid spline, it is readily understood by those skilled in the art that other parameterizations are also possible.

[0085] In general, a clothoid is a 2D curve with a constant rate of change of curvature along the curve. A clothoid spline is thus a connection of multiple clothoids that are constrained so that the curvature is continuous. To find the geometry of a clothoid spline, the following differential equation can be solved:

[0086]

[0087] where x(l), y(l) are the Cartesian coordinates of a point at distance l along the curve, φ(l) is the heading angle of the curve, and k(l) is a continuous piecewise linear function describing the curvature of each segment (within each segment, the slope k (i.e., curvature) is a constant).

[0088] In some embodiments, the centerline parameterization includes 3D clothoid splines. This may be advantageous in scenarios where the perception data obtained from the output of one or more onboard sensors includes 3D road feature coordinates. Accordingly, equation (1.15) may be expanded to 3D according to

[0089]

[0090] Therein, the azimuth angle θ(l) and its corresponding piecewise linear curvature γ(l) have been introduced.

[0091] exist Figure 2 A parameterized visualization of a road (ie, a “road model”) according to some embodiments is depicted in FIG. Figure 2Each of the sub-figures (a) to (d) in FIG. 1 shows a different subset of parameterization parameters according to some embodiments. Figure 2 In the example depicted in , the road has two main road lanes and one branch lane. The centerline curvature and width of the lanes are described by splines. As shown in sub-figure (b), the main road spline has four segments and the branch has three segments, i.e., in the upper right sub-figure. In this bird's-eye view (or top view), the parameters controlling the vertical position of the road are omitted for visual clarity. In short, the parameters y, x, and a i is the distance, φ S is the angle, k i is the curvature, and is the width. As mentioned above, curve and width parameterizations other than clothoids and linear splines can be used.

[0092] The ego lane center is described by the curvature of the edge at each segment. ego lane center, offset parallel to the lane center by half the lane width. Also, the adjacent main road lanes, i.e. other lanes of the main road, can be modeled using the same approach by describing them as parallel offsets to the ego lane using their widths. Each so-called main road lane m can be given its own width profile as a function of the segment edge widths. Parameterized piecewise linear function. The length of both the clothoid and the width segment can be defined as And there are N ( Finally, the number of main road lanes can be defined as M, and the index of the ego lane within these m lanes is m0.

[0093] The width parameterization may include a constant value, which may be predefined, for example, based on regulations in the vehicle's geographic jurisdiction (e.g., 10 to 11 feet in the United States, or 2.5 to 3.25 meters in Europe). However, in some embodiments, the width parameterization may include a function of the length (of the lane).

[0094] In addition, branch lanes can be introduced into the parameterization of the road in order to estimate the center of the lanes that merge into or diverge from the main road. b Attached to main road lane m b and makes an angle φ9 with the connecting main road lane. Starting from the root of the branch, its curve and similar curves of the ego lane can be represented by a curve with curvature The width of the branches can also be described by a piecewise linear function, which is given by parameterized, but constrained to have the same width as the main lane it is connected to (i.e., the main lane of the connecting road). Therefore, you can omit The lengths of the branching clothoid and the width segment are And there are N9 branch lane segments in total. The total number of branches is B.

[0095] In addition, the pose of the ego vehicle relative to the road can be parameterized. This can be done by describing the pose of the vehicle at the starting point 201 relative to the center of the ego vehicle road. The pose of the ego vehicle can be given by three spatial coordinates x, y and z and three Tait-Bryan angles φ, θ and ψ representing yaw, pitch and roll respectively.

[0096] As described above, the road parameterization proposed in this article includes a lane center parameterization for each lane of the road and a width parameterization for each lane of the road. This is in contrast to a variety of traditional road model estimation techniques, in which the road parameterization of traditional road model estimation techniques focuses on lane demarcations. However, the advantage of focusing the road parameterization on lane center parameterization and width parameterization rather than lane demarcation parameterization is that improved performance is obtained in the absence of lane demarcations near the edge of the road. Moreover, the lane center estimation is directly consumed by downstream functions of the ADS such as path planning functions because they are typically configured to drive the vehicle based on lane center estimation. Moreover, the tracking filter is more robust because it allows the use of measurement data (i.e., perception data) in the form of vehicle detections because it can be assumed that other vehicles try to place themselves in the center of the lane, and therefore vehicle detections can be associated with the lane center of the road parameterization. Moreover, another advantage of focusing the road parameterization on lane center parameterization and width parameterization rather than lane demarcation parameterization is that complex road geometries with branch lanes are easier to model, while when using road parameterization focused on lane demarcations, it is usually faced with the challenge of connecting the branch lanes to the appropriate main road lanes.

[0097] In order to formulate the dynamic model, a parameterized state space representation of the road (i.e., the road model) can be defined. All continuous parameters of the parameterized road (they are x, y, z, φ, θ, ψ, φ9, θ9, and a b ) are concatenated into the state vector x. In addition, all discrete parameters of the road model M, B, N and m i is concatenated into the metadata vector ξ. Furthermore, the road model can be represented by auxiliary variables ρ c∈[0,1], these auxiliary variables model the presence of each lane c=1,…,M+B of the road geometry, the main road lanes, and the branch lanes. In a sense, given the rest of the road model, lane c only exists with probability ρ independent of the presence of other lanes c exists. This is similar to the existence probabilities introduced for the MBM filter, but in contrast to the MBM filter, the object states of the tracking filter proposed in this paper are not described by a Bernoulli random finite set. These existence variables are transformed into a vector ρ. Therefore, the complete enhanced road model state is [x T ,ξ T ,ρ T ] T The control signal u at time step k is k Including vehicle speed v k , and optionally including the yaw rate ω z,k Moreover, the control signal u at time step k is k You can also include the scroll rate ω x,k and pitch rate ω y,k .

[0098] Going one step further, a geometric state transition model can be defined to describe the expected change in parameter values ​​from one time step to the next based on the control signal. Ignoring process noise, the motion model can include updating the ego vehicle pose,

[0099] x k =x k-1 +Δtv cosφ k-1 cosθ k-1 (1.17a)

[0100] y k =y k-1 +Δtv sinφ k-1 cosθ k-1 (1.17b)

[0101] z k =z k-1 +Δtv sinθ k-1 (1.17c)

[0102] φ 0,k =φ 0,k-1 +Δtω z,k (1.17d)

[0103] θ 0,k =θ 0,k-1 +Δtω y,k (1.17e)

[0104] ψ 0,k =ψ0,k-1 Δtω x,k (1.17f)

[0105] where Δt is the time step, and all other components of x remain constant except for process noise. Furthermore, to compensate for the motion of the ego vehicle along the main road, whenever the vehicle has traveled too far along the main road, a segment is removed from the back of the road and simultaneously a segment is added to the front. When this happens, all state variables of the segments connected to the main road are shifted backwards in the state vector to reflect the new segment layout. To facilitate merging, branches do not change their length or number of segments, but simply always keep the length they were initialized with.

[0106] Considering that the number of lanes in the road may increase from one time step to the next, the state transition will add some new lanes to the state at each time step. In some embodiments, the tracking filter can apply a lane generation model, where lane generation is based on the assumption that the number of lanes does not increase very quickly. Therefore, these new lanes are given a low probability of existence ρ ^ In some embodiments, a single additional main road lane is generated at the leftmost and rightmost sides of the main road. The state parameters controlling the width of the main road lane and the index m S Also shifted accordingly. In addition, regarding the generation of branch lanes, a forked branch lane and a merging branch lane can be generated starting from the rightmost main lane with sufficient width and existence probability. For reasons of computational performance, the generation of branches on the left side of the main road can be ignored, especially for highway scenarios, because such entrance ramps or exit ramps are not common in places where people drive on the right side of the road. Therefore, the addition or generation of branch lanes may depend on local traffic rules (i.e., whether the current geographical jurisdiction of the vehicle applies right-hand or left-hand driving).

[0107] In addition, in some embodiments, the tracking filter can apply a lane disappearance model. In more detail, the survival probability P is introduced. j , the survival probability represents the probability of each individual lane surviving from one time step to the next time step. The survival probability is then used to update the existence probability of each lane according to the following formula

[0108] ρ ^,k =P j ρ ^,k-1 c=1,…,M k-1 +B k-1 (1.18)

[0109] Additionally, lanes whose width is below a certain threshold and branches where a9 has become negative may be removed from the filter state.

[0110] The geometric state transition model, lane generation model, and lane extinction model can be summarized into three transition equations:

[0111] x k =f f (x k-1 ,ξ k-1 ,ρ k-1 ,u k )(1.19a)

[0112] ξ k =f k (x k-1 ,ξ k-1 ,ρ k-1 ,u k )(1.19b)

[0113] ρ k =f l (x k-1 ,ξ k-1 ,ρ k-1 ,u k )(1.19c)

[0114] where process noise has been excluded. In particular, f f Generally given by equations 1.17a to 1.17f, ignoring the generation and shifting of road segments to avoid negative and when the number of segments in the branch increases. The complete dynamic model including noise is given by

[0115] x k =f f (x k-1 ,ξ k-1 ,ρ k-1 ,u k )+q k (1.20a)

[0116] ξ k =f k (x k-1 ,ξ k-1 ,ρ k-1 ,u k )(1.20b)

[0117] ρ k =f l (x k-1 ,ξ k-1 ,ρ k-1 ,u k )(1.20c)

[0118] in, is the process noise.

[0119] In addition, the description of the measurement value is provided , where are the coordinates of points of road features (e.g., lane markings) that would be expected to be received given the value of the state vector. In some embodiments of the model proposed herein, lane marking measurements may occur only at either edge of the lane, apart from clutter, separating the center of the lane by half the width of the lane. These lines may be referred to as demarcations. The process of associating true measurements with the demarcations of the model is handled by filters and is discussed further below. For the sake of notational simplicity, the time step index k is discarded in reference to the measurement model discussion. In addition, for purposes of illustration, the following discussion will focus on road features in terms of lane markings or demarcations. However, this should not be construed as meaning that the measurement model is limited to measurement data including road features in terms of lane markings, but may additionally or alternatively include other road features and even other objects (i.e., other vehicles).

[0120] Figure 3a is a schematic diagram of an image captured by a front-facing onboard camera, to which re-projected lane marking detections have been added. Detection shows the typical output of a 3D lane marking detection algorithm. Figure 3b is a bird's-eye view of the same 3D lane marking points reprojected into the camera image. In more detail, the output of a 3D lane marking detection algorithm is a set of 3D coordinates along with uncertainty estimates for these coordinates. Since uncertainty estimates are typically provided in the output of 3D lane marking detection, they are suitable for use as measurements in the update step of the tracking filter proposed in this paper.

[0121] Moreover, only existing boundaries can be detected. From ρ, only the probability of each lane center existing can be directly obtained, but it can be assumed here that given the existence of a lane, both its right boundary and left boundary also exist. Each existing boundary has a probability of being detected P m A detected boundary gives at least one measured value, whereas an undetected boundary gives no measured value. The measured value given by each detected boundary is described below.

[0122] like Figure 2 As depicted in , the parameterization of the road model according to some embodiments provides a description of the geometry of the demarcation in the local coordinate frame of the ego road. Since the demarcation of the host road is described relative to the ego lane center, the geometry of this center line is required. This can be done, for example, by integrating equation (1.16) for the lane center parameter using the Euler method with a step size Δl, with the initial conditions for the coordinates and angles being set to zero when working in the local coordinate frame of the ego road. From this we get

[0123]

[0124] Among them, k ((l) and γ ( (l) respectively represent relative to and Linear interpolation of parameters.

[0125] Similar differential equations, this time for linear splines instead of clothoid splines, are solved in the same way to obtain the corresponding to the different main lanes The width (w (U,S ) S Now it is possible to give the coordinates of points on the main road boundary.

[0126]

[0127] in, is a rotation matrix representing the rotation angle φ around the z-axis, and is the matrix used to calculate the offset of the boundary d from the ego lane. Since there are M main road lanes, there are M+1 main road boundaries. For M=2 and ego lane index m0=2, for example,

[0128] A1=[-1 -0,5],A2=[0 -0,5],A1=[0 0,5] (1.23)

[0129] The coordinates of points on the branch lanes can be calculated in a similar way, first solving the differential equations for the centerline parameterization (e.g., clothoid) and the lane width parameterization using the Euler method. As initial conditions for the centerline parameterization, the position is calculated by linear interpolation of the center polyline of the parent main road lane (similar to equation (1.22), but with the A matrix modified accordingly), with the distance a along the center of the ego lane b (See e.g. Figure 2 The initial angle is similarly obtained by interpolating the main road lane angle and then adding the initial angle φ of the branch b and θ b The initial curvature is directly given as and For the initial width of the branch lane, this is interpolated from the width of the parent main road lane (i.e. the lane connecting the main road). In general, taking into account both the main road boundary and the branch lane boundary, there will be a sequence of coordinates r b,i There are D=M+1+2B boundaries.

[0130] In the previous description, the description of the road geometry is given relative to the road frame. However, measurements (i.e., perception data) are usually given in the ego vehicle frame. This frame is aligned with the vehicle, with its origin at the center of the rear axle, the x-axis pointing forward, the y-axis pointing to the left along the rear axle, and the z-axis pointing up. However, the parameters x, y, z and φ, θ, ψ of the ego vehicle pose can be converted to the road-aligned lane demarcation geometry (r) in the ego vehicle frame. d,i ) i , generating the sequence (r′ d,i ) i .

[0131] Assume that lane marking detection is performed along (r′ d,i ) i According to some embodiments, it is assumed that given the measured coordinates y of the lane marking originating from the demarcation d j , which is closest to y on the dividing polyline j points. Furthermore, as mentioned before, the raw measurements are received in the form of 3D coordinates. In order to reduce the dimensionality of the measurement vector, a more compact representation of the innovation than the differences between 3D points can be sought. An example representation of the innovation is the distance vector between the measured lane marking coordinates and their corresponding delimiting polylines. In that case, the measurement function h(x,ξ) is always the zero vector. However, the measurement model can also include additional measurement noise where R is the 3D coordinate y j The covariance matrix of R is y, and T is the correlation matrix, which is used to transform R into y j Covariance of the distance to the cutoff.

[0132] Having defined the road model, a filter must then be designed to estimate the parameters of the road model. According to some embodiments, the tracking filter is a multiple hypothesis tracker. In each hypothesis h, the x part of the state consists of and the covariance matrix P h is represented by a single Gaussian density. Therefore, all lanes of the road are coupled. The remaining states, ξ and ρ, are assumed to be known and are equal to and Therefore, the uncertainty in these variables comes from having multiple assumptions.

[0133] The prior state density of the filter can be described by

[0134]

[0135] in, is the hypothesis weight.

[0136] To simplify the calculation, we can assume that the survival probability P S is constant and independent of the lane, and can be used with a constant strength λ C The filter algorithm itself can further support state-dependent survival probabilities and other clutter models besides PPP. For the detection probability, it can be made state-dependent so that it is larger for the lanes in front of and near the ego vehicle and tapers off outside this area. Moreover, to simplify the computation, the detection probability does not need to be estimated for the entire state density, but only needs to be estimated to estimate the mean

[0137] Example

[0138] Figure 4 is a schematic flow chart representation of a method S100 for estimating the road geometry of a road on which a vehicle is traveling. The road geometry is described by a filter state tracked by a tracking filter, wherein the filter state includes a plurality of road model hypotheses. The tracking filter may be referred to as a multiple hypothesis multi-object tracking filter, which is designed to track filter states. Moreover, each road model hypothesis is associated with a probability and also includes a parameterization of the road describing the geometry of one or more lanes of the road, and each lane of each road model hypothesis is associated with an existence probability. Accordingly, the filter state contains a probability density of parameters relative to the road parameterization. In other words, the filter state is described by a set of hypotheses corresponding to the configuration of the road model (i.e., the parameterization of the road), and each hypothesis has an associated probability. However, in each hypothesis, each lane has an existence probability.

[0139] Furthermore, as described above, according to some embodiments, the "objects" that are tracked by the tracking filter are the lane center and lane width. Therefore, the tracking filter does not track lane markings. However, knowing the geometry of the lane center and width, it follows that the left and right boundaries of the lane can be described, which is where lane markings would be expected to appear. Moreover, the tracking filter is hypothesis oriented, not tracking oriented. Furthermore, all parameters of the road model (i.e., parameterization of the road) are coupled.

[0140] The method S100 is preferably a computer-implemented method S100 executed by a processing system of a vehicle equipped with an ADS. The processing system may, for example, include one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories store one or more programs that, when executed by the one or more processors, perform the steps, services, and functions of the method S100 disclosed herein.

[0141] Method S100 is executed as a loop, and depending on set requirements and / or available computing power and / or available memory resources, the method steps are repeated at a certain sampling frequency (e.g., 10 Hz, 20 Hz, 50 Hz, etc.). Thus, for each time step in a plurality of consecutive time steps, method S100 includes predicting S101 filter states by determining a probability density for each road model hypothesis based on vehicle motion data and a priori filter states determined at a previous time step. In some embodiments, the predicting S101 step includes an extended Kalman filter (EKF) prediction of state estimation and covariance estimation for all hypotheses using a motion model that describes the a priori filter states (such as, for example, the motion model defined in equation (1.20a)). In this article, vectors ξ and ρ can be updated according to equations (1.19a) and (1.19b), respectively. The weight of each hypothesis can remain unchanged. In some embodiments, the vehicle motion data includes the vehicle speed at the time step. However, the vehicle motion data may also include the yaw rate, roll rate, and / or pitch rate at the time step.

[0142] In some embodiments, method S100 includes adding S108 one or more main road lanes and / or one or more branch lanes to each road model hypothesis of the filter state of prediction S101. Each of these lanes added S108 includes a corresponding parameterization (e.g., having a lane center parameterization and a lane width parameterization). The lanes added S108 can be interpreted as candidates for new lanes that have not yet been detected. Moreover, according to some embodiments, method S100 also includes marking S109 one or more main lanes and / or one or more branch lanes added S108. By marking S109 the lanes added S108, it is convenient to subsequently remove the lanes added S108 that are not associated with any road features obtained from the perception data.

[0143] Method S100 also includes associating S102 a parameterized representation of a roadway obtained from perception data derived from outputs of one or more onboard sensors with a roadway assumed by each road model for predicting filter states. This may also be referred to as data association S102. In some embodiments, the roadway features obtained from the perception data derived from outputs of one or more onboard sensors include lane marking detections obtained from a lane tracking algorithm configured to output coordinates of lane markings depicted in an image output from an onboard camera.

[0144] The term "perception data" refers to information collected by sensors and other technologies that ADS-equipped vehicles use to detect and interpret their environment. This includes data collected from cameras, lidar, radar, and other sensors that help the vehicle "perceive" its surroundings and make decisions based on that information. Perception data collected by a vehicle may include the location, speed, and direction of nearby objects, the location and type of road markings, the location and type of traffic signs, and other relevant information. This data can then be processed by the vehicle's onboard computer to help it make decisions about steering, acceleration, braking, and other actions required to safely navigate the environment. Accordingly, the term "perception" data may refer to "environmental estimation" data, "spatial perception" data, "processing sensory" data, and / or "time-correlation" data, while perception "data" may refer to perception "information" and / or "estimation". On the other hand, the term "obtained" from a perception module or perception system may refer to "obtained" from a perception model and / or "based on output data" from a perception module or system. A perception module / system configured to "generate a perception data set" may refer to a perception module / system adapted and / or configured to "estimate the surroundings of the vehicle", "estimate at least a portion of the surroundings of the vehicle", "determine the surroundings of the vehicle", "interpret sensor information related to autonomous manipulation of the vehicle" and / or "estimate the surroundings of the vehicle and perform model predictions on future states of the surroundings of the vehicle".

[0145] In the context of the present disclosure, data association S102 refers to the process of associating measurements from various sensors with corresponding features in the parameterization of each hypothesized road. It involves determining which measurements correspond to which objects or elements in the environment, such as lane boundaries. In the data association step S102, the tracking filter attempts to match the incoming sensor measurements with features in the parameterization of the road by considering the probability of each potential association. In general, the goal is to find the most likely correspondence between the measurements and the parameters of the road parameterization.

[0146] Furthermore, according to some embodiments, prior to the data association step S102, one or more main road lanes and / or one or more branch lanes are generated or added S108 for each road model hypothesis predicting S101 the filter state. This may be counter-intuitive, as it would be assumed that the generated lane candidates may be better estimates once measurement data is available. However, by performing lane generation S108 prior to data association S102, the tracking algorithm is easier to implement. Moreover, performing lane generation S108 prior to data association S102 enables the tracking filter to better handle complex scenarios (i.e., complex road geometries) as the tracking filter is not limited by measurement data during lane generation, and also provides improved accuracy as the lane generation after the data association process will be a coarser mathematical approximation, which leads to the risk of missing potential lanes.

[0147] Furthermore, the method S100 comprises determining S104 the probability of existence of each lane of the road model hypothesis based on the prior probability of existence of each corresponding lane determined at the previous time step and the road feature association S102. In other words, the probability of existence of a particular lane is calculated based on the road feature association, taking into account the degree of match between the road features (obtained from the perception data) and the lane parameterization within the hypothesis, and the prior probability of existence of the lane determined at the previous time step is merged. In more detail, determining the probability of existence of a lane can be interpreted as estimating the likelihood or probability that the lane actually exists in the road environment within the road model hypothesis. This concept is particularly useful when dealing with complex road scenes, in which lane markings may be partially obscured, faded or missing. In order to determine the probability of existence of a lane, the tracking filter can be configured to take into account factors such as the reliability and consistency of the measurements over time (i.e., the road features obtained from the perception data). For example, if the measurements are always consistent with the predicted lane state, the probability of existence can be increased. Conversely, if the measurements are sporadic, inconsistent or significantly deviate from the predicted state, the probability of existence may be reduced.

[0148] Further, method S100 includes determining S105 the probability of each road model hypothesis of a plurality of road model hypotheses based on a prior filter state determined at a previous time step and the determined probability of existence of each lane of the road model hypothesis. The determination S104 of the probability of existence and the determination S105 of the probability of each hypothesis can be considered as an "update" step of Bayesian filtering, such as an EKF update step. However, in some embodiments, the data association S102 forms part of the update step. It should be noted that the update "step" may include several parts or sub-steps. Moreover, the update step S103 or the data association step S102 may also include updating, correcting or changing the parameters of the parameterization of the road (i.e. the parameters of the road model) to better fit the road characteristics obtained from the perception data (i.e. better fit the measurement data).

[0149] In some embodiments, the method S100 further comprises removing S110 any main road lanes and branch lanes added S108 without road feature association. In other words, if the newly added potential candidate lanes do not have any road features associated with them, they are removed from the filter state.

[0150] The method S100 further comprises selecting S106 the road model hypothesis associated with the highest probability based on the determined probability of each road model hypothesis. The method S100 may further comprise forming S111 a road model based on the parameterization of the road of the selected road model hypothesis by converting the road model parameterization into polylines describing lane centers of lanes of the road. In other words, an appropriate representation of the road geometry and its uncertainty is extracted from the filter state.

[0151] In addition, method S100 includes outputting S107 a road model based on a parameterization of a road assumed by the selected road model. In some embodiments, the road model is output to a path planning function of an autonomous driving system of the vehicle, and the path planning function is configured to generate a path for the vehicle to execute. However, as will be readily appreciated by those skilled in the art, the path planning function is only an example of several "downstream" functions from the ADS that rely on road model input. Other examples include trajectory planning functions, emergency action functions, pilot functions, lane change assistance functions, and the like.

[0152] Furthermore, according to some embodiments, after determining the hypothesis probability S105, the tracking filter recursively returns to the prediction step S101. However, in some embodiments, after determining S104 of the existence probability S104, or after adding the removal of the lane S110, the tracking filter recursively returns to the prediction step S101.

[0153] In addition, with respect to the updating step S103, in some embodiments, the input to the updating step S103 is respectively a priori filter states (e.g., a set of a priori hypotheses) and measurement data, which includes coordinates of road features (e.g., lane marking detections) and their corresponding covariances. In the following, the updating step of a single hypothesis according to some embodiments will be described. The same process can be applied to all hypotheses, and the resulting a posteriori hypotheses are concatenated at the end of the updating step. For the updating, the road features obtained from the perception data may include J lane marking coordinate measurements. and their joint covariance matrix

[0154] In some embodiments, the first half of the update step is used to generate multiple data association hypotheses. For simplicity, the measurement point y can be gated first. {This can be done by considering each point individually and comparing it to all available cutoffs d. Points are only allowed to be associated with cutoffs whose Mahalanobis distance is below a threshold, and only with cutoffs that maximize the probability of The probability of an approximate association is determined by a limited number of demarcation associations, where Δ {,t Yes { and the boundary d and The nearest distance between the corresponding innovation covariances. Next, a clustering algorithm can be used to cluster these points, and a similar gating process can be performed for each cluster. However, clusters are also allowed to be associated with both the branch boundary and the main road boundary of the branch's parent lane. Therefore, multiple lane marking detections can be associated with each boundary. After performing these steps, each cluster has a set of possible associations for points within the cluster, either as boundaries or as clutter. Next, all possible combinations of cluster associations are generated to create a first set of data association candidates. Finally, these candidates are further restricted by not allowing two potentially incompatible lane marking points to be associated with the same boundary. Compatibility between lane marking clusters is checked by fitting a cubic polynomial to the markings of the two clusters and checking the scores of the inner layer from each cluster.

[0155] Then, we focus on a single data association hypothesis, and under this hypothesis, d(j) = 0 if y j associated with clutter, let d(j) be the measured value y in this hypothesis j The goal is to perform an EKF update step. To achieve this, we start from Δ j,d(j) The connection calculation and construction of the innovation vector, for d(j) ≠ 0, the matrix H (see equation (1.14)) is obtained by finding j Direction a j,d(j) The closest to y j Finally, the measurement noise is TRT T , where T is a subset of row j such that the block diagonal matrix d(j)≠0. Using these vectors and matrices, we can calculate the innovation covariance S and the posterior mean and covariance Here h′ is the index of the posterior hypothesis derived from h associated with the data under consideration.

[0156] Next, calculate the updated hypothesis weights and the probability of existence Given the data association, we know which subset of all boundaries D is detected, that is,

[0157] D D={d(j)∶j∈J,d(j)≠0} (1.25)

[0158] where J = {1,…,J}. If we let C be the set of all lanes c, then there may be D The combination of lanes in which the measurements are obtained is

[0159]

[0160] Where P(C) is the set C E The power set of , and f(C E ) belongs to C E The set of boundaries for any lane in . E ∈C D The lanes that do exist in the image and the D that is detected accurately outside the boundaries of those lanes D The probability is

[0161]

[0162] Note that lane-independent detection probabilities have been assumed in this equation. However, the filter can have different detection probabilities for each lane. Given a known C E Must be C D The elements of can be calculated by normalizing the corresponding conditional probability

[0163]

[0164] Accordingly, C D Contains possible combinations of lanes that are compatible with data association. Each combination has a weight However, in order to obtain a recursive filter recursion, it is desirable to make the presence of lanes in the posterior independent. This can be done for all |C D | cases, but this seems inefficient since the state densities in these hypotheses all have the same mean and covariance. Therefore, all cases can be summarized in a single hypothesis in which lane existence is approximated as independent. The updated existence probability is calculated as follows

[0165]

[0166] Then, to complete the update step, all that remains is to calculate the new hypothesis weights It has four contributing factors: prior weight The probability of data association, clutter strength, and measurement likelihood. Putting all of this together, we get

[0167]

[0168] Then, by combining all the generated hypotheses and normalizing the weights So that their sum is 1, we get the posterior density.

[0169] Furthermore, in order to keep the number of road model hypotheses manageable, a capping, pruning and / or merging process may be employed after the updating step S103. In more detail, for the merging process, all hypotheses with compatible geometries may be merged. That is, the hypotheses contain the same number of main road lanes and branch lanes and the same type of branch lanes (entry ramps or exit ramps (determined according to the angle φ9)), and all lanes have the same number of segments.

[0170] Executable instructions for performing these functions are optionally included in non-transitory computer-readable storage media or other computer program products configured to be executed by one or more processors.

[0171] Figure 5 is a schematic diagram of an ADS-equipped vehicle 1 according to some embodiments, the ADS-equipped vehicle 1 comprising an apparatus 10 for estimating a road geometry of a road on which the vehicle 1 is traveling. The road geometry is described by a filter state tracked by a tracking filter, and the filter state comprises a plurality of road model hypotheses. Moreover, each road model hypothesis is associated with a probability and comprises a parameterization of the road describing the geometry of one or more lanes of the road, and wherein each lane of each road model hypothesis is associated with a probability of existence. As used herein, a "vehicle" is any form of motorized transport. For example, the vehicle 1 may be any road vehicle, such as a car (as shown herein), a motorcycle, a (freight) truck, a bus, etc.

[0172] The device 10 includes a control circuit 11 and a memory 12. The control circuit 11 may physically include a single circuit device. Alternatively, the control circuit 11 may be distributed over several circuit devices. As an example, the device 10 may share its control circuit 11 with other parts of the vehicle 1 (e.g., the ADS 310). Moreover, the device 10 may form a part of the ADS 310, that is, the device 10 may be implemented as a module or feature of the ADS. The control circuit 11 may include one or more processors, such as a central processing unit (CPU), a microcontroller, or a microprocessor. One or more processors may be configured to execute program codes stored in the memory 12 so as to perform various functions and operations of the vehicle 1 in addition to the methods disclosed herein. The processor may be or include any number of hardware components for performing data or signal processing or for executing computer codes stored in the memory 12. The memory 12 may optionally include a high-speed random access memory, such as a DRAM, SRAM, DDR RAM, or other random access solid-state memory device; and may optionally include a non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 12 may include database components, object code components, script components, or any other type of information structure for supporting the various activities described herein.

[0173] In the example shown, the memory 12 also stores map data 308. The map data 308 may be used, for example, by the ADS 310 of the vehicle 1 in order to perform autonomous functions of the vehicle 1. The map data 308 may include high-definition (HD) map data. It is contemplated that the memory 12 may be provided as an integrated element of the ADS 310 even if it is shown as a separate element from the ADS 310. In other words, according to exemplary embodiments, any distributed or local memory device may be utilized in the implementation of the present invention. Similarly, the control circuit 11 may be distributed, for example, so that one or more processors of the control circuit 11 are provided as an integrated element of the ADS 310 or any other system of the vehicle 1. In other words, according to exemplary embodiments, any distributed or local control circuit device may be utilized in the implementation of the present invention. The ADS 310 is configured to perform functions and operations of the automatic or semi-automatic functions of the vehicle 1. The ADS 310 may include a plurality of modules, each of which is responsible for a different function of the ADS 310.

[0174] The vehicle 1 includes a number of elements commonly found in autonomous or semi-autonomous vehicles. It will be appreciated that the vehicle 1 may have Figure 5 Any combination of the various elements shown in FIG. Moreover, the vehicle 1 may include Figure 5. Although various elements are shown herein as being located inside the vehicle 1, one or more elements may be located outside the vehicle 1. For example, map data may be stored in a remote server and accessed by various components of the vehicle 1 via the communication system 326. In addition, even if various elements are described herein in a certain arrangement, the various elements may also be implemented in a different arrangement, as will be readily understood by those skilled in the art. It should also be noted that the various elements may be communicatively connected to each other in any suitable manner. Figure 5 The vehicle 1 should be regarded as merely an illustrative example, as elements of the vehicle 1 may be implemented in several different ways.

[0175] The vehicle 1 also includes a sensor system 320. The sensor system 320 is configured to acquire sensory data about the vehicle itself or its surroundings. The sensor system 320 may, for example, include a global navigation satellite system (GNSS) module 322 (such as GPS) configured to collect geographic location data of the vehicle 1. The sensor system 320 may also include one or more sensors 324. The sensor 324 may be any type of onboard sensor, such as a camera, a lidar and radar, an ultrasonic sensor, a gyroscope, an accelerometer, an odometer, etc. It should be understood that the sensor system 320 may also provide the possibility of acquiring sensory data directly or via a dedicated sensor control circuit in the vehicle 1.

[0176] The vehicle 1 also includes a communication system 326. The communication system 326 is configured to communicate with external units, such as other vehicles (i.e., via a vehicle-to-vehicle (V2V) communication protocol), a remote server (e.g., a cloud server), a database, or other external devices (i.e., a vehicle-to-infrastructure (V2I) or a vehicle-to-everything (V2X) communication protocol). The communication system 326 can communicate using one or more communication technologies. The communication system 326 may include one or more antennas (not shown). Cellular communication technology can be used for remote communications, such as a remote server or a cloud computing system. In addition, if the cellular communication technology used has low latency, it can also be used for V2V, V2I, or V2X communications. Examples of cellular radio technologies are GSM, GPRS, EDGE, LTE, 5G, 5G NR, etc., also including future cellular solutions. However, in some solutions, medium- and short-range communication technologies, such as wireless local area networks (LANs), such as solutions based on IEEE 802.11, can be used to communicate with other vehicles in the vicinity of the vehicle 1 or with local infrastructure elements. ETSI is developing cellular standards for vehicle communications, and 5G, for example, is seen as a suitable solution due to its low latency and efficient handling of high bandwidth and communication channels.

[0177] The communication system 326 may accordingly provide the possibility to send outputs to a remote location (e.g. a remote operator or a control center) and / or receive inputs from a remote location by means of one or more antennas. Moreover, the communication system 326 may further be configured to allow the various elements of the vehicle 1 to communicate with each other. As an example, the communication system may provide a local network setup such as CAN bus, I2C, Ethernet, optical fiber, etc. The local communication within the vehicle may also be of wireless type with protocols such as WiFi, LoRa, Zigbee, Bluetooth or similar medium / short range technologies.

[0178] The vehicle 1 also includes a steering system 320. The steering system 328 is configured to control the steering of the vehicle 1. The steering system 328 includes a steering module 330, which is configured to control the heading of the vehicle 1. The steering system 328 also includes a throttle module 332, which is configured to control the actuation of the throttle of the vehicle 1. The steering system 328 also includes a brake module 334, which is configured to control the actuation of the brakes of the vehicle 1. The various modules of the steering system 328 can also receive manual inputs from the driver of the vehicle 1 (i.e., from the steering wheel, accelerator pedal, and brake pedal, respectively). However, the steering system 328 can be communicatively connected to the ADS 310 of the vehicle to receive instructions on how the various modules of the steering system 328 should act. Therefore, the ADS 310 can control the steering of the vehicle 1, for example, via the decision and control module 318.

[0179] The ADS 310 may include a positioning module 312 or positioning block / system. The positioning module 312 is configured to determine and / or monitor the geographic location and heading of the vehicle 1 and may utilize data from a sensor system 320, such as data from a GNSS module 322. Alternatively or in combination, the positioning module 312 may utilize data from one or more sensors 324. The positioning system may alternatively be implemented as a real-time kinematic (RTK) GPS for improved accuracy.

[0180] The ADS 310 may also include a perception module 314 or a perception block / system 314. The perception module 314 may refer to any known module and / or functionality, such as included in one or more electronic control modules and / or nodes of the vehicle 1, adapted and / or configured to interpret sensory data (related to driving of the vehicle 1) to identify, for example, obstacles, vehicle lanes, relevant signs, appropriate navigation paths to be executed by the path planning module 316, etc. Thus, the perception module 314 may be adapted to rely on and obtain inputs from multiple data sources, such as vehicle imaging, image processing, computer vision, and / or in-vehicle networking, etc., in combination with, for example, sensory data from the sensor system 320.

[0181] The positioning module 312 and / or the perception module 314 may be communicatively connected to the sensor system 320 to receive sensing data from the sensor system 320. The positioning module 312 and / or the perception module 314 may also send control instructions to the sensor system 320.

[0182] Accordingly, the apparatus 10 includes a control circuit 11 configured to predict a filter state for each of a plurality of consecutive time steps by determining a probability of each road model hypothesis based on vehicle motion data and a priori filter states determined at a previous time step. The control circuit 11 is further configured to associate road features obtained from perception data with a parameterization of a road for each road model hypothesis for predicting the filter state, the perception data originating from outputs of one or more onboard sensors. Moreover, the control circuit 11 is configured to determine a probability of existence of each lane for each road model hypothesis based on a priori probability of existence of each corresponding lane determined at a previous time step and the road feature association. The control circuit is further configured to determine a probability of each road model hypothesis among a plurality of road model hypotheses based on a priori filter states determined at a previous time step and the determined probability of existence of each lane of the road model hypothesis, select a road model hypothesis associated with a highest probability based on the determined probability of each road model hypothesis, and output a road model based on the parameterization of the road of the selected road model hypothesis.

[0183] The present invention has been presented above with reference to specific embodiments. However, other embodiments than those described above are possible and within the scope of the present invention. Within the scope of the present invention, method steps different from the above-described method steps of executing the method by hardware or software may be provided. Therefore, according to an exemplary embodiment, a non-transitory computer-readable storage medium is provided that stores one or more programs configured to be executed by one or more processors of a processing device, the one or more programs comprising instructions for executing a method according to any of the embodiments discussed above. Alternatively, according to another exemplary embodiment, a cloud computing system may be configured to perform any of the methods presented herein. The cloud computing system may include distributed cloud computing resources that jointly perform the methods presented herein under the control of one or more computer program products.

[0184] In general, a computer-accessible medium may include any tangible or non-transitory storage medium or storage medium such as electronic, magnetic, or optical media (e.g., a disk or CD / DVD-ROM coupled to a computer system via a bus). As used herein, the terms "tangible" and "non-transitory" are intended to describe a computer-readable storage medium (or "memory") that does not include propagating electromagnetic signals, but are not intended to limit the type of physical computer-readable storage device encompassed by the phrase computer-readable medium or memory. For example, the term "non-transitory computer-readable medium" or "tangible memory" is intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in non-transitory form on a tangible computer-accessible storage medium may also be transmitted via a transmission medium or signals (such as electrical, electromagnetic, or digital signals), which may be transmitted via a communication medium (such as a network and / or wireless link).

[0185] The processor 11 (associated with the device 10) may be or may include any number of hardware components for performing data or signal processing or for executing computer code stored in the memory 12. The device 10 has an associated memory 12, and the memory 12 may be one or more devices for storing data and / or computer code, which are used to complete or promote the various methods described in this description. The memory may include volatile memory or non-volatile memory. The memory 12 may include a database component, an object code component, a script component, or any other type of information structure for supporting the various activities described herein. According to an exemplary embodiment, any distributed or local memory device may be used with the system and method described herein. According to an exemplary embodiment, the memory 12 is communicatively connected to the processor 11 (e.g., via a circuit or any other wired, wireless or network connection) and includes computer code for executing one or more processes described herein.

[0186] Accordingly, it should be understood that parts of the described solutions may be implemented in the vehicle 1, in a system located external to the vehicle 1, or in a combination of the inside and outside of the vehicle; for example in a server communicating with the vehicle, a so-called cloud solution. The different features and steps of the embodiments may be combined in other combinations than those described.

[0187] It should be noted that any reference signs do not limit the scope of the claims, that the present invention may be implemented at least partially by means of both hardware and software, and that several "means" or "units" may be represented by the same item of hardware.

[0188] Although the accompanying drawings can show the specific order of method steps, the order of these steps may be different from the order of the steps depicted. In addition, two or more steps may be performed simultaneously or partially simultaneously. Such a change will depend on the selected software and hardware system and the designer's choice. All such changes are within the scope of the present invention. Similarly, software implementation can be completed with standard programming techniques, which have rule-based logic and other logic to complete various prediction steps, association steps, determination steps, selection steps, addition steps, removal steps, marking steps and output steps. The embodiments mentioned and described above are given as examples only, and should not be limited to the present invention. For those skilled in the art, other solutions, uses, goals and functions within the scope of the present invention claimed for protection by the patent claims should be obvious.

Claims

1. A computer-implemented method (S100) for estimating a road geometry of a road on which a vehicle is traveling, wherein: The road geometry is described by a filter state tracked by a tracking filter, the filter state comprising a plurality of road model hypotheses, wherein each road model hypothesis is associated with a probability, and comprising a parameterization of the road describing the geometry of one or more lanes of the road, and wherein each lane of each road model hypothesis is associated with a probability of existence, the method comprising: For each of a number of consecutive time steps: predicting (S101) filter states by determining a probability density for each road model hypothesis based on vehicle motion data and a priori filter states determined at a previous time step; Associating road features obtained from sensory data originating from outputs of one or more on-board sensors with the parameterization of the road assumed by each road model of the predicted filter state (S102); determining (S104) an existence probability of each lane of each road model hypothesis based on the a priori existence probability of each corresponding lane determined at the previous time step and the road feature association; determining (S105) a probability of each of the plurality of road model hypotheses based on the a priori filter state determined at the previous time step and the determined existence probability of each lane of the road model hypothesis; Based on the determined probabilities of each road model hypothesis, selecting (S106) the road model hypothesis associated with the highest probability; and A road model is output (S107) based on the parameterization of the road of the selected road model hypothesis.

2. The method (S100) according to claim 1, wherein: The parameterization of the road includes a lane center parameterization of each lane of the road and a width parameterization of each lane of the road.

3. The method (S100) according to claim 1 or 2, wherein: The parameterization of the road comprises: The ego lane center curvature (k0) of each segment of the ego lane of the main road, The width of each segment of the ego lane (ω 01 ), The width (ω) of each segment of one or more adjacent main road lanes of the main road 02 ),as well as The pose (x, y, φ0) of the vehicle relative to the starting point of the ego lane center.

4. The method (S100) according to claim 3, wherein: The parameterization of the road comprises: The classification of each lane as a main road lane or a branch lane; The branch lane distance (a1) of each branch lane is from the starting point connecting the center of the main road lane to the connection point of the branch lane to the center of the main road lane, The branch lane angle (φ1) of each branch lane at the connection point relative to the center of the connecting main road lane, the branch lane curvature (k1) of each segment of one or more branch lanes, and A branch lane width (ω1) of each segment of the one or more branch lanes.

5. The method (S100) according to claim 1, further comprising: For each time step in the plurality of consecutive time steps: adding (S108) one or more main road lanes and / or one or more branch lanes to each road model hypothesis of the predicted filter state; Any added main road lanes and branch lanes that have no road feature association are removed ( S110 ).

6. The method (S100) according to claim 5, further comprising: For each time step in the plurality of consecutive time steps: The one or more main road lanes and / or the one or more branch lanes added are marked ( S109 ).

7. The method (S100) according to claim 1, wherein: The road features obtained from perception data originating from outputs of one or more onboard sensors include lane marking detections obtained from a lane tracking algorithm configured to output coordinates of lane markings depicted in an image output from an onboard camera.

8. The method (S100) according to claim 1, wherein: The vehicle motion data includes vehicle speed.

9. The method (S100) according to claim 1, further comprising: For each time step in the plurality of consecutive time steps: The road model is formed (S111) based on the parameterization of the road selected by the road model hypothesis by converting the road model parameterization into polylines describing lane centers of the lanes of the road.

10. The method (S100) according to claim 1, wherein: The road model is output to a path planning function of an autonomous driving system of the vehicle, which is configured to generate a path for the vehicle to execute.

11. A computer program product having a computer program thereon, the computer program comprising program instructions which, when executed by a computing device of a vehicle, cause the computing device to perform the method (S100) according to any one of claims 1 to 10.

12. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device of a vehicle, cause the computing device to perform the method (S100) according to any one of claims 1 to 10.

13. A device (10) for estimating the road geometry of a road on which a vehicle (1) is traveling, wherein: The road geometry is described by a filter state tracked by a tracking filter, the filter state comprising a plurality of road model hypotheses, wherein each road model hypothesis is associated with a probability, and comprising a parameterization of the road describing the geometry of one or more lanes of the road, and wherein each lane of each road model hypothesis is associated with a probability of existence, the apparatus comprising a control circuit (11) configured to: For each of a number of consecutive time steps: predicting a filter state by determining a probability of each road model hypothesis based on the vehicle motion data and a priori filter states determined at a previous time step; associating road characteristics obtained from sensory data originating from outputs of one or more onboard sensors with said parameterization of said road for each road model hypothesis predicted for said filter state; determining an existence probability of each lane of each road model hypothesis based on the a priori existence probability of each corresponding lane determined at the previous time step and the road feature association; determining a probability of each of the plurality of road model hypotheses based on the a priori filter state determined at the previous time step and the determined existence probability of each lane of the road model hypothesis; Based on the determined probabilities for each road model hypothesis, selecting the road model hypothesis associated with a highest probability; and A road model is output based on the parameterization of the road selected for the road model hypothesis.

14. The device (10) according to claim 13, wherein: The road model is output to a path planning function of an autonomous driving system of the vehicle, the path planning function (316) being configured to generate a path for execution by the vehicle (1).

15. A vehicle (1) comprising a device (10) according to any one of claims 13 and 14.

Images