Method and processing unit for determining the state of an object

Abstract

Method (700) for determining an object state of an object (150) at a sequence of time points, wherein the object state comprises values ​​for a plurality of state variables, wherein the plurality of state variables comprises a position in a spatial area, an orientation, and a velocity of the object (150), wherein the spatial area comprises a plurality of cells (201); wherein the method (700) comprises iteratively determining, for each time point (t) from the sequence of time points, - Determine (701), based on sensor data from one or more sensors (111) in relation to the spatial area, an occupancy grid (200) that indicates for each of the multitude of cells (201) evidence that the respective cell (201) is free at time (t) or is occupied by the object (150); - Determine (702), based on the occupancy grid (200), a subset of one or more cells (201) from the multitude of cells (201) that belong to the object (150) at time (t); - Determining a measured value of a first state variable of the plurality of state variables for time (t) based on the determined subset of one or more cells (201) of the object (150); - Determine (703) a predicted object state of the object (150) for time (t) based on the object state for a previous time (t - 1); - Determining a predicted value of the first state variable from the predicted object state of object (150) for time (t); - Determining a deviation between the predicted value and the measured value of the first state variable; and - Determining (704) the object state for time (t) based on the predicted object state for time (t), based on the determined subset of one or more cells (201) of the object (150) and based on the determined deviation.

Description

[0001] The invention relates to a method and a corresponding processing unit that enables, for example, a vehicle to recognize an object in the vehicle's environment based on sensor data and / or to determine information relating to the recognized object.

[0002] A vehicle typically includes several different environmental sensors designed to collect sensor data about its surroundings. Examples of environmental sensors include radar sensors, ultrasonic sensors, LiDAR sensors, image sensors, etc. Based on the sensor data from one or more of a vehicle's environmental sensors, one or more objects (e.g., one or more other vehicles) in the vehicle's vicinity can be detected.

[0003] DE 10 2014 208 009 A1 describes a method for detecting static and dynamic objects. DE 10 2018 200 683 A1 describes a method for detecting an object.

[0004] This document addresses the technical challenge of providing a method and a corresponding processing unit for the precise and robust detection of an object, particularly one in the vicinity of a vehicle, and / or the determination of information relating to that object. Particular emphasis is placed on precisely determining the dimensions and / or condition of an object and tracking or monitoring it over time.

[0005] The problem is solved by each of the independent claims. Advantageous embodiments are described, inter alia, in the dependent claims. It should be noted that additional features of a claim dependent on an independent claim, without the features of the independent claim itself or only in combination with a subset of the features of the independent claim, can constitute a separate invention independent of the combination of all features of the independent claim, which can be made the subject of an independent claim, a divisional application, or a subsequent application. This applies equally to technical teachings described in the description, which can constitute an invention independent of the features of the independent claims.

[0006] According to one aspect, a method for determining the state of an object at a sequence of points in time is described. Specifically, the object state is to be updated at successive points in time to provide the current object state at each point in time in a precise and robust manner. The object may be located in the environment of a vehicle. The method can be executed by an evaluation unit of the vehicle. The determined object state can be used to provide a vehicle function within the vehicle (e.g., for at least partially automated driving).

[0007] The object state can include values ​​for one or more state variables. These one or more state variables can include, in particular, the object's position within a spatial area (in the vicinity of the vehicle), its orientation, and / or its speed. The spatial area can be divided into multiple cells. The object can belong to a specific object class or type. Examples of object classes are: a car (passenger vehicle), a truck (lorry), a pedestrian, a cyclist, a motorcyclist, and / or another road user.

[0008] The procedure can be repeated iteratively for a sequence of consecutive time points. The procedural steps described in this document can be executed repeatedly for successive time points t - 1, t, t + 1, etc. The following describes the steps performed at or for a given time point t.

[0009] The method involves determining an occupancy grid based on sensor data from one or more (environmental) sensors in relation to the spatial area. For each of the numerous cells, the occupancy grid can display evidence or a measure of evidence indicating that the respective cell is free or occupied by the object at time t. Examples of measures of evidence that can be determined or provided for each cell at time t are the (actual) measures of evidence m(SD) described in this document. t ), m(S t ), m(D t), m(F t ) and / or m(FD t ).

[0010] Furthermore, the procedure includes determining, based on the occupancy grid, a subset of one or more cells from the multitude of cells that belong to the object at time t. For this purpose, a clustering algorithm, as described in this document, can be used, which is configured to assign one or more cells of the occupancy grid to a specific object. Additionally, object attributes or object mappings can be used to filter or consider the object membership of cells across a sequence of time points. Thus, the one or more cells of the occupancy grid that are part of the (specific) object at time t can be precisely determined.

[0011] Furthermore, the method includes determining a predicted object state for or at time t. The predicted object state can be determined based on the object state at the (possibly directly) preceding time t - 1. In particular, predicted values ​​for one or more state variables can be predicted or determined. One or more prediction equations can be used for this purpose.

[0012] The predicted object state for time t (in particular the predicted value of the object's orientation) can be determined based on the prediction equation. φ^t=φt−1+ωΔt

[0013] where φ̂ t the predicted value of the orientation for or at time t, φ t-1is the determined value of the orientation for or at the previous time t - 1, ω is a value of the rotation rate of the object and Δt is the time interval between the previous time t - 1 and the time t.

[0014] Alternatively or additionally, the predicted object state for or at time t (in particular the predicted value of the object's velocity) can be determined based on the prediction equation. ν^t=νt−1+aΔt where ν̂ t the predicted value of the velocity for or at time t, ν t-1 is the determined value of the velocity for or at the previous time t - 1, and α is the acceleration of the object.

[0015] Alternatively or additionally, the predicted object state for or at time t (in particular the predicted value of a first component (e.g. the x-component) of the object's position) can be determined based on the prediction equation. x^tx=xt−1x+1ω2(ων^tsin(φ^t)+a cos(φ^t) −ωνt−1sin(φt−1)−a cos(φt−1))

[0016] This is x^tx the predicted value of the first component of the position for or at time t, and xt−1x is the determined value of the first component of the position for the current time t - 1.

[0017] Alternatively or additionally, the predicted object state for or at time t (in particular the predicted value of a second component (e.g. the y-component) of the object's position) can be determined based on the prediction equation. x^ty=xt−1y+1ω2(−ων^tcos(φ^t)+a sin(φ^t) +ωνt−1cos(φt−1)−a sin(φt−1))

[0018] This is x^ty the predicted value of the second component of the position for or at time t, and xt−1y is the determined value of the second component of the position for the previous time t - 1.

[0019] Furthermore, the method includes determining the object state for time t based on the predicted object state for time t and on the determined subset of one or more cells of the object. The object state for the previous time t - 1 (determined in the previous iteration) can be updated by means of an innovation or a measurement update. This innovation or measurement update can be based on the deviation between the predicted object state for time t and a currently measured object state. The currently measured object state can be determined based on the determined subset of one or more cells of the object. In particular, a measured value of one or more state variables can be determined based on the determined subset of one or more cells of the object.

[0020] The object's state can thus be determined or updated iteratively at a sequence of time points. This can be done in particular using a Kalman filter, especially an unscented Kalman filter. The use of an unscented Kalman filter enables the efficient and precise application of non-linear prediction equations or system equations.

[0021] To determine the predicted values ​​of one or more state variables, i.e., to determine the predicted object state, a prediction of several distinct sigma points in the vicinity of the determined object state can be performed for the current or preceding time t - 1. Specifically, the method can involve determining a plurality of sigma points in the vicinity of the object state for the current or preceding time t - 1. Here, a sigma point comprises values ​​for the one or more state variables. In particular, a sigma point can correspond to an object state in the vicinity of the object state for the preceding time t - 1. Typically, for L distinct state variables, 2L + 1 distinct sigma points are used. The number L of distinct state variables can be two or more, three or more, or four or more.

[0022] Furthermore, the procedure can include determining a plurality of weights for a plurality of sigma points. The weight for a sigma point can depend on how far the sigma point is from the object state at the previous time t - 1. Typically, the weight decreases with increasing distance.

[0023] Furthermore, the procedure can be used to determine a corresponding plurality of predicted sigma points for time t based on the plurality of sigma points for the preceding time t - 1. In particular, the one or more prediction equations for the different state variables described in this document can be applied to the individual state variables of a sigma point in order to determine the predicted sigma point.

[0024] Furthermore, the method can include determining the predicted object state for time t based on the plurality of predicted sigma points and on the plurality of weights, in particular as a weighted average of the plurality of predicted sigma points. Thus, even when using non-linear prediction equations, a predicted object state can be determined efficiently and reliably.

[0025] The object state can include a longitudinal acceleration α of the object. Determining the predicted object state for time t, in particular determining a predicted value α̂ t the longitudinal acceleration can thus be calculated based on the determined value α t-1 the longitudinal acceleration for the preceding time t - 1, such that for the magnitudes of the longitudinal acceleration |â t | ≤ |α t-1It can therefore be specified that the magnitude of the predicted longitudinal acceleration decreases compared to the longitudinal acceleration at the previous time and / or achieves a (preferably relatively slow) convergence towards zero. This allows for a robust update of the object's state.

[0026] Alternatively or additionally, the object state can include a rotation rate ω of the object. Determining the predicted object state for time t, in particular determining a predicted value ω̂ t The rotation rate can be calculated based on the determined value ω. t-1 The rotation rate for the preceding time t - 1 is such that |ω̂ holds for the magnitudes. t | ≤ |ω t-1It can therefore be specified that the magnitude of the predicted rotation rate decreases compared to the rotation rate at the previous time and / or achieves a (preferably relatively slow) convergence towards zero. This allows for a robust update of the object's state.

[0027] The procedure can include determining a measured value of a first state variable for time t based on the determined subset of one or more cells of the object. The first state variable can be, for example, the object's position, orientation, and / or velocity.

[0028] Furthermore, the procedure can include determining a predicted value of the first state variable from the predicted object state at time t. In other words, a predicted value of the first state variable can be determined (e.g., using a prediction equation). Additionally, a deviation between the predicted value and the measured value of the first state variable can be determined. This deviation can also be considered or referred to as an innovation or a measurement update. Such a deviation can be determined for a plurality of state variables (in particular, for all state variables of the object state).

[0029] The object state for time t can then be precisely determined based on the calculated deviation. In particular, the object state for the preceding time t - 1 can be updated based on the calculated deviation (for one or more state variables). This allows the object state of an object to be determined for a sequence of time points in a particularly reliable and robust manner.

[0030] The method can involve determining a boundary, in particular a bounding box, of the object based on the identified subset of one or more cells of the object. Specifically, a boundary can be determined that is as small as possible and that encloses the identified subset of one or more cells of the object as completely as possible. Thus, a (possibly rectangular) boundary with a specific length and width can be determined.

[0031] The (possibly rectangular) border can then be used as a kind of mask, oriented differently on the grid to determine a current measurement of the object's orientation. In particular, multiple subsets of cells from the grid can be determined for a corresponding number of border orientations. For each border orientation, it can be determined which subset of cells from the grid is enclosed by the (differently oriented) border.

[0032] For each of the multiple different subsets of cells, a cumulative and / or mean level of evidence or evidence measure can be determined to show that the respective subset of cells is either unoccupied or occupied. For each individual border orientation, a subset of cells in the occupancy grid that are at least partially enclosed by the border can be identified. Furthermore, the sum and / or mean of the evidence or evidence measures for the cells in the identified subset of cells can be determined as cumulative and / or mean evidence or evidence measure.

[0033] Furthermore, based on the determined majority of cumulative and / or average evidence or evidence measures, a measurement for the object's orientation at time t can be calculated. Specifically, the boundary orientation can be selected that exhibits the maximum cumulative and / or average evidence or evidence measure (for the presence of an object). The boundary orientation determined in this way can be used as a measurement for the object's orientation at time t. Thus, based on the occupancy grid, a current measurement for the object's orientation at time t can be determined efficiently and precisely.

[0034] The measured value can then be compared with the predicted orientation value to determine the deviation between the measured value and the predicted orientation value. This deviation can be used to determine the object's state at time t. This enables reliable, precise, and robust tracking of an object's state.

[0035] The one or more sensors can include at least one velocity-measuring sensor (in particular a radar sensor) configured to detect a (measured) value of the radial and / or tangential velocity of at least one first cell of the object at time t. The radial velocity is the component of the object's (longitudinal) velocity that runs radially to the position of the velocity-measuring sensor (i.e., along the line connecting the position of the velocity-measuring sensor and the position of the first cell). The tangential velocity is the component of the object's (longitudinal) velocity that runs tangentially and / or perpendicularly to the line connecting the position of the velocity-measuring sensor and the position of the first cell.A speed-measuring sensor (especially a radar sensor, a lidar sensor and / or a camera sensor) typically enables a precise and direct determination of the radial and / or tangential velocity.

[0036] The method can include determining a predicted and / or expected value of the radial and / or tangential velocity of the first cell at time t. The predicted and / or expected value of the radial and / or tangential velocity can be determined based on the predicted object state of the object at time t. In particular, the predicted and / or expected value of the radial and / or tangential velocity (relative to the position of the velocity-measuring sensor) can be determined based on the predicted value of the object's (longitudinal) velocity displayed in the predicted object state. Thus, a predicted and / or expected value of the radial and / or tangential velocity can be calculated based on the predicted value of the object's (longitudinal) velocity (resulting from the predicted object state).For this purpose, a geometric relationship between the position of the first cell and the position of the speed-measuring sensor can be taken into account.

[0037] The deviation between the predicted and / or expected value and the measured value of the radial and / or tangential velocity of the first cell at time t can then be determined. This deviation can be used as an innovation or measurement update to determine the object's state at time t. In this way, a measured value of the velocity of one or more cells of the object can be considered efficiently and precisely.

[0038] The predicted and / or expected value of the radial velocity of the first cell at time t can be determined based on the equation hir(ξ^τ,t)=ν^τ,t,ξcos(θz,ir−φ^τ,t,ξ) +ω^τ,t,ξ(sin(θz,ir)(xs,ix−x^τ,t,ξx) −cos(θz,ir)(xs,iy−x^τ,t,ξy)

[0039] where ν̂ τ,t,ξ the predicted value of the longitudinal velocity of the first cell and / or the object, x^τ,t,ξx is the predicted value of the first component of the position of the first cell and / or object, x^τ,t,ξy is the predicted value of the second component of the position of the first cell and / or object. Furthermore, φ̂ shows τ,t,ξ the predicted value of the orientation of the first cell and / or the object, hir(ξ^τ,t) displays the predicted and / or expected value of the radial velocity of the first cell and / or object, xs,ix displays the value of the first component of the position of the speed-measuring sensor, xs,iy displays the value of the second component of the position of the speed-measuring sensor, and θz,iτ is the azimuth angle of the position of the first cell and / or the object relative to the position of the velocity-measuring sensor. In particular, θz,iτ The azimuth angle of a measurement of the velocity-measuring sensor mapped in the first cell and / or in the object relative to the position of the velocity-measuring sensor (i.e., the radial measurement direction of the velocity-measuring sensor). The above equation results, for example, from geometric considerations regarding Fig. 6.

[0040] In a preferred example, predicted and / or expected values ​​are used. hir(ξ^τ,t) The radial velocity of the first cell is determined for the majority of sigma points. The effective predicted and / or expected value of the radial velocity of the first cell is then precisely obtained as the weighted average of the predicted and / or expected values. hir(ξ^τ,t) the radial velocity for the majority of sigma points.

[0041] The method can determine, based on the positions of the identified subset of one or more cells of the object, as well as on the object's orientation, the length and / or width of a (rectangular) border or bounding box of the object. The (previously determined) orientation of an object can thus be used to precisely determine the length and / or width of a bounding box.

[0042] Furthermore, the procedure can include determining the position of a reference point on or within the bounding box as a measured value of the object's position at time t. A predicted value for the object's position is then derived from the predicted object state. A deviation between the predicted value and the measured value of the object's position at time t can then be determined as an innovation or measurement update. Finally, the object's state at time t can be determined precisely and robustly based on this calculated deviation.

[0043] Based on the occupancy grid, an occupancy measure can be determined indicating that the immediate vicinity of edges within the object's bounding box is unoccupied. Specifically, this occupancy measure can show how clear the immediate vicinity of each edge of the bounding box is (and thus how visible and detectable the individual edges of the bounding box are). Depending on the determined occupancy measure, the reference point for determining and / or updating an object's position can then be selected or determined as a specific point on a particular edge of the bounding box. In particular, at least one edge can be identified (based on the occupancy measure) that is relatively isolated from one or more other objects and therefore easily detectable. The reference point can then be placed on this edge.By selecting a suitable reference point, a particularly precise and robust determination of the position of an object can be made possible.

[0044] At least some of the cells in the occupancy grid can display motion information related to movement within the respective cell. This motion information can, for example, indicate the direction and / or speed of movement. The motion information can be determined, for example (as described in this document), based on particles and / or a particle filter.

[0045] Within this process, a new object can be detected at a specific time (based on the occupancy grid). Furthermore, a subset of cells for the new object can be determined (also based on the occupancy grid). The object state can then be determined based on the positions of the identified subset of cells and / or the movement information of that subset. Specifically, the position and / or orientation of the new object can be determined based on the positions of the identified subset of cells. Alternatively or additionally, the speed of the new object can be determined based on the movement information of the identified subset of cells. This allows for precise initialization of the object state for a new object.

[0046] Furthermore, a method for determining the value of an object's size within a spatial area is described. The spatial area can comprise a multitude of cells. The method can be executed by an evaluation unit of a vehicle. The spatial area can encompass or be the vehicle's surroundings. The object's size can be its length and / or width (of a bounding box). The aspects described in connection with this method can be combined in any way with individual aspects of the method described above (and / or vice versa).

[0047] The procedure can be executed iteratively for successive points in time within a sequence of points in time. The procedure steps described below are executed at a point in time t.

[0048] The method involves determining, based on sensor data from one or more sensors covering the spatial area, an occupancy grid that indicates, for each of the multitude of cells, evidence or a measure of evidence that the respective cell is free or occupied by the object at time t. Furthermore, the method involves determining, based on the occupancy grid, a subset of one or more cells from the multitude of cells that belong to the object at time t. The occupancy grid and / or the subset of cells can be determined as described in this document and updated at time t.

[0049] Furthermore, the method includes detecting, based on the grid structure, two opposing bounding edges of the object. In particular, a (rectangular) boundary or bounding box can be determined based on the identified subset of cells of the object. Based on the boundary or bounding box, two opposing pairs of edges are then derived, specifically the front and back edges or the left and right edges.

[0050] Furthermore, the method includes determining a measurement of the object's size at time t, based on the distance between the object's detected edges. Specifically, the object's length (bounding box) can be determined based on the distance between the front and rear edges. Alternatively or additionally, the object's width (bounding box) can be determined based on the distance between the left and right edges.

[0051] Furthermore, the process includes determining, based on the occupancy grid, a quality measure that indicates how well the two opposite edges of the object could be detected. This quality measure can, for example, indicate how clear the immediate surroundings of the individual edges are. A relatively clear surroundings typically indicate that an edge can be detected with relatively high accuracy.

[0052] Furthermore, the procedure includes determining, based on the accuracy measure, a measurement probability distribution for the object's size at time t. This measurement probability distribution indicates the probability of different object size values ​​around the measured value. In particular, the measurement probability distribution can indicate the accuracy with which the measured value for the object's size could be determined based on its edges. If the edges were captured relatively well (indicated, for example, by a relatively high accuracy measure), the measurement probability distribution may be relatively narrow around the measured value. Conversely, the measurement probability distribution may be relatively wide, especially for values ​​above the measured value, if the edges were captured relatively poorly (indicated, for example, by a relatively low accuracy measure).

[0053] Furthermore, the procedure can include determining a cumulative probability distribution for time t based on the measurement probability distribution for time t and starting from the cumulative probability distribution for a previous time t - 1. In particular, the cumulative probability distribution for the previous time t - 1 can be updated using the current measurement probability distribution for time t to determine the cumulative probability distribution for time t. This makes it possible to consider the accuracy of the measured object size at different times in order to determine the object size in a particularly precise manner. Specifically, the value of the object size for time t can be determined precisely based on the cumulative probability distribution for time t.

[0054] The measurement probability distribution can, at least in some areas, follow a normal distribution of values ​​around the measured value (especially for values ​​below the measured value). Alternatively or additionally, the measurement probability distribution can be such that, as the precision decreases, the probability distribution for values ​​greater than the measured value increasingly approximates a uniform distribution. The measurement probability distribution can be asymmetric. In particular, the probability distribution for values ​​below the measured value can be independent of the precision, while the probability distribution for values ​​above the measured value depends on the precision. This allows for the consideration that, based on the edges, a lower limit of the object's size (especially its length and / or width) can typically be determined precisely, but not its upper limit.

[0055] The measurement probability distribution can depend on, or be given by, the following function: p(l|ztl,θ)∝exp(−(l−lz)22ςl,θσi2), ςl,ϑ={(ϑzfront ϑzrear)−1,if l≥lz1,else,

[0056] Here, l is a value representing the size of the object and l z is the measured value of the object's size. σl2 is the variance of the measured value, which results, for example, from the measurement procedure used to determine the measured value. ϑzfrontϑzrear is the measure of the quality of edge detection, and can be derived, for example, from a measure of the visibility of the two edges, i.e. ϑzfront and ϑzrear, to assemble. Furthermore, p(l|ztl,ϑ) the measurement probability distribution for time t.

[0057] The cumulative probability distribution can be represented by a multitude of probability values. plτ,ti for a corresponding number of size intervals i, with i = 1, ..., I l , can be described. The number I l The size intervals can be, for example, 5 or more, 10 or more, or 20 or more.

[0058] The cumulative probability distribution p(lτ|z1:tl,ϑ) The time t can then be approximated piecewise by p(lτ|z1:tl,ϑ)≈plτ,tiδi, lτ∈(li−δi2,li+δi2]

[0059] This is l τ the value of the object's size, δ, determined for time t i is an interval size of the size interval i, and l i is a mean value of the size interval i. The piecewise calculation of the cumulative probability distribution enables an efficient and precise determination of the cumulative probability distribution.

[0060] The probability value plτ,ti For a size interval i at time t, starting from the probability value plτ,t−1i The value for the size interval i can be updated for the previous time t-1. In particular, an iterative determination of the cumulative probability distribution is enabled. The probability value can be... plτ,ti for a size interval i for time t, can be determined in a precise manner, as plτ,ti=p(li|ztl,ϑ)plτ,t−1i∑j=1Ilp(lj|ztl,ϑ)plτ,t−1j where p(li|ztl,ϑ) the measurement probability distribution for time t for a mean value l i of the size interval i is.

[0061] As explained above, the object can be described by a bounding box, where the bounding box has a length and a width. The size of the object can be the length and / or the width of the bounding box. Furthermore, the two opposite bounding edges of the object can be the front and back edges of the bounding box if the length of the bounding box is defined as the size of the object. Alternatively or additionally, the two opposite bounding edges of the object can be the right and left edges of the bounding box if the width of the bounding box is defined as the size of the object.

[0062] The procedure can involve assigning the object to a class from a plurality of road user classes. Examples of such classes are: a car, a truck, a pedestrian, a motorcycle, a bicycle, etc. The assignment to a class can be carried out using the procedure described in this document.

[0063] Furthermore, the procedure can include determining a class probability distribution for the size of objects from the assigned class. The class probability distributions for different object classes can be determined and provided in advance (e.g., based on measurements).

[0064] Furthermore, the procedure can involve combining the class probability distribution for the assigned class with the cumulative probability distribution for time t to obtain a combined probability distribution for time t. Specifically, the class probability distribution can be multiplied by the cumulative probability distribution to determine the combined probability distribution. The value of the object's size for time t can then be precisely determined based on the combined probability distribution for time t (specifically, as the expected value of the combined probability distribution).By taking into account the class to which the object has been assigned, the value of the object's size can be determined with increased accuracy and / or reliability (even if the edges of the object can only be detected relatively poorly based on the sensor data and / or if only a lower limit of the object size is visible).

[0065] As explained above, the cumulative probability distribution can be represented by a multitude of probability values. plτ,ti for a corresponding number of size intervals i, with i = 1, ..., I l , can be described. p(li|kτ*) can the class probability for a mean value l i of the size interval i of the object size for the object class kτ* It can then be based on the respective product. p(li|kτ*)plτ,ti, for i = 1, ...,I l, a multitude of combined probability values pcombi,l The corresponding number of size intervals i can be determined. Determining the value of the object's size can then involve selecting a size interval from the multitude of size intervals, based on the multitude of combined probability values. pcombi,l, This includes the ability to efficiently and precisely determine the size of the object.

[0066] The method described in this document can involve determining a plurality of class probability distributions p(l|k) for the size of objects from a corresponding plurality of different classes k ∈ K (e.g., based on measurements of actual objects). In other words, for a plurality of different object classes k ∈ K, class-specific class probability distributions p(l|k) for the size of objects of the respective class can be determined or provided, e.g., for the expected length and / or width of an object.

[0067] Furthermore, the procedure can include determining a plurality of combined probability distributions for time t for the plurality of different classes k ∈ K. The plurality of combined probability distributions can be determined based on the cumulative probability distribution for time t and on the plurality of class probability distributions p(l|k).

[0068] It is then possible to assign the object to a class in a precise and efficient manner. kτ* The cumulative probability distribution can be derived from the plurality of different classes k ∈ K based on the plurality of combined probability distributions for time t. The cumulative probability distribution can thus (if necessary) also be used to efficiently classify the object.

[0069] The procedure can involve determining a plurality of prior probabilities p(k) for the plurality of distinct classes k ∈ K. These prior probabilities p(k) can be determined based on measurements taken prior to the execution of the procedure. The plurality of combined probability distributions for time t can then also be determined based on these prior probabilities p(k). By considering the prior probabilities p(k) of the different classes, the accuracy of the object classification can be further improved.

[0070] The procedure can include determining the object's velocity based on the occupancy grid and / or sensor data. In particular, a maximum velocity value can be determined (e.g., using the procedure described in this document). Assigning the object to a class. kτ* The selection from the plurality of different classes k ∈ K can then also be based on the determined velocity of the object. In particular, the determined velocity of the object can be used to exclude one or more object classes (which typically cannot reach the determined velocity of the object). By taking the velocity of the object into account, the classification can be further improved.

[0071] The procedure can involve determining a plurality of expected values ​​for the plurality of combined probability distributions for time t. The expected value E can be... L [p(L τ |k)] for class k are calculated as EL[p(Lτ|k)]=∑i=1llp(lτk)plτ,ti where p(l i |k) the class probability of class k for a mean value l iThe size interval i is the size of the object. Assigning the object to a class kτ* From the plurality of different classes k ∈ K, it can then be done in a particularly precise way on the basis of the plurality of expected values.

[0072] The procedures described in this document can each include providing a vehicle's driving function based on the determined object state and / or the determined size of the object. The driving function can, for example, include at least partially automated longitudinal and / or lateral guidance of the vehicle. In other words, a vehicle can be operated based on the determined object state and / or the determined size of an object in the vehicle's environment. This enables reliable vehicle operation.

[0073] The one or more sensors can include a radar sensor and / or a lidar sensor of a vehicle. The spatial area can encompass the vehicle's surroundings that can be detected by the vehicle's one or more sensors.

[0074] Another aspect described is a software (SW) program. The SW program can be configured to run on a processor (e.g., on a vehicle's control unit) and thereby execute one of the procedures described in this document.

[0075] According to another aspect, a storage medium is described. The storage medium can contain a software program that is configured to run on a processor and thereby execute one of the procedures described in this document.

[0076] According to another aspect, a processing unit for a vehicle is described, wherein the processing unit is configured to execute at least one of the procedures described in this document.

[0077] According to another aspect, a (road) motor vehicle (in particular a passenger car or a truck or a bus or a motorcycle) is described that includes the processing unit described in this document.

[0078] It should be noted that the methods, devices, and systems described in this document can be used both alone and in combination with other methods, devices, and systems described in this document. Furthermore, any aspect of the methods, devices, and systems described in this document can be combined with one another in a variety of ways. In particular, the features of the claims can be combined with one another in a variety of ways.

[0079] The invention will now be described in more detail using exemplary embodiments. Fig. 1 an exemplary vehicle with a plurality of different environmental sensors; Fig. 2 an exemplary grid of a vehicle's environment; Fig. 3 exemplary particles of a cell of a grid; Fig. 4. An exemplary orientation of an object; Fig. 5 exemplary particles in a grid for the environment of a vehicle; Fig. 6 an exemplary geometric relationship between a radar sensor and a detected object; Fig. 7a a flowchart of an exemplary procedure for determining the object state of an object; Fig. 7b a flowchart of an exemplary procedure for determining the size of an object; Fig. 8 exemplary measurement probability distributions for different values ​​of edge visibility; Fig. 9 exemplary class probability distributions for the length, width, and speed of different object classes.

[0080] As stated at the outset, this document deals with the detection and, in particular, the tracking of the state of at least one object based on sensor data from one or more environmental sensors. In this context, it shows Fig. 1. A vehicle 100 with one or more environmental sensors 111 for acquiring sensor data. The vehicle 100 further includes a processing unit 101, which is configured to detect an object 150 in the vicinity of the vehicle 100 based on the sensor data. A detected object 150 can be taken into account in a vehicle function 102 (e.g., for the partially automated or highly automated driving of the vehicle 100).

[0081] This document deals in particular with the consistent multisensory modeling of the environment of a vehicle 100, which maps static obstacles, drivable or free areas and / or dynamic objects 150 at different levels of abstraction and estimates them holistically. The local environment of a vehicle 100 can be estimated or represented as a Dynamic Occupancy Grid Map or raster 200 (see Fig. 2).

[0082] Fig. Figure 2 shows an exemplary grid 200 of the environment of vehicle 100 with a multitude of grid cells, or cells 201 for short. The grid 200 can divide the environment or surroundings of vehicle 100 into the multitude of two- or three-dimensional cells 201. A two-dimensional cell 201 can have a rectangular shape (for example, with an edge length of 10 cm or less, 5 cm or less, 2 cm or less, or 1 cm or less).

[0083] The processing unit 101 can be configured to determine, based on the sensor data for one or more of the cells 201 (in particular for each cell 201), measurement data indicating whether a cell 201 is occupied at a specific time t or not. In particular, the measurement data can be used to determine whether a cell 201 is occupied at a specific time t. c Display for cell c 201 zc=(m(SDz,t),m(Fz,t)), where m(SD) = m({S, D}) is evidence or a measure of evidence that cell c 201 is occupied by an object 150 that cannot be uniquely identified as static ("S") or dynamic ("D"), and where m(F) is evidence that cell c 201 is free ("F") and therefore not occupied by an object 150 (and is thus a free cell 201). The evidence m(SD) that cell 201 is occupied by an object 150 can be considered the object probability or the lower bound of the object probability that cell 201 is occupied by an object 150 (especially in the sense of Dempster-Shafer theory). The measures of evidence can each be values ​​between 0 and 1. The sum of the evidence measures of a cell 201 (for the different hypotheses) can be normalized to the value 1.

[0084] Typically, based on a temporally isolated measurement at a specific time t, only the evidence or evidence measures m(SD) and m(F) can be determined, since it cannot be ascertained whether the object is occupied by a static or a dynamic object 150. However, it can be assumed that after a sequence of measurements at a corresponding sequence of time points, an occupancy grid will be available at the current time t. Mt 200 can be provided, which shows different evidence for different hypotheses for the different cells 201, Mt={m(St),m(Dt),m(SDt),m(Ft),m(FDt)}, where m(FD t ) = m({F, D} t) the evidence for the hypothesis shows that a cell 201 that was previously unoccupied, i.e., measured as free ("F") in a measurement, is still free ("F") or occupied by a dynamic object 150 ("D") at time t, thus representing an area that is fundamentally traversable or passable for a dynamic object 150. Furthermore, m(S) shows t ) the evidence or mass of evidence that cell c 201 is occupied by a static object 150 at time t. Furthermore, m(D) shows t ) the evidence or mass of evidence that cell c 201 is occupied by a dynamic object 150 at time t. The occupancy grid Mt Describes the status or state of cells 201 of grid 200 at a specific time t. The evidence or evidence measures of the occupancy grid. Mt−1 The 200 values, which were determined taking into account the current sensor data at the current time t, are also referred to in this document as actual evidence measures.

[0085] The movement of a possible object 150 can be described by so-called particles. For this purpose, the evidence mass of a cell 201 for the hypothesis that a moving object 150 is present can be divided among a multitude of particles. In particular, the actual evidence mass m(D) can be t-1 ) and / or the actual measure of evidence m(SD) t-1 ) for time t - 1, divided into a plurality of particles 300 (see Fig. 3) The particles 300 can exhibit different movements 301 or different movement hypotheses. In particular, the different movements 301 can have different speeds and / or different directions of movement. For example, with a spatial resolution of 360° / RR, different directions of movement can be considered, and with a specific value resolution V, different speeds can be taken into account, so that different particles 300 are initialized in a cell 201 R x V. Thus, when initializing particles 300 for a cell 201, it can be assumed that the object 150 to which the cell 201 belongs can move arbitrarily.

[0086] The (actual) occupancy grid Mt−1 200 (at a previous time t - 1) and the particles 300 can be used to create an occupancy grid. M¯t to predict for a (current) time t. In other words, the (predicated) occupancy grid. M¯t For time t, the (actual) occupancy grid can be used. Mt−1 and / or the particles 300 are predicted for a previous time t - 1. For the evidence measures, it can be assumed that m(St')=m(St−1). m(SDt')=m(SDt−1), m(Dt')=0, m(Ft')=0, m(FDt')=m(FDt−1)+m(Ft−1)1−m(Dt−1).

[0087] The checkmark (') indicates that this is a (statically) predicted evidence measure. Therefore, static aspects can be derived from the (actual) occupancy grid. Mt−1 of the previous time t - 1 are transferred to time t to create a statically predicted occupancy grid. M't to determine.

[0088] The dynamic aspects can be considered using particles 300. In particular, for each cell 201, the sum of the (particle) evidence masses of the particles 300 that fall into the respective cell 201 at time t can be determined. Based on this, the evidence mass for the fact that a cell 201 is a dynamic object 150 can then be determined, e.g. m(D^tc)=min(1−εo,∑χ∈X^tcoχ),εo∈(0,1) where Σ o χ the sum of the (particle) evidence masses of the particles 300 that fall into a particular cell 201, and where ε o is an adjustable constant between 0 and 1. The min() function ensures that the (dynamically) predicted evidence measure m(D̂) t ) of a cell c 201 is not larger than one. This results in a dynamically predicted occupancy grid based on particle 300. M^t.

[0089] The dynamically predicted occupancy grid M^t and the statically predicted occupancy grid M't can be superimposed to create a predicted occupancy grid M¯t to determine, e.g. by means of M¯t=(1−ε¯)(M^t⊕M't)

[0090] In this combination, a temporal uncertainty factor ε̅ can be taken into account.

[0091] Combining the dynamically predicted occupancy grid M^t and the statically predicted occupancy grid M't For compatible hypotheses or measures of evidence, this can be achieved by multiplying the measures of evidence. On the other hand, a conflict exists in the case of contradictory hypotheses. This can be the case, in particular, if dynamic prediction is used for cell 201. M^t a measure of evidence m(D̂ t ) and via static prediction M't a measure of evidence m(S') t ) is predicted. A robust approach to conflict resolution could be that in this case the actual measure of evidence m(S) t-1 ) of the preceding time t - 1 as predicted evidence measure m(S) t ) is used.

[0092] Thus, based on the actual occupancy grid, Mt−1 and the particle 300 for a previous time point (t - 1) a predicted occupancy grid M¯t to be determined.

[0093] Furthermore, at time t, 201 new measurements are available for the individual cells, which are arranged in a measured occupancy grid. Mz,t can be summarized. The measured occupancy grid can be used for this purpose. Mz,t for each cell 201 the evidence measures m(SD) z,t ) and m(F z,t ) display. The predicted occupancy grid M¯t The measured occupancy grid can then be used Mz,t They can be combined to create the actual occupancy grid. Mt to determine at time t, i.e. Mt=M¯t⊕Mz,t

[0094] The combination of evidence measures can, in turn, be achieved by multiplying evidence measures for compatible hypotheses. Conversely, conflicts can arise with incompatible hypotheses. Such a conflict can occur, in particular, if the current measurement data provide an evidence measure for an unoccupied cell 201, i.e., m(F). z,t ), display, while the predicted occupancy grid M¯t Evidence measures for a static, a dynamic, or an object that is not clearly static or dynamic are displayed as 150. The resulting conflict evidence measures ζ(M¯t,Mz,t)=m(S¯t)m(Fz,t)︸ζ1+m(D¯t)m(Fz,t)︸ζ2+ m(SD¯t)m(Fz,t)︸ζ3 can be assigned to the different hypotheses as follows, for example: m(St)←12ζ1, m(Ft)←12ζ1+ζ2+ζ3.

[0095] As part of determining a measured occupancy grid Mz,t Typically, no measure of evidence is determined for the hypotheses "static object" or "dynamic object", but only the measure of evidence m(SD). z,t ), indicating that cell 201 is occupied by an unspecified object 150. As a consequence, when combining the predicted occupancy grid... Mt with the measured occupancy grid Mz,t Only the evidence for hypothesis SD is increased, but not the evidence for hypotheses S or D.

[0096] To ensure the accuracy of an occupancy grid Mt Increasing the evidence mass for the hypothesis SD at time t can increase the evidence mass for the hypothesis SD, i.e. m(SD|M¯t⊕cMz,t)=m(SD¯t)m(Θz,t)︸λ1+ m(SD¯t)m(SDz,t)︸λ2+m(Θt)m(SDz,t)︸λ3 The data can be divided between the hypotheses SD, S, and D. Here, it is possible to... m(SD|M¯t⊕cMz,t) Describe the Dempster-Shafer evidence combination, in this case for the SD hypothesis, given the predicted occupancy grid. M¯t and the measured occupancy grid Mz,t without standardization or consideration of conflicts. In other words, m(SD|M¯t⊕cMz,t) can describe the multiplication of the evidence measures of the hypotheses whose intersection corresponds to the described hypothesis, in this case hypothesis SD, with the associated combined set {S, D} from the hypotheses of a static object S or a dynamic object D. The division can be carried out as follows, m(SDt)←λ1 m(St)←λ2 m(Dt)←fDλ3, m(SDt)←(1−fD)λ3 where f Da positive number less than one, which may depend on the number of particles 300 in the cell 201 under consideration. Alternatively or additionally, f D depend on the measured value of the radial velocity of the cell 201 under consideration, the measured value of which can be acquired, for example, by means of a radar sensor 111. In this case, f D increase with increasing speed.

[0097] On the other hand, evidence that would be assigned to hypothesis 150 when combining the hypothesis for a dynamic object can also be at least partially redistributed to hypothesis SD in order to account for measurement errors or faulty simplifications of the measurement models of the respective sensors. For the evidence for hypothesis D, this results in… m(D|M¯t⊕cMz,t)=m(D¯t)(1−m(Fz,t))+ m(FD¯t)m(SDz,t)︸λ4

[0098] The corresponding term λ4, which is the combination of the predicted evidence measure m(FD) t) the FD hypothesis, i.e. the hypothesis that cell 201 is free or dynamically occupied, and the measured evidence measure m(SD) z,t ) of the SD hypothesis for an unclassified allocation, can be redistributed as follows. m(Dt)←(1−γ)λ4+fDγλ4, m(SDt)←(1−fD)γλ4, where γ is a design parameter between 0 and 1.

[0099] The resulting evidence measures of the actual occupancy grid Mt At time t, the following results are obtained: m(St)=m(S|Mt⊕cMz,t)+12ζ1+λ2, m(Dt)=m(D|M¯t⊕cMz,t)−(1−fD)γλ4 +fDλ3, m(SDt)=m(SD|M¯t⊕cMz,t)−λ2−dDλ3 +(1−fD)γλ4, m(Ft)=m(F|M¯t⊕cMz,t)+12ζ1+ζ2+ζ3, m(FDt)=m(FD|M¯t⊕cMz,t).

[0100] In addition to determining an actual occupancy grid Mt For time t, the distribution of particles 300 can be updated to create a current picture of the dynamic aspects, in particular the direction of movement of one or more objects 150, of an environment. For this purpose, the evidence mass for the hypothesis of a dynamic object 150, i.e., m(D), can be used. t ), are distributed across a large number of particles 300. The number of particles 300 for a cell 201 can be proportional to the evidence mass m(D). t ) of cell 201.

[0101] The particle density of 300 for a cell of 201 can be chosen, for example, as ρc=m(Dtc)+(1−fD)(λ3+γλ4) and can therefore depend on the evidence mass for a dynamic object D as well as on a measurement obtained through the current measurement Mz,t An increased level of evidence m(SD), which may correspond to a dynamic object, can be selected. Additionally, the level of evidence for a static object S can also be considered, with a higher level of evidence m(S). t The density of particles 300 is reduced because the particles 300 correspond to the hypotheses of dynamic objects. The density of particles 300 can be multiplied by a maximum value of particles 300 for a cell 201 to determine the adaptive number of particles 300 for a cell 201. Alternatively, the number of particles 300 in a cell 201 can be chosen independently of the evidence measures and, for example, defined as a constant fixed value.

[0102] Of the available particles 300, at least some can be derived from the predicted particles 300 of the previous time (t - 1) that were predicted into the respective cell 201. This allows filtering along the direction of movement of an object 150. Furthermore, a (relatively small) number of randomly distributed new particles 300 can be generated. This increases the robustness of the iterative method for updating an actual occupancy grid. Mt be increased. The total dynamic evidence mass m(D̂ t ) of a cell 201 can be evenly distributed over the total number of particles 300 of cell 201, so that the sum of the (particle) evidence masses Σ o χ all particles 300 of the cell 201 of the dynamic evidence mass m(D) t) this corresponds to cell 201. The updated particles 300 can then be used to create a dynamically predicted occupancy grid. M^t+1 to determine for a subsequent time t + 1.

[0103] An (actual) occupancy grid Mt (that for each cell 201 of the grid 200 actual evidence measures m(S) t ), m(D t ), m(SD t ), m(F t ), m(FD t The system (which includes) can thus be updated iteratively based on sensor data from one or more environmental sensors. The iterative process can be repeated for a sequence of time points t - 1, t, t + 1, t + 2, ..., e.g., with a repetition rate or sampling frequency of 1 Hz, 10 Hz, 100 Hz, or more. At a given time point t, a measured occupancy grid can be generated based on the sensor data. Mz,t to be determined (the evidence measures m(SD) measured for each cell 201 of the grid 200z,t ), m(F z,t ) includes). The measured occupancy grid Mz,t can be used with a predicted occupancy grid M¯t They can be combined to create the actual occupancy grid. Mt to determine. The predicted occupancy grid can be used for this purpose. M¯t by combining a statically predicted occupancy grid M't and a dynamically predicted occupancy grid M^t to be determined. The statically predicted occupancy grid M't can be derived from the (actual) occupancy grid through a mapping process Mt−1 The occupancy grid is predicted or determined from the previous time point (t - 1). M^t can be derived from the actual D-evidence mass m(D) using a particle tracking process t-1) of the individual cells 201 at the previous time point (t - 1). The particle tracking process can include a resampling step in which the actual D-evidence mass m(D) t-1 ) of a cell 201 is distributed among old and / or new particles 300. Based on the movement 301 of the particles 300, a movement and / or distribution of the particle evidence masses within the grid 200 at the current time t can then be predicted. From the distribution of the particle evidence masses at the current time t, the dynamically predicted D-evidence mass m(D̂) can then be determined. t ) of the individual cells 201 are determined (e.g. based on the sum of the particle evidence mass in the respective cell 201).

[0104] Thus, a corresponding sequence of actual occupancy grids can be determined from a sequence of time points t. Mt 200 with actual evidence measures, e.g. m(SD) t ), m(S t ), m(Dt ), m(F t ) and / or m(FD t ). Based on these actual evidence measures for the individual cells 201, (possibly new) objects 150 can be detected using a clustering algorithm, and the individual cells 201 can be assigned to one or more (dynamic) objects 150. Furthermore, the object assignment determined at a previous time t - 1 can be used to predict an object assignment for the current time t. The predicted object assignment can be compared with the one based on a current occupancy grid. Mt The object assignment is determined by superimposing various representations to ascertain the (actual) object assignment for the current time t. The individual (dynamic) objects 150 can each be described, for example, by a (rectangular) bounding box or another geometric shape, whereby such a box representation is used in the subsequent description without loss of generality. In general, a bounding box of an object 150 can be described as the boundary of the object 150. Examples of boundary shapes are a (rectangular) box, a box with rounded edges, an elliptical boundary, etc.

[0105] Based on the predicted object assignment of cells 201, predicted bounding boxes (or borders) for the individual objects 150 can be determined. Furthermore, based on the actual object assignment, actual bounding boxes (or borders) for the individual objects 150 can be determined.

[0106] To identify an object 150, contiguous dynamic cells 201 (which exhibit a relatively high dynamic evidence mass) can be grouped into a single, contiguous object 150 using a clustering algorithm. However, when objects 150 are located relatively close to each other, misassignments of individual cells 201 to different objects 150 can occur. To improve the accuracy with which individual cells 201 are assigned to different objects 150, each particle 300 can be assigned an additional attribute or label indicating its membership in a specific object 150. In particular, a particle 300 can have an object attribute that indicates whether the cell 201 in which the particle 300 is located during particle 300 initialization is assigned to a (dynamic) object 150 or not.Furthermore, an object identifier can be used to display the exact object 150 to which cell 201 is assigned.

[0107] τ t-1 can represent the number of dynamic objects 150 that are in the actual occupancy grid at time t - 1 Mt−1 are known. Each particle 300 can have an object 150 τ ∈ τ as an object attribute. t-1 It may be assigned, or it may be displayed as an object attribute that particle 300 is not assigned to any object 150.

[0108] A particle 300 is assigned to at most exactly one object 150 or to no object 150. A particle 300 newly initialized at a specific time t may not be assigned to any object 150. Furthermore, a particle 300 can retain its current object attribute or object assignment during the particle tracking process, particularly during the resampling step. Additionally, the process of filtering the particles 300 can remain unchanged; that is, the distribution of the evidence mass of a cell 201 between existing particles 300 and new particles 300 can be independent of the object affiliation of the individual particles 300. This enables robust tracking of objects 150 at the particle level (without distortion from a model-based object level).

[0109] As outlined above, the particle tracking process can include a resampling step where the actual D-evidence mass m(D) t-1The data from a cell 201 is distributed among old and / or new particles 300. These particles 300 each have an object affiliation as an object attribute. Based on the movement or movement hypothesis 301 of the individual particles 300, the position of the particles 300 at the (current) time t can then be predicted.

[0110] To determine the membership of a cell 201 to a (dynamic) object 150, it is possible to determine at time t, based on the predicted positions, which particles 300 fall into the cell 201, and what proportion of the particles 300 belong to the respective objects 150 τ ∈ τ t-1 The cell 201 can then be assigned to one or possibly no object 150 based on the distribution of the object assignment of particles 300. In particular, the cell 201 can be assigned to object 150 τ ∈ τ t-1The cell 201 with the highest relative proportion of particles 300 can be assigned to the cell 201. This assignment may only occur if the highest relative proportion reaches or exceeds a certain minimum threshold (e.g., 50%, 60%, 70%, 80%, or more). Alternatively, it may be determined that cell 201 is not part of an object 150 or that cell 201 cannot be uniquely assigned to an object 150.

[0111] It can therefore be checked at time t, based on the object attributes of particle 300, whose predicted position falls into a cell 201, whether the cell 201 is part of an already known object 150 τ ∈ τ t-1Furthermore, for the cells 201 that cannot be assigned to any object 150 (but which exhibit a relatively high dynamic evidence mass), a clustering algorithm can be used to identify new objects 150. This algorithm groups neighboring, unassigned cells 201 into a contiguous (new) object 150. Thus, the set τ t 150 dynamic objects will be updated.

[0112] After taking into account a current measurement at time t, updated, actual bounding boxes (or borders) can be calculated for a specific quantity τ. t provided by (dynamic) objects 150. Based on these actual boxes (or outlines) for the different objects 150, the object attributes of the particles 300 can be updated. This is exemplified in Fig. 5 shown. Fig. Figure 5 shows on the left a grid 200 with a multitude of particles 300 (where the particles 300 are arranged at their respective predicted positions). Furthermore, it illustrates Fig. 5, left side, the object attributes of particle 300, wherein the particles 300 in area 500 are not assigned to any dynamic object 150, wherein the particles 300 in area 501 are assigned to a first dynamic object 150, and wherein the particles 300 in area 502 are assigned to a second dynamic object 150.

[0113] Fig. Figure 5, right side, shows the actual bounding boxes (or outlines) 511, 512 for the first and second object 150, respectively, which were determined at time t as part of an (possibly model-based) object recognition process. Furthermore, it shows Fig. 5, right side, enlarged tolerance zones 513, 514 around the actual bounding boxes (or boundaries) 511, 512. It can be checked whether a particle 300 is located outside an actual box (or boundary) 511, 512 and / or outside the tolerance zone 513, 514. If this is the case, the object attribute indicating that the particle 300 is not part of a detected and / or extracted dynamic object 150 can be assigned to this particle 300. Conversely, a particle 300 that lies within exactly one actual box (or boundary) 511, 512 can be assigned the membership of the corresponding object 150 as an object attribute (possibly only if the particle 300 has a movement 301 that lies within a tolerance zone of the movement of the object 150).

[0114] In an overlap area of ​​actual boxes (or borders) 511, 512 of different objects 150, a new assignment to an object 150 typically does not occur. In this case, a newly initialized particle 300 is assigned an object attribute indicating that particle 300 is not part of a dynamic object 150. However, a particle 300 that already has an object attribute to an existing object 150 and remains within the tolerance range of the assigned object 150 typically retains this assignment. Thus, the particle-object membership existing from the previous time can be used to continue to enable a unique assignment of cells 201 to objects 150, even in the case of overlap areas of multiple objects 150 occurring at the current time.

[0115] Thus, at time t, the object attributes of the individual particles 300 can be updated. Particles 300 that are outside the actual boxes (or boundaries) 511, 512 can lose their assignment to an object 150. Conversely, particles 300 that are not assigned to any object 150 and are clearly within a box (or boundary) 511, 512 can be assigned to an object 150. Particles 300 that are already assigned to an object 150 and remain within the tolerance range 513, 514 can retain this object attribute. The updated particles 300 can then be used in the particle tracking process for the subsequent time t + 1. The particles 300 can retain their respective updated object attributes.

[0116] Thus, one or more dynamic objects 150 can be detected at a time t. Each object τ 150 can have a specific object state s at time t. τ,t exhibit: sτ,t=[xtx,xty,νt,at,φt,ωt]T where x=[xtx,xty] displays the 2D position of object 150, where ν t The longitudinal velocity of the object is shown as 150, where α t the corresponding longitudinal acceleration of the object 150 is displayed, where φ t the alignment or orientation of the object 150 indicates, and where ω t = φ̇ t The rotation or yaw rate of object 150 is displayed (relative to a reference point of object 150, for example, relative to the center of the rear axis of object 150). Furthermore, at time t, object 150 has an actual bounding box (or boundary) 511, 512 with a specific width w and a specific length l.

[0117] The goal is to determine the object's condition. τ,t To robustly determine the time t of an object 150 for a sequence of consecutive time points t, this can be achieved using an unscented Kalman filter (UKF). A UKF comprises the following steps: a) Initialization of the object state s τ,t Upon initial detection of object 150, the individual state variables of the object state of object 150 can be determined or initialized based on the particle 300 and / or the bounding box or border 511, 512 of a newly detected object 150. b) Prediction of the object state ŝ τ,t based on the object's condition s τ,t-1 at a previous time t - 1. This can be done using prediction equations for the individual state variables of the object's state. The prediction is made for 2 · L + 1 sigma points ξ. τ,t-1, where L (in the present example L = 6) is the number of state variables in the object state s τ,t-1 is. The 2 · L + 1 sigma points ξ τ,t-1 are thereby carried out in a specific environment of the determined object state s τ,t-1 selected (possibly using the covariance of the object state s) τ,t-1 ). Within the framework of the prediction, 2 · L + 1 predicted sigma points ξ̂ are thus obtained. τ,t determined. Furthermore, the following can be determined for the individual sigma points ξ τ,t-1 each weight w ξ be determined. The predicted object state ŝ τ,t This is then calculated as the weighted mean of the predicted sigma points ξ̂ τ,t . c) Performing a measurement update or an innovation step using a current measurement s z of the object state. The measurement error, i.e., the difference between the predicted object state ŝ, is taken into account. τ,t and the measured object condition s z, used to at least partially determine the predicted object state ŝ τ,t to correct in order to update the object state s τ,t to determine.

[0118] The object's condition s τ,t-1 The previous time t - 1 can be used to predict the object state at a current time, in order to predict an object state ŝ. τ,t to determine. In the following, predicted quantities are described using the hat symbol “̂”. Furthermore, the index τ for identifying a specific object 150 is at least partially omitted to simplify the notation. The prediction of the current object state ŝ τ,t based on the actual previous condition of the object τ,t-1This can be done using a motion model for object 150. In this motion model, it can be assumed, for example, that object 150 has a constant rotation rate and a constant longitudinal acceleration. The following prediction equations then result. φ^t=φt−1+ωΔt ν^t=νt−1+aΔt x^tx=xt−1x+1ω2(ων^tsin(φ^t)+a cos(φ^t) −ων−1sin(φt−1)−a cos(φt−1)) x^ty=xt−1y+1ω2(−ων^t cos(φ^t)+a sin(φ^t) +ωνt−1 cos(φt−1)−a sin(φt−1)) where Δt is the time interval between two consecutive time steps. Furthermore, deviating from the basic assumption of a constant longitudinal acceleration and / or rotation rate, it can be assumed that the longitudinal acceleration and / or rotation rate converge relatively slowly to zero. This increases the robustness of the prediction. The following prediction equations then result: ω^t=(1−εω)ωt−1 a^t=arg mina1,a2(|a1|,|a2|),a1=(1−εa)at−1,a2=−νt−1thorizon where the reduction factors ε ω and ε α greater than zero and less than one, and where the term −νt−1thorizon, with t horizon ≥ Δt, sign changes of the longitudinal velocity can be avoided (e.g., in the case of a relatively strong deceleration of an object 150). In the above prediction equations, ω = ω̂ t , as well as α = â t .

[0119] Furthermore, process noise can be taken into account in the prediction, which can be described by a UKF matrix Q for process noise, with Q=[Q100Q2] and Q1=σω,a2[Δt44cos2φΔt44sinφcosφΔt32cosφΔt22cosφ.Δt44sin2φΔt32sinφΔt22sinφ..Δt2Δt...1] Q2=σω,ω˙2[Δt44Δt32Δt32Δt2] where Q1=Q1T (The points in the above matrix thus represent the corresponding terms from the rows).

[0120] As explained above, based on current measurements and the grid occupancy 200, a subset of cells 201 belonging to a specific object 150 can be determined. For these cells 201, a current bounding box (or boundary) 511, 512 can be determined. In particular, based on a boundary 511, 512 of object 150, the measured length l can be calculated. z and the measured width w z These quantities are determined, depending on the orientation φ of the boundary 511, 512. The measured length l z and the measured width w z can be determined as lz=maxc∈Cτ,t(x˜cφ,x)−minc∈Cτ,t(x˜cφ,x)+δdφ wz=maxc∈Cτ,t(x˜cφ,y)−minc∈Cτ,t(x˜cφ,y)+δdφ where the rotated cell positions x˜cφ,x or x˜cφ,y by rotation R φ according to the orientation φ from the cell positions xcx or xcy The individual cells 201 of object 150 result in: [x˜cφ,x,x˜cφ,y]T=Rφ[xcx,xcy]T

[0121] The operators "max" and "min" refer to the maximum and minimum positions of the individual cells 201 in the x and y directions, respectively. Thus, the length l can be determined using these operators. z the distance between the front edge and the back edge of the border 511, 512, and to determine the width w z The distance between the left and right edges of borders 511 and 512 can be determined. Furthermore, the cell size d can be determined. c × d c the individual cells 201 are taken into account in order to calculate the additional size term δdφ=dc(|sin(φ)|+|cos(φ)|)∈[dc,2dc] to calculate.

[0122] Thus, based on the current grid 200, a current bounding box or boundary 511, 512 of an object 150 can be determined. The current box or boundary 511, 512 can then be used to determine the current position of the object 150. The current position of the object 150 can be, for example, the center point of the current box or boundary 511, 512, or another reference point of the current box or boundary 511, 512. The current position can (as in Fig. 4 shown) e.g. determined as xzcenter=RφT[minc∈Cτ,t(x˜cφ,x)+12(lz−δdφ)minc∈Cτ,t(x˜cφ,y)+12(wz−δdφ)].

[0123] However, using the center point as a reference point is disadvantageous, since the size of a bounding box 511, 512 can change due to occlusions, which would lead to a change in the position of object 150 (even if object 150 is not moving). Therefore, a (back) corner or a point (e.g., the midpoint) of a (back) edge is preferred.

[0124] The reference point is then determined as xzref=xzcenter+RφT[δz,lreflzδz,wrefwz],δz,lref,δz,wref∈{0,12;−12}

[0125] Where the parameters δz,lref, δz,wref 150 can be selected for each object (e.g. to select a reference point that enables the most robust possible position determination).

[0126] To select a suitable reference point for an object 150, the four edges of a box 511, 512 can be considered, and the proportion of free cells in the vicinity of each edge can be determined. A relatively high proportion of free cells indicates relatively good visibility of the respective edge. If only one edge has relatively high visibility, a midpoint on that edge can be chosen as the reference point. Conversely, if two adjacent edges have relatively high visibility, the corner between these two edges can be chosen as the reference point.

[0127] It can be the ratio r F (A) are calculated to estimate the safety of an edge, with rF(A)=1|A|∑c∈Am(Fz,tc),A⊆G where A is the set of cells in the immediate outer neighborhood of an edge and |A| is the number of cells in the set A. The ratio rF (A) thus shows the average level of evidence per cell 201 for the hypothesis of a free cell 201 in the immediate vicinity of an edge. It can be assumed that the certainty ts The visibility of an edge e (where zero indicates low visibility and one indicates high visibility) can be approximated by the following value. ϑze≈rF(Ae)

[0128] The reference point for the front or rear edge running in the transverse direction can then be determined as δz,lref={+12,if ϑzfront≥ϑmin∧ϑzrear<ϑmin−12,if ϑzfront<ϑmin∧ϑzrear≥ϑmin0,else and by corresponding formulas for the longitudinal left or right edge.

[0129] Based on the particles 300 assigned to a cell 201 and on the basis of the movement 301 (especially the speed) of the individual particles 300, the (directional) speed of a cell 201 can be determined (as a weighted average of the directional speeds of the individual particles 300): vtc=(∑χ∈χtcoχ)−1⋅∑χ∈χtcoχvχ.

[0130] This is χtc the quantity of all particles 300 of a cell c 201 at time t, and o χ displays the dynamic evidence mass of particle 300.

[0131] Based on the velocities of the cells 201 of an object 150, an average velocity of the object 150 can be determined, e.g. as a weighted average of the cell velocities using the evidence measures of the individual cells 201: v¯=(∑c∈Cτ,tm(Dz¯,tc))−1⋅∑c∈Cτ,tm(Dz¯,tc)vtc From this directed velocity (i.e., from this 2D velocity vector), the magnitude of the velocity and the direction of motion can be derived as υz¯=‖v¯‖, φz¯=arctan(v¯).

[0132] The uncertainties or variances can be determined as συ,z¯2=ησ∑c∈Cτ,tm(Dz¯,tc)(‖vtc‖−υz¯)2 σφ,z¯2=ησ∑c∈Cτ,tm(Dz¯,tc)((arctan(vtc)−φz¯)mod 2π)2 with the normalization factor ησ=∑c∈Cτ,tm(Dz¯,tc)(∑c∈Cτ,tm(Dz¯,tc))2−∑c∈Cτ,t(m(Dz¯,tc))2.

[0133] Thus, (pseudo-)measurements of the longitudinal velocity ν can be performed. z̃ and the orientation φ z̃ These (pseudo-)measurements can be used, at least for initializing the object state of a newly detected object 150.

[0134] Furthermore, based on the (pseudo-)measurement of the orientation φ z̃A confidence interval for the orientation φ can be defined. This confidence interval can be used to restrict a (differently determined) measurement of the orientation φ to prevent the orientation φ from converging to a false local minimum. The convergence interval can be defined as Iφχ={φ∈[−π,π]||φ−φz¯|mod π≤γσφ,z¯+σφ,minχ} where γ is a scaling factor, and where σφ,minχ defines an additional uncertainty.

[0135] The use of particles 300 to determine a current measurement of the object velocity ν z and the object orientation φ z This can lead to a time delay, as the iterative update of particles 300 corresponds to a filtering process, which typically results in a time delay. Therefore, methods are described below for adjusting the object velocity ν. zand the object orientation φ z can be determined without filtering and without time delay.

[0136] For a specific orientation φ of the bounding box 511, 512 of an object 150, an area can be defined. Aiφ be defined, encompassing all cells 201 that are bordered by boxes 511 and 512 with length l z and with the width w z be enclosed. For this area, the average level of evidence for the hypothesis of an unoccupied cell can be determined, i.e. rF(Aiφ). To determine the current measured object orientation φ z The following optimization problem can then be solved. φz*=arg minφ∈Iφχ κ(φ), κ(φ)=rF(Aiφ) to determine the orientation φ for which the proportion of unoccupied cells or unoccupied evidence mass is minimal. To solve this optimization problem, values ​​for the occupancy measure can be gradually determined. rF(Aiφ) for different orientations φi=φ0+i⋅δφ∈Iφχ and i∈ℤ can be determined. The orientation φ determined based on particle 300 can then be used. z̃ The starting point φ0 can be used. Bounding boxes 511 and 512 can therefore be oriented in different ways to define different areas. Aiφ (i.e., different subsets of cells 201). For each of the different areas Aiφ A value for the occupancy measure can then be determined. rF(Aiφ) can be determined. If a specific orientation φ is chosen. i∗ a minimum value for rF(Aiφ) If this results in the current measured value being used, then the current measured value can be used. φz* φ z̃ for object orientation are determined as φz*≈(∑φ'1κ(φ'))−1⋅∑φ'1κ(φ')⋅φ', φ'∈{φi*,φi*+δφ,φi*−δφ}. with the variance or uncertainty σφ,z2=(∇r−∇i)−2 with the right- and left-hand gradients, e.g. determined by means of ∇r=κ(φimax,r)−κ(φi*)φimax,r−φi*,φimax,r=maxi>i*φi. (and for the left-hand gradient in a corresponding manner).

[0137] Thus, a measured value can φz* for object orientation in an efficient and robust manner. The current measured value φz* For object orientation, a current measured value can be used (as explained above). xzref to determine the position (of the reference point) of object 150.

[0138] A vehicle 100 can include one or more radar sensors as environmental sensors 111. The sensor data of a radar sensor 111 can be taken into account within the framework of the occupancy grid 200. In particular, speed measurements can be zv,tc 201 will be displayed for each (occupied) cell, showing a speed measurement. zv,tc especially can display • the radial velocity vzr cell 201 (in radial direction relative to position x) s of radar sensor 111; • the position x s of radar sensor 111; and / or • the azimuth angle θzr cell 201 relative to position x s of radar sensor 111.

[0139] The velocity ν τ The velocity of an object 150 can be considered as a scalar velocity in the longitudinal direction of a (velocity) reference point (e.g., the center of the rear axle) of the object 150 (see Fig. 6). Also the orientation φ τ and the rotation rate ω τThe speed of object 150 can be specified with respect to this (speed) reference point. This (speed) reference point may differ from the reference point described above for determining the position of object 150.

[0140] The 2D speed ν i at any point x i The object 150 can be determined as [vixviy]=vτ[cos(φτ)sin(φτ)]+wτ[−(xiy−xτy)xix−xτx] where the pivot point (ICR, instant center of rotation) of object 150 was taken into account with an Ackermann steering geometry and a drift-free operation of object 150.

[0141] The radial velocity vir at point x i This then results as vir=[cos(θir),sin(θir)][vixviy] =vτ(cos(θir)cos(φτ)+sin(θir)sin(φτ)) +wτ[cos(θir),sin(θir)][−(xiy−xτy)xix−xτx]

[0142] When using a discrete grid layout 200 with a discretized position x i However, for a cell 201, e.g., the cell center, this calculation method typically leads to a distortion, as it does not contain the exact continuous position of the (radar) detection. Such a distortion can be avoided by using the exact sensor position x s the measuring sensor 111 is taken into account. To derive an alternative calculation method, the position of a (radar) detection can thus be described as [xixxiy]=di[cos(θir)sin(θir)]+[xsxxsy] where d i = ||x i - x s || the distance between position x s of radar sensor 111 and the considered position x i of object 150 (or of cell 201 of object 150) or the position of the original measurement. Taking into account the relation [cos(θir).sin(θir)][−xiyxix]=[cos(θir),sin[θir]][−xsyxsx] This results in a precise calculation for the radial velocity. vir=vrcos(θir−φτ) +ωτ(sin(θir)(xsx−xτx)−cos(θir)(xsy−xτy)) without regard to the potentially distorting position of the discretized cell. This relationship between the longitudinal velocity ν τ of an object 150 or of a cell 201 of an object 150 and the radial velocity vir Cell 201 of object 150 can be used to precisely determine a predicted or expected value of the radial velocity based on a predicted value of the longitudinal velocity.

[0143] As part of the prediction step, the predicted object state ŝ is determined. τ,t a plurality of predicted sigma points ξ̂ τ,t determined. Each predicted sigma point ξ̂ τ,tThis indicates a specific predicted value for the individual state variables. In particular, a predicted sigma point ξ̂ can be used. τ,t include • a predicted value ν̂ τ,t,ξ the longitudinal velocity of the object 150; • a predicted value x^τ,t,ξx the x-component of the object's position 150; • a predicted value x^τ,t,ξy the y-component of the object's position 150; • a predicted value φ̂ τ,t,ξ the orientation of the object 150; and / or • a predicted value ω̂ τ,t,ξ The object's rotation rate is 150.

[0144] Using the above formula for radial velocity, it is therefore possible to calculate the radial velocity based on the predicted and / or expected values ​​of a predicted sigma point ξ̂. τ,t the corresponding predicted and / or expected value hir(ξ^τ,t) the radial velocity for this sigma point ξ̂ τ,t be determined as hir(ξ^τ,t)=v^τ,t,ξcos(θz,ir−φ^τ,t,ξ) +ω^τ,t,ξ(sin(θz,ir)(xs,ix−x^τ,t,ξx) −cos(θz,ir)(xs,iy−x^τ,t,ξy))

[0145] The predicted and / or expected value v^ir The radial velocity is then calculated as the weighted average of the values ​​ξ̂ for the individual predicted sigma points. τ,t determined predicted values hir(ξ^τ,t) the radial velocity as v^ir=∑ξ∈Ξwξhir(ξ^τ,t) where Ξ is the set of all sigma points ξ, and where w ξ the weights of the different sigma points ξ are.

[0146] The innovation step of the UKF can be implemented using the radial velocity (instead of the longitudinal velocity). In particular, the deviation between the predicted and expected values ​​can be considered an error, innovation, or measurement update. v^ir the radial velocity for a cell 201 or for an object 150 and the measured value vz,ir the radial velocity, i.e. vz,ir−v^ir, can be used. In this way, the measured velocity can be efficiently used to determine the object's state. τ,t of an object 150 are taken into account, whereby a measurement update of the (radial) velocity components can affect all components of the object state, allowing the orientation and / or rotation rate of an object to be estimated in addition to the longitudinal velocity.

[0147] This method can be applied analogously to other velocity-measuring sensors. For example, a lidar sensor can be used to measure radial velocity. Alternatively or additionally, the angular velocity of an object can be measured based on data from camera sensors. In this case, a corresponding formula for the tangential velocity can be provided.

[0148] Fig. Figure 7a shows a flowchart of an exemplary procedure 700 for determining the state of an object 150 at a sequence of time points. The procedure 700 can be executed by an evaluation unit 101 of a vehicle 100. The object state to be determined can include values ​​for one or more state variables, where the one or more state variables can include a position in a spatial area, an orientation, and / or a velocity of the object 150. The spatial area can comprise a plurality of cells 201.

[0149] The procedure 700 comprises, iteratively for each time point t from the sequence of time points, determining 701, based on sensor data from one or more sensors 111 with respect to the spatial area, an occupancy grid 200. The occupancy grid 200 can be determined as described in this document. The occupancy grid 200 can indicate, for each of the plurality of cells 201, evidence, a measure of evidence, or a probability that the respective cell 201 is free at time point t or occupied by the object 150. The occupancy grid 200 can be iteratively updated for each sequence of time points based on a current measurement of the sensor data. In particular, as described in this document, an actual occupancy grid can be Mt 200 will be determined.

[0150] Furthermore, the procedure 700 includes determining 702, based on the occupancy grid 200, a subset of one or more cells 201 from the multitude of cells 201 that belong to the object 150 at time t. This can be done, for example (as in connection with Fig. (described in section 5) using a clustering algorithm. Furthermore, particles 300 can be taken into account when assigning cells 201 to object 150.

[0151] Furthermore, the procedure 700 can include determining 703 a predicted object state of object 150 for time t based on the object state for a previous time t - 1. For this purpose, one or more of the prediction equations described in this document can be used.

[0152] Furthermore, the procedure 700 can include determining the object state 704 for or at time t based on the predicted object state for time t and on the basis of the determined subset of one or more cells 201 of the object 150. In particular, the subset of cells 201 of the object 150 determined for or at time t based on a current measurement of the sensor data can be used to provide a measurement update or innovation to update the object state for the previous time t - 1. Specifically, as described in this document, measured values ​​for the position of the object 150, for the orientation of the object 150, and / or for the (radial) velocity of the object 150 can be determined based on the determined subset of cells 201 of the object 150.In this way, the object state of an object 150 can be determined reliably and robustly for a sequence of successive time points.

[0153] The following describes one or more measures by which the size of an object 150 (in particular the length and / or the width of an object 150) can be determined in a precise manner.

[0154] The length l τ and / or the width w τ The length of an object (150) cannot typically be precisely determined with a single measurement at a specific time t. Rather, based on one or more measurements, typically only a lower limit of the length l can be established. τ and / or the width w τ to be determined. The fact that a measurement also determines the upper limit (and thus the actual value) of the length l τ and / or the width w τThis corresponds to the visibility of the individual edges of the bounding box 511, 512 of object 150.

[0155] Visibility ts The visibility of an edge e of bounding box 511, 512 at a given measurement can be defined as a value between zero and one, where zero indicates no visibility and one indicates full visibility. The visibility of an edge can be determined, as explained above, based on the free space in the immediate vicinity of the edge, as follows: ϑze≈rF(Ae)

[0156] Within the scope of a measurement (i.e., at a specific time t), the following measurement vectors can therefore be provided. ztl,ϑ=[lz,ϑzfront,ϑzrear]T, ztw,ϑ=[wz,ϑzleft,ϑzright]T.

[0157] A measurement vector can represent a current measured value l z , w zthe length or width, which results, for example, from the distance between the front and back edges (for length) or between the left and right edges (for width). Furthermore, the measurement vectors can determine the visibility. ts display the relevant edges (i.e. ϑzfront for the front edge, ϑzrear for the back edge, ϑzleft for the left edge, and tzright (for the right edge).

[0158] The goal is to increase the length l τ and the width w τ The aim is to determine the probability distribution of an object 150 based on all available measurements up to the current time t, i.e., all measurements from 1: t. In particular, a probability distribution for the length l is to be determined. τ and the width w τ of an object 150 can be determined as p(lτ,wτ|z1:tl,ϑ,z1:tw,ϑ)=p(lτ|z1:tl,ϑ)p(wτ|z1:tw,ϑ).

[0159] It can be assumed that the length l τ and the width w τ are independent of each other and can therefore be determined independently of each other. The following describes how to determine the length, while the width can be determined analogously. The following model can be used as an inverse sensor model for the length l. τ be accepted p(l|ztl,ϑ)∝exp(−(l−lz)22ςl,ϑσl2), ςl,ϑ={(ϑzfrontϑzrear)−1,if l≥lz1,else,

[0160] Through the scaling variable ς l,ϑ can be used to control the influence of visibility ts to consider an edge e. Fig. Figure 8 shows the measurement probability distribution p(l|ztl,ϑ) for different values ​​of the scaling variable ς l,ϑ .

[0161] The scaling variable ςl,ϑ is dependent on the visibility of the data used to determine the length l τ The relevant front edge and rear edge depend on the scaling variable ς. l,ϑ This therefore represents a (possibly inverse) measure of the quality with which the length l is produced. τ The length of an object can be measured at 150. If both edges are clearly visible, the upper limit of the length l can be determined. τ be determined in a precise manner, such that ς l,ϑ ≈ 1 and such that a normal distribution can be obtained for the measurement probability distribution p(l|ztl,ϑ) the length. However, if at least one of the two edges is not visible or only slightly visible, then ς tends to l,ϑ → ∞ and the measurement probability distribution results p(l|ztl,ϑ) The length is a piecewise function that is below the measured length l z still corresponds to a normal distribution, but above this measured length l τis approximately uniformly distributed, thus representing an approximate sigmoid function overall.

[0162] To determine the length l τ Based on all available measurements up to the current time t, i.e., for all time points 1:t, a one-dimensional (1D) histogram filter can be used. For this purpose, the continuous measurement probability distribution can be used. p(l|ztl,ϑ) the length for a specific number I l of different length intervals with the respective mean lengths l1, ..., l Il be discretized, where the distance between two mean lengths l i and l i+1 by the interval width δ i The probability distribution over a length interval can then be approximated by the following probability value. p(lτ|z1:tl,ϑ)≈plτ,tiδi, lτ∈(li−δi2;li+δi2]

[0163] The (cumulative) probability for a length interval i can then be determined recursively as follows: plτ,ti=p(li|ztl,ϑ)plτ,t−1i∑j=1llp(lj|ztl,ϑ)plτ,tj where the (cumulative) probabilities plτ,0i for the different length intervals i at time t = 0 can be initialized according to a uniform distribution, with plτ,0i=1ll,∀i. To avoid singularities and ensure reliable convergence, a condition can be set in each iteration that the individual probability values plτ,t−1i a specific minimum value ε min Do not fall below 0.

[0164] Filtering at a time t can be performed in such a way that the visibility ts the edges relevant for the length, and based on that the value of the scaling variable ς l,ϑis determined. This results in a measurement probability distribution. p(l|ztl,ϑ) the length for the current measurement. Based on the individual (cumulative) probability values plτ,t−1i For the previous time t - 1, the updated cumulative probability values ​​can then be calculated. plτ,ti The probability distribution for the current time t can be determined (using the above recursive formula). p(lτ|z1:tl,ϑ) This is then approximated based on the probability values. plτ,ti using the above formula.

[0165] By using a recursive histogram filter, the length and, correspondingly, the width of the bounding box 511, 512 of an object 150 can be determined efficiently and robustly and updated with the latest measurements. The length lτ of an object 150 (and correspondingly the width w) τ of an object (150) can then be used as the expected value of the cumulative probability distribution p(lτ|z1:tl,ϑ) The determined length and / or width can then be taken into account when updating the object status of object 150, as described above.

[0166] Furthermore, the determined length and / or width of the bounding box 511, 512 of an object 150 can be considered when classifying that object 150. Examples of classes include: a car, a truck, a pedestrian, a cyclist, a motorcyclist, and / or another type of road user. Thus, a set K of different classes of road users can be provided, and based on the length l and / or the width w of the bounding box 511, 512 of an object 150, it can be determined which class k ∈ K an object 150 belongs to. The speed v of the object 150 (indicated by its state) can also be taken into account. This results in the estimation problem. p(k|l,w,v)=p(k)p(l,w,v|k)p(l,w,v).

[0167] This problem can be reformulated under the assumption of conditional independence of l, w, v, as p(k|l,w,v)∝p(k)p(l|k)p(w|k)p(v|k).

[0168] Fig. Figure 9 shows exemplary class probability functions p(l|k), p(w|k), and p(v|k) for different classes. For the class "other road users," a uniform distribution for l, w, and v can be assumed. The class probability functions p(l|k), p(w|k), and p(v|k) for different classes can be determined experimentally. Alternatively or additionally, analytical approximations for the class probability functions p(l|k), p(w|k), and p(v|k) can be provided. For a pedestrian, a cyclist, and a motorcyclist, a normal distribution can be assumed for the length and / or width. Furthermore, normal distributions can be assumed for the width of a car and / or a truck. On the other hand, for a truck and / or a car, due to the relatively large differences in possible lengths, a combination of two sigmoid functions (one for the minimum possible length) can be used. lmink and one for the maximum possible length lmaxk will be accepted: p(l|k)∝S(l,lmink,αl.mink)(1−S(l,lmaxk,αl,maxk)) e.g. with each S(x,x0,α)=11+e−α(x−x0).

[0169] Furthermore, a probability distribution in the form of a negative sigmoid function can be assumed for the speed of the different classes: p(v|k)∝1−S(v,vmaxk,αvk).

[0170] The class probability function decreases for speeds above the maximum speed. vmaxk the respective class abruptly ends, so that the speed of a class can be used as an exclusion criterion.

[0171] The best class k* given known values ​​for length, width, and speed can be identified by the following optimization problem: k*=arg maxk∈Kp(k|l,w,v)=arg maxk∈Kp(k)p(l|k)p(w|k)p(v|k).

[0172] The classification can be based on each individual measurement of length, width, and / or speed (i.e., at each time t). The class determined at each time t for an object 150 can then be filtered to enable a robust classification of that object 150 over time.

[0173] Alternatively, the classification can be based on the cumulative probability distributions. p(lτ|z1:tl,ϑ) The length and / or width can be determined using the histogram filtering described above. In particular, (cumulative) probability values ​​can be calculated at a given time t. plτ,ti for the different length intervals i = 1, ..., I l can be determined. Similarly, (cumulative) probability values ​​can be calculated. pw,ti for the different width intervals i = 1, ... , I w to be determined.

[0174] The expected value of the length can then be determined at a time t. EL[p(Lτ|k)]=∑i=1llp(li|k)plτ,ti and in a corresponding way the expected value E[p(W τ |k) ] of the width can be determined. For the speed of an object 150, the maximum speed observed up to the current time t, i.e., between times 1: t, can be considered (as an exclusion criterion for a class): vτmax=maxt'=1,…,t(vτ,t').

[0175] The class kτ* The size of object 150 can then be calculated based on the filtered length and / or width using the following optimization problem: kτ*=arg maxk∈KEL,W[p(k|Lτ,Wτ,vτmax)]=arg maxk∈Kp(k)p(vτmax|k)EL,W[p(Lτ|k)p(Wτ|k)]=arg maxk∈K(p(k)p(vτmax|k)∑i=1llp(li|k)plτ,ti ∑i=1lwp(wi|k)pwτ,ti).

[0176] Therefore, the class kτ* 150 can be determined for an object, for which the product of • the a priori probability p(k) of a class, • the likelihood probability p(vτmax|k) for an object of this class to have a speed of 150. vτmax exhibits, and • the expected value of the likelihood probability for the length or width of the object 150 given this class The classification concept is enlarged, specifically maximized, i.e., exhibits the highest a posteriori probability p(k |l, w, v). This classification concept can be combined arbitrarily with further features of object 150 (which were determined, for example, based on image data from a camera) to further increase the classification accuracy.

[0177] The class determined for an object 150 kτ* This can be used to determine the most probable length of object 150 (especially if only the minimum length of object 150 can be measured based on the sensor data from one or more sensors 111), and / or to determine the most probable width of object 150 (especially if only the minimum width of object 150 can be measured based on the sensor data from one or more sensors 111). In particular, the most probable length (or width) of object 150 can be determined, assuming that object 150 is an object of class kτ* it.

[0178] The probability values ​​at a specific time t result from the measurements (especially from the determined actual occupancy grid 200). plτ,ti for the different length intervals i = 1, ..., I lFurthermore, the expected or likelihood probabilities are derived. p(li|kτ*) for the length of object 150, assuming that object 150 belongs to the class kτ* belongs. Consequently, combined probabilities result for the different length intervals i = 1, ..., I. l pcombi,l∝p(li|kτ*)⋅plτ,ti

[0179] The length interval can now be ilτ* to determine which has the highest combined probability ilτ*=arg maxi=1,…,Il{pcombi,l:pcombi,l>((1+ε) maxj=1,…,ipcombi,l)} where it can be stipulated as a constraint that larger lengths are only selected if the combined probability for that length is significant (i.e., higher by at least a factor of (1 + ε) than the preceding maximum of a lower length). The selection of a length interval ilτ* This can therefore be done by choosing the smallest length interval if several length intervals with similar combined probabilities are present. The (average) length lilτ* of the determined length interval ilτ* can then be considered the length lτ* The object 150 can be selected. The width can also be selected in a similar way. wτ* of object 150 taking into account the determined class kτ* of object 150 will be determined.

[0180] Regarding the determination of the length or width of an object 150, taking into account a class kτ* For the classification of object 150, a priori probabilities p(k) for relatively small and / or slow classes (especially for "pedestrians" and / or "cyclists") can be strengthened to increase the probability of assigning a class to a relatively small object 150 (provided this is not then excluded by a larger measured object size or maximum permissible speed). As a consequence, a particularly robust and / or conservative length and / or width determination can be achieved. This adjustment or conservative estimation can be used, in particular, to conservatively estimate the area relevant for an association of a predicted object in such cases.

[0181] Fig. Figure 7b shows a flowchart of an exemplary procedure 710 for determining the value of a quantity (in particular the length and / or width) of an object 150 in a spatial area. The procedure 700 can be executed by an evaluation unit 101 of a vehicle 100. The spatial area can be the vicinity of the vehicle 100. The spatial area can comprise a plurality of cells 201.

[0182] Method 710 comprises iteratively determining, for each time point t from the sequence of time points, an (actual) occupancy grid 200 based on sensor data from one or more sensors 111 with respect to the spatial area. The occupancy grid 200 can be determined as described in this document. The occupancy grid 200 can indicate, for each of the plurality of cells 201, evidence, a measure of evidence, or a probability that the respective cell 201 is free at time point t or occupied by the object 150. The occupancy grid 200 can be iteratively updated for each sequence of time points based on a current measurement of the sensor data.

[0183] Furthermore, procedure 710 includes determining, based on the occupancy grid 200, a subset 712 of one or more cells 201 from the multitude of cells 201 that belong to object 150 at time t. This can be done, for example (as in connection with Fig. 5 described) using a clustering algorithm.

[0184] Furthermore, the method 710 can include the detection 713, based on the grid 200, of two opposite bounding edges of the object 150 (in particular, a boundary or bounding box 511, 512 of the object 150). Specifically, the front and back edges can be detected (to determine the length of the object 150) or the left and right edges can be detected (to determine the width of the object 150). The edges can be determined based on the arrangement of the subset of cells 201 of the object 150 (e.g., such that the boundary or bounding box 511, 512 is as small as possible while enclosing the subset of cells 201 as completely as possible).

[0185] Furthermore, the procedure 710 can include determining 714 a measured value of the size of the object 150 for time t, based on a distance between the detected edges of the object 150.

[0186] Furthermore, procedure 710 includes determining, based on the occupancy grid 200, a quality measure that indicates how well the two opposite edges of object 150 could be detected. In particular, the visibility of the edges can be determined (as described in this document).

[0187] Furthermore, procedure 710 can include determining, based on the precision measure, a measurement probability distribution of the size of object 150 for time t. The measurement probability distribution can indicate the probability of different values ​​of the size of object 150 around the measured value for time t. The measurement probability distribution depends on the precision measure (especially for values ​​of the size of object 150 that are above the measured value). As the precision measure decreases (i.e., as the visibility of the edges decreases), the measurement probability distribution can increasingly become a uniform distribution.

[0188] Furthermore, procedure 710 includes determining 717 a cumulative probability distribution for time t based on the measurement probability distribution for time t and starting from the cumulative probability distribution for a previous time t - 1. In other words, the cumulative probability distribution can be updated using the measurement probability distribution for time t.

[0189] Finally, procedure 710 includes determining 718 a value for the size of object 150 for time t based on the cumulative probability distribution for time t. In particular, the size of object 150 can be determined as the expected value or as the mean of the cumulative probability distribution for time t.

[0190] Method 710 thus makes it possible to iteratively combine current measurements relating to the size of an object 150 with previous measurements relating to the size in order to gradually increase the accuracy of the determined size value. This enables precise environmental perception.

[0191] The present invention is not limited to the embodiments shown. In particular, it should be noted that the description and the figures are intended only to illustrate the principle of the proposed methods, devices, and systems.

Claims

[1] Method (700) for determining an object state of an object (150) at a sequence of time points, wherein the object state comprises values ​​for a plurality of state variables, wherein the plurality of state variables comprises a position in a spatial area, an orientation and a velocity of the object (150), wherein the spatial area comprises a plurality of cells (201); wherein the method (700) comprises iteratively determining, for each time point (t) from the sequence of time points, - Determine (701), based on sensor data from one or more sensors (111) in relation to the spatial area, an occupancy grid (200) that indicates for each of the multitude of cells (201) evidence that the respective cell (201) is free at time (t) or is occupied by the object (150); - Determine (702), based on the occupancy grid (200), a subset of one or more cells (201) from the multitude of cells (201) that belong to the object (150) at time (t); - Determining a measured value of a first state variable of the plurality of state variables for time (t) based on the determined subset of one or more cells (201) of the object (150); - Determine (703) a predicted object state of the object (150) for time (t) based on the object state for a previous time (t - 1); - Determining a predicted value of the first state variable from the predicted object state of object (150) for time (t); - Determining a deviation between the predicted value and the measured value of the first state variable; and - Determining (704) the object state for time (t) based on the predicted object state for time (t), based on the determined subset of one or more cells (201) of the object (150) and based on the determined deviation. [2] Method (700) according to claim 1, wherein the object state of the object (150) is determined at the sequence of time points using an unscented Kalman filter. [3] Method (700) according to any one of the preceding claims, wherein the method (700) comprises, - Determining a plurality of sigma points in a neighborhood of the object state for the previous time (t - 1); where a sigma point comprises values ​​for the plurality of state variables; - Determining a plurality of weights for the plurality of sigma points; - Determining a plurality of predicted sigma points for time (t) based on the corresponding plurality of sigma points for the preceding time (t - 1); and - Determining the predicted object state for time (t) based on the plurality of predicted sigma points and the plurality of weights, in particular as a weighted mean of the plurality of predicted sigma points. [4] Method (700) according to any one of the preceding claims, wherein the predicted object state is determined for time (t) based on φ^t=φt−1+ωΔt where φ̂ t a predicted value of the orientation for the time (t), φ t-1 a determined value of the orientation for the previous time (t - 1), ω a value of the rotation rate of the object (150) and Δt a time interval between the previous time (t - 1) and the current time (t); and / or based on v^t=vt−1+aΔt where v̂ t a predicted value of the velocity for time (t), v t-1 a determined value of the velocity for the previous time (t - 1) and a is a value of the acceleration of the object (150); and / or based on x^tx=xt−1x+1ω2(ωv^tsin(φ^t)+a cos(φ^t) −ωvt−1sin(φt−1)−a cos(φt−1)) where x^tx a predicted value of a first component of the position for time (t) and xt−1x a determined value of the first component of the position for the previous time (t - 1); and / or based on x^ty=xt−1y+1ω2(−ωv^tcos(φ^t)+a sin(φ^t) +ωvt−1cos(φt−1)−a sin(φt−1)) where x^ty a predicted value of a second component of the position for time (t) and xt−1y a determined value of the second component of the position for the previous time (t - 1). [5] Method (700) according to any one of the preceding claims, wherein - the object state includes a longitudinal acceleration a of the object (150); and - determining (703) the predicted object state for time (t) includes determining a predicted value â t the longitudinal acceleration for time (t) is calculated based on a determined value a t-1 the longitudinal acceleration for the previous time (t - 1), that |â t | ≤ |a t-1 |; and / or - the object state includes a rotation rate ω of the object (150); and - determining (703) the predicted object state for time (t) includes determining a predicted value ω̂ t the rotation rate for time (t) such that it is based on a determined value ω t-1 the rotation rate for the previous time (t - 1), such that |ω̂ t | ≤ |ω t-1 | [6] Method (700) according to any one of the preceding claims, wherein the method (700) comprises, - Determining a boundary (511, 512), in particular a bounding box, of the object (150) based on the determined subset of one or more cells (201) of the object (150); - Determining a plurality of different subsets of cells (201) of the occupancy grid (200) for a corresponding plurality of border orientations (511, 512); - Determine, for each of the plurality of distinct subsets of cells (201), a cumulative and / or mean level of evidence that the respective subset of cells (201) is unoccupied or occupied; and - Determining a measurement value for the orientation of the object (150) for time (t) based on the determined majority of cumulative and / or mean evidence. [7] Method (700) according to claim 6, wherein the method (700) comprises for an orientation of the border (511, 512), - Determining a subset of cells (201) of the occupancy grid (200) that are at least partially enclosed by the border (511, 512); and - Determining a sum and / or mean of the evidence of the cells (201) of the identified subset of cells (201). [8] Method (700) according to any one of the preceding claims, wherein - the one or more sensors (111) comprising a velocity-measuring sensor configured to measure a value of a radial and / or tangential velocity of the first cell (201) of the object (150) at time (t); - the procedure (700) comprises determining a predicted value of the radial and / or tangential velocity of the first cell (201) for time (t) based on the predicted object state of the object (150) for time (t); and - the procedure (700) comprises determining a deviation between the predicted value and the measured value of the radial and / or tangential velocity of the first cell (201) at time (t); and - the procedure (700) includes determining (704) the object state for time (t) based on the determined deviation. [9] Method (700) according to claim 8, wherein the predicted value of the radial velocity of the first cell (201) for time (t) is determined based on hir(ξ^τ,t)=v^τ,t,ξcos(θz,ir−φ^τ,t,ξ) +ω^τ,t,ξ(sin(θz,ir)(xs,ix−x^τ,t,ξx) −cos(θz,ir)(xs,iy−x^τ,t,ξy) where v̂ τ,t,ξdisplays a predicted value of the object's longitudinal velocity (150), x^τ,t,ξx displays a predicted value of a first component of the object's position (150), x^τ,t,ξy indicates a predicted value of a second component of the object's position (150), φ̂ τ,t,ξ displays a predicated value of the object's orientation (150); and ω̂ τ,t,ξ displays a predicted value of the object's rotation rate (150); where hir(ξ^τ,t) the predicted value of the radial velocity of the first cell (201); where xs,ix a value of the first component of a position of the speed-measuring sensor is, xs,iy a value of the second component of the position of the speed-measuring sensor, and θz,iτ an azimuth angle of a measurement of the velocity-measuring sensor shown in the first cell (201) relative to the position of the velocity-measuring sensor. [10] Method (700) according to any one of the preceding claims, wherein the method (700) comprises, - Determine, based on positions of the determined subset of one or more cells (201) of the object (150), as well as based on an orientation of the object (150), a length and a width of a bounding box (511, 512) of the object (150); - Determining the position of a reference point on or in the bounding box (511, 512) as a measurement of the position of the object (150) at time (t); - Determining a predicted value of the object's position (150) from the predicted object state; - Determining a deviation between the predicted value and the measured value of the object's position at time (t); and - Determine (704) the object state for time (t) based on the determined deviation. [11] Method (700) according to claim 10, wherein the method (700) comprises, - Determine, based on the occupancy grid (200), an occupancy measure that an area of ​​edges of the bounding box (511, 512) of the object (150) is unoccupied; and - Selecting the reference point as a point on an edge of the bounding box (511, 512), depending on the determined occupancy measure. [12] Method (700) according to any one of the preceding claims, wherein at least some of the cells (201) of the occupancy grid (200) display motion information relating to a movement of the respective cell (201); and wherein the method (700) comprises, - Detect, at a specific time, a new object (150) based on the occupancy grid (200) for the specific time; - Determining a subset of cells (201) for the new object (200); and - Initializing the object state for the new object (200) based on positions of the determined subset of cells (201) for the new object (200) and based on the movement information of the determined subset of cells (201) for the new object (200). [13] Method (700) according to any one of the preceding claims, wherein - the procedure (700) comprises providing a driving function of a vehicle (100) depending on the determined object state and / or the determined value of the object's size (150); and - the driving function includes at least partially automated longitudinal and / or lateral guidance of the vehicle (100). [14] Method (700) according to any one of the preceding claims, wherein - comprising one or more sensors (111) including a radar sensor and / or a lidar sensor; and / or - the spatial area encompasses the environment of a vehicle (100). [15] Processing unit (101) for a vehicle (100), wherein the processing unit (101) is configured to perform the method (700) according to one of the preceding claims.

Images