Mesh-based road model with multiple layers

Abstract

This document describes techniques, apparatus, and systems for grid-based road models with multiple layers. An example road perception system generates a road grid representation comprising a plurality of cells. The road perception system uses data from a plurality of information sources to generate a road model comprising a plurality of layers. Each layer represents a road property for each cell in the grid and comprises one or more layer hypotheses. In this manner, the described techniques and systems can provide accurate and reliable road models and quantify uncertainty therein.

Description

[0001] This invention application is a divisional application of the invention patent application with application number 202210170435.5, application date February 23, 2022, entitled "Grid-based road model with multiple layers". Background Technology

[0002] To control vehicles, road perception systems provide information about road conditions, road geometry, and lane geometry to vehicle-based systems. Various vehicle-based systems that rely on these road perception systems exist, including, for example, adaptive cruise control (ACC), traffic jam assist (TJA), lane centering assist (LCA), and Level 3 / Level 4 autonomous driving on highways. Some safety regulations require such vehicle-based systems (including Level 3 / Level 4 systems) to generate road models that can conceptualize or model multiple road attributes. Additionally, some safety standards require road models to quantify the uncertainties associated with them. Existing road perception systems typically do not provide information about multiple road attributes (if any). These systems also tend to rely on parametric methods (e.g., representing lane boundaries and lane centers as polylines), which are often inaccurate, making them unreliable for road modeling; furthermore, their uncertainties cannot be quantified. In addition, even when acquiring sensor data from multiple different sensors, some road perception systems still cannot accurately fuse sensor data to build a road model, especially when sensor data from two sensors conflict. Summary of the Invention

[0003] This document describes techniques, apparatus, and systems for grid-based road models with multiple layers. For example, this document describes a road perception system configured to generate a grid representation of a road comprising multiple cells. The road perception system uses data from multiple information sources to generate a road model for the road. Multiple layers exist beneath the road model. Each layer represents the road properties of each cell in the grid. The road model also includes layer hypotheses that indicate the potential attribute states for cells in each layer.

[0004] The road perception system then determines a quality value associated with the layer hypothesis. The quality value indicates the confidence level associated with the data contributing to the corresponding layer hypothesis. The road perception system uses the quality value to determine a confidence parameter and a plausibility parameter associated with the layer hypothesis. The confidence parameter indicates the confidence level of the latent attribute state for this unit, and the plausibility parameter indicates the probability that the latent attribute state is accurate for this unit. The road perception system determines whether the confidence parameter and plausibility parameter associated with the layer hypothesis meet corresponding thresholds. Depending on whether the confidence parameter and plausibility parameter of at least one layer hypothesis meet the corresponding thresholds, the autonomous driving system or the driver assistance system may or may not use at least one layer hypothesis as input to operate vehicles on the road.

[0005] This document also describes other operations of the systems, techniques, and apparatuses summarized above, as well as other methods set forth herein, and apparatus for performing these methods.

[0006] This invention presents a simplified concept of a grid-based road model with multiple layers, which is further described in the detailed description and accompanying drawings. This invention is not intended to identify essential features of the claimed subject matter, nor is it intended to define the scope of the claimed subject matter. Attached Figure Description

[0007] The following figures illustrate details of one or more aspects of a multi-layered mesh-based road model in this document. The same numbers are used throughout the figures to refer to similar features and components:

[0008] Figure 1 This example environment shows how the road perception system generates a grid-based road model with multiple layers.

[0009] Figure 2 An example configuration of a road perception system that generates a grid-based road model with multiple layers is shown;

[0010] Figure 3 An example architecture for a road perception system used to generate a grid-based road model with multiple layers is shown.

[0011] Figure 4 An example static mesh is shown, generated from a grid-based road model with multiple layers, for which a road perception system can determine lane boundary cells.

[0012] Figure 5A and Figure 5B The input processing module of a grid-based road model with multiple layers is shown to be used to offset the input evidence for determining quality values ​​for assumptions about other layers;

[0013] Figure 6 The input processing module of a grid-based road model with multiple layers is shown to be used to determine a set of sample lanes for assigning quality values ​​to lane numbers; and

[0014] Figure 7 The flowchart illustrates an example process performed by a road perception system configured to generate a grid-based road model with multiple layers. Detailed Implementation

[0015] Overview

[0016] Road perception systems can be an important technology for driver assistance and autonomous driving systems. Some vehicle-based systems (e.g., L3 / L4 systems) and some safety standards (e.g., Safety of Intended Functions (SOTIF) (ISO / PAS21448:2019 "Road vehicles - Safety of Intended Functions")) may require road perception systems not only to model the road, but also to quantify the uncertainties in the model and maintain one or more assumptions.

[0017] Road models may be necessary for many functions of vehicle-based systems, including ACC, LCA, and L3 / L4 systems. For example, road modeling is crucial for these systems to understand the road environment, plan driving trajectories, and control vehicles. Some road perception systems define a road as any street, avenue, road, highway, highway, driveway, ramp, path, parking lot, garage, side road, or other path that can be traversed by vehicles. These systems typically define a road as a set of parametric lanes (e.g., represented by a set of polyline boundaries). Therefore, road curvature is usually not considered. These systems often also provide incomplete road models with only a single layer or Framework of Identification (FOD) and cannot fuse data from different sources.

[0018] In contrast, this document describes a road perception technique for accurately modeling roads and road elements as a cell mesh. This meshing approach allows the described road perception technique to avoid inaccurate lane assumptions associated with the parameterized representation of lane boundaries. Road elements are represented as one or more layer assumptions for each cell. The described road perception technique fuses information about roads from multiple sources (e.g., camera systems, LiDAR systems, maps, radar systems, ultrasound) to develop a static mesh with multiple layers. These layers represent different road attributes (e.g., cell category, lane marking type, lane marking color, lane numbering assignment, pavement marking type, road sign type). The road perception technique can also extract quality values ​​from information sources to estimate confidence and plausibility parameters associated with each layer assumption. In this way, the described road perception technique can provide critical information related to the road environment and model uncertainties to enable safe path planning and maneuver control for vehicle-based systems.

[0019] This section describes only one example of how the described techniques and systems can generate grid-based road models with multiple layers. Other examples and implementations are described in this document.

[0020] Operating environment

[0021] Figure 1 An example environment 100 is illustrated, in which a road perception system 108 generates a mesh-based road model with multiple layers. In the depicted environment 100, the road perception system 108 is mounted on a vehicle 102 or integrated within a vehicle 104. The vehicle 102 can travel on a road 120, which includes lanes 122 (e.g., a first lane 122-1 and a second lane 122-2). In this implementation, the vehicle 102 travels in the first lane 122-1.

[0022] Although shown as a passenger car, vehicle 102 could represent other motorized vehicles (e.g., motorcycles, buses, tractor-trailers, semi-trailer trucks, or construction equipment). Typically, the manufacturer can mount the road perception system 108 onto any mobile platform that can travel on the road 120.

[0023] In the depicted implementation, a portion of the road perception system 108 is mounted in the rearview mirror of the vehicle 102 to have a field of view of the road 120. The road perception system 108 can project the field of view from any external surface of the vehicle 102. For example, the vehicle manufacturer may integrate at least a portion of the road perception system 108 into a side mirror, bumper, roof, or any other internal or external location where the field of view includes the road 120. Generally, the vehicle manufacturer may design the location of the road perception system 108 to provide a specific field of view that sufficiently encompasses the road 120 on which the vehicle 102 may travel.

[0024] Vehicle 102 also includes one or more sensors 104 to provide input data to one or more processors (not shown) of road perception system 108. Figure 1 (In the middle). Sensor 104 may include a camera, radar system, Global Positioning System (GPS), Global Navigation Satellite System (GNSS), lidar system, or any combination thereof. The camera may capture still images or videos of road 120. The radar system or lidar system may use electromagnetic signals to detect objects in road 120 or features of road 120. GPS or GNSS may determine the position or heading of vehicle 102. Vehicle 102 may include additional sensors to provide road perception system 108 with input data relating to road 120 and lane 112. Road perception system 108 may also use vehicle-to-everything (V2X) or cellular communication technologies to acquire input data from external sources (e.g., nearby vehicles, nearby infrastructure, the Internet).

[0025] Vehicle 102 also includes map 106. Map 106 may include a high-resolution map or database providing information related to road 120 and lane 122. Map 106 may be stored in the memory of road perception system 108 or the memory of vehicle 102, and may be a map retrieved from a map or navigation service communicating with road perception system 108, or a map obtained from a mobile phone or other device communicatively coupled to road perception system 108.

[0026] Road perception system 108 may use fused-grid module 110 and layer module 112 to represent road 120 as a grid-based road model. Road perception system 108 represents the attributes and elements of road 120 in the form of cells. In this way, road perception system 108 can provide a non-parametric representation of road 120 without making assumptions about lane shapes. Road perception system 108 uses multiple Frameworks of Identification (FODs) to define and describe possible states or assumptions for specific attributes of road 120. FODs may include at least two of the following: cell category, lane number, lane marking type, lane marking color, traffic signs, pavement markings, and lane type. Road perception system 108 may define additional FODs as required by vehicle-based system 114.

[0027] The fusion grid module 110 can extract quality values ​​from input data from sensor 104 and map 106 using a quality extraction formula. The quality values ​​indicate the confidence level associated with the data. The fusion grid module 110 can then fuse data from various sources (e.g., sensor 104, map 106) to identify layer hypotheses for each cell. Layer hypotheses identify possible values ​​for each FOD (Foreign Object Dependency) for each cell. The corresponding quality values ​​determine the uncertainty associated with the layer hypotheses.

[0028] Layer module 112 can use corresponding quality values ​​to determine a confidence parameter and a plausibility parameter associated with each layer hypothesis. The confidence parameter represents the evidence supporting the hypothesis (e.g., the sum of quality values ​​of a subset of the hypothesis) and provides a lower bound. The confidence parameter of a layer hypothesis indicates the confidence level of the road perception system 108 in the latent attribute state for that unit. The plausibility parameter is represented by subtracting the evidence not supporting the hypothesis (e.g., subtracting the sum of quality values ​​of sets that intersect the hypothesis but are empty) and is an upper bound. The plausibility parameter indicates the probability that the latent attribute state is accurate for that unit. The values ​​of the confidence parameter and the plausibility parameter may be different for each unit of the layer hypothesis. (This document references...) Figure 2 and Figure 3 The components and operation of the road perception system 108 are described in more detail, including the fused mesh module 110 and the layer module 112.

[0029] Vehicle 102 also includes one or more vehicle-based systems 114 that can use data from road perception system 108 and a grid-based road model to operate vehicle 102 on road 120. Vehicle-based systems 114 may include driver assistance systems 116 and autonomous driving systems 118 (e.g., adaptive cruise control (ACC), traffic jam assist (TJA), lane centering assist (LCA), and Level 3 / Level 4 autonomous driving (L3 / L4) systems on highways). Typically, vehicle-based systems 114 can use the grid-based road model provided by road perception system 108 to operate the vehicle and perform specific driving functions. For example, driver assistance system 116 can provide adaptive cruise control and monitor the presence of objects in the first lane 122-1 in which vehicle 102 is traveling (as detected by another system on vehicle 102). In this example, road perception data from road perception system 108 identifies lane 122. As another example, the driver assistance system 116 can provide an alert when the vehicle 102 crosses the lane markings of the first lane 122-1.

[0030] The autonomous driving system 118 can move the vehicle 102 to a specific location on the road 120 while avoiding collisions with objects detected by other systems on the vehicle 102 (e.g., radar systems, lidar systems). Road perception data provided by the road perception system 108 can provide information about the position of lane 122 and the uncertainty of lane 122's position, thereby enabling the autonomous driving system 118 to perform lane changes or maneuver the vehicle 102.

[0031] Figure 2 An example configuration of a road perception system 108 for generating a grid-based road model with multiple layers is shown. The road perception system 108 may include one or more processors 202 and a computer-readable storage medium (CRM) 204.

[0032] As a non-limiting example, processor 202 may include a system-on-a-chip (SoC), an application processor (AP), a central processing unit (CPU), or a graphics processing unit (GPU). Processor 202 may be a single-core or multi-core processor implemented using a homogeneous or heterogeneous core architecture. Processor 202 may include a hardware-based processor implemented as hardware-based logic, circuitry, processing cores, etc. In some aspects, the functionality of processor 202 and other components of the road perception system 108 is provided via an integrated processing, communication, and / or control system (e.g., SoC) to enable various operations of vehicle 102. Processor 202 may generate road perception data for vehicle-based system 114 based on a grid-based road model of road perception system 108.

[0033] The CRM 204 described herein does not include propagation signals. The CRM 204 may include any suitable memory or storage device that can be used to store device data (e.g., map 106) of the road perception system 108, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), non-volatile RAM (NVRAM), read-only memory (ROM), or flash memory.

[0034] Processor 202 executes computer-executable instructions stored within CRM 204. As an example, processor 202 may execute fusion mesh module 110 to generate a mesh-based road model of road 120. Fusion mesh module 110 may use data from map 106 stored in CRM 204 or data acquired from sensor 104 to generate the mesh-based road model.

[0035] The fused mesh module 110 includes an input processing module 206 and an evidence fusion module 208. The processor 202 can execute the input processing module to extract quality values ​​associated with different input data. For example, the input processing module 206 can use quality extraction formulas to determine the corresponding quality values ​​associated with data from different sources (e.g., sensor 104, map 106). The quality values ​​indicate the confidence level associated with the data, which contributes to the layer assumptions of the grid-based road model. The input processing module 206 can also validate the input data.

[0036] The processor 202 can execute the evidence fusion module 208 to recursively update the quality values ​​associated with the input data. For example, the evidence fusion module 208 can compare data from one or more previous time points with data from the current time point to update the quality values ​​associated with the input data.

[0037] Layer module 112 may include output processing module 210. Output processing module 210 may estimate confidence parameters and plausibility parameters associated with one or more layer hypotheses for each unit. Processor 202 may also execute output processing module 210 to verify the confidence parameters and plausibility parameters.

[0038] Layer module 112 can also group layer hypotheses into one or more path hypotheses based on Dempster-Shafer theory. Dempster-Shafer theory provides a framework for reasoning about a set of layer hypotheses affected by uncertainty. A generalization of Bayesian probability theory, Dempster-Shafer theory explains the lack of evidence or its omission when estimating the probability of a layer hypothesis being true. (See also...) Figures 3 to 6 The generation of the layer hypothesis and the road hypothesis is described in more detail.

[0039] The road perception system 108 may also include a communication component 212, which includes a sensor interface 214 and a vehicle-based system interface 216. The sensor interface 214 and the vehicle-based system interface 216 can transmit data via the communication bus of the vehicle 102. For example, when the various components of the road perception system 108 are integrated within the vehicle 102, the sensor interface 214 and the vehicle-based system interface 216 can communicate via a wired or wireless vehicle bus (e.g., a controller-based vehicle network (CAN)).

[0040] The processor 202 can also receive measurement data from one or more sensors 104 via the sensor interface 214 as input to the road perception system 108. As an example, the processor 202 can receive image or video data from a camera system via the sensor interface 214. Similarly, the processor 202 can send configuration data or requests to one or more sensors 104 via the sensor interface 214.

[0041] The vehicle-based system interface 216 can transmit road perception data (including a grid-based road model) to the vehicle-based system 114 or another component of the vehicle 102. Generally, the road perception data provided by the vehicle-based system interface 216 is in a format usable by the vehicle-based system 114. In some implementations, the vehicle-based system interface 216 can send information to the road perception system 108, which, as a non-limiting example, includes the speed or heading of the vehicle 102. The road perception system 108 can use this information to appropriately configure itself. For example, the road perception system 108 can adjust the frame rate or scan rate of one or more sensors 104 via the sensor interface 214 based on the speed of the vehicle 102 to maintain the performance of the road perception system 108 under changing driving conditions.

[0042] The operation of merging mesh module 110, layer module 112 and their corresponding sub-components will refer to Figures 3 to 7 It was described in more detail.

[0043] Figure 3 An example architecture 300 for a road perception system 108 used to generate a grid-based road model 322 with multiple layers is shown. Architecture 300 is shown. Figure 2 The example components and functions of the fusion mesh module 110, input processing module 206, evidence fusion module 208, output processing module 210, and layer module 112 are described. In other implementations, the road perception system 108 includes... Figure 3 The components and functions shown are compared to additional or fewer components and functions.

[0044] Architecture 300 illustrates example information sources input to the fusion mesh module 110. Input sources may include mesh 302, LiDAR data 304, visual data 306, vehicle status data 308, and map 106. Other data may include radar data, ultrasonic data, data from online sources, and data received via V2X communication. Mesh 302 provides a mesh representation of road 120 and may be a static occupancy mesh and / or a dynamic mesh. Mesh 302 may include mesh size, mesh resolution, and cell center coordinates. Each cell of mesh 302 may also indicate the speed of vehicle 102. LiDAR data 304 provides information related to objects on road 120 and road features. Visual data 306 may include still images or videos of road 120 and provide information related to lane boundaries, objects on road 120, and other road features. Vehicle status data 308 can provide information related to the speed, position, and heading of vehicle 102. Vehicle status data 308 may be provided by, for example, steering sensors, yaw rate sensors, heading sensors, or position sensors. Map 106 can provide information related to the characteristics of road 120 and lane 122. Road perception system 108 may include additional (e.g., radar data) or less input to fusion grid module 110.

[0045] The output of layer module 112 is a grid-based road model 322. The grid-based road model 322 includes multiple layer hypotheses 324 for each layer or recognition framework defined by the road perception system 108. The grid-based road model 322 also includes cell values ​​326, which indicate the corresponding confidence parameters, plausibility parameters, and probabilities associated with the layer hypotheses 324.

[0046] A Framework of Identification (FOD) is defined as a set of cell values ​​within a layer of a grid-based road model 322 that describes possible states or hypotheses for a specific problem. Generally, the elements in a FOD layer are mutually exclusive and exhaustive. For example, Lane Number FOD and Lane Marking Type FOD address different problems. From a computational perspective, the number of layer hypotheses 324 that need to be evaluated for each FOD grows exponentially with the number of elements within them. Therefore, the described road perception system 108 defines multiple FOD layers within the grid-based road model 322 to minimize the computational overhead for any given layer.

[0047] Layer module 112 can generate multiple layer hypotheses 324 for multiple FOD layers. For example, a grid-based road model 322 may include seven FODs, including a cell category layer, a lane number layer, a lane marking type layer, a lane marking color layer, a traffic sign layer, a pavement marking layer, and a lane type layer. Depending on the needs of the vehicle-based system 114, the driving environment, or any other factors, layer module 112 may define additional or fewer FOD layers.

[0048] The cell category layer describes the possible categories of cells within the mesh-based road model 322. Cell category layer Ω C It can be restricted in equation (1) as follows:

[0049] Ω C ={L b L c B a ,O} (1)

[0050] Where L b Indicates lane boundaries, L c Indicates the center of the lane, B a The symbol indicates an obstacle, and O indicates an "other" value.

[0051] The lane numbering layer describes the possible lane numbering assignments for each unit. Lane Numbering Layer Ω LA It can be restricted in equation (2) as follows:

[0052] Ω LA ={-1, 0, 1, 2} (2)

[0053] Where -1 represents the adjacent lane on the left, 0 represents the current lane (e.g., the vehicle's own lane) in which vehicle 102 is traveling, 1 represents the adjacent lane on the right, and 2 represents other lanes.

[0054] The lane marking type layer describes the possible lane marking types associated with lane boundary units. Lane Marking Type Layer Ω MT It can be restricted in equation (3) as follows:

[0055] Ω MT ={M 实线 M 双 M 虚线 M WF M O} (3)

[0056] Where M 实线 Indicating solid lane markings, M 双 Indicating dual-lane lane markings, M 虚线 Indicating dashed lane markings (e.g., skip lines), M WF The thick white lane markings indicate the M O Additional lane marking types can be specified for different regions and countries.

[0057] The lane marking color layer describes the possible lane marking colors associated with lane boundary cells. Lane Marking Color Layer Ω MC It can be restricted in equation (4) as follows:

[0058] Ω MC ={C Y C W C O} (4)

[0059] Where C Y The indicator color is yellow for lane markings, C W Indicates white color, and C O Indicates another color. Additional lane marking colors can be specified for other regions and countries (e.g., blue for South Korea).

[0060] The traffic sign layer describes the possible types of traffic signs associated with a unit. Traffic sign layer Ω TS It can be defined in equation (5) as:

[0061] Ω TS ={TS1, TS2, ..., TS n ,NTS} (5)

[0062] Where n is the number of possible traffic signs, and NTS represents non-traffic signs. If road 120 is a highway, possible traffic signs may include temporary traffic control signs, regulatory signs, directional signs, driver service signs, and recreational signs.

[0063] The pavement marking layer describes the possible types of pavement markings associated with a unit. Pavement marking layer Ω PM It can be restricted in equation (6) as follows:

[0064] Ω PM={PM1, PM2, ..., PM m , NPM} (6)

[0065] Where m represents the number of possible road markings, and NPM represents non-road markings. Possible road markings may include highway entrance and exit markings, work zone markings, and HOV lane markings.

[0066] The lane type layer describes the possible lane types associated with lane number assignments. Lane Type Layer Ω LT It can be restricted in equation (6) as follows:

[0067] Ω LT ={L t L d ,L a L O} (7)

[0068] Where L t Indicates a straight-ahead lane, L d Indicates a deceleration lane, L a Indicates the acceleration lane, and L O Indicates other lane types.

[0069] The following describes an example operation of the fusion mesh module 110 and layer module 112 generating a mesh-based road model 322 based on input data. At function 310, input processing module 206 verifies the input data, including mesh 302, LiDAR data 304, visual data 306, vehicle status data 308, and map 106. Input processing module 206 can verify that the inputs used to model the boundaries provide a reasonable shape for the boundaries. For example, on a highway, a radius of curvature below 300m is considered low reasonableness. Input processing module 206 can also compare various inputs to each other to verify whether the inputs are consistent or whether they include possible outliers. For example, sensor inputs where lanes intersect each other are unreasonable. As another example, on a Class I limited access highway, stop signs are unreasonable unless a construction zone exists.

[0070] At function 312, input processing module 206 extracts quality values ​​for each type of input data. For example, input processing module 206 can extract the quality value for the current moment. This document references Figure 4 and Figure 6 Describes the extraction of quality values ​​assigned to lane numbers.

[0071] Visual data 306 can directly provide information for the lane marking type layer. Input processing module 206 can determine the quality value for each type of lane marking based on the sensor confidence level used for classification. Therefore, the quality value for each cell with coordinates (x, y) is given by equations (8) and (9):

[0072] m MT，视觉 (α, (x, y)) = p MT (α, (x, y)) (8)

[0073] m MC，视觉 (β, (x, y)) = p MC (β, (x, y)) (9)

[0074] Where p indicates the classification confidence level given by the sensor (e.g., a camera sensor), α∈Ω MT And β∈Ω MC .

[0075] The input processing module 206 can also use visual data 306 to extract quality values ​​associated with each possible traffic sign, road marking, and lane type. Visual data 306 can indicate the type of traffic sign and road marking. Similarly, visual data 306 can provide the lane type directly or indirectly. The input processing module 206 can determine the quality value m for each possible traffic sign, road marking, and lane type based on the sensor confidence p used for classification. Therefore, the quality value for each unit can be given by equations (10) to (12):

[0076] m TS,视觉 (γ, (x, y)) = p TS (γ, (x, y)) (10)

[0077] m PM，视觉 (χ, (x, y)) = p PM (χ, (x, y)) (11)

[0078] m LT,视觉 (ζ,(x,y))=p LT (ζ,(x,y)) (12)

[0079] where γ∈Ω TS , χ∈Ω PM And ζ∈Ω LT .

[0080] In some implementations, the input processing module 206 can simplify the grid-based road model 322 and extract only the two highest probability quality values ​​for the traffic sign layer and the pavement marking layer from the visual data 306. In these implementations, the probabilities of other cell categories are extracted into the "Other" category. Therefore, the input processing module 206 can extract quality values ​​for three categories (or any other number of categories) for the traffic sign layer and the pavement marking layer.

[0081] As described above, the input from mesh 302 includes mesh size, mesh resolution, and cell center coordinates. Each cell of mesh 302 indicates the quality value of each layer hypothesis 324 formed for the FOD layer. To distinguish it from the FOD layer of the mesh-based road model 322, the FOD layer of mesh 302 can be defined in equation (13) as follows:

[0082] Θ = {F, SO, DO} (13)

[0083] Where F indicates free space, SO indicates statically occupied cells, and DO indicates dynamically occupied cells. Each cell in mesh 302 also indicates velocity V, which the input processing module 206 can use for quality value extraction.

[0084] To reduce complexity, the input processing module 206 can use a pignistic transformation to convert the quality values ​​provided by the mesh 302 into probability values. The input processing module 206 then assigns each available cell the following probability value: p F (·), which indicates the probability that the unit becomes free space; p SO (·), which indicates the probability that the indicator unit is statically occupied; and p DO (·), which indicates the probability that the unit is dynamically occupied.

[0085] The input processing module 206 can design a mapping function based on the mesh 302 and its cell values ​​using the following rules: Cells with low speed and high static occupancy probability are more likely to be obstacles. Cells with high speed and high static occupancy probability are more likely to be lane centers. If nearby cells (e.g., 0.5W) L W L If a lane width has high speed and high dynamic occupancy probability, then a cell with a high free space probability is more likely to be a lane boundary.

[0086] Consider the input processing module 206 will p l (V) is defined as a function of velocity, and if the velocity is large, then p l (V) has a relatively small value. The input processing module 206 can also process p... h (V) is defined as a function of velocity, and if the velocity is small, then p h(V) has a relatively small value. The function p l (V) can be any distribution that is a function of velocity, has a peak at zero, and decays monotonically. For example, the input processing module 206 can select this function p. l (V; α, β) represents a gamma distribution with shape parameter α and scaling parameter β. The input processing module 206 can select this function p. h (V) is a Gaussian distribution representing the average road speed (e.g., 65 km / h) with a default standard deviation (e.g., 8 km / h). The input processing module can adjust p based on specific scenarios. l (V) and p h The parameters of (V). For different information sources, the input processing module 206 can use different parameters for the attenuation function based on how much information should be retained during quality value extraction and fusion.

[0087] Based on the mapping rules for mesh 302, the input processing module 206 can extract the mass value of the obstacle unit for the unit category layer based on equation (14):

[0088] m C,DG (B a (x, y))=p l (V(x, y); α, β)×p SO (x, y) (14)

[0089] Where x is the center coordinate of the evaluation unit, V(x, y) represents the velocity of the evaluation unit, and α and β are adjusted based on the performance of the data.

[0090] The input processing module 206 can use equations (15) and (16) respectively to extract the quality values ​​for the lane center and lane boundary:

[0091] m C，DG (L c (x, y))=p h (V(x, y); α, β)×p DO (x, y) (15)

[0092]

[0093] Where max(a, b) represents the maximum value of a and b. Δ is the total shift distance, and θ l and θ r These are the angles of displacement to the left and right, respectively.

[0094] Similar to the processing of visual data 306, the input processing module 206 uses the mass value m based on the obstacle layer hypothesis.C，DG (B a (x, y)), the assumed mass value m of the lane center layer C，DG (L c (x, y)), the mass value m of the lane boundary layer assumption C，DG (L b The sum of (x, y) To assign quality values ​​to other cell values. Input processing module 206 can use equation (17) to determine the quality value for the cell designated as another cell:

[0095]

[0096] Generally, the map 106 is validated before being processed by the input processing module 206. The confidence level of the map accuracy is represented by p. 地图 (·). Map 106 can provide information for each FOD input into different FODs. Input processing module 206 can use a similar equation to obtain the cell category m. C，地图 (·) and lane assignment m LA，地图 (·). For tag type m MT,地图 (·), Mark color m MC,地图 (·), Traffic signs m TS，地图 (·), Road markings m PM,地图 (·), and lane type m LT，地图 (·), the input processing module 206 can assign quality values ​​based on the confidence level associated with the map 106.

[0097] Inputs from visual data 306 or lidar data 304 include polylines or other parametric or non-parametric models that indicate lane markings and the edges of road 120. Visual data 306 and lidar data 304 may also indicate semantic information for lane 122. Input processing module 206 can use a mapping function to map visual data 306 or lidar data 304 to cell confidence levels to become specific categories within a cell category layer. A mapping function provides that cells intersecting with lane marking polylines (e.g., y = φ(x)) are more likely to be lane boundaries L. b Another mapping function provides a distance of 0.5W from the solid lane marking unit. L The cell that intersects with the lane markings and faces the vehicle 102 is more likely to be the lane center cell L. c The input processing module 206 can process W L The lane width can be assigned as a default width for a specific area (e.g., 3.66 meters in the United States) or based on visual data 306. LFor dashed lane markings, the input processing module 206 can process the signal at a certain distance (e.g., 0.5W). L The units facing the vehicle 102 and the units away from the vehicle 102 are both assigned to be lane center units.

[0098] Based on these rules, the input processing module 206 can limit the credibility quality mapping function in equation (18) to:

[0099]

[0100] Where the standard deviation represents half the width of the lane marking (e.g., σ = 0.075 m), (x, y) are the coordinates of the element. Lane markings The confidence level of the lane marker. Typically provided by visual data 306.

[0101] If the lane marking is a solid lane marking on the right, where the distance Δ is less than W L Then the input processing module 206 can move the measurement point to the left as evidence of the lane center provided in equation (19):

[0102]

[0103] in and

[0104] If the lane marking is a solid lane marking on the left, the input processing module 206 can use equation (20):

[0105]

[0106] in and

[0107] If two sets of lane markings are available, the input processing module 206 can use equation (21):

[0108]

[0109] Visual data 306 can also provide information on edge and obstacle types. By using edge and obstacle polylines, input processing module 206 can similarly obtain the quality value for obstacles using equation (22):

[0110]

[0111] Input processing module 206 can represent the sum of quality values ​​for lane center category, lane boundary category, and obstacle category based on visual data 306. The mass value of a cell belonging to another class of states is expressed by equation (23):

[0112]

[0113] At function 314, evidence fusion module 208 updates and fuses the quality values ​​for each cell of each layer. Evidence fusion module 208 can use the Dempster-Schafer fusion rule to fuse various data about cells of a layer. The data may include data from past measurements and data from current sensor measurements. For past measurements, previous quality values ​​are propagated via a decay function representing the decay of information over time. If both types of information are available, evidence fusion module 208 performs fusion to update the quality values.

[0114] The evidence fusion module 208 can also use the Dempster-Shaffer fusion rule to fuse information from various data hypotheses. The evidence fusion module 208 can first use the Dempster-Shaffer method to obtain a fused quality value for a specific hypothesis based on quality values ​​for different sources. The evidence fusion module 208 can recursively calculate the quality value. In this way, the evidence fusion module 208 uses equation (24) to apply the previous quality values ​​and decay function of a specific layer hypothesis:

[0115]

[0116] Wherein, the subscript FT∈{C, LA, LT, TS, PM, MT, MC} represents the fusion type, and H is the layer hypothesis corresponding to different fusion types.

[0117] The evidence fusion module 208 can use equations (25) to (27) to perform Dempster-Schaffer fusion of two information sources and obtain the joint quality value m. 1，2 :

[0118]

[0119]

[0120] in

[0121]

[0122] Furthermore, the value (1-K) represents the normalization coefficient, K represents the conflict between pieces of evidence, m1(·) and m2(·) represent the quality values ​​corresponding to specific hypotheses for the first and second pieces of evidence, respectively, and B and C are the layer hypotheses determined by FOD. If a single source of information is available, the evidence fusion module 208 does not need to perform evidence fusion.

[0123] At function 316, evidence fusion module 208 uses the fused quality value to calculate the probability for each cell for each layer.

[0124] At function 318, output processing module 210 calculates the confidence parameter and rationality parameter for each unit of each layer. The confidence parameter bel(H) and rationality parameter pl(H) are given by equations (28) and (29), respectively:

[0125] bel(H)=∑ B|B∈H m(B) (28)

[0126]

[0127] For a specific hypothesis, the probability is calculated via a pignistic transformation. At function 320, output processing module 210 verifies the confidence parameter, the reasonableness parameter, and the probability. For example, output processing module 210 can verify that the lane width is within applicable standards (e.g., area policies) and is reasonable. Output processing module 210 can also verify that lanes do not intersect each other, have similar widths, conform to divergence or merging signs, and are concentric. Function 320 can be similar to the verification checks performed at function 310.

[0128] Layer module 112 can output layer hypothesis 324 as a structured array. Each structure within the array defines a corresponding FOD layer. Layer module 112 can provide components in the structure in the following ways:

[0129]

[0130] Layer module 112 can also provide grid information and FOD layer information as a structure array. Layer module 112 can use a unique "cell_index" for each cell of grid 302 to limit "Grid_Info" to:

[0131]

[0132] "Layer_Info" typically depends on "Layer_Type", which is an enumeration type given by equation (30):

[0133] Layer_Type∈{Standalone_Layer,Reduced_Layer,Shared_Layer}({Standalone_Layer,Reduced_Layer,Shared_Layer}) (30)

[0135] For "Standalone_Layer", layer module 112 can limit "Layer_Info" to:

[0136]

[0137] For "Reduced_Layer", layer module 112 can limit "Layer_Info" to:

[0138]

[0139] For “Shared_Layer”, layer module 112 can limit “Layer_Info” to:

[0140]

[0141] For both Standalone_Layer and Shared_Layer, the road perception system 108 can assign cells with different resolutions. For example, the road perception system 108 can use a higher resolution for cells with lane marking evidence and a lower resolution for other cells.

[0142] Layer module 112 can use only units specific to Standalone_Layer to represent the confidence, rationality, and probability parameters for units of a particular unit category. Alternatively, layer module 112 can use structures to connect different road elements (e.g., L...). b , ∈ Ω MT , and ∈ Ω MC The credibility, rationality, and probability parameters are used for the cells of the Shared_Layer. Shared_Layer cells can be connected because lane markings are associated with lane boundary cells (opposite to the shoulder or lane center), and lane marking information and lane boundary information are often used together.

[0143] Layer module 112 can be used for ∈ ∈ Ω LTSpecial cases. For layer module 112, using Standalone_Layer to represent the lane type of each unit may be inefficient because lane types are associated with existing lane number assignments, and the number of lane types is relatively small. Therefore, layer module 112 can define a dictionary associated with lane assignments. Table 1 provides an example of a dictionary defined by layer module 112, where the "key" is formed by "lane assignment number" + "lane type", and the "value" is a probability parameter value for a lane to be a specific lane type.

[0144] Table 1

[0145] <![CDATA[1_L t ]]> <![CDATA[p(1,L t )]]> <![CDATA[-1_L t ]]> <![CDATA[p(-1,L t )]]> <![CDATA[1_L d ]]> <![CDATA[p(1,L d )]]> <![CDATA[-1_L d ]]> <![CDATA[p(-1,L d )]]> <![CDATA[1_L a ]]> <![CDATA[p(1,L a )]]> <![CDATA[-1_L a ]]> <![CDATA[p(-1,L a )]]> <![CDATA[0_L t ]]> <![CDATA[p(0,L t )]]> <![CDATA[2_L t ]]> <![CDATA[p(2,L t )]]> <![CDATA[0_L d ]]> <![CDATA[p(0,L d )]]> <![CDATA[2_L d ]]> <![CDATA[p(2,L d )]]> <![CDATA[0_L a ]]> <![CDATA[p(0,L a )]]> <![CDATA[2_L a ]]> <![CDATA[p(2,L a )]]>

[0146] Table 2 summarizes the layer representations of different outputs of layer module 112 for the mesh-based road model 322.

[0147]

[0148] Figure 4 An example static mesh 400 is shown, generated from a mesh-based road model with multiple layers, for which the road perception system 108 can determine lane boundary cells 404. The road perception system 108 can use the static mesh 400 to represent the properties of the road (e.g., road 120) on which the vehicle 102 is traveling. The static mesh 400 may include or contain references Figure 3 The described mesh 302. The static mesh 400 comprises multiple elements 402. Figure 4 In this implementation, cell 402 is represented as having a uniform resolution. In other implementations, cell 402 may have a higher resolution in the region of interest (e.g., lane boundaries, lane markings, lane center).

[0149] The road perception system 108 can extract quality values ​​for static grid 400. As previously discussed, the input processing module 206 can determine quality values ​​for lane boundaries 404 using equation (16). The road perception system 108 can treat each cell 402 that is determined to possibly include lane boundaries 404 as a grayscale image.

[0150] Input processing module 206 can apply a Canny edge detector or other edge or object detector methods to the grayscale image to extract the edges of lane boundary 404. The Canny edge detector generally includes the following operations: noise reduction, gradient calculation, non-maximum suppression, dual-threshold analysis, and hysteresis edge tracking. Because the Canny edge detector may be sensitive to image noise, input processing module 206 can perform noise reduction. Gradient calculation involves determining the intensity variation in different directions (e.g., the probability for lane boundary 404). Input processing module 206 can apply non-maximum suppression to provide thin edges to lane boundary 404, thus preserving cells with maximum values ​​in the edge directions. Input processing module 206 can use dual-thresholding to identify different types of cells 402, specifically cells that strongly contribute to edges, cells that weakly contribute to edges, and cells that are unrelated to edges. Hysteresis edge detection tracking is used to identify the true edges of lane boundary 404 as the output.

[0151] The input processing module 206 can then perform a shape closure operation to fill the small gaps between different elements of the lane boundary 404. The shape closure operation may include an expansion operation preceding the erosion operation.

[0152] By comparing the position (v0) of vehicle 102 with edge information for the row closest to vehicle 102, input processing module 206 can define the left lane boundary C with the left adjacent lane. la,l (0) The left lane boundary of the self-lane C e,l (0) The right lane boundary of the self-lane C e,r (0), and the right lane boundary C of the adjacent lane on the right. ra,r (0) The corresponding unit. Specifically, the input processing module 206 can process C e,l (0) is limited to the range [v0-W L The edge within [v0] has a maximum value in cell 402, where W L This indicates the lane width. The input processing module 206 can process C... e,r (0) is limited to the range [v0, v0+W] L The cell 402 with the maximum value at the edge within the []. The input processing module 206 can process C la，l (0) is limited to the range [v0-2W] L v0-W L The edge within the [ ] contains the unit 402 with the largest mass. The input processing module 206 can process C ra，r (0) is limited to the range [v0+W L v0+2W LThe edge of the cell containing the largest mass is cell 402.

[0153] These four units are defined as initial units. The input processing module 206 can use these initial units to initialize four lane boundary unit lists: C e,l ={C e，l (0)}、C e，r ={C e，r (0)}、C la,l ={C la，l (0)} and C ra,r ={C ra,r (0)}. By following the edge lines, the input processing module 206 can iteratively expand the list of cells of the lane boundary 404 by appending line by line, such as Figure 4 As shown. For each cell, the input processing module 206 can represent the cell using the center coordinates.

[0154] The input processing module 206 can then use the lane boundary cell list as evidence to assign quality values ​​to different cells 402. Left-side self-lane list C e,l It can be along the curvature direction of lane boundary 404 at a distance Δ = δ, 2δ, ..., nδ (where Δ ≤ W) L ) Offset to the right. For relative to element C e，l (i) For unit 402 with distance Δ, input processing module 206 uses equation (31) to calculate the mass value:

[0155]

[0156] in For unit C e The coordinates of (i), and p lb (·) is the mass value of unit 402, which serves as lane boundary 404.

[0157] For those with Δ≤0.5W L The ambiguity of the self-lane relative to the left adjacent lane is given by equation (32):

[0158] m e , ((-1, 0), (x, y)) = 1 - m e，l (0, (x, y)) (32)

[0159] Similarly, the cell list C of the left lane boundary of the self-lane. e,l Alternatively, distance Δ≤W L Shift to the left. Input processing module 206 can extract the quality value using equation (33):

[0160]

[0161] in For unit C e，l The coordinates of (i) and p lb (·) is the mass value of unit 402, which serves as lane boundary 404.

[0162] For those with Δ≤0.5W L The ambiguity of the self-lane relative to the left adjacent lane is given by equation (34):

[0163] m e , ((-1, 0), (x, y)) = 1 - m e，l (-1, (x, y)) (34)

[0164] Cell list C of the right lane boundary of the self-lane e，r It can be offset to the right by distances Δ = δ, 2δ, ..., nδ. For relative to element C... e，r (i) The mass value of unit 402 with distance Δ is calculated by input processing module 206 using equation (35):

[0165]

[0166] in For unit C e，r The coordinates of (i) and p lb (·) is the mass value of unit 402, which serves as lane boundary 404.

[0167] For those with Δ≤0.5W L The ambiguity of the self-lane relative to the right-adjacent lane is given by equation (36):

[0168] m e，r ((0, 1), (x, y)) = 1 - m e，r (0, (x, y)) (36)

[0169] List of units for the right-hand self-lane C e，r It can be offset to the right by distances Δ = δ, 2δ, ..., nδ. For relative to element C... e，r (i) The mass value of unit 402 with distance Δ is calculated by input processing module 206 using equation (37):

[0170]

[0171] in For unit C e,r The coordinates of (i) and p lb (·) is the mass value of unit 402, which serves as lane boundary 404.

[0172] For those with Δ≤0.5W L The ambiguity of the self-lane relative to the right-adjacent lane is given by equation (38):

[0173] m e,r ((0, 1), (x, y)) = 1 - m e,r (1, (x, y)) (38)

[0174] Similar to the self-lane boundary cell list, the input processing module 206 can use the left adjacent lane boundary cell list C lal As evidence, the quality values ​​-1 and 2 were assigned for lane number assignment. Right adjacent lane boundary unit C ra，r It can be used as evidence to assign quality values ​​1 and 2 for lane number assignment.

[0175] List of cells for the left adjacent lane boundary C la，l It can be offset to the right by distances Δ = δ, 2δ, ..., nδ. For relative to element C... la，l (i) The mass value of unit 402 with distance Δ is calculated by input processing module 206 using equation (39):

[0176]

[0177] in For unit C la，l The coordinates of (i) and p lb (·) is the mass value of unit 402, which serves as lane boundary 404.

[0178] For those with Δ≤0.5W L The ambiguity of other lanes relative to their left adjacent lanes is given by equation (40):

[0179] m la，l ((-1, 2), (x, y)) = 1 - m la，l (-1, (x, y)) (40)

[0180] Cell list C of the right-side adjacent lane boundary ra，r It can be offset to the left by distances Δ = δ, 2δ, ..., nδ. For relative to element C... ra，r (i) The mass value of unit 402 with distance Δ is calculated by input processing module 206 using equation (41):

[0181]

[0182] in For unit C ra,rThe coordinates of (i) and p lb (·) is the mass value of unit 402, which serves as lane boundary 404.

[0183] For those with Δ≤0.5W L The ambiguity of other lanes relative to the right-adjacent lane is given by equation (42):

[0184] m ra，r ((1,2),(x,y))=1-m ra，r (1, (x, y)) (42)

[0185] Because lane assignment numbers can be inferred from multiple sets of evidence (lane markings), input processing module 206 can use the Dempster-Schafer fusion rule to fuse the quality values ​​for static mesh 400. Input processing module 206 can normalize the quality value for each cell 402. Input processing module 206 can use equations 43(a) to 43(g) to determine the quality value of cell 402:

[0186]

[0187]

[0188]

[0189] m LA,DG ((0, 1), (x, y)) = m e，r ((0, 1), (x, y)) (43d)

[0190] m LA，DG ((-1, 0), (x, y)) = m e，l ((-1, 0), (x, y)) (43e)

[0191] m LA，DG ((-1, 2), (x, y)) = m la，l ((-1, 2), (x, y)) (43f)

[0192] m LA，DG ((1,2),(x,y))=m ra,r ((1,2),(x,y)) (43g)

[0193] The input processing module 206 can assign other units to a "2" with a quality value of 1. The above equation assumes C... e，l C e，r C la，l C ra，r It is available for road 120. If there is no C... la，lThen the input processing module 206 can reduce FOD to {0, 1, 2}. Similarly, if there is no C ra，r Then the input processing module 206 can reduce FOD to {-1, 0, 2}. If C la，l and C ra，r If neither is available, the input processing module 206 can reduce the FOD to {0, 2}. The static grid 400 generally does not provide information regarding lane marking type, lane marking color, traffic signs, road markings, or lane type.

[0194] Figure 5A and Figure 5B The input processing modules for a grid-based road model with multiple layers are shown, which can be used to offset input evidence for determining example lane sets 500 and 550 for assumptions about other layers. For example, input processing module 206 can offset lane markings around half the width of the lane to provide evidence for the lane centering assumption. Thus, depending on the curvature of lane 122, input processing module 206 can routinely offset the quality values ​​concentrically to the left or right.

[0195] Figure 5 illustrates the lateral offset that can be used by the input processing module 206. The lane set 500 includes lane markers 502, 504, 506, 508, 510, and 512, separated by a distance 518 (δ). The input processing module 206 can establish a coordinate system identical to the one used for the vehicle 102. For example, the x-axis 520 points forward of the vehicle 102, and the y-axis 522 points to the left.

[0196] The road perception system 108 can determine whether the curvature of road 120 or lane 122 is greater than a curvature threshold. If the curvature of road 120 or lane 122 is relatively small or less than the curvature threshold, a mass value from the origin 514 (e.g., a cell) can be directly offset along the y-axis 522 to provide a mass value or evidence for the offset point 516. The coordinates of the origin 514 are represented as follows: The input processing module 206 can represent the coordinates of the offset point 516 as follows: Additionally, the offset step size is denoted as δ, and the total offset distance of 524 is denoted as Δ (e.g., as...). Figure 5A The dashed line 526 indicates the possible offset direction. Based on the coordinate system, the input processing module 206 uses equation (44) to obtain the coordinates of the offset point 516 from the origin 514:

[0197]

[0198] Use "+" when the offset direction is to the left and "-" when the offset direction is to the right.

[0199] The origin 514 and the offset point 516 have inherent uncertainties. To describe these uncertainties, the input processing module 206 can use a Gaussian distribution. The uncertainty of a cell with coordinates (x, y) can be calculated by the input processing module 206 using equation (47):

[0200]

[0201] Where σ is the standard deviation of y.

[0202] Figure 5B The concentric offset that can be used by the input processing module 206 is shown. The lane set 550 includes lane markers 552, 554, 556, 558, 560 and 562, which are separated by a distance of 518 (δ).

[0203] If the curvature of road 120 or lane 122 is relatively large, the mass value from origin 514 can be offset along the curvature direction 566 to provide evidence for other units. Input processing module 206 can use three points to determine the local curvature and normal. In other implementations, input processing module 206 can use other methods to identify the local curvature of lane set 550. Based on the coordinate system, the coordinates of point 516 offset from origin 514 are obtained using equation (46).

[0204]

[0205] Here, angle θ is defined as the angle between the x-axis (520°) and the offset direction. When the offset direction is to the right, the angle is 180 < θ ≤ 360°.

[0206] To describe the uncertainty of the offset point 516, the input processing module 206 can use a Gaussian distribution. The uncertainty of the cell with coordinates (x, y) can be calculated by the input processing module 206 using equation (47).

[0207]

[0208] Where σ 2 It is the variance of the Gaussian distribution.

[0209] When offsetting the origin 514 to provide evidence for other elements, it is possible that multiple origins are available. The input processing module 206 can determine the quality value of the element based on the evidence from multiple origins, using the average of these points.

[0210] Figure 6An example lane set 600 is shown. An input processing module 206 of a grid-based road model with multiple layers can use the example lane set 600 to determine quality values ​​for lane number assignments. Specifically, the input processing module 206 can use lane marking data (e.g., from grid 302, visual data 306, or LiDAR data 304) and vehicle status data 308 to determine the lane assignment layer Ω. LA The quality value.

[0211] The input processing module 206 can compare the position of the vehicle 102 with visual data 306 (including lane marking polylines) to obtain relevant polylines. For example, the polylines may include left-side lane markings 602 (e.g., φ). e，l ,), the broken line of the right-hand lane marker 604 (e.g., φ) e，r ), the broken line of the adjacent lane marking 606 on the left (e.g., φ) la，l ), and the broken line of the adjacent lane marking 608 on the right (e.g., φ ra，r The input processing module 206 can assume the polyline φ of the adjacent lane marking on the right. la，r The broken line is the same as the broken line of lane marker 602 on the left. Assume the broken line φ of the adjacent lane marker on the right is... ra，l The broken line is the same as the one marked 604 on the right-hand side of the self-lane. The input processing module 206 can also assume that the broken line is a function of y = φ(x).

[0212] The input processing module 206 can use self-lane markers as evidence to assign quality values ​​to different units. (Left self-lane marker point) This can be used to provide evidence for elements moving to the right along the curvature direction. For elements with coordinates (x, y) (the element is relative to...) With distance (Δ≤W) L The input processing module 206 can use equation (48) to calculate the quality value:

[0213]

[0214] in Furthermore, δ is an adjustable parameter. For lane number assignment, the standard deviation σ is set to the lane width W. L The value of . Additionally, the mapping function D((x, y), It could be the peak value at Any function at a given location, and decreases to (x, y) as the distance increases.

[0215] For having relative The distance Δ≤0.5W L The ambiguity of the self-lane relative to the left adjacent lane is given by equation (49):

[0216] m e,l ((-1,0), (x,y))=1-m e，l (0, (x, y)) (49)

[0217] Similarly, the input processing module 206 can use the left-side self-lane marker. To provide evidence for elements moving to the left along the curvature direction. For elements with coordinates (x, y) (this element is relative to...) With distance (Δ≤W) L The input processing module 206 can use equation (50) to calculate the quality value:

[0218]

[0219] in

[0220] For having relative The distance Δ≤0.5W L The ambiguity of the self-lane relative to the left adjacent lane is given by equation (51):

[0221] m e，l ((-1, 0), (x, y)) = 1 - m e，l (-1, (x, y)) (51)

[0222] Input processing module 206 can use right-side self-lane markers. To provide evidence for elements moving to the left along the curvature direction. For elements with coordinates (x, y) (this element is relative to...) With distance (Δ≤W) L The input processing module 206 can use equation (52) to calculate the quality value:

[0223]

[0224] in

[0225] For those with Δ≤0.5W L The ambiguity of the self-lane relative to the right-adjacent lane is given by equation (53):

[0226] m e,r ((1,0),(x,y))=1-m e，r (0, (x, y)) (53)

[0227] Input processing module 206 can use right-side self-lane markers. To provide evidence for elements moving to the right along the curvature direction. For elements with coordinates (x, y) (this element is relative to...) With distance (Δ≤W) L The input processing module 206 can use equation (54) to calculate the quality value:

[0228]

[0229] in

[0230] For those with Δ≤0.5W L The ambiguity of the self-lane relative to the right-adjacent lane is given by equation (55):

[0231] m e,r ((1,0), (x,y))=1-m e,r (1, (x, y)) (55)

[0232] Similar to self-marking lane markers, the input processing module 206 can use the marker points of the adjacent lane on the left. As evidence, quality values ​​of -1 and 2 are assigned for lane number assignment. The input processing module 206 can also use the marker points of the adjacent lane on the right. As evidence, assign quality values ​​1 and 2 for lane assignment.

[0233] Input processing module 206 can use the left adjacent lane point This provides evidence for elements moving to the right along the curvature direction. For elements with coordinates (x, y) (this element is relative to...) With distance (Δ≤W) L The input processing module 206 can use equation (56) to calculate the quality value:

[0234]

[0235] in

[0236] For those with Δ≤0.5W L The ambiguity of other lanes relative to their left adjacent lanes is given by equation (57):

[0237] m la，l ((-1, 2), (x, y)) = 1 - m la，l (-1, (x, y)) (57)

[0238] Input processing module 206 can use the right adjacent lane point To provide evidence for elements moving to the left along the curvature direction. For elements with coordinates (x, y) (this element is relative to...) With distance (Δ≤W) L The input processing module 206 can use equation (58) to calculate the quality value:

[0239]

[0240] in

[0241] For those with Δ≤0.5W L The ambiguity of other lanes relative to the right-adjacent lane is given by equation (59):

[0242] m ra，r ((1,2),(x,y))=1-m ra，r (1, (x, y)) (59)

[0243] Because lane assignment numbers can be inferred from multiple pieces of evidence (lane markings), the evidence fusion module 208 can use the Dempster-Schafer fusion rule to combine quality values ​​for visual data 306. Before combining the quality values, the evidence fusion module 208 can normalize the quality values. Node quality values ​​can be calculated using equations (60a) to (60g):

[0244]

[0245]

[0246]

[0247] m LA，视觉 ((0, 1), (x, y)) = m e，r ((0, 1), (x, y)) (60d)

[0248] m LA，视觉 ((-1, 0), (x, y)) = m e,l ((-1, 0), (x, y)) (60e)

[0249] m LA,视觉 ((-1,2),(x,y))=m la,l ((-1,2),(x,y)) (60f)

[0250] m LA,视觉 ((1,2),(x,y))=m ra,r ((1,2),(x,y)) (60g)

[0251] The input processing module 206 can assign a unit that has no evidence to lane number category 2 with a quality value of 1.

[0252] Example Method

[0253] Figure 7 A sample flowchart depicts an example process performed by a road perception system configured to generate a grid-based road model with multiple layers. Flowchart 700 is shown as multiple sets of operations (or actions) performed, but is not limited to the order or combination of operations shown herein. Furthermore, any one or more of the operations may be repeated, combined, or reorganized to provide other methods. References may be made in the various sections discussed below. Figures 1 to 6 The road perception system 108 and the entities detailed therein are referred to by way of example only. The technology is not limited to being performed by one or more entities.

[0254] At 702, the road grid representation is generated by the vehicle's road perception system. The grid representation comprises multiple cells. For example, the road perception system 108 of vehicle 102 generates a grid 302 or a static grid 400 for road 120. The static grid 400 comprises multiple cells 402.

[0255] At 704, a road model of the road is generated by a road perception system. The road model is based on data from multiple information sources. The road model includes at least two distinct layers, which represent different corresponding road attributes of multiple cells in a grid representation of the road. For each cell in each of the multiple cells in each of the distinct layers, the at least two distinct layers include at least one corresponding layer hypothesis indicating a potential attribute for that cell. For example, road perception system 108 generates a grid-based road model 322 based on data from multiple information sources. The multiple information sources may include grid 302, LiDAR data 304, visual data 306, vehicle status data 308, and map 106. The grid-based road model 322 includes at least two distinct FOD layers, which represent different corresponding attributes of road 120. For example, the FOD layers may include a cell category layer, a lane number assignment layer, a lane marking type layer, a lane marking color layer, a traffic sign layer, a pavement marking layer, and a lane type layer. For each cell 402 of grid 302 in each layer, the FOD layer includes at least one layer hypothesis 324. Layer hypothesis 324 indicates the potential attribute state for each of the multiple cells.

[0256] At 706, a corresponding quality value is determined for each of at least one corresponding layer hypothesis. The corresponding quality value indicates the confidence level associated with the data that contributes to at least one corresponding layer hypothesis. For example, the road perception system 108 or the input processing module 206 may determine a quality value associated with layer hypothesis 324 for each cell in the FOD layer.

[0257] At 708, a confidence parameter and a plausibility parameter are determined for each of the at least one layer hypothesis. The confidence parameter and plausibility parameter are determined using at least one corresponding quality value associated with the layer hypothesis. The confidence parameter indicates the confidence level of the potential attribute state for that unit. The plausibility parameter indicates the likelihood that the potential attribute state is accurate for that unit. For example, the road perception system 108 or the evidence fusion module 208 can use the quality values ​​associated with the layer hypotheses to determine the confidence parameter and plausibility parameter associated with each layer hypothesis 324.

[0258] At 710, it is determined whether the confidence parameter and reasonableness parameter associated with at least one layer hypothesis meet the corresponding thresholds. For example, the road perception system 108 can determine whether the confidence parameter and reasonableness parameter associated with the layer hypothesis are greater than or less than the corresponding thresholds.

[0259] At point 712, in response to determining that the confidence parameter and reasonableness parameter of at least one layer hypothesis meet the corresponding thresholds, the autonomous driving system or the driver assistance system can use at least one layer hypothesis as input to operate the vehicle on the road. For example, in response to determining that the confidence parameter and reasonableness parameter are greater than the corresponding thresholds, the vehicle 102 can be operated by the autonomous driving system 118 or the driver assistance system 116. The autonomous driving system 118 or the driver assistance system 116 can use at least one layer hypothesis as input to operate the vehicle 102 on the road 120. Alternatively, in response to determining that at least one of the confidence parameter or reasonableness parameter is less than the corresponding threshold, the road perception system 108 can interrupt the operation of the vehicle 102 using the autonomous driving system 118 or the driver assistance system 116 and switch the operation of the vehicle 102 to driver control.

[0260] Example

[0261] Examples are provided in the following sections.

[0262] Example 1: A method comprising: generating a grid representation of a road by a road perception system of a vehicle, the grid representation comprising a plurality of cells; generating a road model for the road by the road perception system and based on data from a plurality of information sources, the road model comprising at least two distinct layers, the at least two distinct layers representing different corresponding road attributes of the plurality of cells in the grid representation of the road, the at least two distinct layers comprising at least one corresponding layer hypothesis for each of the plurality of cells in each of the distinct layers, the at least one corresponding layer hypothesis indicating a potential attribute state of the cell; and determining a corresponding quality value associated with each of the at least one corresponding layer hypothesis, the corresponding quality value indicating a corresponding attribute state of the cell. The confidence level associated with the data contributing to the at least one corresponding layer hypothesis; by the road perception system and using corresponding quality values, determining a confidence parameter and a plausibility parameter associated with each of the at least one corresponding layer hypothesis, the confidence parameter indicating the confidence level for the potential attribute state of the unit, the plausibility parameter indicating the probability that the potential attribute state is accurate for the unit; determining whether the confidence parameter and plausibility parameter associated with the at least one layer hypothesis meet corresponding thresholds; and in response to determining that the confidence parameter and plausibility parameter associated with the at least one layer hypothesis meet the corresponding thresholds, using the at least one layer hypothesis as input to an autonomous driving system or an assisted driving system that operates a vehicle on the road.

[0263] Example 2: The method of Example 1, wherein the plurality of information sources include at least one of the following: a lidar system, a vision system, a camera system, a vehicle status system, a position sensor, a steering sensor, a yaw rate sensor, or a map.

[0264] Example 3: A method from any of the preceding examples, wherein the corresponding quality value is recursively updated based on data from multiple information sources.

[0265] Example 4: The method of any of the preceding examples, wherein determining the corresponding quality value associated with each of at least one corresponding layer hypothesis includes: fusing multiple quality values ​​using the Dempster-Schafer fusion rule, the multiple quality values ​​contributing to each of at least one corresponding layer hypothesis.

[0266] Example 5: A method from any of the preceding examples, where the multiple cells represented by the grid have a non-uniform resolution.

[0267] Example 6: The method of any of the preceding examples, wherein at least two distinct layers are defined by a corresponding identification framework (FOD), the corresponding FOD identifying a corresponding set of possible layer hypotheses.

[0268] Example 7: The method of Example 6, wherein at least two distinct layers include at least two of the following: a cell category layer, a lane number layer, a lane marking type layer, a lane marking color layer, a traffic sign layer, a road marking layer, or a lane type layer.

[0269] Example 8: The method of Example 7, wherein: the unit category layer includes at least two of lane boundary, lane center, obstacle, or other values; the lane number layer includes at least two of left adjacent lane, own lane, right adjacent lane, or other lane; the lane marking type layer includes at least two of solid line lane marking, double line lane marking, dashed line lane marking, thick lane marking, or other lane marking types; the lane marking color layer includes at least two of yellow, white, or other colors; and the lane type layer includes at least two of straight lane, deceleration lane, acceleration lane, or other lane types.

[0270] Example 9: The method of Example 7, the method further comprising: determining whether the curvature of the road is greater than a curvature threshold; and in response to determining that the curvature of the road is greater than the curvature threshold, offsetting a mass value relative to the origin along a y-axis perpendicular to the vehicle to determine a mass value at the offset point; or in response to determining that the curvature of the road is less than the curvature threshold, offsetting a mass value relative to the origin along the curvature direction to determine a mass value at the offset point.

[0271] Example 10: A method of any of the foregoing examples, the method further comprising: subsequently determining whether a confidence parameter and a reasonableness parameter associated with at least one layer hypothesis meet a corresponding threshold; and, in response to subsequently determining that the confidence parameter or reasonableness parameter associated with the at least one layer hypothesis does not meet the corresponding threshold, interrupting another operation of the vehicle using an autonomous driving system or an assisted driving system.

[0272] Example 11: A method of any of the foregoing examples, the method further comprising: subsequently determining whether a confidence parameter and a reasonableness parameter associated with at least one layer hypothesis meet a corresponding threshold; and in response to subsequently determining that a confidence parameter or reasonableness parameter associated with the at least one layer hypothesis does not meet a corresponding threshold, switching another operation of the vehicle to be controlled by the driver.

[0273] Example 12: A method of any of the preceding examples, wherein the autonomous driving system or the assisted driving system includes at least one of the following: an automatic cruise control system, a traffic jam assist system, a lane centering assist system, or an L3 / L4 autonomous driving system on a highway.

[0274] Example 13: A computer-readable storage medium comprising computer-executable instructions, which, when executed, cause a processor in a vehicle to: generate a grid representation of a road, the grid representation comprising a plurality of cells; generate a road model for the road based on data from a plurality of information sources, the road model comprising at least two distinct layers, the at least two distinct layers representing different corresponding road attributes of the plurality of cells in the grid representation of the road, wherein for each of the plurality of cells in each of the distinct layers, the at least two distinct layers include at least one corresponding layer hypothesis, the at least one corresponding layer hypothesis indicating a potential attribute state of the cell; and determine a corresponding quality value associated with each of the at least one corresponding layer hypothesis. The corresponding quality value indicates the confidence level associated with data contributing to the at least one corresponding layer hypothesis; the corresponding quality value is used to determine a confidence parameter and a plausibility parameter associated with each of the at least one corresponding layer hypothesis, the confidence parameter indicating the confidence level for the potential attribute state of the unit, the plausibility parameter indicating the probability that the potential attribute state is accurate for the unit; it is determined whether the confidence parameter and plausibility parameter associated with the at least one layer hypothesis meet a corresponding threshold; and in response to determining that the confidence parameter and plausibility parameter associated with the at least one layer hypothesis meet the corresponding threshold, the at least one layer hypothesis is used as input to an autonomous driving system or an assisted driving system that operates a vehicle on the road.

[0275] Example 14: A computer-readable storage medium of Example 13, wherein at least two distinct layers comprise at least two of the following: a cell category layer, a lane number layer, a lane marking type layer, a lane marking color layer, a traffic sign layer, a road marking layer, and a lane type layer.

[0276] Example 15: A computer-readable storage medium of any one of Examples 13 to 14, wherein: the cell category layer includes at least two of lane boundary, lane center, obstacle, or another value; the lane number layer includes at least two of left adjacent lane, own lane, right adjacent lane, or other lane; the lane marking type layer includes at least two of solid line lane marking, double line lane marking, dashed line lane marking, thick lane marking, or other lane marking types; the lane marking color layer includes at least two of yellow, white, or other colors; and the lane type layer includes at least two of straight lane, deceleration lane, acceleration lane, or other lane types.

[0277] Example 16: A computer-readable storage medium of Example 14, the computer-readable storage medium including additional computer-executable instructions that, when executed, cause a processor in a vehicle to: determine whether the curvature of a road is greater than a curvature threshold; and, in response to determining that the curvature of the road is greater than the curvature threshold, offset a mass value relative to the origin along a y-axis perpendicular to the vehicle to determine a mass value at the offset point; or, in response to determining that the curvature of the road is less than the curvature threshold, offset a mass value relative to the origin along a curvature direction to determine a mass value at the offset point.

[0278] Example 17: A computer-readable storage medium of any of Examples 13 to 16, wherein at least two distinct layers are defined by a corresponding identification frame (FOD), the corresponding FOD identifying a corresponding set of possible layer hypotheses.

[0279] Example 18: A computer-readable storage medium of any of Examples 13 to 17, wherein a plurality of information sources include at least one of the following: a lidar system, a vision system, a camera system, a vehicle status system, a position sensor, a steering sensor, a yaw rate sensor, or a map.

[0280] Example 19: A computer-readable storage medium of any of Examples 13 to 18, wherein the corresponding quality value is recursively updated based on data from multiple information sources.

[0281] Example 20: A computer-readable storage medium of any of Examples 13 to 19, wherein determining a corresponding quality value associated with each of at least one corresponding layer hypothesis comprises: fusing multiple quality values ​​using the Dempster-Schafer fusion rule, the multiple quality values ​​contributing to each of at least one corresponding layer hypothesis.

[0282] Example 21: A computer-readable storage medium comprising computer-executable instructions that, when executed, cause a processor in a vehicle to perform any of the methods of Examples 1 to 12.

[0283] Example 22: A system comprising a processor configured to perform a method of any one of Examples 1 to 12.

[0284] Example 23: The system of Example 22, wherein the system is installed in a vehicle.

[0285] Conclusion

[0286] While various embodiments of the present disclosure have been described in the foregoing description and illustrated in the accompanying drawings, it should be understood that the present disclosure is not limited thereto, but can be practiced in various ways within the scope of the following claims. It will be apparent from the foregoing description that various modifications can be made without departing from the spirit and scope of the present disclosure as defined by the following claims.

Claims

1. A method for a grid-based road model for roads, the grid-based road model comprising one or more layers, and the method comprising: Generate a grid representation of the road on which the vehicle is traveling, the grid representation having multiple cells; The grid-based road model is generated based on data from one or more information sources. The grid-based road model includes one or more layers representing different corresponding road attributes of multiple units. For each unit in each layer of the multiple units, the one or more layers include at least one corresponding layer hypothesis indicating the potential attribute state of that unit. For each cell in each layer, determine a corresponding quality value associated with each hypothesis of the at least one corresponding layer hypothesis; The corresponding quality value is used to determine a confidence parameter or reasonableness parameter associated with each of the at least one corresponding layer assumption for each unit in each layer; Determine whether the credibility parameter or the reasonableness parameter associated with each of the at least one corresponding layer hypothesis is greater than a threshold; and In response to determining that the confidence parameter or reasonableness parameter associated with each of the at least one corresponding layer hypothesis is greater than the threshold, the at least one corresponding layer hypothesis is used as input to the autonomous driving system or the driver assistance system to operate the vehicle on the road.

2. The method as described in claim 1, characterized in that, The corresponding quality value is updated by comparing previous corresponding quality values ​​from one or more previous moments with current corresponding quality value data from the current moment.

3. The method as described in claim 2, characterized in that, The frame rate between the corresponding information sources in one or more information sources is adjusted based on the speed of the vehicle.

4. The method as described in claim 2, characterized in that, The previous corresponding quality value is updated to the current corresponding quality value using a decay function.

5. The method as described in claim 1, characterized in that, The credibility parameter or the rationality parameter associated with the corresponding layer hypothesis in at least one of the corresponding layer hypotheses of two or more units in each layer has a different value.

6. The method as described in claim 1, characterized in that, The method further includes grouping the at least one corresponding layer hypothesis of each layer into road hypotheses based on the Dempster-Schaffer theory.

7. The method as described in claim 1, characterized in that, The grid representation of the road is a statically occupied grid.

8. The method as described in claim 1, characterized in that, The grid representation of the road is a dynamically occupied grid, and each cell in the grid representation indicates the speed of the vehicle.

9. The method as described in claim 1, characterized in that, Determining the corresponding quality value associated with each hypothesis includes: For two layer hypotheses that have the highest probability for a corresponding information source among the one or more information sources, the corresponding quality values ​​of the two layer hypotheses are set to be equal to their corresponding probabilities for the corresponding information source; and For other layer hypotheses in the at least one corresponding layer hypothesis, the corresponding quality value of the other layer hypothesis is set to zero for the corresponding information source.

10. The method as described in claim 1, characterized in that, The method further includes For each cell in each layer, the corresponding probability associated with each hypothesis is determined by calculating the pignistic transformation of the corresponding quality value.

11. A computer-readable storage medium comprising computer-executable instructions, which, when executed, cause a processor in a vehicle to: Generate a grid representation of the road, the grid representation having multiple cells; A road model of the road is generated based on data from one or more information sources. The road model includes one or more layers representing different corresponding road attributes of multiple units. For each unit in each layer of the multiple units, the one or more layers include at least one corresponding layer hypothesis indicating the potential attribute state of that unit. For each cell in each layer, determine a corresponding quality value associated with each hypothesis of the at least one corresponding layer hypothesis; The corresponding quality value is used to determine a confidence parameter or reasonableness parameter associated with each of the at least one corresponding layer assumption for each unit in each layer; Determine whether the credibility parameter or the reasonableness parameter associated with each of the at least one corresponding layer hypothesis is greater than a threshold; and In response to determining that the confidence parameter or reasonableness parameter associated with each of the at least one corresponding layer hypothesis is greater than the threshold, the at least one corresponding layer hypothesis is used as input to the autonomous driving system or the driver assistance system to operate the vehicle on the road.

12. The computer-readable storage medium as claimed in claim 11, characterized in that, The computer-readable storage medium further includes additional computer-executable instructions that, when executed, cause the processor to: update the corresponding quality value by comparing previous corresponding quality values ​​from one or more previous times with current corresponding quality value data from the current time.

13. The computer-readable storage medium as claimed in claim 12, characterized in that, The computer-readable storage medium further includes additional computer-executable instructions that, when executed, cause the processor to: adjust the frame rate between the times of the respective information sources among the one or more information sources based on the speed of the vehicle.

14. The computer-readable storage medium as claimed in claim 12, characterized in that, The computer-readable storage medium includes additional computer-executable instructions that, when executed, cause the processor to: update the previous corresponding quality value to the current corresponding quality value using a decay function.

15. The computer-readable storage medium as claimed in claim 11, characterized in that, The credibility parameter or the rationality parameter associated with the corresponding layer hypothesis in at least one of the corresponding layer hypotheses of two or more units in each layer has a different value.

16. The computer-readable storage medium as claimed in claim 11, characterized in that, The computer-readable storage medium includes additional computer-executable instructions that, when executed, cause the processor to: Based on the Dempster-Schaffer theory, the at least one corresponding layer hypothesis of each layer is grouped into road hypotheses.

17. The computer-readable storage medium as claimed in claim 11, characterized in that, The grid representation of the road is a statically occupied grid.

18. The computer-readable storage medium as claimed in claim 11, characterized in that, The grid representation of the road is a dynamically occupied grid, and each cell in the grid representation indicates the speed of the vehicle.

19. The computer-readable storage medium as claimed in claim 11, characterized in that, The computer-readable storage medium includes additional computer-executable instructions that, when executed, cause the processor to: determine the corresponding quality value associated with each hypothesis by means of the following steps: For two layer hypotheses that have the highest probability for a corresponding information source among the one or more information sources, the corresponding quality value of the two layer hypotheses is set to be equal to their corresponding probability for the corresponding information source; and For other layer hypotheses in the at least one corresponding layer hypothesis, the corresponding quality value of the other layer hypothesis is set to zero for the corresponding information source.

20. The computer-readable storage medium as claimed in claim 11, characterized in that, The computer-readable storage medium includes additional computer-executable instructions that, when executed, cause the processor to perform... For each cell in each layer, the corresponding probability associated with each hypothesis is determined by calculating the pignistic transformation of the corresponding quality value.

Images