Correlate perceived and mapped lane edges for localization

Abstract

A system for associating perceived and mapped lane edges may include a processor and a memory. The memory includes instructions that configure the processor to: receive sensor data representing a perceived object; receive map data representing a map object; determine a cost matrix indicating an association cost for associating the map object to the perceived object; compare the association cost to an association cost threshold; and associate the perceived object with the map object based on the association cost.

Description

Technical Field

[0001] The technical field generally relates to perception systems and methods, and more particularly to systems and methods for associating perceived objects with corresponding map data. Background Art

[0002] Autonomous and semi-autonomous vehicles often require accurate perception of object-to-object trajectories. The perception of object trajectories can be affected by perspective distortion. This results in incorrect lane object identification. In addition, sensor data from vehicle sensors and / or received map data may contain significant noise, further reducing the accuracy of identifying lane edges. This discrepancy can be problematic because autonomous vehicles, among other things, need to correctly identify lane edges on the road. Summary of the invention

[0003] A system includes a controller, the controller including a processor and a memory. The memory includes instructions that configure the processor to: receive sensor data representing a sensed object; receive map data representing a map object; determine a cost matrix indicating an association cost for associating the map object to the sensed object; compare the association cost to an association cost threshold; and associate the sensed object with the map object based on the association cost.

[0004] Among other features, the perceived object represents the edge of a lane disposed on the road.

[0005] Among other features, the map object represents the edge of a lane disposed on the road.

[0006] In other features, the controller is further configured to ij ] Determine the cost matrix, where C represents the calculated cost matrix and element c ij Represents the associated cost.

[0007] Among other features, Calculate element c ij ,in represents the distance cost between the edge of the i-th map lane and the edge of the j-th sensed lane, where represents the type mismatch cost between the i-th map lane edge and the j-th perceived lane edge.

[0008] Among other features, the distance cost is determined based on the mean squared Mahalanobis distance calculation.

[0009] In other features, a type mismatch cost represents a calculated difference between a plurality of lane marking types, wherein the type mismatch cost represents a calculated difference between a plurality of lane marking types, wherein the cost difference is defined as a negative log-likelihood of a probability that the lane marking types represent the same underlying lane edge.

[0010] A vehicle may include a system. The system may include a controller having a processor and a memory. The memory includes instructions that configure the processor to: receive sensor data representing a sensed object; receive map data representing a map object; determine a cost matrix indicating an association cost for associating the map object to the sensed object; compare the association cost to an association cost threshold; and associate the sensed object with the map object based on the association cost.

[0011] Among other features, the perceived object represents the edge of a lane disposed on the road.

[0012] Among other features, the map object represents the edge of a lane disposed on the road.

[0013] In other features, the controller is further configured to ij ] Determine the cost matrix, where C represents the calculated cost matrix and element c ij Represents the associated cost.

[0014] Among other features, Calculate element c ij ,in represents the distance cost between the edge of the i-th map lane and the edge of the j-th sensed lane, where The type mismatch cost between .

[0015] Among other features, the distance cost is determined based on the mean squared Mahalanobis distance calculation.

[0016] In other features, a type mismatch cost represents a calculated difference between a plurality of lane marking types, wherein the type mismatch cost represents a calculated difference between a plurality of lane marking types, wherein the cost difference is defined as a negative log-likelihood of a probability that the lane marking types represent the same underlying lane edge.

[0017] A method may include: receiving, by a controller, sensor data representing a sensed object; receiving map data representing a map object; determining a cost matrix indicating association costs for associating the map object to the sensed object; comparing the association costs to an association cost threshold; and associating the sensed object with the map object based on the association costs.

[0018] Among other features, the perceived object represents the edge of a lane disposed on the road.

[0019] Among other features, the map object represents the edge of a lane disposed on the road.

[0020] In other features, the controller is further configured to ij ] Determine the cost matrix, where C represents the calculated cost matrix and element c ij Represents the associated cost.

[0021] Among other features, Calculate element c ij ,in represents the distance cost between the edge of the i-th map lane and the edge of the j-th sensed lane, where represents the type mismatch cost between the i-th map lane edge and the j-th perceived lane edge.

[0022] Among other features, the distance cost is determined based on the mean squared Mahalanobis distance calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Exemplary embodiments will be described hereinafter with reference to the following drawings, in which like reference numerals represent like elements, and in which:

[0024] Figure 1 An exemplary environment is shown having a vehicle including a system associated with lane edges being sensed and mapped.

[0025] Figure 2 A schematic plan view of a vehicle travelling on a road is shown.

[0026] Figure 3 is a flow chart illustrating a process for associating perceived and mapped lane edges. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure may be described herein according to functions and / or logic block components and various processing steps. It should be appreciated that these block components may be implemented by any number of hardware, software and / or firmware components configured to perform a specified function. For example, embodiments of the present disclosure may employ various integrated circuit components (e.g., storage elements, digital signal processing elements, logic elements, or lookup tables, etc.), which may perform various functions under the control of one or more microprocessors or other control devices. In addition, it will be appreciated by those skilled in the art that embodiments of the present disclosure may be practiced in conjunction with any number of systems, and the systems described herein are merely exemplary embodiments of the present disclosure.

[0028] Autonomous and semi-autonomous vehicles are able to sense their environment and navigate based on the sensed environment. Such vehicles utilize various types of sensing devices such as radar, lidar, image sensors, etc. to sense their environment. In such vehicles, the sensed data can be fused with map data to identify and track objects near the vehicle.

[0029] As discussed in more detail herein, a system can associate sensed objects, such as lane edges, with corresponding objects within map data. For example, temporary sense or map imperfections may cause future sensed object data and / or map data to not match. The presently disclosed system associates sensed object data and corresponding map data based on prior association probabilities. For example, the system can implement one or more cost matrices such that the system does not rely on future sensed object data and / or map data that may be corrupted.

[0030] Figure 1 A side view of a vehicle 10 is shown, which is disposed on and capable of traveling on a driving surface 70, such as a paved road. The vehicle 10 includes an onboard navigation system 24, a computer-readable storage device or medium (memory) 23 including a digitized road map 25, a spatial monitoring system 100, a vehicle controller 50, a global positioning system (GPS) sensor 52, a human-machine interface (HMI) device 60, and in one embodiment, an autopilot controller 65 and a telematics controller 75. The vehicle 10 may include, but is not limited to, a mobile platform in the form of a commercial vehicle, an industrial vehicle, an agricultural vehicle, a bus, an airplane, a boat, a train, an all-terrain vehicle, a personal mobility device, a robot, etc., to achieve the purposes of the present disclosure.

[0031] In one embodiment, the spatial monitoring system 100 includes one or more spatial sensors and systems and a controller 110, which are configured to monitor a visible area 32 in front of the vehicle 10. The spatial sensors configured to monitor the visible area 32 in front of the vehicle 10 include, for example, a lidar sensor 34, a radar sensor 36, a digital camera 38, etc. Each of the spatial sensors is disposed on the vehicle to monitor all or a portion of the visible area 32 to detect nearby remote objects, such as road features, lane markings, buildings, pedestrians, road signs, traffic control lights and signs, other vehicles, and geographic features near the vehicle 10. The controller 110 generates a digital representation of the visible area 32 based on data input from the spatial sensors.

[0032] In view of each nearby remote object, the controller 110 can evaluate the input from the spatial sensors to determine the linear distance, relative speed and trajectory of the vehicle 10. The spatial sensors can be located at different locations on the vehicle 10, including the front corners, rear corners, rear sides and mid-sides. In one embodiment, the spatial sensors can include front radar sensors and cameras, but the present disclosure is not limited to this. The placement of the spatial sensors allows the controller 110 to monitor traffic flow, including nearby vehicles, intersections, lane markings, and other objects around the vehicle 10. The data generated by the controller 110 can be used by a lane marking detection processor (not shown) to estimate the road.

[0033] The spatial sensors of the vehicle spatial monitoring system 100 may include object location sensing devices including distance sensors such as FM-CW (frequency modulated continuous wave) radar, pulse and FSK (frequency shift keying) radar, LiDAR (light detection and ranging) devices, and ultrasonic devices that rely on effects such as Doppler effect measurements to locate forward objects. Possible object location devices include charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) video image sensors and other cameras / video image processors that utilize digital photography methods to "see" forward objects including one or more vehicles.

[0034] The lidar sensor 34 uses a pulsed reflected laser beam to measure the range or distance to an object. The radar sensor 36 uses radio waves to determine the distance, angle and / or speed of an object. The camera 38 includes an image sensor, a lens, and a camera controller. The image sensor is an electro-optical device that uses a multi-dimensional array of light-sensitive sensing elements to convert an optical image into an electrical signal. The camera controller is operably connected to the image sensor to monitor the visible area 32. The camera controller is configured to control the image sensor to capture an image of a field of view (FOV) associated with the visible area 32, which is projected onto the image sensor through the lens.

[0035] The optical lens may be configured to include features such as a pinhole lens, a fisheye lens, a stereo lens, a telephoto lens, etc. The camera 38 periodically captures image files associated with the visible area 32 via an image sensor at a desired rate (e.g., 30 image files per second). Each image file consists of a 2D or 3D pixelated digital representation of all or a portion of the visible area 32 captured at the native resolution of the camera 38. In one embodiment, the image file is in the form of a 24-bit image, including RGB (red, green, blue) visible spectrum values ​​and depth values ​​representing the visible area 32. Other embodiments of the image file may include, but are not limited to, a 2D or 3D image showing a black and white or grayscale visible spectrum representation of the visible area 32, an infrared spectrum representation of the visible area 32, or other image representations at a certain resolution level.

[0036] In one embodiment, the image representations of multiple image files can be evaluated for parameters related to brightness and / or luminance. The image representations can be evaluated based on RGB color components, brightness, texture, contour, or a combination thereof. The image sensor communicates with an encoder that performs digital signal processing (DSP) on each image file. The image sensor of the camera 38 can be configured to capture images at a nominal standard definition resolution (e.g., 640×480 pixels). The image sensor of the camera 38 can be configured to capture images at a nominal high definition resolution (e.g., 1440×1024 pixels) or other suitable resolution. The image sensor of the camera 38 can capture still images or digital video images at a predetermined image capture rate. In one embodiment, the image files are transmitted to the camera controller as encoded data files stored in a non-transitory digital data storage medium for on-board or off-board analysis.

[0037] The camera 38 is advantageously mounted and positioned on the vehicle 10 in a position that allows for capturing images of the visible area 32, wherein at least a portion of the visible area 32 includes a portion of the driving surface 70 in front of the vehicle 10 and includes the trajectory of the vehicle 10. The visible area 32 may also include the surrounding environment, including, for example, traffic, roadside objects, pedestrians and other features, the sky, the horizon, the driving lane, and oncoming traffic in front of the vehicle 10. Other cameras (not shown) may also be employed, including, for example, a second camera disposed at the rear or side of the vehicle 10 to monitor the rear of the vehicle 10 and one of the right or left sides of the vehicle 10.

[0038] The autopilot controller 65 is configured to implement autopilot or advanced driver assistance system (ADAS) vehicle functions. Such functions may include an onboard control system capable of providing a certain level of driving automation. The terms "driver" and "operator" describe a person who is responsible for directing the operation of the vehicle 10 (whether actively participating in controlling one or more vehicle functions or directing the operation of the autopilot vehicle). Driving automation may include a range of dynamic driving and vehicle operations. Driving automation may include a certain level of automatic control or intervention associated with a single vehicle function (such as steering, acceleration and / or braking), while the driver continues to have full control of the vehicle 10. Driving automation may include a certain level of automatic control or intervention associated with simultaneous control of multiple vehicle functions (such as steering, acceleration and / or braking), while the driver continues to have full control of the vehicle 10. Driving automation may include simultaneous automatic control of vehicle driving functions (including steering, acceleration and braking), where the driver relinquishes control of the vehicle for a period of time during travel.

[0039] Driving automation may include simultaneous automatic control of vehicle driving functions (including steering, acceleration, and braking) where the driver relinquishes control of the vehicle 10 throughout the trip. Driving automation includes hardware and controllers configured to monitor the spatial environment in various driving modes to perform various driving tasks during dynamic operation of the vehicle. As non-limiting examples, driving automation may include cruise control, adaptive cruise control, lane change warning, intervention and control, automatic parking, acceleration, braking, etc.

[0040] As non-limiting examples, automated driving vehicle functions include adaptive cruise control (ACC) operations, lane guidance and lane keeping operations, lane change operations, steering assist operations, object avoidance operations, parking assist operations, vehicle braking operations, vehicle speed and acceleration operations, vehicle lateral motion operations (e.g., as part of lane guidance, lane keeping, and lane change operations), etc. As such, braking commands may be generated by the automated driving controller 65 independent of vehicle operator actions and in response to automated driving control functions.

[0041] Operator controls may be included in the passenger compartment of the vehicle 10 and may include, as non-limiting examples, a steering wheel, an accelerator pedal, a brake pedal, and operator input devices as elements of the HMI device 60. The operator controls enable the vehicle operator to interact with and direct the operation of the vehicle 10 in passenger transport operations. In some embodiments of the vehicle 10, operator controls including a steering wheel, an accelerator pedal, a brake pedal, a transmission range selector, etc. may be omitted.

[0042] The HMI device 60 provides human-machine interaction for directing the operation of the infotainment system, the global positioning system (GPS) sensor 52, the navigation system 24, etc., and includes a controller. The HMI device 60 monitors the operator's requests and provides information to the operator, including vehicle system status, service and maintenance information. The HMI device 60 communicates with and / or controls the operation of multiple operator interface devices, wherein the operator interface devices are capable of transmitting messages associated with the operation of one of the autonomous vehicle control systems.

[0043] The HMI device 60 may also communicate with one or more devices that monitor biometric data associated with the vehicle operator, including, for example, eye gaze position, posture, and head position tracking. For ease of description, the HMI device 60 is depicted as a single device, but may be configured as multiple controllers and associated sensing devices in one embodiment of the system described herein. The operator interface device may include a device capable of transmitting a message that prompts the operator to act, and may include an electronic visual display module, such as a liquid crystal display (LCD) device, a head-up display (HUD), an audio feedback device, a wearable device, and a tactile seat.

[0044] Operator interface devices capable of prompting operator actions are preferably controlled by or through the HMI device 60. The HUD may project information reflected off the inside of the vehicle windshield into the operator's field of view, including conveying a confidence level associated with operating one of the automated vehicle control systems. The HUD may also provide augmented reality information, such as lane position, vehicle path, directions, and / or navigation information.

[0045] The onboard navigation system 24 uses the digitized road map 25 to provide navigation support and information to the vehicle operator. The digitized road map 25 is used by the autonomous driving controller 65 to control autonomous vehicle operation or ADAS vehicle functions.

[0046] The vehicle 10 may include a telematics controller 75 including a wireless telematics communication system capable of off-vehicle communications, including communications with a communications network 90 having both wireless and wired communications capabilities.

[0047] The telematics controller 75 is capable of off-vehicle communications, including short-range vehicle-to-vehicle (V2V) communications and / or vehicle-to-everything (V2x) communications, which may include communications with infrastructure monitors (e.g., traffic cameras). The telematics controller 75 has a wireless telematics communication system that is capable of short-range wireless communications with a handheld device (e.g., a cellular phone, satellite phone, or other telephone device). In one embodiment, the handheld device includes a software application that includes a wireless protocol for communicating with the telematics controller 75, and the handheld device performs off-vehicle communications, including communicating with an off-vehicle server 95 via the communication network 90. ​​The telematics controller 75 performs off-vehicle communications directly by communicating with the off-vehicle server 95 via the communication network 90.

[0048] The term "controller" and related terms (such as microcontroller, control unit, processor and similar terms) refer to one or various combinations of application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), electronic circuits, central processing units (e.g., microprocessors), and related non-transitory memory components in the form of memory and storage devices (read-only memory, programmable read-only memory, random access memory, hard disk drives, etc.) indicated by memory 23. The non-transitory memory components are capable of storing machine-readable instructions in the form of one or more software or firmware programs or routines, combinational logic circuits, input / output circuits and devices, signal conditioning and buffering circuits, and other components that can be accessed by one or more processors to provide the described functionality.

[0049] Input / output circuits and devices include analog / digital converters and related devices that monitor inputs from sensors, either at a preset sampling frequency or in response to a triggering event. Software, firmware, programs, instructions, control routines, code, algorithms and similar terms refer to sets of instructions executable by a controller, including calibration and lookup tables. Each controller executes a control routine to provide the desired functionality. Routines may be executed periodically, such as every 100 microseconds during ongoing operation.

[0050] The routine may be executed in response to the occurrence of a triggering event. Communications between controllers, actuators, and / or sensors may be accomplished using direct wired point-to-point links, networked communication bus links, wireless links, or other suitable communication links. Communications include exchanging data signals in an appropriate form, including, for example, electrical signals via a conductive medium, electromagnetic signals via air, optical signals via optical waveguides, and the like.

[0051] Data signals may include discrete, analog or digitized analog signals representing inputs from sensors, actuator commands, and communications between controllers. The term "signal" refers to a physically discernible indicator that conveys information and may be a suitable waveform (e.g., electrical, optical, magnetic, mechanical, or electromagnetic) capable of propagating through a medium, such as DC, AC, sine, triangle, square, vibration, etc. A parameter is defined as a measurable quantity that represents a physical characteristic of a device or other element that can be discerned using one or more sensors and / or physical models. A parameter may have discrete values ​​(e.g., "1" or "0"), or may be an infinitely variable value.

[0052] As used herein, the terms "dynamic" and "dynamically" describe steps or processes performed in real time that are characterized by monitoring or otherwise determining the states of parameters and regularly or periodically updating the states of those parameters during execution of a routine or between repeated executions of a routine.

[0053] Figure 2 An exemplary top view of a road segment 200 is shown, which may be part of a digitized road map 25 that may be employed by a vehicle 10. The exemplary road segment 200 includes road elements 202, such as lane markings, with an upcoming turn 204. It should be understood that the road segment 200 and concepts associated therewith may be applied to road segments configured as multi-lane roads. An edge portion 205 of the road segment is shown and represents a perceived lane marking.

[0054] A first set of points 210 is shown and represents geophysical locations of a plurality of lane markings 202 disposed at edge portion 205 of road segment 200 as sensed by one or more spatial sensors of vehicle spatial monitoring system 100. First set of points 210 form sensed lane markings that define edge portion 205. Second set of points 220 represent map-based locations of lane markings (e.g., lane edges 205) that are identified on digitized road map 25 and correspond to individual ones of lane markings 202.

[0055] As described in further detail herein, the controller 110 may calculate one or more cost matrices to determine whether the perceived object represented by the first set of points 210 is associated with, i.e., corresponds to, the map object represented by the second set of points 220. Based solely on geometric distance determinations, certain distortions within the map data and / or sensor data may result in erroneous associations or mismatches.

[0056] As used herein, for ease of reading, the present disclosure may refer to the second set of points 220 as map objects and / or map segments, and may refer to the first set of points 210 as perceived objects and / or perceived segments. As used herein, the term "type" may be defined as a lane marking type, and the term "color" may be defined as a color or hue of a lane marking. By employing the techniques described herein, the controller 110 may determine similarities between lane edge types and colors to improve the correct association between perceived objects and corresponding objects within the map data.

[0057] The sensed object data (e.g., the first set of points 210) may be received by the controller 110 and converted from a host vehicle reference (i.e., the ego vehicle 10) to a global reference frame representation. For example, the controller 110 may convert the sensed data according to Equation 1:

[0058] x g =Rx e +x p Equation 1

[0059] where x g is the position vector of the point in the global reference frame, x eis the position vector of the point in the main vehicle coordinates, x p is the position vector of the host vehicle in the global reference frame, and R is the rotation matrix that maps the sensory data from the host vehicle reference to the corresponding global reference frame representation, which depends on the vehicle's heading in the global reference frame.

[0060] The controller 110 may calculate the cost matrix according to Equation 2,

[0061] C=[c ij Equation 2

[0062] Where C is the calculated cost matrix, element c ij represents the cost of associating the mapped object with the perceived object j, i.e., the association cost. The cost may include the distance between i (where i represents the i-th mapped lane edge or segment) and j (where j represents the j-th perceived lane edge or segment) and type mismatch cost The type mismatch cost is calculated by combining i (where i represents the i-th map lane edge or segment) with type t according to Equation 3. i and j (where j represents the jth perceived lane edge or segment) with type t j The associated costs:

[0063]

[0064] If two objects (e.g., the sensed object and the map object) are of the same type, the type mismatch cost It can usually be small, while if the two objects are incompatible, the type mismatch cost is infinite. The cost can be interpreted as the negative log-likelihood of the data under the assumed probability distribution.

[0065] The controller 110 can determine the point {m i，k} and the points in the perception segment j The mean squared Mahalanobis distance between (The subscripts i and j are omitted for convenience):

[0066]

[0067] in Including point {m i，k} and {p j，k}, and ∑ is the covariance term.

[0068] The lateral distance can be calculated according to Equation 5:

[0069]

[0070] where p k′ Including distance k and The nearest (e.g., closest) perception point, where θ k is the same as point m k The longitudinal distance d between the points lon，k is also calculated by the controller 110, and if the longitudinal distance is greater than the threshold T log , then the distance d is set to infinity, where and

[0071] Covariance ∑ can be defined as ∑ = ∑ percep +∑ pose , where ∑ percep Is has A diagonal matrix, where is closest to m k The lateral variance of the perception point, and ∑ pose =U∑ t U T , where ∑ t is the posture of vehicle 10 and U T =(u0 ... u M ). The average weighted transverse Euclidean distance r can be calculated based on To calculate, It should be understood that each map segment i and perception segment j can be associated with d ij and r ij Associated.

[0072] The controller 110 may further determine a lane marking type and / or color cost. The type and / or color cost is calculated such that when the perception and mapping types are determined to be the same, the type and / or color cost is relatively small; and when the perception and mapping types are determined to be different, the type and / or color cost is relatively high. Type cost of lane edge It can be calculated according to equation 6,

[0073]

[0074] in represents the probability distribution of the true type of map lane edge i conditional on the reported type, represents the probability distribution of the true color of map lane edge i conditioned on the reported color, represents the probability distribution of the type of lane edge j perceived conditioned on the reported type, is the probability distribution of the perceived color of lane edge j conditioned on the reported color, represents the dot product of the map distribution and the perceived distribution with respect to type, and denotes the type-based similarity of the two map and perceived lane edges to each other.

[0075] Each perceived lane edge and mapped lane edge includes a plurality of points associated with a corresponding type of lane edge. i The pattern can be defined as the segmentation type of the lane edge, and the type of lane edge segment t i The online confidence of can be defined as p(m i =t i ), which is calculated by the controller as the average of the confidence types for each point that matches the type pattern in the lane edge segment.

[0076] The controller 110 can use the marginal probability to determine the probability of the actual type and color under the conditions of the reported type, the reported color, the confusion matrix, and the priors about the existing types. For example, the probability of the actual type can be calculated according to Equation 7:

[0077]

[0078] where p(t=a|r=b) represents the entry from the transposed confusion matrix and describes the probability of the actual type conditional on the reported type, and l i (r b ) represents an entry from the type likelihood vector for the i-th map lane edge, and the type likelihood vector represents the probability of the reported type. It should be understood that what is shown in Equation 7 is the calculation of the actual type corresponding to the map; however, the same formula can be used to calculate the perception or map and / or color or type.

[0079] In this context, the confusion matrix describes the probability of the reported type based on the exemplary data test set under the condition of the actual type. For example, the controller 110 can implement Bayes' theorem to transpose the mapping and perception type confusion matrix and the color confusion matrix to identify the probability of the actual type under the condition of the reported type. The controller 110 can determine the transposed confusion matrix according to Equation 8:

[0080]

[0081] where p(r=b|t=a) represents the entry from the map type confusion matrix corresponding to the actual type a and the reported type b, and where p(t=a) is a prior on the existing type distribution. In some cases, a uniform distribution is used for the prior unless additional information on the type is available.

[0082] The controller 110 may also segment the sensed lane edge at the boundary of the mapped lane segment. In some cases, the sensed lane edge may span more than one mapped lane segment, which may violate the 1:1 mapping rule for the output data. In various embodiments, the controller 110 segments the sensed lane edge into segments when the closest mapped lane edge segment point is the last point in the segment.

[0083] In some embodiments, the controller 110 longitudinally segments the sensed and mapped lane edges according to type to achieve type-based association. For example, a type lane segmentation protocol can address noise in lane classification, such as temporary type changes over short distances. To address classification noise, a suitable filtering process is used for each lane edge within a sliding window of n points (where more than k points are of different types), which divides the segment into two. The ratio of n to k and the size of n determine the sensitivity of the segmentation protocol to noise. For example, n=5 and k=2 can correspond to points within 5 points, and segmentation will only be performed if they are within a window of 5 points (where 3 or more points are of different types).

[0084] q pm The term can be defined as the perceived lane index Lane edge index with map The associated probability, where the special case express There is no association between the perceived lane edge index and the mapped lane edge index. As used herein, the distance and / or type measurements at time t may be represented by the vector z (t) To express.

[0085] As described herein, the controller 110 may employ Bayes' theorem to determine the probability that the perceived lane edge corresponds to the mapped lane edge. The prior probability of the perceived lane edge being associated with the mapped lane edge m may be calculated according to Equation 9:

[0086]

[0087] Where m = 0 corresponds to an outlier, i.e., no association, and P inlier Includes inlier probabilities.

[0088] exist (i.e., under the condition of the associated probability of the prior time step), the predicted probability It can be defined according to equation 10:

[0089]

[0090] in is a normalization term, and 1 / α is a pseudo-count. The generated predictions include the attributes (i.e., prior posterior) and Based on these properties, α specifies how much of the previous probability is retained and how much of the prediction reverts to the prior probability. The controller 110 can calculate the direction of the adjacent association pair according to Equation 11 Contribution to the evolution of To calculate the predicted probability :

[0091]

[0092] where ∑ = αI is a parameter with adjustment parameters α and In order to predict The probability of association with Marginalize on and use the corresponding posterior probabilities of the candidates To weight the candidates, Equation 8 adds all possible candidates at time step t-1 Included. Mahalanobis distance Additional spatial weighting is provided, giving higher contribution weights to nearby candidates during prediction.

[0093] z t The likelihood of m =P(z (t) |M (t) =m) can be given by equation 12:

[0094]

[0095] where u(p|a,b) is the likelihood that the multivariate uniform distribution has lower bounds a and b at p, respectively, and and are the type and distance likelihoods, respectively.

[0096] The outlier likelihood (e.g., where m=0) consists of the product of two independent uniform distributions, one corresponding to the distance and the other corresponding to the type. In one exemplary embodiment, the type likelihood can be written as Where N i is the cardinality of the discrete variables in dimension i, e.g., for dimensions color∈yellow,white and road and style∈dashed,solid, then l t = 1 / 6. The range of distances [a, b] should be large enough to cover practical observations. Cost and relief are related by c = -ln l, so that They are the sum of type cost and squared Mahalanobis distance cost respectively.

[0097] In the prior probability The posterior probability that the perceived lane edge p is a noisy realization of the mapped lane edge m is given by Equation 13:

[0098]

[0099] With reference to the perceptual index p, all probabilities can be collected into a P x M posterior probability matrix Q = [q pm ], where the outlier probabilities corresponding to m = 0 are removed. The probability matrix can then be converted into a cost matrix according to C = -ln Q, where ln(·) is applied element-wise.

[0100] In some embodiments, the controller 110 may calculate d by identifying candidates from the map. ij , by selecting the maximum distance D from each candidate candidate The k nearest neighbors within a node are used to find the candidate, and the maximum distance can be determined using a suitable Euclidean distance and calculated using a kd tree.

[0101] In some embodiments, the controller 110 longitudinally segments the sensed and mapped lane edges according to type to achieve type-based association. For example, a type lane segmentation protocol can address noise in lane classification, such as temporary type changes over short distances. To address classification noise, a suitable filtering process is used for each lane edge within a sliding window of n points (where more than k points are of different types), which divides the segment into two. The ratio of n to k and the size of n determine the sensitivity of the segmentation protocol to noise. For example, n=5 and k=2 can correspond to points within 5 points, and segmentation will only be performed if they are within a window of 5 points (where 3 or more points are of different types).

[0102] The controller 110 selects an association among the plurality of associations based on the calculated cost matrix. The cost matrix represents the cost of a plurality of potential associations between the first group of points 210 and the second group of points 220. In an exemplary embodiment, the cost matrix may be calculated according to the Kuhn Munkres algorithm. In an exemplary embodiment, using a cost matrix having a value c skip The cost matrix is ​​augmented by virtual rows and columns, and the value c skip denotes a cost of no assignment for all perception and map candidates, so that objects from perception and map potentially remain unassigned. The controller 110 may ignore solutions involving virtual entries, and if only virtual entries are included in the calculated solution, the controller 110 does not perform an association determination.

[0103] If the controller 110 determines that there are multiple solutions with similar costs, the controller 110 stops determining the association between the first set of points 210 and the second set of points 220 to avoid selecting an ambiguous solution. The controller 110 can determine whether there are multiple solutions with similar costs by comparing the difference between the total cost c1 from the solution S1 and the total cost c2 from the solution S2. The controller 110 compares the difference with the cost threshold Δc ignore For example, the total cost is calculated according to Equation 14:

[0104]

[0105] where c k Indicates that based on c ij If the controller 110 determines that |c1-c2|<Δc ignore , the controller 110 does not associate the perceived object (e.g., the first set of points 210) with the object depicted in the map data (e.g., the second set of points 220). Otherwise, the controller 110 associates the perceived object with the object depicted in the map data, indicating that the perceived object is an object in the map data. In various embodiments, the |c1-c2| value may represent the association cost, and Δc ignore can represent the associated cost threshold. By setting the entry in the cost matrix corresponding to S1 to infinity, And by repeating the cost calculation according to Equation 7, a second solution S2 that has no common allocation with the optimal solution S1 can be found.

[0106] When the Euclidean distance r ij When neither is statistically significantly greater than a predetermined maximum distance, the controller 110 selects solution S1. , the controller 110 does not associate the first set of points 210 with the second set of points 220, where i,j = argmax i,j∈S1 r ij and k represents some standard deviation used to determine significance, for example, for a confidence level of about 95%, k = 2. Solution S1 may represent the association between the perceived lane edge and the lane edge represented in the map.

[0107] Figure 3 An exemplary process 300 is shown for associating sensed lane edges with corresponding lane edges represented in map data (eg, determining that each lane edge is the same). Process 300 may include software instructions executable by controller 110.

[0108] In step 302, the controller 110 segments the map lane edges based on perception. In step 304, the controller 110 initializes the perception / map recognition mapping table. In step 306, the controller 110 initializes the association probability matrix using the prior probability data.

[0109] Using the techniques described in more detail above, the controller 110 calculates the distance likelihood between each map / perception pair at step 308. At step 310, the controller 110 calculates the type likelihood between each map / perception pair. At step 312, the controller 110 updates the probability matrix using the overall likelihood calculated in steps 308 and 310. At step 314, the controller 110 converts the probability matrix into a cost matrix using a suitable negative logarithmic transformation function.

[0110] At step 316, the controller 110 selects the best association using the total cost calculation described above in conjunction with Equation 7. At step 318, the controller 110 determines whether the timestamp has increased. For example, the controller 110 determines whether the received perception data includes a timestamp that has increased relative to a previously received timestamp. If the timestamp has not increased, the process 300 returns to step 318. Otherwise, at step 320, the controller 110 generates a prediction probability matrix. Based on the prediction probability matrix, the controller 110 updates the perception / map recognition mapping table at step 322. At step 324, the controller 110 segments the lane edges based on the received perception data.

[0111] At step 326, the controller 110 determines whether the number of perceived lanes has changed. If the number of perceived lanes has changed, then at step 328, the controller 110 may add or remove existing probability matrix rows and / or reallocate new probability matrix rows with prior data. At step 330, the controller 110 determines whether the number of lanes in the map data has changed. If the number of lanes within the map data has changed, the controller 110 adds or removes columns and reallocates probabilities. The process 300 may then return to step 308.

[0112] Although at least one exemplary embodiment has been presented in the above specific description, it should be understood that there are a large number of variations. It should also be understood that one or more exemplary embodiments are merely examples and are not intended to limit the scope, applicability or configuration of the present disclosure in any way. On the contrary, the above specific description will provide a convenient roadmap for those skilled in the art to implement one or more exemplary embodiments. It should be understood that various changes may be made to the function and arrangement of elements without departing from the scope of the present disclosure set forth in the attached claims and their legal equivalents.

[0113] This detailed description is merely exemplary in nature and is not intended to limit the present application and uses. In addition, the purpose is not to be bound by any explicit or implicit theory presented in the aforementioned technical field, technical background, invention content or the following detailed description. As used herein, the term "module" refers to any hardware, software, firmware, electronic control component, processing logic and / or processor device that can be used alone or in combination with each other, including but not limited to: application-specific integrated circuits (ASICs), electronic circuits, processors (shared processors, dedicated processors or processor groups) and memories that execute one or more software or firmware programs, combinational logic circuits and / or other suitable components that provide the functions.

[0114] Embodiments of the present disclosure may be described herein in terms of functional and / or logical block components and various processing steps. It should be understood that these block components may be implemented by any number of hardware, software, and / or firmware components configured to perform specified functions. For example, an embodiment of the present disclosure may employ various integrated circuit components (e.g., storage elements, digital signal processing elements, logic elements, or lookup tables, etc.), which may perform various functions under the control of one or more microprocessors or other control devices. In addition, it will be appreciated by those skilled in the art that embodiments of the present disclosure may be practiced in conjunction with any number of systems, and the systems described herein are merely exemplary embodiments of the present disclosure.

Claims

1. A system associated with a sensed and mapped lane edge, comprising a controller, the controller comprising a processor and a memory, the memory comprising instructions such that the processor is configured to: receiving sensor data representing a sensed object; receiving map data representing a map object; Determine a cost matrix indicating the association cost for associating the map object to the sensed object, wherein the cost matrix is ​​based on C=[c ij ] is determined, where C represents the calculated cost matrix and element c ij represents the associated cost; Calculate the element c ij ,in represents the distance cost between the edge of the i-th map lane and the edge of the j-th sensed lane, where represents a type mismatch cost between the i-th mapped lane edge and the j-th sensed lane edge, wherein the type mismatch cost represents a calculated difference between a plurality of lane marking types; comparing the association cost to an association cost threshold; and The sensed object is associated with the map object based on the association cost.

2. The system associated with a perceived and mapped lane edge of claim 1, wherein the perceived object represents a lane edge disposed on a road.

3. The system associated with sensed and mapped lane edges of claim 1, wherein the map object represents a lane edge disposed on a road.

4. The system associated with sensed and mapped lane edges of claim 1, wherein the distance cost is determined based on a mean squared Mahalanobis distance calculation.

5. The system associated with sensed and mapped lane edges of claim 1, wherein the calculated difference between a plurality of lane marking types is defined as a negative log likelihood of the probability that the lane marking types represent the same underlying lane edge.

Images