Lane Level Map Matching
The integration of a hidden Markov model with vehicle and sensor data enhances the accuracy and robustness of map matching for autonomous vehicles, addressing the limitations of existing systems by providing precise localization and perception.
Patent Information
- Application Number
- CN202010511519.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-06-07
- Filing Date
- 2020-06-08
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-06-08
AI Technical Summary
Existing map matching systems for autonomous vehicles are not accurate and robust, particularly due to noisy GPS signals and imperfect visual sensor detections, necessitating improved methods for precise localization and perception of the vehicle's environment.
A method involving a state space model, specifically a hidden Markov model, is used to integrate vehicle data, sensor data, and high-definition map data to determine the most likely path of the vehicle, incorporating lane geometry and road references, and calculating costs based on predefined probabilities to enhance accuracy and robustness.
The method provides reliable and accurate lane-level map matching, capable of handling error-prone data and enabling real-time, computationally efficient navigation for autonomous vehicles.
Smart Images

Figure CN112050817B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This patent application claims priority to European Patent Office application entitled “LANE LEVEL MAP MATCHING” (application number 19179083.1) filed on June 7, 2019, which is assigned to the assignee and is specifically incorporated herein by reference. Technical Field
[0003] The present disclosure generally relates to a process of matching geographic location data of a vehicle to a digital map location, i.e., map matching. More specifically, the present disclosure relates to a solution for determining a lane path for an autonomous vehicle (AV) or a semi-autonomous vehicle (i.e., a vehicle equipped with an advanced driver assistance system). Background Art
[0004] Today's maps are primarily designed for human use. More specifically, they are intended for turn-by-turn navigation purposes. As the map user navigates the road, he / she visually acquires additional environmental information, such as the type and location of lane markings, any debris located on the road, and road maintenance obstacles. However, autonomous vehicles require very different maps. More specifically, these maps need to be high definition (HD), providing the "robot" with the possibility of very precise localization and perception of the environment. HD maps also need to be constantly updated to track events such as traffic accidents or traffic jams.
[0005] Furthermore, just as a vehicle needs minute information about its environment, it also needs to understand its location on the road. This problem of identifying locations on the road is called map matching, and can be more formally described as the process of matching location data to a digital map to obtain the true location on the road network. Map matching can be considered an important aspect of navigation and route guidance systems.
[0006] Existing solutions built on, for example, probability theory, fuzzy logic theory and belief theory are used to map GPS coordinates to a certain road segment to give information about the surroundings of the vehicle.
[0007] However, there is a need for new and improved solutions that provide more accurate and robust map matching by, for example, compensating for the noisy characteristics of GPS signals. Summary of the invention
[0008] It is therefore an object of the present disclosure to provide a method, computer readable storage medium, control device, and vehicle for lane level map matching of a vehicle that alleviates all or at least some of the drawbacks of currently known systems.
[0009] This object is achieved via a method, a computer readable storage medium, a control device and a vehicle for lane level map matching of a vehicle as defined in the appended claims.The term exemplary is in the present context to be understood as serving as an example, instance or illustration.
[0010] According to a first aspect of the present disclosure, a method for lane-level map matching of a vehicle is provided. The method includes receiving vehicle data, the vehicle data including the geographic location of the vehicle, the orientation of the vehicle, and the speed of the vehicle. The method further includes receiving sensor data from a perception system of the vehicle. The sensor data includes information about the location of at least one road reference in the surrounding environment of the vehicle. Moreover, the method includes receiving map data including lane geometry of the surrounding environment of the vehicle, the lane geometry including a set of candidate lanes. Next, the method includes forming a state space model including a set of states, wherein each state in the set of states represents a candidate lane in the set of candidate lanes, and defining a cost from each state in the set of states to each other state based on the received vehicle data and the received sensor data. Moreover, the method includes determining a possible path of the vehicle based on the formed state space model and the defined cost.
[0011] The proposed method enables reliable and accurate lane-level map matching. Moreover, the proposed method is robust to error-prone data such as noisy GPS measurements and imperfect visual sensor detections. Furthermore, the method can be implemented as an online algorithm, considering global optimality at each stage of the computation.
[0012] In addition, the inventors have recognized that sensor observations of the surrounding environment can be fused with location data to add robustness to conventional GPS-based map matching. Furthermore, by forming a state space model and calculating the most likely path by observing the probability of a vehicle being in a particular state at a given time, a computationally efficient and accurate map matching solution can be provided.
[0013] According to an exemplary embodiment of the present disclosure, each state is a hidden state in a hidden Markov model, and the cost is defined based on the following terms: m At time t k Make observations k The first predefined probability E k and at time t k From the first state x in a set of states k Change state to another state x in a set of states k The second predefined probability T k .
[0014] According to a second aspect of the present disclosure, there is provided a (non-transitory) computer-readable storage medium storing one or more programs, the one or more programs being configured to be executed by one or more processors of a vehicle control system, the one or more programs comprising instructions for executing a method according to any of the embodiments disclosed herein. For this aspect of the present disclosure, similar advantages and preferred features exist as in the first aspect of the present disclosure discussed previously.
[0015] The term "non-transitory" as used herein is intended to describe a computer-readable storage medium (or "memory") that excludes propagating electromagnetic signals, but is not intended to otherwise limit the type of physical computer-readable storage device encompassed by the phrase computer-readable medium or memory. For example, the term "non-transitory computer-readable medium" or "tangible memory" term is intended to include types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data storage stored in non-transitory form on tangible, computer-accessible storage media may be further transmitted by a transmission medium or signals such as electrical, electromagnetic or digital signals, which may be transmitted via a communication medium such as a network and / or wireless link. Therefore, the term "non-transitory" as used herein is a limitation of the medium itself (i.e., tangible, non-signal), rather than a limitation on data storage persistence (e.g., RAM vs. ROM).
[0016] According to a third aspect of the present disclosure, a control device for lane-level map matching of a vehicle is provided, the control device comprising a control circuit configured to receive vehicle data, the vehicle data comprising the geographic location of the vehicle, the orientation of the vehicle and the speed of the vehicle. The control circuit is configured to receive sensor data from a perception system of the vehicle. The sensor data comprises information about the position of at least one road reference in the surrounding environment of the vehicle. In addition, the control circuit is configured to receive map data including lane geometry of the surrounding environment of the vehicle, the lane geometry comprising a set of candidate lanes. Next, the control circuit is configured to form a state space model comprising a set of states, wherein each state in the set of states indicates a lane of the set of candidate lanes. The control circuit is further configured to define a cost from each state in a set of states to each other state based on the received vehicle data and the received sensor data, and determine a possible path of the vehicle based on the formed state space model and the defined cost. For this aspect of the present disclosure, similar advantages and preferred features exist as in the first aspect of the present disclosure discussed previously.
[0017] According to a fourth aspect of the present disclosure, there is provided a vehicle including a perception system, the perception system including at least one sensor for monitoring the surrounding environment of the vehicle. The vehicle also includes an inertial measurement unit (IMU) for measuring the inertial motion of the vehicle, a localization system for monitoring the geographic location and orientation of the vehicle, and a control device according to any embodiment disclosed herein. For this aspect of the present disclosure, similar advantages and preferred features exist as in the first aspect of the present disclosure discussed previously.
[0018] Further embodiments of the invention are defined in the dependent claims. It should be emphasized that the term "comprises / comprising" when used in this specification is used to specify the presence of a specified feature, integer, step or component. It does not exclude the presence or addition of one or more other features, integers, steps, components or groups thereof.
[0019] These and other features and advantages of the present invention will be further elucidated hereinafter with reference to the embodiments described below. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Further objects, features and advantages of embodiments of the present disclosure will emerge from the following detailed description with reference to the accompanying drawings, in which:
[0021] Figure 1 is a schematic flow chart representation of a method for lane level map matching for a vehicle traveling on a road according to an embodiment of the present disclosure.
[0022] Figure 2 is a schematic graphical representation of a discrete Hidden Markov Model (HMM) having three states and three observations with corresponding transmission and emission probabilities according to an embodiment of the present disclosure.
[0023] Figure 3 yes Figure 2 Schematic lattice diagram representation of the HMM.
[0024] Figure 4 is a schematic side view of a vehicle having a control device for lane-level map matching of the vehicle traveling on a road according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0025] Those skilled in the art will understand that the steps, services and functions explained herein can be implemented using a single hardware circuit, using software running in conjunction with a programmed microprocessor or general purpose computer, using one or more application specific integrated circuits (ASICs) and / or using one or more digital signal processors (DSPs). It is also understood that when the present disclosure is described in terms of a method, it can also be embodied in one or more processors and one or more memories connected to the one or more processors, wherein the one or more memories store one or more programs that, when executed by the one or more processors, perform the steps, services and functions disclosed herein.
[0026] In the following description of exemplary embodiments, identical reference numerals refer to identical or similar components.A vehicle is understood in the present context to be a road vehicle, such as a car, bus, truck, construction vehicle or the like.
[0027] Figure 1 is a schematic flow chart representation of a method 100 for lane level map matching of a vehicle. Map matching can be understood as the process of matching the position data of a vehicle to a digital map to obtain the true position in the road network. Figure 1 Further included are illustrations of the various method steps 101 to 106 to the right of the flowchart blocks 101 to 106. In addition, the method 100 provides "lane level" matching, meaning that the vehicle position data is matched not only to a detailed road, but also to a detailed lane on that road.
[0028] Continuing, method 100 includes receiving vehicle data from, for example, a localization system of a vehicle. The vehicle data includes a geographic location (latitude, longitude), a vehicle orientation (yaw angle), and a speed of the vehicle. Vehicle data can be received 101 from a localization system of the vehicle (e.g., a global navigation satellite system, GNSS), an inertial measurement unit (IMU) or a combination of systems together with high-definition map data. Method 100 further includes receiving sensor data from a perception system of the vehicle. The sensor data includes at least information about the position of at least one road reference in the surrounding environment of the vehicle. The road reference may be, for example, a lane marking, a traffic sign, a road edge, a road obstacle, or any other suitable landmark. In addition, depending on the application and implementation selection, the position of one or more road references can be determined with reference to a local coordinate system of the vehicle or with reference to a global coordinate system. The perception system is understood in the current context as a system responsible for acquiring raw sensor data from sensors such as cameras, lidars, radars, ultrasonic sensors, and converting this raw data into scene understanding. Naturally, the sensor data may be received 102 directly from one or more suitable sensors, such as, for example, a camera, a lidar sensor, a radar, an ultrasonic sensor, or the like.
[0029] Additionally, with respect to road references, the sensor data may include lane marking data, which can include the distance to one or more lane markings (relative to the ego vehicle) and the type of lane marking (dashed, semantic, solid, etc.). The sensor data may further include lane marking geometry (which can be represented as a polynomial or a clothoid, depending on the camera software).
[0030] The method 100 also includes receiving 103 map data including lane geometry of the vehicle's surroundings. The lane geometry includes a set (i.e., a plurality of) candidate lanes. Candidate lanes are understood in the present context as drivable lanes on roads in the vehicle's surroundings, i.e., lanes on which the vehicle may or may not be drivable. The map data may be in the form of high-definition (HD) map data, i.e., the map data has a high degree of accuracy (centimeter level). HD maps are maps built specifically for robots that manipulate themselves around 3D space. In more detail, these maps need to be precise and include a lot of information that humans may take for granted. Not only should the map include information about the location of lanes, the location of road boundaries, one would also like to know the location of curves and the height of curves.
[0031] The lane geometry may be retrieved 103, for example, from a defined portion of the high definition map, wherein the defined portion may be all lane geometries within a circle of a predefined radius. The center of the circle may be determined, for example, based on the geographic location of the vehicle previously received 101. The radius may be defined based on a measurement error of the geographic location received 101, such as 20 meters or 50 meters in an urban canyon. To improve computational efficiency, the method may include forming a spatial database based on the map data received 103 and a spatial indexing method (e.g., Geohash, quadtree, M-tree, X-tree, R-tree, etc.). For example, an R-tree of the retrieved 103 lane geometry may be constructed and further compressed using a sorting method (e.g., Sort-Tile-Recursive (STR), Nearest-X, Overlap Minimization Top-Down (OMT), etc.). This spatial indexing and compression allows one to quickly find a set of candidate lanes from the high definition map received 103.
[0032] Next, a state space model is formed 104 comprising a set of states. Each state in the set of states represents a candidate lane in the set of candidate lanes (retrieved from map data). State space models are understood in the present context as a class of probabilistic graphical models that describe the probabilistic dependencies between latent state variables and observed measurements. States or measurements can be either continuous or discrete. State space models are used to provide a general framework for analyzing deterministic and stochastic dynamic systems measured or observed by stochastic processes.
[0033] More specifically, the method 100 may include forming a hidden Markov model (HMM), which is considered in the present disclosure to be a subtype of a state space model. In more detail, each candidate lane is considered to be represented by a hidden state in the HMM, and the received vehicle data and sensor data are used to estimate the hidden state, i.e., the lane in which the vehicle is actually traveling. Figure 2 The discussion relates to further details of hidden Markov models and embodiments of the present disclosure related thereto.
[0034] In addition, the method 100 includes defining 105 a cost from each state in a set of states in the state space model to each other state. The cost is calculated based on the received vehicle data, the received sensor data, and the received map data. In more detail, the cost can be defined 105 as a probability, i.e., the likelihood that the vehicle will go from a first state to a second state at time t or maintain a first state at time t.
[0035] Continuing, the method further includes determining 106 possible paths for the vehicle based on the formed state-space model and the defined costs. In other words, the possible paths for the vehicle are determined by calculating probabilities of a plurality of possible paths (state transitions), the probabilities being determined based on the sensor observations received 102 and the vehicle data received 101. For example, the calculations for determining 106 possible paths may be performed, for example, by a numerical optimization solver.
[0036] The initial state probabilities may also be calculated, i.e., the probability that the vehicle is in each state at time t = 0. For this calculation, one may use the vehicle data received 101, the sensor data received 102, and the high definition map acquired 103. Alternatively, one may assign equal probabilities to starting in each state.
[0037] In addition, method 100 may further include sending the possible paths determined 106 to a control system for controlling a driver assistance or autonomous driving feature of the vehicle, or to a map generation system for updating a map of the surrounding environment. With reference to the former, the possible paths may be used to activate, deactivate, or adjust specific ADAS or AD features of the vehicle based on which lane the vehicle is traveling in. In more detail, the possible paths may be used to more accurately estimate the position of the vehicle so that appropriate adjustments to the ADAS or AD features may be performed (e.g., adjusting the subsequent distance to guide the vehicle, adjusting the emergency braking threshold, etc.). With reference to a map generation system that may be local or remote to the vehicle, the possible paths can be used as input to identify new lanes or rerouted lanes. Therefore, a vehicle implementing the disclosed method may be used as a detector for map updates.
[0038] refer to Figure 1, depicts a stretch of road with three lanes x1, x2, x3. The first lane x1 and the second lane x2 are parallel and separated by a double lane marking. The second lane x2 is divided into the third lane x3. In the schematic diagram, the corresponding state transitions are illustrated as circles interconnected with solid arrows. Figure 1 In the illustrated use case of , at time t-1, there are only two possible lane candidates at times x1 and x2. In other words, at time t-1, the vehicle can only travel in one of the two possible lanes (retrieved from the map data received 103 with the help of the vehicle data received 101, the vehicle data including the GPS position with the associated measurement error radius as described above).
[0039] The arrows indicate possible transitions between lanes x1, x2, and x3. For example, since the vehicle is in lane x1 at time t-1, it may stay in lane x1, or it may change lanes from x1 to x2. Since the dual lane markings are real / imaginary, one could argue that the probability of the vehicle changing lanes from the first lane x1 to the second lane x2 is relatively small (because a lane change from the first lane x1 to the second lane x2 would mean that the vehicle violated traffic regulations, which is assumed to be less likely here). On the other hand, the vehicle is more likely to change lanes from x2 to x1 (perhaps there is a vehicle in front of it that slows down to turn to the exit lane x3). Of course, the car may also stay in its lane. Furthermore, in this example, the topology of the road network restricts us from going directly from the first lane x1 to the third lane x3 (without passing through the second lane x2 first).
[0040] In addition, by calculating the cost at time t>0 from any candidate lane at time t-1 to any other candidate state at time t, a possible path 106 can be determined. For this calculation, the vehicle data received 101 and the sensor data received 102 and the measurements included in the map data received 103 are used. The cost can be obtained, for example, by multiplying a value such as P(y t-1 |x1 t-1 )*P(x t |x1 t-1 ) to determine / calculate 105, P(y t-1 |x1 t-1 ) can be defined as the difference between the lane marking type reported by the perception system and the lane marking type of the candidate HD map, and P(x t |x1 t-1 ) may be defined as the difference between the measured vehicle orientation and the orientation of the candidate lanes in the HD map. An optimization algorithm (such as a maximum sum or maximum product algorithm) may then be used to determine 106 the most likely path for the vehicle.
[0041] As mentioned earlier, the candidate lanes can be represented as hidden states in a Hidden Markov Model (HMM). Therefore, Figure 2 A schematic graphical representation of a discrete HMM with three states s1, s2, s3 and three observations y1, y2, y3 with corresponding transmission probabilities T i,j , i,j∈{1,2,3} and emission probability E k,i , k∈{1,2,3}, i,j∈{1,2,3}.
[0042] Hidden Markov Model (HMM) can be interpreted as a statistical model in which the modeled system is assumed to be a random process with unobserved states, i.e. hidden states. The states are included in the set In Describes a sequence of states. More specifically, a set of states can be A state sequence x might look like x = [x1, x2, x3] = [s2, s2, s1], where x1, x2, x3 ∈ x, s1, The HMM states are not directly visible to the observer, which is why they are called hidden. Instead, there are observations from these hidden states. The observation sequence is generated by a second random process. In that sense, HMM is a double random process. HMM includes three main parameters: transition probability, emission probability, and initial state distribution. As a subcategory of Markov model, HMM also satisfies Markov property.
[0043] exist Figure 2 In FIG. 1 , each hidden state 21 , 22 , 23 represents a candidate lane obtained from the lane geometry in the received map data, and the bottom boxes 31 , 32 , 34 , 34 represent different “observations”, such as the position of lane markings, lane marking type, orientation, geographic location, etc. Thus, the received vehicle data and sensor data form a time t k The observation y k .
[0044] The cost from each state 21, 22, 23 in the set of states to each other state 21, 22, 23 is based on the received vehicle data and the received sensor data. The cost may be defined, for example, based on two probabilities, namely a first predefined probability E k and the second predefined probability T k According to an exemplary embodiment of the state space model in the form of a hidden Markov model, the first and second predefined probabilities may be referred to as the emission probability and the transmission probability, respectively. k Defined at x k state, at time t kMake observations k The transmission probability T k Defined at time t k From the first state x in a set of states k Change state to another state x in a set of states k+1 probability.
[0045] In more detail, the emission probability E k is associated with a second random process that models the distribution of the observations. k All have emission probabilities,
[0046] E k =P(y k |x k )x k ∈χ. #(1)
[0047] This is the probability distribution function, which, as mentioned, reflects the k At time t k Make observations k Probability k When considering explicit states, the emission probability is more appropriately expressed as:
[0048]
[0049] The transmission probability T, as mentioned, is the k From the first state x in a set of states k Change state to another state x k+1 The probability of . It can be written as:
[0050] T k =P(x k+1 |x k )x k ,x k+1 ∈χ. #(3)
[0051] When considering explicit states, the transition probabilities are more appropriately expressed as:
[0052]
[0053] The distribution of initial state probabilities describes the likelihood of starting in each state:
[0054]
[0055] According to an exemplary embodiment of the present disclosure, the sensor data includes lane marking data. The lane marking data includes at least a distance to at least one lane marking and a lane marking type of at least one lane marking. In addition, the vehicle data also includes a yaw rate of the vehicle (obtained from, for example, an inertial measurement unit (IMU) or a steering wheel angle sensor of the vehicle). Accordingly, the first predefined probability (emission probability) E k The emission probability E may be based on one or more of a distance to the at least one lane marking, a lane marking type of the at least one lane marking, a geographic location of the vehicle, and a speed of the vehicle. k The transmission probability T may naturally be based on further parameters such as, for example, the location and type of other landmarks (e.g. traffic signs), lane text (e.g. bus lane), the location of other vehicles in the vehicle's surroundings, the confidence of the left / right lane markings, the distance between the left and right markings, etc. On the other hand, the second predefined probability (transmission probability) T k May be based on the vehicle's yaw rate.
[0056] In more detail, yaw rate can be used as a lane change indicator because the steering angle over time during a lane change roughly resembles a sine wave. This wave can be modeled by fitting a parameterized sine function s(ω) to the yaw rate ω:
[0057] s(ω)=a·sin(fω+p)+c #(6)
[0058] The yaw rate corresponding to the left lane change has an opposite sign compared to the right lane change. Naturally, the transmission probability can be further based on other lane change indicators to improve redundancy. For example, visual data obtained from the vehicle perception system can be used to estimate when the lane change occurs (e.g., based on the lane marking type, the distance to the lane marking, the change in the distance to the lane marking, etc.). In addition, a lane tracking model (e.g., represented as a polynomial or a clothoid) can be used as a lane change indicator. In more detail, the lateral offset parameter (typically represented as a0 for polynomial lane boundary representation) can be used to indicate a lane change. For example, if the lateral offset parameter of the right lane trajectory (the lateral offset between the vehicle and the lane trajectory) becomes smaller and smaller over time, a right lane change is likely to occur.
[0059] Discrete HMM can be represented as a trellis diagram where time steps t1-t4 are merged, as Figure 3 Each node 30a-30l in the diagram corresponds to a different state s1-s3 at a given time t1-t4, and the edges (interconnecting lines) represent possible state transitions at the next time step t1-t4. The edge weights are chosen to be proportional to the transition and emission probabilities T k 、E kIn the example shown, some of these products are zero, so the edges are eliminated to avoid clutter. A useful property of this particular representation is that every possible sequence of states in the model corresponds to a unique path through the lattice, and vice versa. Because of this, it is a useful representation when applying dynamic programming algorithms (such as the Viterbi algorithm) to HMMs to find the most likely path through the model.
[0060] In more detail, we can say that inferring which sequence of states produces a particular sequence of observations is called decoding. The Viterbi algorithm is an example of such a decoder. More specifically, we can say that the Viterbi algorithm solves the problem of estimating the maximum likelihood of a sequence of states, i.e., it finds the most likely sequence of states. The set of transition sequences can be defined as ξ = {ξ1,…,ξ K-1}, with a transition ξ at a specified time k k ={x k+1 ,x k}. These are mapped one-to-one to the state sequence x = {x1,…,x K}. Use this notation to refer to Figure 2 The observation in question y={y1,…,y K}、 can be described as the output of a channel whose input is a sequence of transformations. The channel is memoryless, that is, That is, each observation depends only on the transition ξ in probability k .
[0061] Executable instructions for performing these functions are optionally included in non-transitory computer-readable storage media or other computer program products configured to be executed by one or more processors.
[0062] Figure 4 is a schematic side view of a vehicle 1 including a control device 10 for lane level map matching. The vehicle 1 further includes a perception system 6, an inertial measurement unit (IMU) 7 and a localization system 5. The perception system 6 is understood in the current context as a system responsible for acquiring raw sensor data from sensors 6a, 6b, 6c such as cameras, lidars and radars, ultrasonic sensors, and converting this raw data into scene understanding. The localization system 5 is configured to monitor the geographic location and orientation of the vehicle and can be in the form of a global navigation satellite system (GNSS), such as GPS. However, the localization system can alternatively be implemented as a real-time dynamic (RTK) GPS to improve accuracy. The IMU 7 is understood as an electronic device configured to measure the inertial motion of the vehicle 1. The IMU 7 typically has six degrees of freedom, three accelerometers and three gyroscopes.
[0063] The control device 10 includes one or more processors 11, a memory 12, a sensor interface 13, and a communication interface 14. The processor 11 may also be referred to as a control circuit 11 or a control loop 11. The control circuit 11 is configured to execute instructions stored in the memory 12 to perform a lane-level map matching method for a vehicle according to any of the embodiments disclosed herein. In other words, the memory 12 of the control device 10 can include one or more (non-temporary) computer-readable storage media for storing computer-executable instructions, for example, when the less common executable instructions are executed by one or more computer processors 11, they can cause the computer processor 11 to perform the technology described herein. The memory 12 optionally includes a high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes a non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.
[0064] In more detail, the control circuit 11 is configured to receive vehicle data, including the geographic location of the vehicle, the orientation of the vehicle, and the speed of the vehicle. For example, the vehicle data can be obtained from the GPS unit of the vehicle 1. The control circuit 11 is further configured to receive sensor data from the perception system 6 of the vehicle 1. The sensor data includes information about the position of at least one road reference in the surrounding environment of the vehicle. The position of the road reference can be referenced to the vehicle, or in a "global" coordinate system, depending on the specification. In more detail, the perception system 6 is preferably a forward-looking camera 6c configured to detect lane markings on the road. Conventional automotive-grade cameras are capable of detecting lane markings within a range of 25 meters. The detection gives information about the distance to the nearest markings on the left and right sides of the vehicle and their corresponding types. The types of markings that the system 6 or camera 6c can recognize include, for example, real and virtual. Once the lane marking is detected, the perceived projected image can be transformed into its corresponding bird's eye vision.
[0065] The control circuit 11 is further configured to receive map data including lane geometry of the vehicle's surroundings. The lane geometry includes a set of candidate lanes. The map data may be in the form of a high-definition map including information about a road with multiple parallel lanes, each lane having a centerline represented as a polyline. Polylines are typically two-dimensional, defined by the longitude and latitude of the start and end points of each line segment. In addition, left and right lane markings representing lane boundaries are polylines. They are also associated with their marking type (e.g., virtual or real). The map data may also include road separators such as guardrails, speed limits, lane orientations, etc.
[0066] In addition, the control circuit 11 is configured to form a state space model including a set of states. Each state in the set of states represents a candidate lane in the set of candidate lanes. Next, the control circuit 11 is configured to define a cost from each state in the set of states to each other state based on the received vehicle data, the received sensor data, and the received map data. Various embodiments for calculating costs have been discussed in detail above and are similarly applicable to this aspect of the present disclosure.
[0067] Furthermore, the control circuit 11 is configured to determine a possible path for the vehicle based on the formed state space model and the defined costs. In other words, the control circuit 11 is configured to calculate the most likely path for the vehicle 1 to travel based on the formed state space model and the defined costs of moving between states (i.e., moving between lanes). Although the control device 11 is illustrated here as an onboard system, some or all of the components may be located remotely from the vehicle (e.g., a cloud-based solution) to increase computing power.
[0068] In addition, the vehicle 1 can be connected to an external network 2 via, for example, a wireless link (for example, in order to retrieve map data). The same or some other wireless link can be used to communicate with other vehicles 2 in the vicinity of the vehicle or with local infrastructure elements. Cellular communication technology can be used for long-range communications such as to external networks, and if the cellular communication technology used has low latency, it can also be used for communication between vehicles, vehicle to vehicle (V2V), and / or vehicle to infrastructure, V2X. Examples of cellular radio technologies include GSM, GPRS, EDGE, LTE, 5G, 5G NR, etc., also including future cellular solutions. However, in some solutions, short and medium-range communication technologies are used, such as wireless local area networks (LANs), such as solutions based on IEEE 802.11. ETSI is working on cellular standards for vehicle communications, and 5G is seen as a suitable solution, for example due to high bandwidth and low latency and efficient processing of communication channels.
[0069] The contents of the present disclosure have been introduced above with reference to detailed embodiments. However, other embodiments than those described above are possible and within the scope of the present disclosure. Method steps different from those described above may be provided within the scope of the present disclosure, and the method may be performed by hardware or software. Therefore, according to an exemplary embodiment, a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a vehicle control system is provided, and the one or more programs include instructions for performing a method according to any of the embodiments discussed above. Alternatively, according to another exemplary embodiment, a cloud computing system can be configured to perform any of the methods presented herein. The cloud computing system may include distributed cloud computing resources that jointly perform the methods presented herein under the control of one or more computer program products.
[0070] In general, computer-accessible media may include any tangible or non-transitory storage media or memory media, such as electronic, magnetic or optical media - for example, a disk or CD / DVD-ROM connected to a computer system via a bus. As used herein, the terms "tangible" and "non-transitory" are intended to describe computer-readable storage media (or "memory") that exclude propagating electromagnetic signals, but are not intended to otherwise limit the types of physical computer-readable storage devices covered by the phrase computer-readable media or memory. For example, the term "non-transitory computer-readable media" or "tangible memory" is intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in non-transitory form on tangible computer-accessible storage media may be further transmitted by transmission media or signals such as electrical, electromagnetic or digital signals, and may be transmitted via communication media such as networks and / or wireless links.
[0071] Processor 11 (associated with control device 10) can be or include any number of hardware components for performing data or signal processing or for executing computer code stored in memory 12. Device 10 has associated memory 12, and memory 12 can be one or more devices for storing data and / or computer code, which are used to complete or promote the various methods described in this specification. The memory may include volatile memory or non-volatile memory. The memory 12 may include database components, object code components, script components, or any other type of information structure for supporting various activities of this specification. According to an exemplary embodiment, any distributed or local memory device can be used with the system and method of this specification. According to an exemplary embodiment, the memory 12 is communicatively connected to the processor 11 (for example, via a circuit or any other wired, wireless or network connection) and includes computer code for executing one or more processes described herein.
[0072] It should be understood that the sensor interface 13 may also provide the possibility of acquiring sensor data directly or via a dedicated sensor control circuit 6 in the vehicle. The communication / antenna interface 14 may further provide the possibility of sending the output to a remote location (e.g. a remote operator or a control center) via the antenna 8. In addition, some sensors in the vehicle may communicate with the control device 10 using a local network setting, such as a CAN bus, I2C, Ethernet, optical fiber, etc. The communication interface 14 may be arranged to communicate with other control functions of the vehicle and may therefore also be considered a control interface; however, a separate control interface (not shown) may be provided. Local communications within the vehicle may also be of a wireless type with protocols such as WiFi, LoRa, Zigbee, Bluetooth or similar medium / short range technologies.
[0073] Accordingly, it should be understood that parts of the described solutions may be implemented in the vehicle, in a system located external to the vehicle, or in a combination of internal and external to the vehicle; for example, in a server in communication with the vehicle, a so-called cloud solution. For example, sensor data may be sent to an external system and that system performs the steps that define the cost of going from one state to another. The different features and steps of the embodiments may be combined in other combinations than those described.
[0074] Exemplary methods, computer readable storage media, control devices, and vehicles are set forth in the following items:
[0075] 1. A method for lane-level map matching of a vehicle, the method comprising:
[0076] receiving vehicle data, the vehicle data including a geographic location of the vehicle, a direction of the vehicle, and a speed of the vehicle;
[0077] receiving sensor data from a perception system of the vehicle, the sensor data comprising information regarding a location of at least one road reference in a surrounding environment of the vehicle;
[0078] receiving map data including lane geometry of a surrounding environment of a vehicle, the lane geometry including a set of candidate lanes;
[0079] forming a state space model comprising a set of states, wherein each state in the set of states represents a candidate lane for the set of candidate vehicles;
[0080] defining a cost from each state in a set of states to each other state based on the received vehicle data and the received sensor data;
[0081] Based on the formed state-space model and the defined costs, possible paths for the vehicle are determined.
[0082] 2. The method of claim 1, wherein each state is a hidden state in a hidden Markov model, and wherein the cost is defined based on:
[0083] When in state x m At time t k Make observations k The first predefined probability E k ;
[0084] At time t k From the first state x in a set of states k Change state to another state x in a set of states k+1 The second predefined probability T k .
[0085] 3. The method according to item 2, wherein the sensor data comprises lane marking data, the lane marking data comprising a distance to at least one lane marking and a lane marking type of at least one lane marking;
[0086] Wherein the vehicle data further includes a yaw rate of the vehicle;
[0087] The first predefined probability E k based on at least one of a distance to at least one lane marking, a lane marking type of at least one lane marking, a geographic location of the vehicle, and a speed of the vehicle;
[0088] The second predefined probability T k Based on the vehicle's yaw rate.
[0089] 4. The method according to item 1 or 2, wherein the sensor data comprises lane marking data, the lane marking data comprising a distance to at least one lane marking and a type of at least one lane marking.
[0090] 5. A method according to item 1, 2 or 3, wherein the vehicle data further comprises a yaw rate of the vehicle.
[0091] 6. The method according to any one of items 1, 2 or 3, further comprising:
[0092] A spatial database is formed based on map data and spatial indexing methods.
[0093] 7. The method according to any one of items 1, 2 or 3, further comprising:
[0094] sending the possible paths to a control system to control a driver assistance or autonomous driving feature of the vehicle based on the possible paths; or
[0095] Send the possible paths to the map generation system to update the map of the surrounding environment.
[0096] 8. A method according to item 1, 2 or 3, wherein the step of determining possible paths comprises decoding the formed state space model to estimate the maximum likelihood of multiple state sequences and determine the most likely state sequence.
[0097] 9. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a vehicle control system, the one or more programs comprising instructions for executing the method according to any one of items 1 to 3.
[0098] 10. A control device comprising:
[0099] at least one processor;
[0100] at least one memory;
[0101] at least one sensor interface;
[0102] at least one communication interface;
[0103] At least one of the processors is configured to execute instructions stored in the memory to perform a method for lane level map matching for a vehicle, wherein the at least one processor is configured to:
[0104] receiving vehicle data, including a geographic location of the vehicle, a heading of the vehicle, and a speed of the vehicle;
[0105] receiving sensor data from a perception system of the vehicle, the sensor data comprising information regarding a location of at least one road reference in a surrounding environment of the vehicle;
[0106] receiving map data including lane geometry of a surrounding environment of a vehicle, the lane geometry including a set of candidate lanes;
[0107] forming a state space model comprising a set of states, wherein each state in the set of states represents a candidate lane;
[0108] defining a cost from each state in a set of states to each other state based on the received vehicle data and the received sensor data;
[0109] Based on the formed state-space model and the defined costs, possible paths for the vehicle are determined.
[0110] 11. The control device of claim 10, wherein each state is a hidden state of a hidden Markov model, and wherein the control circuit is configured to define a cost based on:
[0111] When in state x k At t k Make observations k The first predefined probability E k , the state is represented by a candidate lane in the set of candidate lanes; and
[0112] When at time t k From the first state x in a set of states k Change state to another state x in a set of states k+1 The second predefined probability T k .
[0113] 12. The control device according to item 10, wherein the sensor data comprises lane marking data, the lane marking data comprising a distance to at least one lane marking and a lane marking type of at least one lane marking;
[0114] Wherein the vehicle data further includes a yaw rate of the vehicle;
[0115] The first predefined probability E k Based on at least one of a distance to the at least one lane marking, a lane marking type of the at least one lane marking, a geographic location of the vehicle, and a speed of the vehicle.
[0116] The second predefined probability T k Based on the vehicle's yaw rate.
[0117] 13. The control device according to item 11 or 12, wherein the control circuit is further configured to:
[0118] A spatial database is formed based on map data and spatial indexing methods.
[0119] 14. The control device according to item 11 or 12, wherein the control circuit is further configured to:
[0120] sending the possible paths to a control system to control a driver assistance or autonomous driving feature of the vehicle based on the possible paths; or
[0121] Send the possible paths to the map generation system to update the map of the surrounding environment.
[0122] 15. A vehicle comprising:
[0123] A perception system, the perception system comprising at least one sensor for monitoring the surrounding environment of the vehicle;
[0124] An inertial measurement unit (IMU), which is used to measure the inertial motion of the vehicle;
[0125] Localization systems that monitor the geographic location and orientation of the vehicle; and
[0126] A control device according to any one of items 10 to 12.
[0127] It should be noted that the word "comprising" does not exclude the presence of other elements or steps than those listed, and the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. It should be further noted that any reference signs do not limit the scope of the claims, the present invention may be implemented at least in part by hardware and software, and several "components" or "units" may be represented by the same item of hardware.
[0128] Although the accompanying drawings can show a specific order of method steps, the order of steps may be different from that described. In addition, two or more steps can be performed in parallel or partially in parallel. For example, the step of receiving a signal including information about movement and information about current road scenario can be interchanged based on a specific implementation. Such changes will depend on the selected software and hardware system and depend on the designer's choice. All such changes are within the scope of the present disclosure. Similarly, software embodiments can be implemented with standard programming techniques, which have rule-based logic and other logic that implements various connection steps, processing steps, comparison steps and decision steps. The embodiments mentioned above and described are given only as examples, and should not be restrictive for the present invention. Other solutions, uses, targets and functions within the scope of the present invention as claimed in the patent embodiments described below should be apparent to those skilled in the art.
Claims
1. A method for lane-level map matching of a vehicle, the method comprising: receiving vehicle data, the vehicle data comprising a geographic location of the vehicle, a heading of the vehicle, and a speed of the vehicle; receiving sensor data from a perception system of the vehicle, the sensor data comprising information regarding a location of at least one road reference in a surrounding environment of the vehicle; receiving map data including lane geometry of a surrounding environment of the vehicle, the lane geometry including a set of candidate lanes; forming a state space model comprising a set of states, wherein each state in the set of states represents a candidate lane in the set of candidate lanes; defining a cost from each state in the set of states to each other state based on the received vehicle data, the received sensor data, and the received map data; determining possible paths for the vehicle based on the formed state-space model and the defined costs; Determining the possible paths includes decoding the formed state space model to estimate the maximum likelihood of multiple state sequences and determine the most likely state sequence.
2. The method of claim 1, wherein each state is a hidden state in a Hidden Markov Model, and wherein the cost is defined based on: When in state x m At time t k Make observations k The first predefined probability E k ;as well as At the time t k From the first state x in the set of states k Change the state to another state x in the set of states k+1 The second predefined probability T k .
3. The method of claim 2, wherein the sensor data comprises lane marking data, the lane marking data comprising a distance to at least one lane marking and a lane marking type of the at least one lane marking; wherein the vehicle data further includes a yaw rate of the vehicle; The first predefined probability E k based on at least one of the distance to the at least one lane marking, the lane marking type of the at least one lane marking, the geographic location of the vehicle, and the speed of the vehicle; The second predefined probability T k Based on the yaw rate of the vehicle. 4 . The method according to claim 1 , wherein the sensor data comprises lane marking data, the lane marking data comprising a distance to at least one lane marking and a type of the at least one lane marking.
5. The method of claim 1 or 2, wherein the vehicle data further comprises a yaw rate of the vehicle.
6. The method according to claim 1 or 2, further comprising: A spatial database is formed based on the map data and the spatial index method.
7. The method according to claim 1 or 2, further comprising: sending the possible paths to a control system to control driver assistance or autonomous driving features of the vehicle based on the possible paths; or The possible paths are sent to a map generation system to update a map of the surrounding environment.
8. A non-transitory computer-readable storage medium storing one or more programs, wherein the one or more programs are configured to be executed by one or more processors of a vehicle control system, and the one or more programs include instructions for executing the method according to any one of claims 1 to 2.
9. A control device for lane level map matching of a vehicle, the control device comprising a control circuit, the control circuit being configured to: receiving vehicle data, the vehicle data comprising a geographic location of the vehicle, a heading of the vehicle, and a speed of the vehicle; receiving sensor data from a perception system of the vehicle, the sensor data comprising information regarding a location of at least one road reference in a surrounding environment of the vehicle; receiving map data including lane geometry of a surrounding environment of the vehicle, the lane geometry including a set of candidate lanes; forming a state space model comprising a set of states, wherein each state in the set of states represents a candidate lane in the set of candidate lanes; defining a cost from each state in the set of states to each other state based on the received vehicle data, the received sensor data, and the received map data; determining possible paths for the vehicle based on the formed state-space model and the defined costs; When determining the possible path, the control circuit is further configured to: decode the formed state space model to estimate the maximum likelihood of multiple state sequences and determine the most likely state sequence.
10. The control device of claim 9, wherein each state is a hidden state of a Hidden Markov Model, and wherein the control circuit is configured to define the cost based on: When in state x k At time t k Make observations k The first predefined probability E k , the state is represented by a candidate lane in the set of candidate lanes; and At time t k From the first state x in the set of states k Change the state to another state x in the set of states k+1 The second predefined probability T k .
11. The control device according to claim 10, wherein the sensor data comprises lane marking data, the lane marking data comprising a distance to at least one lane marking and a lane marking type of the at least one lane marking; wherein the vehicle data further includes a yaw rate of the vehicle; The first predefined probability E k based on at least one of the distance to the at least one lane marking, the lane marking type of the at least one lane marking, a geographic location of the vehicle, and a speed of the vehicle; The second predefined probability T k Based on the yaw rate of the vehicle.
12. The control device according to any one of claims 10 or 11, wherein the control circuit is further configured to: A spatial database is formed based on the map data and the spatial index method.
13. The control device according to any one of claims 10 or 11, wherein the control circuit is further configured to: sending the possible paths to a control system to control driver assistance or autonomous driving features of the vehicle based on the possible paths; or The possible paths are sent to a map generation system to update a map of the surrounding environment.
14. A vehicle comprising: a perception system comprising at least one sensor for monitoring the surrounding environment of the vehicle; An inertial measurement unit IMU, wherein the inertial measurement unit IMU is used to measure the inertial motion of the vehicle; a localization system for monitoring the geographic location and orientation of the vehicle; as well as A control device according to any one of claims 9 to 11.
Citation Information
Patent Citations
Lane-level map matching
EP3748300A1
Vehicle yaw detection device
CN102336163A
HMM-based V2X internet-of-vehicles vehicle anti-collision early warning system
CN109686125A
Inference-aware motion planning
US10074279B1
Systems and Methods for Precise Sub-Lane Vehicle Positioning
US20120150437A1
