ESTIMATION OF A VIRTUAL LANE USING A RECURSIVE SELF-ORGANIZING MAP

Abstract

Virtual lane estimation system (100), including: a storage device (104) configured to store a road map (170) corresponding to a section of a road in front of a vehicle (90); a sensor (102) configured to observe a plurality of trajectories of a plurality of adjacent vehicles crossing the section of road; and a computer (106) trained to initialize a recursive self-organizing map (168) as a plurality of points arranged as a two-dimensional grid aligned with the road map (170), to train the plurality of points in the recursive self-organizing map (168) in response to the plurality of trajectories, to generate a directed graph containing one or more virtual lanes through the road map (170) in response to the points trained on the plurality of trajectories, and to generate a control signal that controls the navigation of the vehicle (90) through the section of the road in response to the one or more virtual lanes in the directed graph.

Description

[0001] The present disclosure relates to a system and method for estimating a virtual lane using a recursive self-organizing map.

[0002] Lane detection of lane markings is a component of many autonomous vehicles. Lane departure warning systems are available in many commercial vehicles. However, commercial warning systems rely on small visual cues and have many false positive detections. Such systems are designed to assist the human driver, not to be fully autonomous. While more accurate lane detection systems are being developed in the autonomous vehicle market, these systems still do not account for cases where lane markings are clear but conditions make normal driving in those lanes difficult. Furthermore, unless autonomous vehicles can detect lane markings or encounter unexpected situations, they may not have a model for reasonable driving behavior.

[0003] DE 10 2018 215 055 A1 discloses a method for determining a lane change indication of a vehicle. The method comprises loading a lane change model and position data of the vehicle. The position data indicate a distance of the vehicle from a lane center of a lane. The method further comprises determining a lane change indication. The determination is performed using the position data and the lane change model.

[0004] US 2016 / 0 358 349 A1 discloses a method for defining a bidirectional road geometry based on a plurality of measurement points. An orientation representative of one or more measurement points is determined, and then a seed point representative of one or more measurement points is determined. The seed point is based on determining a constrained weighted center of gravity of the one or more measurement points. The movement of the constrained weighted center of gravity is limited to a direction based on the respective directions of the measurement points. Polylines representative of a bidirectional road geometry are created.The polylines are created to represent one direction of travel along the bidirectional road geometry by extending a first polyline in the direction from the starting point and a second polyline in an opposite direction from the starting point.

[0005] DE 11 2016 007 047 T5 discloses a lane estimation system using a map database containing the shape of each lane. The system includes a GNSS receiver, a vehicle speed calculator, an angular velocity measuring device, a subject vehicle position determining unit that calculates a reference coordinate of the vehicle and a reference orientation of the vehicle from a GNSS coordinate, and time series data of vehicle positions.The system further includes a map information storage unit that stores map information including position information about a dividing line of each lane; and a lane estimation unit that calculates, as an optimal correction value, an error pattern having a posterior probability that is the highest among a plurality of error patterns, corrects the time series data using the optimal correction value, and compares the as-corrected time series data with the dividing line to estimate the vehicle's lane. The posterior probability is calculated using a product of a prior occurrence probability of the time series data with the plurality of error patterns and a likelihood calculated under the condition that the error pattern has occurred based on a relative positional relationship between the time series data after correction and the dividing line.

[0006] What is desired is a system or technology for virtual lane estimation that determines drivable lanes by learning from the behavior of other vehicles.

[0007] A virtual lane estimation system is provided here. The virtual lane estimation system includes a storage device, a sensor, and a computer. The storage device is configured to store a road map corresponding to a section of road ahead of a vehicle. The sensor is configured to observe a plurality of trajectories of a plurality of neighboring vehicles traversing the section of road.The computer is configured to initialize a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map, train the plurality of points in the recursive self-organizing map in response to the plurality of trajectories, generate a directed graph including one or more virtual lanes through the road map in response to the points trained to the plurality of trajectories, and generate a control signal that controls navigation of the vehicle through the portion of the road in response to the one or more virtual lanes in the directed graph.

[0008] In one or more embodiments of the virtual lane estimation system, training the recursive self-organizing map uses the plurality of trajectories individually.

[0009] In one or more embodiments of the virtual lane estimation system, a recurrent layer of the recursive self-organizing map includes a plurality of weights of the plurality of points at one end of the training.

[0010] In one or more embodiments of the virtual lane estimation system, a grid layer of the recursive self-organizing map contains the plurality of trajectories at the end of training.

[0011] In one or more embodiments of the virtual lane estimation system, the computer is further configured to select a random trajectory from the plurality of trajectories during training.

[0012] In one or more embodiments of the virtual lane estimation system, the computer is further configured to isolate a first point of the random trajectory.

[0013] In one or more embodiments of the virtual lane estimation system, the computer is further configured to apply the first point to the recursive self-organizing map during training.

[0014] In one or more embodiments of the virtual lane estimation system, the computer is further configured to isolate a second point of the random trajectory and apply the second point to the recursive self-organizing map during training.

[0015] In one or more embodiments of the virtual lane estimation system, the recursive self-organizing map is a self-organizing map having an input layer from a current time step, and the input layer is an output layer from a previous time step, and the self-organizing map implements a neural network, wherein the output layer is arranged as a two-dimensional grid.

[0016] A method for estimating a virtual lane is provided herein. The method includes reading a road map corresponding to a section of road ahead of a vehicle, initializing a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map using a computer of the vehicle, and observing a plurality of trajectories of a plurality of neighboring vehicles traversing the section of road using a sensor of the vehicle.The method further includes training the plurality of points in the recursive self-organizing map in response to the plurality of trajectories, generating a directed graph including one or more virtual lanes through the road map in response to the points trained on the plurality of trajectories, and controlling a drive to navigate the vehicle through the portion of the road in response to the one or more virtual lanes in the directed graph.

[0017] In one or more embodiments of the method, training the recursive self-organizing map uses the plurality of trajectories individually.

[0018] In one or more embodiments of the method, a recurrent layer of the recursive self-organizing map includes a plurality of weights of the plurality of points at one end of the training.

[0019] In one or more embodiments of the method, a grid layer of the recursive self-organizing map contains the plurality of trajectories at the end of training.

[0020] In one or more embodiments of the method, training includes selecting a random trajectory from the plurality of trajectories.

[0021] In one or more embodiments of the method, training includes removing a first point of the random trajectory.

[0022] In one or more embodiments of the method, training includes applying the first point to the recursive self-organizing map.

[0023] In one or more embodiments of the method, training includes removing a second point of the random trajectory and applying the second point to the recursive self-organizing map.

[0024] In one or more embodiments of the method, the recursive self-organizing map comprises a self-organizing map with an input layer from a current time step, and the input layer is an output layer from a previous time step.

[0025] In one or more embodiments of the method, the self-organizing map implements a neural network, wherein the output layer is arranged as a two-dimensional grid.

[0026] A vehicle is provided herein. The vehicle includes a storage device, a sensor, a computer, and a power plant. The storage device is configured to store a road map corresponding to a section of road ahead of the vehicle. The sensor is configured to observe a plurality of trajectories of a plurality of neighboring vehicles traversing the section of road.The computer is configured to initialize a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map, train the plurality of points in the recursive self-organizing map in response to the plurality of trajectories, generate a directed graph including one or more virtual tracks through the road map in response to the points trained to the plurality of trajectories, and generate a control signal that controls navigation of the vehicle through the portion of the road in response to the one or more virtual tracks in the directed graph. The power plant is configured to propel the vehicle in response to the control signal. Fig. Figure 1 is a schematic floor plan diagram showing a vehicle implementing an autonomous driving system. Fig. 2 is a schematic diagram of software in a computer in accordance with an example embodiment of the autonomous driving system. Fig. 3 is a flowchart of an estimation method in accordance with an exemplary embodiment of the autonomous driving system. Fig. 4 is a schematic representation of an environment around a map manager according to an exemplary embodiment of the autonomous driving system. Fig. 5 is a schematic plan view of a portion of a road according to an exemplary embodiment of the autonomous driving system. Fig. 6 is a flowchart of a training sequence in accordance with an example embodiment of the autonomous driving system. Fig. 7 is a flowchart of a processing method in accordance with an exemplary embodiment of the autonomous driving system.

[0027] Embodiments of the disclosure implement a virtual lane estimation system for autonomous vehicles by observing the past trajectories of other vehicles traveling on the same road. A virtual lane can be defined as an optional path for moving vehicles. The optional path can be useful in situations where lane markings on the road are unclear or in unexpected road conditions (e.g., an obstacle on the road) that cause vehicles to bypass standard traffic rules of lane travel.

[0028] The system generally uses a recursive self-organizing map (SOM), which takes one or more observed vehicle trajectories as input and represents an averaged trajectory in the form of a directed graph. Using the recursive self-organizing map, the system can find drivable lanes and capture temporal data such as speed, direction, and lane-change preferences.

[0029] The nodes of the directed graph can represent physical coordinates along the trajectory. The directed edges of the directed graph can capture a sequential order in which the nodes were observed. The directed graph can then be used in graph-based techniques to control the movement of the autonomous vehicle and / or predict the trajectories of other vehicles in the future.

[0030] Referring to Fig. 1 illustrates a schematic plan view of a vehicle 90. The vehicle 90 generally includes a plurality of wheels 92a-92d and an autonomous driving system 100. The autonomous driving system 100 generally includes a sensor 102, a memory device 104, a computer 106, and a power plant 108.

[0031] A signal (e.g., SEN) may be generated by sensor 102 and transmitted to computer 106. Sensor signal SEN may contain information about neighboring vehicles in a section of road ahead of vehicle 90. A signal (e.g., CMD) may be exchanged between computer 106 and power plant 108. Command signal CMD may convey commands to power plant 108 to control at least the speed, steering, and braking of vehicle 90. A signal (e.g., DATA) may be exchanged between computer 106 and power plant 108. Data signal DATA may convey data relating to the operation of power plant 108 between power plant 108 and computer 106. A signal (e.g., MEM) may be exchanged between computer 106 and memory device 104. Memory signal MEM may convey information defining the roads in an area used by vehicle 90.In various embodiments, the information may include a road map of a physical road (or surface). Other types of information may be implemented to meet the design criteria of a particular application.

[0032] The vehicle 90 may be embodied as an automobile (or car). In various embodiments, the vehicle 90 may be, but is not limited to, a passenger vehicle, a truck, an autonomous vehicle, a gas-powered vehicle, an electric-powered vehicle, a hybrid vehicle, and / or a motorcycle. Other types of vehicles 90 may be implemented to meet the design criteria of a particular application.

[0033] The wheels 92a-92d may be configured as road wheels. The wheels 92a-92d are generally operable to enable movement of the vehicle 90 around the ground. In various embodiments, each wheel 92a-92d may include a tire mounted on a rim. The wheels 92a-92d may be used to provide traction between the vehicle 90 and the ground on which the vehicle 90 rests.

[0034] The autonomous driving system 100 may implement a power plant and associated electronics suitable for autonomous, semi-autonomous, and / or driver-assisted operation. The autonomous driving system 100 is generally operable to read a road map corresponding to a section (or segment) of the road in front of (or around) the vehicle 90 and initialize a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map. In various embodiments, the road map may range from a few dozen meters to several hundred meters in a direction of travel of the vehicle 90. The road map may extend in front of the vehicle 90 and / or behind the vehicle 90. The road map may also extend from a few dozen meters to a width of approximately 100 meters.The road map may extend to the left of the vehicle 90 and / or to the right of the vehicle 90.

[0035] The autonomous driving system 100 may further operate to observe one or more trajectories of one or more neighboring vehicles traversing the road segment represented by the road map and train the points in the recursive self-organizing map in response to the trajectories. A directed graph containing one or more virtual lanes through the road map may be generated by the autonomous driving system 100 in response to the points along the trajectories. The autonomous driving system 100 may then navigate the vehicle 90 across the road corresponding to the road map in response to the virtual lanes in the directed graph.

[0036] Sensor 102 may include an optical sensor and / or a radio frequency sensor. Sensor 102 is generally operative to detect the movement of neighboring vehicles moving through the portion of the roadway to be traversed by vehicle 90. In various embodiments, sensor 102 may implement one or more cameras configured to observe the neighboring vehicles in front of vehicle 90. In some embodiments, sensor 102 may implement a radar device configured to track the neighboring vehicles. In other embodiments, sensor 102 may be a transceiver configured to collect vehicle tokens generated by the neighboring vehicles. Each vehicle token may identify a particular neighboring vehicle and a corresponding position.The vehicle tokens may be collected from vehicle to vehicle and / or from a database via a wireless network (e.g., a cellular network). In various embodiments, sensor 102 may also be capable of detecting and reporting lane markings painted on the road, if they are visible.

[0037] The storage device 104 may implement a non-volatile memory device. The storage device 104 is generally operable to store and present the road map information to the computer 106. The road map information may be presented to the computer 106 via the memory signal MEM. The road map information stored in the storage device 104 may be updated from time to time via the memory signal MEM to reflect changes to the roads.

[0038] The computer 106 may implement one or more electronic control units. The computer 106 is generally capable of collecting the sensor information from the sensor 102, reading a local portion of the road map from the storage device 104, and determining one or more virtual lanes available to the vehicle 90. The computer 106 may encode the virtual lanes into a directed graph. In one or more embodiments, the computer 106 may then use the information about the virtual lanes in the directed graph to autonomously navigate the vehicle 90 across the road according to the road map, semi-autonomously assist in navigating the vehicle 90, and / or assist the driver in crossing the road.

[0039] The engine 108 may implement an engine, transmission, and drivetrain of the vehicle 90. The engine 108 is generally operable to propel and steer the vehicle 90 in response to one or more commands received from the computer 106 in the CMD signal. The engine 108 may provide operating information back to the computer 106 in the DATA signal.

[0040] With reference to Fig. 2 shows a schematic diagram of an example architecture of the software in computer 106 according to an example embodiment of autonomous driving system 100. The software in computer 106 generally includes a hardware abstraction layer 120, a high-level cognitive system 122, and a sensor abstraction layer 124. Sensor abstraction layer 124 may include a map manager 126. The command signal CMD, the data signal DATA, and the sensor signal SEN may be coupled to hardware abstraction layer 120. The memory signal MEM may be coupled to map manager 126.

[0041] The hardware abstraction layer 120 is generally responsible for enabling the receipt of information into the computer 106 and the transmission of information from the computer 106. The hardware abstraction layer 120 may be responsible for receiving the sensor data in the SEN signal from the sensor 102. The hardware abstraction layer 120 may exchange the command and data with the power plant 108 via the CMD signal and the DATA signal.

[0042] The high-level cognitive system 122 is generally operational to provide the autonomous, semi-autonomous, and / or driver-assisted decision-making processes. The high-level cognitive system 122 may receive the virtual lanes in the directed graphs from the sensor abstraction layer 124. The high-level cognitive system 122 may determine actions to be taken based on the virtual lanes, translate the actions into commands, and forward the commands to the hardware abstraction layer 120 for subsequent transmission to the power plant 108.

[0043] The sensor abstraction layer 124 is generally operable to use the sensor data from sensor 102 and the power plant data from power plant 108 to determine the location of the vehicle 90 and what other vehicles are nearby. The sensor abstraction layer 124 can present the directed graph to the high-level cognitive system 122 to support navigation decisions.

[0044] The map manager 126 is generally capable of generating recursive, self-organizing maps based on the road map data received from the storage device 104 and the trajectories of neighboring vehicles observed by the sensor 102. The map manager 126 can determine one or more virtual lanes available to the vehicle 90 from the recursive, self-organizing map. The virtual lanes can be formatted as a directed graph that is presented to the high-level cognitive system 122.

[0045] A self-organizing map generally defines a way to cluster high-dimensional input data into spatially organized representations. The self-organizing map is typically implemented as a neural network with an input layer for incoming data and an output layer arranged in a two-dimensional grid. The output neurons can receive weight values ​​from the input neuron that represent a cluster center of the output neuron. For each iteration of training, a random point from a training dataset can be selected as input to the network. The values ​​for each output value can then be calculated as a weighted sum of the inputs, with the neuron with the highest value referred to as neuron U. The weights can be updated during training.For an iteration S, a vector of weights going to an output neuron V can be incremented for the next iteration S+1 by the product of a proximity function between U and V, a learning rate, and a distance between the vector of weights going to U and the weights going to V. Over many iterations, the weights of the output neurons shift values ​​to represent the distribution of the input data. For example, if you train a self-organizing map on two-dimensional data points arranged in a ring, the weights of the output neurons can shift to cover this space.

[0046] A recursive self-organizing map generally allows for the clustering of spatial and temporal data within a single representation. Recursive self-organizing maps can capture sequential relationships between output clusters. A recursive self-organizing map contains a regular self-organizing map with an additional input layer that is a copy of the output layer from the previous time step. The value of each output neuron can be calculated as the weighted combination of the output from only the current input and the output from only the copy of the map at the previous time step. The weights from the current input at time t can be denoted as X(t) and the weights from the copy of the map from the previous time step can be denoted as Y(t). The two sets of weights can be trained separately using the same learning rules as regular self-organizing maps.The training iterations are generally performed as before, but with the input data in the original temporal order in which they were found. As a result, the X(t) weights perform clustering as before, and the Y(t) weights encode transition probabilities for moving from one cluster to the next in the sequence.

[0047] With reference to Fig. 3 illustrates a flowchart of an example implementation of an estimation method 140 in accordance with an example embodiment of the autonomous driving system 100. The estimation method (or process) 140 may be implemented by the autonomous driving system 100. The estimation method 140 generally includes a step 142, a step 144, a step 146, and a step 148.

[0048] In step 142, the sensor 102 may observe the paths of other vehicles traveling through an area of ​​interest. The area of ​​interest is generally the road map through which the vehicle 90 is currently traveling. Based on the observations, the computer 106 may initialize and train a recursive self-organizing map, which is overlaid on the area of ​​interest, in step 144. During each time step in training, a current map 150 may be updated using a current input vector 152 and a copy of the map 154 ​​in a previous time step. The current map 150, after being updated, becomes the map 154 ​​in a next time step. The time steps continue for multiple iterations as new input vectors 152 are added to the current map 150.

[0049] In step 146, the computer 106 may determine one or more virtual lanes in the area of ​​interest. The virtual lanes are generally drivable lanes used by other vehicles that have previously traversed the area of ​​interest. Once the virtual lanes are determined, the computer 106 may generate a directed graph in step 148. The directed graph may include spatial points in each virtual lane. The directed graph may then be used by the computer 106 to make navigation decisions for perception, prediction, and / or planning.

[0050] With reference to Fig. 4 illustrates a schematic diagram of an exemplary environment 160 surrounding the map manager 126 in accordance with an exemplary embodiment of the autonomous driving system 100. The environment 160 generally includes the map manager 126, a perception function 164, a high-level planner function 166, a recursive self-organizing map 168, a road map 170, a hypothesizer and decoder function 172, a computer matrix transformation function 174, a lane center inference function 176, a route generation function 178, and an internal map rendering function 180.

[0051] The sensing function 164 is generally operable to provide real-time data and update data to the map manager 126. The real-time data may include, among other things, lane boundaries, neighboring vehicle trajectories, and / or neighboring vehicle tokens determined by the sensor 102. The update data may include map updates to the road data stored in the storage device 104.

[0052] The high-level planner function 166 is generally operable to provide route start points and route end points to the map manager 126. The points are generated according to the final destinations defined by external operators and may also include a sequence of spatially distant waypoints, such as intersections, suitable for reaching the target destinations.

[0053] The recursive self-organizing map 168 can provide a dynamic map of the area of ​​interest and the virtual lanes. The virtual lanes are generally based on the observed neighboring vehicle trajectories and / or vehicle tokens.

[0054] The road map 170 may be an electronic map stored in the storage device 104 and represents the road on which the vehicle 90 is traveling. A segment of the road map 170 may be read by the map manager 126 as part of an initialization of the recursive self-organizing map 168.

[0055] Hypothesizer and decode function 172 may be operative to generate navigation decisions based on data in the directed graphs received from map manager 126. The data in the directed graphs may include, among other things, the virtual lanes, lane boundaries, lane centers, traffic sign locations, and the current lighting status.

[0056] The computer matrix transformation function 174 may be operative to generate updated vehicle tokens. The updates may be based on the vehicle tokens received from the sensor 102 and the lane lines defined in the road map 170. The updated vehicle tokens may be passed to the internal map rendering function 180.

[0057] The lane center inference function 176 generally serves to determine lane center information based on the recursive self-organizing map 168. The lane center information may take map updates into account. The lane center information may be transmitted to the internal map display function 180.

[0058] The route generation function 178 is generally operable to determine a current location and direction of travel of the vehicle 90 based on the route information received from the high-level planner function 166. The current location and direction of travel generally enable the map manager 126 to determine a portion of the road map 170 to be accessed from the storage device 104.

[0059] The internal map rendering function 180 is generally responsible for training the recursive self-organizing map 168. The internal map rendering function 180 may also be used to generate the directed graphs containing the virtual lanes, lane boundaries, lane centers, traffic sign positions, and current light status, etc. The directed graphs may be presented to the hypothesizer and decoder function 172 to support the navigation of the vehicle 90.

[0060] With reference to Fig. 5 is a schematic floor plan of an example portion of a road 200 according to an example embodiment of the autonomous driving system 100. The road 200 may represent an intersection between a north-south road and an east-west road. The road 200 generally includes a southbound lane 202, a center lane 204 north of the intersection, a northbound lane 206, a westbound lane 208, an eastbound lane 210, and a center (or left-turn) lane 212 south of the intersection. The road 200 may be used to explain how the recursive self-organizing map 168 is trained. In the example, the center lane 204 does not officially exist as a northbound lane and would not be an option for autonomous vehicles.Autonomous vehicles operating exclusively on the road map 170 typically wait for gaps in northbound and southbound traffic when making left turns, which can be difficult during rush hour. By observing other vehicles using the center lane 204, the recursive self-organizing map can create a virtual lane to represent common and / or newly discovered movement options.

[0061] With reference to Fig. 6 illustrates a flowchart of an example implementation of a training sequence 220 in accordance with an example embodiment of the autonomous driving system 100. The training sequence 220 may be implemented by the computer 106. The training sequence 220 generally includes an initial condition 222, an iterated condition 224, another iterated condition 226, and an ending condition 228.

[0062] In initial condition 222, map manager 126 may align a uniform grid of points with the road map 170 received from storage device 104 along an intended route. The grid may be a two-dimensional grid of evenly spaced points. The grid points are represented in the example as multiple (e.g., 60) black dots labeled 0-59. Vehicle 90 is generally outside or at an edge of the grid at initial condition 222. In the example, vehicle 90 may be in eastbound lane 210 and approaching the intersection.

[0063] Vehicle 90 may observe other vehicles traveling through a road section corresponding to road map 170, along approximately the same route that vehicle 90 intends to follow. The observations may be used to create a training dataset of observed trajectories. In the example, two other vehicles previously turned left from eastbound lane 80 into northbound lane 206. Map manager 126 may extract two observed trajectories from the observations, one from a left-turning vehicle using center lane 204 and one using regular northbound lane 206, from public map data.

[0064] The observations are generally made before the vehicle 90 reaches the intersection, or they are retrieved from a database before or when the vehicle 90 reaches the intersection. The map manager 126 can overlay the spatial coordinates of the two previous trajectories on the recursive self-organizing map. The spatial coordinates of two trajectories are represented as white squares. The rightmost trajectory of the observed trajectories generally shows the left turn directly into the northbound lane 206. The leftmost trajectory of the observed trajectories shows the left turn into the center lane 204 and a subsequent shift into the northbound lane 206.

[0065] The map manager 126 can train the recursive self-organizing map 168 based on the observed trajectories. Training is generally based on alternating the two trajectories, with discrete point coordinates of the trajectories being input in the observed order. The iteration state 224 generally illustrates the points after several (e.g., 10) iterations of the recursive self-organizing map 168. Several of the points (black points) can be shifted toward a nearest observed discrete point coordinate (white squares) during the iterations.

[0066] The map manager 126 may continue iterating the points if the points do not sufficiently converge to (or overlap or cover) the observed coordinates and / or a predetermined number of iterations have been performed. The further iterated condition 226 generally represents the points after multiple (e.g., 20) iterations of the recursive self-organizing map 168. In the end condition 228, the recursive self-organizing map 168 may converge the points to the discrete points along the two observed trajectories. In the example, the rightmost trajectory may contain points 56, 57, 52, 47, 41, 34, 28, 22, 16, 9, and 3. The leftmost lane can contain the points 56, 51, 45, 39, 33, 27, 21, 15, 9 and 3.

[0067] During training, the map manager can examine 126 recursive weights, y(x), to consider transitions between output nodes. The y(x) weights can be used to create directed edges in a final directed graph. For each non-zero weight in y(x), a directed edge can be added for corresponding nodes. Afterward, a set of nodes connected by directed edges can be returned as the final graph representation of the virtual lanes (represented as white squares).

[0068] The directed graph can then be used in a planning technique. For example, a Dijkstra algorithm can use the directed graph to compute a decision for vehicle 90 to take the leftmost trajectory into the center lane 204 first, rather than turning directly into the northbound lane 206. The directed edges in the graph can be assigned pressure values ​​related to the amount of surrounding traffic. For oncoming vehicles from the south, the pressure values ​​for edges in the northbound lane 206 may be higher, causing vehicle 90 to choose the center lane 204.

[0069] With reference to Fig. 7 illustrates a flowchart of an example implementation of a processing method 240 in accordance with an example embodiment of the autonomous driving system 100. The processing method (or process) 240 may be implemented by the autonomous driving system 100. The processing method 240 generally includes a step 242, a step 244, a step 246, a step 248, and a step 250. The step 246 generally includes a step 252, a step 254, a step 256, and a step 258.

[0070] In step 242, data for the recursive self-organizing map 168 may be collected by the sensor 102, the storage device 104, and the computer 106. The recursive self-organizing map 168 may be initialized by the map manager 126 in step 244. Initialization may include generating the initial grid aligned with a two-dimensional area on the road map 170. During initialization, a number of training sessions may also be specified.

[0071] The training of the recursive self-organizing map 168 may be performed by the map manager 126 in step 246. At the end of the training, the recursive self-organizing map 168 may be finalized. A grid layer in the final recursive self-organizing map 168 may cover the discrete point coordinates of the observed trajectories. A recurrent layer may contain the weights for the recursive self-organizing map 168. In step 250, the map manager 126 generates a directed graph with virtual lanes from the grid layer.

[0072] As part of the training at step 246, the card manager 126 may check at step 252 whether training is in progress or complete. If training is complete, the processing method 240 may proceed to step 248. If training is still in progress, the processing method 240 may proceed to step 254.

[0073] In step 254, the map manager 126 may randomly select a trajectory from a collection of observed trajectories. In step 256, a check may be performed to determine if any points remain on the selected trajectory. If one or more points remain, a first (or end) point (e.g., X,Y) of the selected trajectory may be isolated (or removed). The isolated point may be used as input to the recursive self-organizing map 168. Training equations may be applied to the recursive self-organizing map 168 in step 258. Fig. applied. Training step 246 may then return to step 256 to search for additional points in the selected trajectory.

[0074] Training step 246 may continue around step 256 and step 258 until the points in the selected trajectory have been considered. Step 256 may return to step 246 to determine whether additional observed trajectories should be considered. If additional observed trajectories still need to be considered, training step 246 may continue in a loop around steps 246-258 until the observed trajectories have been processed.

[0075] Various embodiments of the disclosure can assist in the planning and decision-making processes of autonomous vehicles. The use of recursive self-organizing maps for virtual lane estimation generally offers several advantages. For example, the recursive self-organizing map can be trained either with an observation of another vehicle or with a large dataset of observations. Training with a single observation can be performed quickly for online learning in a real-world driving situation. The recursive self-organizing map can also take advantage of larger datasets, such as overhead video footage from traffic cameras and / or a crowdsourced database, to produce more accurate and comprehensive estimates.

[0076] The resulting directed graph format can be compatible with many path planning algorithms that work with directed graphs. This saves computation time because the virtual lanes are already in a usable format that requires no conversion techniques. Furthermore, the recursive part of the self-organizing map can encode temporal information about the direction of travel in addition to the spatial information of the virtual lane.

[0077] Embodiments of the disclosure may handle uncertain conditions, such as accidents or ambiguous visual markers. When such conditions occur, the vehicle 90 may imitate the movements of surrounding vehicles as a starting point for reasonable behavior. Over time, the vehicle 90 may thus gain experience and skills to handle difficult situations unforeseen by the system designers. The observations collected from many vehicles may also form a crowdsourced database to support improved future iterations of autonomous driving systems.

[0078] The use of a recursive self-organizing map for virtual lane estimation generally addresses multiple criteria for autonomous vehicles within the single map manager 126 without requiring conversion between representations. In various embodiments, the system addresses problems with difficult-to-detect lane markings by detecting other vehicles and tracking their movement. The system can also natively address situations where such lane markings are irrelevant to the situation and normal traffic rules are overridden (e.g., a stuck car blocking a lane). Furthermore, current modular approaches to autonomous driving include phases of gathering map information, hypothesizing about other agents' movements, and deciding which output data to use.The system's graph-based format is generally useful for displaying map-based data, such as virtual lanes, and can serve as the basis for many different path planning techniques in commercial use.

Claims

[1] Virtual lane estimation system (100), comprising: a storage device (104) configured to store a road map (170) corresponding to a section of a road ahead of a vehicle (90); a sensor (102) configured to observe a plurality of trajectories of a plurality of adjacent vehicles crossing the section of road; and a computer (106) configured to initialize a recursive self-organizing map (168) as a plurality of points arranged as a two-dimensional grid aligned with the road map (170), train the plurality of points in the recursive self-organizing map (168) in response to the plurality of trajectories, generate a directed graph including one or more virtual lanes through the road map (170) in response to the points trained on the plurality of trajectories, and generate a control signal that controls navigation of the vehicle (90) through the portion of the road in response to the one or more virtual lanes in the directed graph. [2] The virtual lane estimation system (100) of claim 1, wherein training the recursive self-organizing map (168) uses the plurality of trajectories individually. [3] The virtual lane estimation system (100) of claim 2, wherein a recurrent layer of the recursive self-organizing map includes a plurality of weights of the plurality of points at one end of the training. [4] The virtual lane estimation system of claim 3, wherein a grid layer of the recursive self-organizing map (168) contains the plurality of trajectories at the end of training. [5] The virtual lane estimation system (100) of claim 2, wherein the computer (106) is further configured to select a random trajectory from the plurality of trajectories during training. [6] The virtual lane estimation system (100) of claim 5, wherein the computer (106) is further configured to isolate a first point of the random trajectory. [7] The virtual lane estimation system (100) of claim 6, wherein the computer (106) is further configured to apply the first point to the recursive self-organizing map during training. [8] The virtual lane estimation system (100) of claim 7, wherein the computer (106) is further configured to isolate a second point of the random trajectory and apply the second point to the recursive self-organizing map during training. [9] The virtual lane estimation system (100) of claim 1, wherein the recursive self-organizing map (168) comprises a self-organizing map having an input layer from a current time step, and the input layer is an output layer from a previous time step, and the self-organizing map implements a neural network, the output layer being arranged as the two-dimensional grid. [10] A method (140) for estimating the virtual lane, comprising: Reading a road map (170) corresponding to a section of a road in front of a vehicle (90); Initializing (144) a recursive self-organizing map (168) as a plurality of points arranged as a two-dimensional grid aligned with the road map (170) using a computer (106) of the vehicle (90); observing (142) a plurality of trajectories of a plurality of adjacent vehicles (90) crossing the section of road using a sensor (102) of the vehicle (90); Training the plurality of points in the recursive self-organizing map (168) depending on the plurality of trajectories; Generating (148) a directed graph containing one or more virtual lanes through the road map (170) in response to the points trained on the plurality of trajectories; and Controlling an engine (108) to navigate the vehicle (90) through the road segment in response to the one or more virtual lanes in the directed graph.

Images