Method and system for generating lane line and road edge data with empirical path distributions

Abstract

Systems and methods for defining map data used in controlling a vehicle are provided. In one embodiment, a method includes receiving, by a processor, telemetry data; determining, by the processor, distribution data for a path based on the telemetry data; determining, by the processor, a plurality of sample data based on a trained machine learning model and the distribution data; generating, by the processor, at least one of lane line data and road edge data based on the sample data and a second machine learning model; and storing, by the processor, the map data including the lane line data and road edge data for use in controlling the vehicle.

Description

Technical Field

[0001] This disclosure generally relates to autonomous vehicles, and more specifically to systems and methods for generating lane line and road edge data for controlling autonomous vehicles. Background Technology

[0002] Autonomous vehicles are vehicles capable of sensing their environment and navigating with little or no user input. Autonomous vehicles use sensing devices such as radar, lidar, and image sensors to perceive their environment. Autonomous vehicle systems also use information from Global Positioning System (GPS) technology, navigation systems, vehicle-to-vehicle communication, vehicle-to-infrastructure technology, and / or drive-by-wire systems to navigate the vehicle.

[0003] Vehicle automation has been categorized into digital levels, ranging from zero (corresponding to no automation with complete human control) to five (corresponding to complete automation without human control). Various automated driver assistance systems (such as cruise control, adaptive cruise control, and parking assistance systems) correspond to lower levels of automation, while truly "driverless" vehicles correspond to higher levels of automation.

[0004] While significant progress has been made in autonomous vehicles in recent years, such vehicles can still be improved in many ways. For example, for navigation, autonomous vehicles rely on maps that define the roads and lanes within the roads in the environment. Typically, these maps are predefined based on aerial imaging and then transmitted to the autonomous vehicle for use. Creating maps from aerial imaging can be time-consuming and expensive.

[0005] Therefore, it is desirable to provide improved systems and methods for generating lane line and road edge data for maps. Furthermore, other desirable features and characteristics of this disclosure will become apparent from the accompanying drawings and the foregoing technical and background information, based on the following detailed description and the appended claims. Summary of the Invention

[0006] A system and method are provided for generating map data for controlling a vehicle. In one embodiment, the method includes: receiving telemetry data via a processor; determining path distribution data via the processor based on the telemetry data; determining sample data via the processor based on the first machine learning model and the distribution data; generating at least one of lane line data and road edge data via the processor based on the sample data and the second machine learning model; and storing the map data, including the lane line data and road edge data, via the processor for use in controlling the vehicle.

[0007] In various embodiments, the telemetry data includes position, velocity, heading, and time difference observed from a region in the environment.

[0008] In various embodiments, the second machine learning model includes a convolutional neural network, and the first machine learning model includes an expectation-maximization model.

[0009] In various embodiments, determining the distribution data includes: determining a transformation graph by counting the number of paths that sequentially access each pair of nodes; and including edges for each pair of nodes that occur more than a threshold number of times relative to the total number of paths.

[0010] In various embodiments, the method includes controlling the vehicle based on the map data.

[0011] In various embodiments, determining the distribution data further includes: for each node, initializing a conditional distribution to describe the log-likelihood of each successor node; and for each edge between the source node and the target node, initializing a conditional distribution to describe the log-likelihood of the position, velocity, heading, and time difference at the target node based on the position, velocity, and heading at the source node.

[0012] In various embodiments, the method further includes: using an expectation-maximization model to train the distribution data based on the telemetry data.

[0013] In various embodiments, determining the plurality of sample data includes: sampling the distribution data by repeatedly selecting successor nodes from the plurality of nodes based on the log-likelihood at the current node; and selecting parameters based on the distribution data on the edges corresponding to the successor nodes.

[0014] In various embodiments, determining the plurality of sample data includes: repeating the picking and selection multiple times to generate density image data including a density image.

[0015] In another embodiment, a computer-implemented system for generating map data used in controlling a vehicle includes: a non-transitory computer-readable medium configured to store instructions; and a processor configured to execute the instructions to perform a method. The method includes: receiving telemetry data via the processor; determining path distribution data via the processor based on the telemetry data; determining a plurality of sample data via the processor based on the first machine learning model and the distribution data; generating at least one of lane line data and road edge data via the processor based on the sample data and a second machine learning model; and storing the map data, including the lane line data and road edge data, used in control.

[0016] In various embodiments, the telemetry data includes position, velocity, heading, and time difference observed from a region in the environment.

[0017] In various embodiments, the second machine learning model is a convolutional neural network.

[0018] In various embodiments, determining the distribution data includes: determining a transformation graph by counting the number of paths that sequentially access each pair of nodes; and including edges for each pair of nodes that occur more than a threshold number of times.

[0019] In various embodiments, the threshold number is relative to the total number of paths.

[0020] In various embodiments, determining the distribution data further includes: for each node, initializing a conditional distribution to describe the log-likelihood of each successor node; and for each edge between the source node and the target node, initializing a conditional distribution to describe the log-likelihood of the position, velocity, heading, and time difference at the target node based on the position, velocity, and heading at the source node.

[0021] In various embodiments, the method further includes: using expectation maximization to train the distribution data based on the telemetry data.

[0022] In various embodiments, determining the plurality of sample data includes: sampling the distribution data by repeatedly selecting successor nodes from the plurality of nodes based on the log-likelihood at the current node; and selecting parameters based on the distribution data on the edges corresponding to the successor nodes.

[0023] In various embodiments, determining the plurality of sample data includes: repeating the picking and selection multiple times to generate density image data including a density image.

[0024] In another embodiment, the vehicle includes: an autonomous driving system configured to control the vehicle based on map data; and a map definition module configured to receive telemetry data via a processor, determine path distribution data based on the telemetry data, determine multiple sample data based on a first machine learning model and the distribution data, generate at least one of lane line data and road edge data based on the sample data and a second machine learning model, and store the map data including the lane line data and road edge data used when controlling the vehicle.

[0025] In various embodiments, the map definition module is further configured to determine the distribution data by the processor by performing the following operations: determining a transformation map by counting the number of paths that sequentially access each pair of nodes; including edges for each pair of nodes that occur more than a threshold number of times; initializing a conditional distribution for each node to describe the log-likelihood of each successor node; initializing a conditional distribution for each edge between the source node and the target node to describe the log-likelihood of the position, velocity, heading, and time difference at the target node based on the position, velocity, and heading at the source node; and training the distribution data based on the telemetry data using expectation maximization. Attached Figure Description

[0026] Exemplary embodiments will be described below in conjunction with the following figures, wherein the same numerals denote the same elements, and wherein:

[0027] Figure 1 This is a functional block diagram illustrating an autonomous vehicle with a map definition system according to various embodiments;

[0028] Figure 2 This is a functional block diagram illustrating a transportation system associated with an autonomous vehicle according to various embodiments;

[0029] Figure 3 This is a data flow diagram illustrating an autonomous driving system according to various embodiments, the autonomous driving system including a map definition system for autonomous vehicles;

[0030] Figure 4 This is a data flow diagram illustrating a map definition system according to various embodiments;

[0031] Figure 5A , Figure 5B , Figure 5C and Figure 5D These are maps of example data generated by a map definition system according to various embodiments; and

[0032] Figure 6 This is a flowchart illustrating control methods for defining maps and controlling autonomous vehicles according to various embodiments. Detailed Implementation

[0033] The following detailed description is exemplary in nature only and is not intended to limit application and use. Furthermore, it is not intended to be bound by any express or implied theory set forth in the foregoing technical fields, background art, summary of the invention, or the following detailed description. As used herein, the term "module" refers to any hardware, software, firmware, electronic control components, processing logic, and / or processor device, individually or in any combination, including but not limited to: application-specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated, or grouped) and memories executing one or more software or firmware programs, combinational logic circuits, and / or other suitable components providing the described functionality.

[0034] Embodiments of this disclosure are described herein according to functional and / or logical block components and various processing steps. It should be understood that such block components can be implemented by any number of hardware, software, and / or firmware components configured to perform specified functions. For example, embodiments of this disclosure may employ various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, lookup tables, etc., which can perform various functions under the control of one or more microprocessors or other control devices. Furthermore, those skilled in the art will understand that embodiments of this disclosure can be practiced in combination with any number of systems, and the systems described herein are merely exemplary embodiments of this disclosure.

[0035] For the sake of brevity, conventional techniques related to signal processing, data transmission, signaling, control, and other functional aspects of the system (and its various operating components) will not be described in detail herein. Furthermore, the connecting lines shown in the various figures included herein are intended to represent example functional relationships and / or physical connections between various elements. It should be noted that many alternative or additional functional relationships or physical connections may exist in the embodiments of this disclosure.

[0036] Reference Figure 1 According to various embodiments, the map definition system 100, generally shown as 100, is associated with the vehicle 10. Generally, the map definition system 100 generates map data used when controlling the vehicle 10. As will be discussed in more detail below, the map definition system 100 generates map data by: learning the distribution of data observed in a defined area using a machine learning model, sampling that distribution a standard number of times to generate sampled data, and using the sampled data to infer the positions of road edges and lane lines in the map incorporated into that area by applying a convolutional neural network (CNN).

[0037] like Figure 1As described, the vehicle 10 generally includes a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is mounted on the chassis 12 and substantially surrounds the components of the vehicle 10. The body 14 and the chassis 12 may together form a frame. The wheels 16-18 are each rotatably connected to the chassis 12 near a corresponding corner of the body 14.

[0038] In various embodiments, vehicle 10 is an autonomous vehicle, and map definition system 100 is incorporated into autonomous vehicle 10 (hereinafter referred to as autonomous vehicle 10). Autonomous vehicle 10 is, for example, a vehicle automatically controlled to transport passengers from one location to another. In the illustrated embodiment, vehicle 10 is depicted as a passenger car, but it should be understood that any other means of transportation may also be used, including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), boats, aircraft, etc. In exemplary embodiments, autonomous vehicle 10 is a so-called Level 4 or Level 5 automation system. Level 4 system means “high automation,” referring to the performance of the automated driving system in specific driving modes for all aspects of a dynamic driving task, even if the human driver does not respond appropriately to intervention requests. Level 5 system means “full automation,” referring to the full-time performance of the automated driving system for all aspects of a dynamic driving task under all road and environmental conditions that a human driver can manage.

[0039] As shown in the figure, an autonomous vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, at least one controller 34, and a communication system 36. In various embodiments, the propulsion system 20 may include an internal combustion engine, an electric motor such as a traction motor, and / or a fuel cell propulsion system. The transmission system 22 is configured to transmit power from the propulsion system 20 to the wheels 16-18 according to a selectable speed ratio. According to various embodiments, the transmission system 22 may include a step-ratio automatic transmission, a continuously variable transmission (CVT), or other suitable transmission. The braking system 26 is configured to provide braking torque to the wheels 16-18. In various embodiments, the braking system 26 may include friction braking, brake-by-wire braking, a regenerative braking system such as an electric motor, and / or other suitable braking systems. The steering system 24 affects the position of the wheels 16-18. Although depicted as including a steering wheel for illustrative purposes, in some embodiments contemplated within the scope of this disclosure, the steering system 24 may not include a steering wheel.

[0040] Sensor system 28 includes one or more sensing devices 40a-40n that sense observable conditions of the external and / or internal environment of the autonomous vehicle 10. Sensing devices 40a-40n may include, but are not limited to, radar, lidar, GPS, optical cameras, thermal imaging cameras, ultrasonic sensors, inertial measurement units, and / or other sensors. Actuator system 30 includes one or more actuator devices 42a-42n that control one or more vehicle features (such as, but not limited to, propulsion system 20, drivetrain 22, steering system 24, and braking system 26). In various embodiments, vehicle features may also include internal and / or external vehicle features, such as, but not limited to, doors, trunk, and cabin features (such as air, music, lighting, etc.) (not numbered).

[0041] Communication system 36 is configured to wirelessly communicate information with other entities 48, such as, but not limited to, other vehicles (“V2V” communication), infrastructure (“V2I” communication), remote systems and / or personal devices (regarding...). Figure 2 (Described in more detail). In an exemplary embodiment, communication system 36 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communication. However, additional or alternative communication methods, such as Dedicated Short Range Communication (DSRC) channels, are also considered within the scope of this disclosure. A DSRC channel refers to a one-way or two-way short-to-medium range wireless communication channel specifically designed for automotive use, and a corresponding set of protocols and standards.

[0042] Data storage device 32 stores data used when automatically controlling the autonomous vehicle 10. In various embodiments, data storage device 32 stores a defined map of the navigable environment. In various embodiments, the map is defined by map definition system 100. In various embodiments, defining the map may include a map predefined by a remote system and obtained from the remote system (regarding...). Figure 2 (To be described in further detail). For example, the defined map can be assembled by a remote system and transmitted to the autonomous vehicle 10 (wireless and / or wired transmission), and stored in data storage device 32. It is understood that data storage device 32 may be part of controller 34, separate from controller 34, or part of controller 34 and a separate system.

[0043] The controller 34 includes at least one processor 44 and a computer-readable storage device or medium 46. The processor 44 can be any custom or commercially available processor, central processing unit (CPU), graphics processing unit (GPU), auxiliary processor among several processors associated with the controller 34, semiconductor-based microprocessor (in the form of a microchip or chipset), macroprocessor, any combination thereof, or any device typically used for executing instructions. For example, the computer-readable storage device or medium 46 can include volatile and non-volatile storage in read-only memory (ROM), random access memory (RAM), and non-fail-to-erase memory (KAM). KAM is persistent or non-volatile memory used to store various operational variables when the processor 44 is powered off. The computer-readable storage device or medium 46 can be implemented using any of many known memory devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or any other electrical, magnetic, optical, or combined memory device capable of storing data, some of which represents executable instructions used by the controller 34 in controlling the autonomous vehicle 10.

[0044] The instructions may include one or more separate programs, each comprising an ordered list of executable instructions for implementing logical functions. When executed by processor 44, the instructions receive and process signals from sensor system 28, execute logic, calculations, methods, and / or algorithms for automatically controlling components of autonomous vehicle 10, and generate control signals for actuator system 30 to automatically control components of autonomous vehicle 10 based on the logic, calculations, methods, and / or algorithms. Figure 1 Only one controller 34 is shown, but embodiments of the autonomous vehicle 10 may include any number of controllers 34 that communicate via any suitable communication medium or combination of communication media and cooperate to process sensor signals, perform logic, calculations, methods and / or algorithms, and generate control signals for automatically controlling the features of the autonomous vehicle 10.

[0045] In various embodiments, one or more instructions of controller 34 are embodied in map definition system 100 and, when executed by processor 44, receive telemetry data from sensor system 28, learn the distribution of telemetry data observed in the defined area using a machine learning model, sample the distribution a standard number of times to generate sampled data, and use the sampled data to infer the positions of road edges and lane lines in the map incorporated into the area by applying a convolutional neural network (CNN) (e.g., by modifying the map based on aerial imagery and / or by creating a new map), which is stored, for example, in data storage device 32.

[0046] The instruction storage and transmission feature a distribution whose size depends solely on the road geometry, replacing the potentially very large raw telemetry data in busy areas. This distribution can be sampled in-situ to obtain data that can be used to infer lane lines and road edges. Furthermore, the model can be sampled multiple times as needed, enabling the retrieval of sufficiently dense sample data even if the original dataset is too sparse to produce a good density map. This process allows for the identification of lane lines and road edges without relying on aerial imaging, while simultaneously increasing computational throughput.

[0047] Now refer to Figure 2 In various embodiments, regarding Figure 1 The described autonomous vehicle 10 can be used in taxi or shuttle bus systems within a specific geographic area (e.g., a city, school or business park, shopping mall, amusement park, event center, etc.), or it can be managed solely by a remote system. For example, the autonomous vehicle 10 can be associated with a remote transportation system based on autonomous vehicles. Figure 2 An exemplary embodiment of an operating environment, generally shown at 50, is illustrated. This operating environment includes an autonomous vehicle-based remote transportation system 52, which is related to... Figure 1 The one or more autonomous vehicles 10a-10n are associated with each other. In various embodiments, the operating environment 50 also includes one or more user equipment 54 communicating with the autonomous vehicle 10 and / or the remote transportation system 52 via a communication network 56.

[0048] Communication network 56 supports communication between devices, systems, and components supported by operating environment 50 as needed (e.g., via physical and / or wireless communication links). For example, communication network 56 may include a wireless carrier system 60, such as a cellular telephone system comprising multiple cell towers (not shown), one or more mobile switching centers (MSCs) (not shown), and any other networking components required to connect the wireless carrier system 60 to a terrestrial communication system. Each cell tower includes transmit and receive antennas and a base station, with base stations from different cell towers connected directly to the MSC or via intermediate devices (such as base station controllers). Wireless carrier system 60 can implement any suitable communication technology, including, for example, digital technologies such as CDMA (e.g., CDMA2000), LTE (e.g., 4G LTE or 5G LTE), GSM / GPRS, or other current or emerging wireless technologies. Other cell tower / base station / MSC arrangements are possible and can be used with wireless carrier system 60. For example, base stations and cell towers may coexist in the same location or they may be far apart from each other; each base station may serve a single cell tower, or a single base station may serve various cell towers, or various base stations may be interconnected to a single MSC; these are just a few possible arrangements.

[0049] In addition to the wireless carrier system 60, a second wireless carrier system in the form of a satellite communication system 64 may be included to provide one-way or two-way communication with autonomous vehicles 10a-10n. This can be accomplished using one or more communication satellites (not shown) and an uplink transmitting station (not shown). One-way communication may include, for example, a satellite radio service, in which program content (news, music, etc.) is received by the transmitting station, packaged, uploaded, and then transmitted to a satellite, which broadcasts the program to subscribers. Two-way communication may include, for example, a satellite telephone service, which uses a satellite to relay telephone communication between vehicle 10 and the station. Satellite telephones may be used in conjunction with or in place of the wireless carrier system 60.

[0050] It may also include a terrestrial communication system 62, which is a conventional terrestrial telecommunications network connected to one or more fixed-line telephones and connecting a wireless carrier system 60 to a remote transportation system 52. For example, the terrestrial communication system 62 may include a Public Switched Telephone Network (PSTN) such as that providing infrastructure for hard-line telephone, packet-switched data communications, and the Internet. One or more portions of the terrestrial communication system 62 may be implemented using standard wired networks, fiber optic or other optical networks, cable networks, power lines, other wireless networks (such as wireless local area networks (WLANs) or networks providing broadband wireless access (BWAs), or any combination thereof. Furthermore, the remote transportation system 52 does not need to be connected via the terrestrial communication system 62, but may include wireless telephone equipment that allows it to communicate directly with wireless networks such as the wireless carrier system 60.

[0051] Despite Figure 2Only one user device 54 is shown, but embodiments of the operating environment 50 can support any number of user devices 54, including multiple user devices 54 owned, operated, or otherwise used by a single person. Each user device 54 supported by the operating environment 50 can be implemented using any suitable hardware platform. In this regard, user devices 54 can be implemented in any common form factor, including but not limited to: desktop computers; mobile computers (e.g., tablet computers, laptop computers, or netbook computers); smartphones; video game devices; digital media players; home entertainment devices; digital cameras or camcorders; wearable computing devices (e.g., smartwatches, smart glasses, smart clothing), etc. Each user device 54 supported by the operating environment 50 is implemented as a computer-implemented or computer-based device having the hardware, software, firmware, and / or processing logic required to perform the various techniques and methods described herein. For example, user device 54 includes a microprocessor in the form of a programmable device, which includes one or more instructions stored in an internal memory structure and applied to receive binary input to create binary output. In some embodiments, user device 54 includes a GPS module capable of receiving GPS satellite signals and generating GPS coordinates based on these signals. In other embodiments, user equipment 54 includes cellular communication capabilities, enabling the device to perform voice and / or data communications on communication network 56 using one or more cellular communication protocols, as discussed herein. In various embodiments, user equipment 54 includes a visual display, such as a touchscreen graphic display or other display.

[0052] The remote transportation system 52 includes one or more backend server systems that may be cloud-based, network-based, or reside on a specific campus or geographic location served by the remote transportation system 52. The remote transportation system 52 may be operated by human advisors, automated advisors, or a combination of both. The remote transportation system 52 can communicate with user equipment 54 and autonomous vehicles 10a-10n to schedule rides, dispatch autonomous vehicles 10a-10n, etc. In various embodiments, the remote transportation system 52 stores account information such as subscriber authentication information, vehicle identifiers, profile records, behavioral patterns, and other relevant subscriber information.

[0053] According to a typical use case workflow, a registered user of the remote transportation system 52 can create a ride request via user equipment 54. The ride request typically indicates the passenger's desired pick-up location (or current GPS location), desired destination location (which may identify predefined vehicle stops and / or user-specified passenger destinations), and pick-up time. The remote transportation system 52 receives the ride request, processes it, and dispatches a selected vehicle (if any) from autonomous vehicles 10a-10n to pick up the passenger at the specified pick-up location and at the appropriate time. The remote transportation system 52 may also generate and send an appropriately configured confirmation message or notification to user equipment 54 to inform the passenger that the vehicle is en route.

[0054] It is understood that the subject matter disclosed herein provides certain enhanced features and functionalities for what can be considered a standard or baseline autonomous vehicle 10 and / or an autonomous vehicle-based long-distance transportation system 52. Therefore, autonomous vehicles and autonomous vehicle-based long-distance transportation systems can be modified, enhanced, or otherwise supplemented to provide the additional features described in more detail below.

[0055] According to various embodiments, such as Figure 3 As shown, controller 34 implements autonomous driving system (ADS) 70. That is, suitable software and / or hardware components of controller 34 (e.g., processor 44 and computer-readable storage device 46) are used to provide autonomous driving system 70 for use in conjunction with vehicle 10.

[0056] In various embodiments, the instructions of the autonomous driving system 70 can be organized by function, module, or system. For example, as Figure 3 As shown, the autonomous driving system 70 may include a computer vision system 74, a positioning system 76, a guidance system 78, and a vehicle control system 80. It is understood that in various embodiments, instructions may be organized into any number of systems (e.g., combined, further divided, etc.), as this disclosure is not limited to this example.

[0057] In various embodiments, the computer vision system 74 synthesizes and processes sensor data and predicts the presence, location, classification, and / or path of objects and features in the environment of the vehicle 10. In various embodiments, the computer vision system 74 may combine information from multiple sensors, including but not limited to cameras, lidar, radar, and / or any number of other types of sensors.

[0058] The positioning system 76 processes sensor data and other data to determine the position of the vehicle 10 relative to its environment (e.g., local location relative to a map, exact position relative to a road lane, vehicle heading, speed, etc.). The guidance system 78 processes sensor data and other data to determine the path that the vehicle 10 should follow. The vehicle control system 80 generates control signals to control the vehicle 10 according to the determined path.

[0059] In various embodiments, the controller 34 implements machine learning techniques to assist in the functions of the controller 34, such as feature detection / classification, obstacle mitigation, route traversal, mapping, sensor integration, and ground condition determination.

[0060] As briefly described above, Figure 1 The map definition system 100 is included within the ADS 70, for example, as a separate system or as part of, for example, a guidance system 78. For example, as per [reference to...] Figure 4 For more details, please refer to [link / reference]. Figure 3 The map definition system 100 includes a distribution module 102, a sample module 104, a prediction module 106, a map generation module 108, and a convolutional neural network data storage 110.

[0061] Distribution module 102 receives telemetry data 112 as input. In various embodiments, telemetry data 112 includes position data, velocity data, heading data, and time difference data, and can be received from sensors of autonomous vehicle 10 and / or from other vehicles in the area. Distribution module 102 uses a machine learning model to learn the distribution of telemetry data 112 observed in the defined area and generates distribution data 114 based on this.

[0062] For example, such as Figure 5A As shown, in various embodiments, telemetry data 112 is assembled into a path of points traveled by vehicle 10 along a road segment, and this path is associated with the normal of the road segment. The normal may be equidistant and / or spaced according to a defined increment along the road segment. The distribution module 102 then generates a transformation map by evaluating multiple paths generated for the road segment (e.g., from multiple vehicles).

[0063] For example, such as Figure 5B As shown, in various embodiments, the distribution module 102 defines each intersection of a path and a normal as a node in the graph. The distribution module 102 maintains a count of how many paths sequentially visit each pair of nodes. For each pair of nodes that appears more often than a number defined relative to the total number of paths, the distribution module 102 includes an edge in the transformation graph.

[0064] Then, for each node, distribution module 102 initializes a conditional distribution to describe the log-likelihood of each successor node. Then, for each edge, distribution module 102 initializes a conditional distribution to describe the log-likelihood of the position, velocity, heading, and time difference at the target node based on the position, velocity, and heading at the source node. Distribution module 102 trains these distributions using a machine learning module (such as an expectation-maximization model) based on the observed telemetry data 112, and generates, for example... Figure 5C The distribution data shown is 114.

[0065] Return to reference Figure 3 The sampling module 104 receives distribution data 114 as input. The sampling module 104 samples the distribution data multiple times to generate density image data 116. For example, the sampling module 104 samples the distribution data 114 by repeatedly selecting successor nodes based on the log-likelihood determined at the current node. Then, the sampling module 104 selects parameters based on the distribution on the edges corresponding to the selected successor nodes, and repeats this process multiple times as needed to generate density image data 116 including a density image, for example, as... Figure 5D As shown.

[0066] The prediction module 106 receives density image data 116 as input. The prediction module 106 uses a trained machine learning model to predict road edges and lane lines, and generates road edge data 120 and lane line data 122 based on this. For example, the prediction module 106 retrieves a trained convolutional neural network 118 from the CNN data storage 110. The prediction module 106 processes the density image data 116 using the trained CNN 118 to predict the positions of lane lines and road edges and generate data 120 and 122.

[0067] The map generation module 108 receives road edge data 120 and lane line data 122 as input. Based on the road edge data 120 and lane line data 122, the map generation module 108 generates map data 124 of the local environment. The map data 124 is then stored in the data storage device 32 and / or used by the ADS 70 to directly control the autonomous vehicle 10.

[0068] Now refer to Figure 6 And continue to refer to Figures 1 to 4 The flowchart illustrates the process that can be performed according to this disclosure. Figure 1 and Figure 4 The map definition system 100 executes a control method 200. As can be understood from this disclosure, the order of operations within this method is not limited to, for example... Figure 6The method 200 may be executed in the order shown, but may be executed in one or more different orders as applicable and in accordance with this disclosure. In various embodiments, method 200 may be scheduled to run based on one or more predetermined events, and / or may run continuously during the operation of the autonomous vehicle 10.

[0069] In various embodiments, method 200 may begin at 205. At 210, telemetry data 112 is received and assembled into a path. At 212, a transformation map is determined based on the telemetry data 112. Then, at 214, a machine learning model is used to learn distribution data 114 based on the telemetry data 112. Then, at 216, the distribution data 114 is sampled a defined number of times to produce density image data 116. Then, at 218, the density image data 116 is processed using a trained CNN 118 to produce predictions of the positions of lane lines and road edges. Subsequently, at 220, this prediction is incorporated into map data at 124, which can be used to control vehicle 10. Method 200 may then end at 222.

[0070] Therefore, the methods, systems, and vehicles described herein provide improved ways to generate map data for controlling vehicles, and accordingly, the claimed embodiments are improved within the scope of this disclosure.

[0071] Although at least one exemplary embodiment has been presented in the foregoing detailed description, it should be understood that numerous variations exist. It should also be understood that the exemplary embodiments or multiple exemplary embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of this disclosure in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient roadmap for implementing the exemplary embodiments or multiple exemplary embodiments. It should be understood that various changes may be made to the function and arrangement of the elements without departing from the scope of this disclosure as set forth in the appended claims and their legal equivalents.

Claims

1. A method for generating map data used when controlling a vehicle, the method comprising: Telemetry data is received via a processor; The processor determines path distribution data based on the telemetry data, wherein determining the distribution data includes: determining a transition graph by counting the number of paths that sequentially visit each pair of nodes; including edges for each pair of nodes that occur more than a threshold number of times relative to the total number of paths; initializing a conditional distribution for each node to describe the log-likelihood of each successor node; and initializing a conditional distribution for each edge between the source node and the target node to describe the log-likelihood of the position, velocity, heading, and time difference at the target node based on the position, velocity, and heading at the source node. The processor determines the sample data based on the first machine learning model and the distribution data. The processor generates at least one of lane line data and road edge data based on the sample data and the second machine learning model; and The map data, including lane line data and road edge data, is stored by the processor and used when controlling the vehicle.

2. The method according to claim 1, wherein, The telemetry data includes position, velocity, heading, and time difference observed from a region in the environment.

3. The method according to claim 1, wherein, The second machine learning model includes a convolutional neural network.

4. The method according to claim 1, wherein, The first machine learning model includes the expectation-maximization model.

5. The method according to claim 1, wherein, The method further includes using an expectation-maximization model to train the distribution data based on the telemetry data.

6. The method according to claim 1, wherein, Determining the sample data includes: sampling the distribution data by repeatedly selecting successor nodes from multiple nodes based on the log-likelihood at the current node; and selecting parameters based on the distribution data on the edges corresponding to the successor nodes.

7. The method according to claim 6, wherein, Determining the sample data includes: repeating the picking and selection multiple times to generate density image data including density images.

8. A computer-implemented system for generating map data used in controlling a vehicle, the system comprising: A non-transitory computer-readable medium configured to store instructions; as well as A processor configured to execute the instructions to perform a method comprising: The processor receives telemetry data. The processor determines path distribution data based on the telemetry data, wherein determining the distribution data includes: determining a transition graph by counting the number of paths that sequentially visit each pair of nodes; including edges for each pair of nodes that occur more than a threshold number of times relative to the total number of paths; initializing a conditional distribution for each node to describe the log-likelihood of each successor node; and initializing a conditional distribution for each edge between the source node and the target node to describe the log-likelihood of the position, velocity, heading, and time difference at the target node based on the position, velocity, and heading at the source node. The processor determines multiple sample data based on a first machine learning model and the distribution data; The processor generates at least one of lane line data and road edge data based on the sample data and the second machine learning model; and The map data, including lane line data and road edge data, is stored by the processor for use during control.

Images