Identifying Parking-Able Areas for Autonomous Vehicles

Abstract

Aspects of the present disclosure provide for the identification of parkable areas. In one example, observations of parked vehicles can be identified from stored data. The observations can be used to determine whether a subpart of an edge of a road map corresponds to a parkable area. In some examples, the edge can define a drivable area in the road map. Additionally, map information is generated based on a determination of whether a subpart of the edge corresponds to a parkable area.

Description

Technical Field

[0001] The present disclosure relates to identifying parking areas for autonomous vehicles. Background Art

[0002] Autonomous vehicles (e.g., vehicles that do not require a human driver) can be used to assist in transporting passengers or items from one location to another. Such vehicles can operate in a fully autonomous mode where the passenger can provide some initial input (such as a pick-up or destination location), and the vehicle maneuvers itself to that location. When approaching that location or at any point during the trip, the autonomous vehicle may constantly seek a location to stop the vehicle. For example, these locations can be used, for example, for short stops to pick up and / or drop off passengers and / or cargo, for example, for transportation services. Typically, an autonomous vehicle may seek such a location when the vehicle is within a certain distance of the destination. Summary of the invention

[0003] Aspects of the present disclosure provide a method for identifying a parking area. The method includes identifying, by one or more processors, observations of a parked vehicle from logged data; using the observations to determine, by the one or more processors, whether a sub-portion of an edge of a road map corresponds to a parking area, wherein the edge defines a drivable area in the road map; and generating, by the one or more processors, map information based on the determination of whether the sub-portion of the edge corresponds to the parking area.

[0004] In one example, the method further includes using the observation to determine whether the second sub-portion of the edge corresponds to a second parkable area. In another example, the method further includes further analyzing the observation to determine whether the parkable area is to the left of the edge. In another example, the method further includes further analyzing the observation to determine whether the parkable area is along the edge. In another example, the method further includes further analyzing the observation to determine whether the parkable area is between the edge and the second edge of the road map. In another example, the method further includes further analyzing the observation to determine the width of the parkable area. In another example, the method further includes further analyzing the observation to determine the percentage of time that the parkable area is occupied by the vehicle. In another example, the method further includes further analyzing the observation to determine the likelihood that the parkable area is available during multiple different time periods. In another example, the method further includes using the observation to train a machine learning model to provide the likelihood that the parkable area is occupied at a future point in time. In this example, the method further includes further analyzing the observation to determine the percentage of time that the parkable area is occupied by the vehicle, and using the percentage of time to train the model. Additionally or alternatively, the method further includes providing the model to an autonomous vehicle to enable the autonomous vehicle to use map information to make driving decisions. In another example, the method further includes using the map information to identify potential locations for the vehicle to stop and pick up or drop off passengers or cargo.

[0005] Another aspect of the present disclosure provides a system for identifying a parking area. The system includes a memory storing stored data and one or more processors. The one or more processors are configured to identify an observation of a parked vehicle from the stored stored data; use the observation to determine whether a sub-portion of an edge of a road map corresponds to a parking area, wherein the edge defines a drivable area in the road map; and generate map information based on the determination of whether the sub-portion of the edge corresponds to the parking area.

[0006] In one example, the one or more processors are further configured to use the observation to determine whether the second sub-portion of the edge corresponds to a second parking area. In another example, the one or more processors are further configured to further analyze the observation to determine whether the parking area is between the edge and the second edge of the road map. In another example, the one or more processors are further configured to further analyze the observation to determine the percentage of time that the parking area is occupied by the vehicle. In another example, the one or more processors are further configured to further analyze the observation to determine the possibility that the parking area is available during multiple different time periods. In another example, the one or more processors are further configured to provide map information to the autonomous vehicle so that the autonomous vehicle can use the map information to make driving decisions. In another example, the one or more processors are further configured to use the observation to train the machine learning model to provide the possibility that the parking area is occupied at a future point in time. In another example, the one or more processors are further configured to use the map information to identify potential locations for the vehicle to stop and pick up or drop off passengers or cargo. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a functional diagram of an example vehicle according to an exemplary embodiment.

[0008] Figure 2 is an example of map information according to aspects of the present disclosure.

[0009] Figure 3 are example exterior views of a vehicle according to aspects of the present disclosure.

[0010] Figure 4 is a schematic diagram of an example system according to aspects of the present disclosure.

[0011] Figure 5 According to various aspects of the present disclosure Figure 4 Functional diagram of the system.

[0012] Figure 6 is an example of log data and map information according to aspects of the present disclosure.

[0013] Figure 7 are examples of edges and sub-portions according to aspects of the present disclosure.

[0014] Figure 8 are examples of log data, map information, and data according to aspects of the present disclosure.

[0015] Fig. 9 are examples of log data, map information, and data according to aspects of the present disclosure.

[0016] Fig.10 is an example of annotated map information in accordance with aspects of the present disclosure.

[0017] Fig.11 is an example flow chart according to aspects of the present disclosure. DETAILED DESCRIPTION

[0018] Overview

[0019] The present technology relates to identifying parking areas for autonomous vehicles. In many cases, parking areas (such as parking spaces and parking lots) can be known from any number of different sources (such as city maps, parking databases, etc.). However, it may be that in many cases, parking areas are not necessarily defined, but rather it is decided by a person to park in such an area at a certain time. For example, parking areas are not always adjacent to curbs or along straight sections of roads, especially in residential areas. As another example, during religious ceremonies or near busy parks on weekends, it may be considered acceptable for a vehicle to park in an area that was originally a non-parking area (such as along a median, a turning lane, or a no-parking zone). In other examples, a person may park a vehicle on the side of a street without a curb.

[0020] Therefore, these parking areas are not necessarily pre-mapped or otherwise known, and therefore many parking areas are not necessarily available or even recognizable to autonomous vehicles. Manually mapping these parking areas on a map is possible, but has many disadvantages: it is costly, requires clear specifications to be defined in advance, requires iterative and slow creation and rollout, requires work every time the road map changes, and requires a way to detect when annotations are out of date and require re-marking. However, many of the above parking areas can be identified from the deposited data generated by the autonomous vehicle.

[0021] As autonomous vehicles drive around, their various systems can detect and identify objects, such as parked or slow-moving vehicles. This stored data can be analyzed to make certain determinations. For example, each time a detected object is identified or marked as a parked vehicle, this can be considered an "observation."

[0022] To determine whether a vehicle has been observed parking at or near various features, the observations can be used to analyze each edge sub-portion, and each edge sub-portion can be classified or labeled. Further, these observations can be analyzed to determine other details about the parking area for a given sub-portion.

[0023] These classifications and details can be used in any number of different ways. In some instances, the details and additional details can be used to annotate map information, and the annotated map information can be used for any number of different purposes as described below. In addition to or in lieu of annotating the map information, the observations, details, and additional details can be used to train a model. The output of the aforementioned annotated map information and / or model can then be used in a variety of ways.

[0024] The features described herein may allow identification of parking areas that are not necessarily defined, but rather are determined by a person to park in such an area at a certain time. In other words, the features described herein allow identification of parking areas that are not necessarily defined by city maps, parking databases, etc.

[0025] Example System

[0026] like Figure 1 As shown, a vehicle 100 according to one aspect of the present disclosure includes various components. Although certain aspects of the present disclosure are particularly useful in conjunction with specific types of vehicles, the vehicle can be any type of vehicle, including but not limited to cars, trucks, motorcycles, buses, recreational vehicles, etc. The vehicle can have one or more computing devices, such as a computing device 110 that includes one or more processors 120, memory 130, and other components typically found in a general-purpose computing device.

[0027] The memory 130 stores information accessible by one or more processors 120, including instructions 132 and data 134 that can be executed or otherwise used by the processor 120. The memory 130 can be any type of memory capable of storing information accessible by the processor, including computing device readable media or other media that stores data that can be read by electronic devices (such as hard drives, memory cards, ROM, RAM, DVDs or other optical disks), as well as other writable memories and read-only memories. The systems and methods may include different combinations of the foregoing, whereby different portions of the instructions and data are stored on different types of media.

[0028] Instructions 132 may be any set of instructions that are directly executed by a processor (such as machine code) or indirectly executed (such as a script). For example, instructions may be stored as computing device code on a computing device readable medium. In this regard, the terms "instructions" and "programs" may be used interchangeably herein. Instructions may be stored in an object code format for direct processing by a processor or in any other computing device language (including a collection or script of independent source code modules that are interpreted on demand or pre-compiled). Below, the functions, methods, and routines of instructions will be explained in more detail.

[0029] Processor 120 may retrieve, store, or modify data 134 according to instructions 132. For example, although the claimed subject matter is not limited to any particular data structure, the data may be stored in a computing device register, in a relational database (as a table with multiple different fields and records), an XML document, or a flat file. The data may also be formatted in any computing device readable format.

[0030] The one or more processors 120 may be any conventional processor, such as a commercially available CPU or GPU. Alternatively, the one or more processors may be a dedicated device, such as an ASIC or other hardware-based processor. Figure 1 The processor, memory, and other elements of computing device 110 are functionally shown as being within the same box, but one of ordinary skill in the art will appreciate that a processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be housed within the same physical housing. For example, the memory may be a hard drive or other storage medium located in a housing different from that of computing device 110. Thus, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0031] Computing device 110 may include all components typically used in conjunction with a computing device, such as the processor and memory described above, as well as user input 150 (e.g., one or more buttons, a mouse, a keyboard, a touch screen, and / or a microphone), various electronic displays (e.g., a monitor with a screen or any other electrical device operable to display information), and speakers 154 for providing information to passengers of vehicle 100 or to others as desired. For example, electronic display 152 may be located within the cabin of vehicle 100 and may be used by computing device 110 to provide information to passengers within vehicle 100.

[0032] The computing device 110 may also include one or more wireless network connections 156 to facilitate communication with other computing devices, such as client computing devices and server computing devices described in detail below. The wireless network connections may include short-range communication protocols such as Bluetooth, Bluetooth Low Energy (LE), cellular connections, and various configurations and protocols, including the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local network, a private network using one or more company-proprietary communication protocols, Ethernet, WiFi, and HTTP, and various combinations of the foregoing.

[0033] The computing device 110 may be part of an autonomous control system of the vehicle 100 or may be capable of communicating with various components of the vehicle in order to control the vehicle in an autonomous driving mode. Figure 1The computing device 110 can communicate with various systems of the vehicle 100 to control the movement, speed, etc. of the vehicle 100 according to the instructions 132 of the memory 130 in the autonomous driving mode. The various systems of the vehicle 100 are such as a deceleration system 160, an acceleration system 162, a steering system 164, a signal system 166, a planning system 168, a routing system 170, a positioning system 172, a perception system 174, a behavior modeling system 176 and a power system 178.

[0034] As an example, computing device 110 may interact with deceleration system 160 and acceleration system 162 to control the speed of the vehicle. Similarly, computing device 110 may use steering system 164 to control the direction of vehicle 100. For example, if vehicle 100 is configured for use on a road (such as a car or truck), the steering system may include components that control the angle of the wheels to turn the vehicle. Computing device 110 may also use signaling system 166 to signal the vehicle's intentions to other drivers or vehicles (e.g., by illuminating a turn signal or brake light when necessary).

[0035] Computing device 110 may use routing system 170 to generate a route to a destination using map information. Computing device 110 may use planning system 168 to generate a short-term trajectory that allows the vehicle to follow the route generated by the routing system. In this regard, planning system 168 and / or routing system 170 may store detailed map information, for example, a highly detailed map identifying a road network including the shape and height of the road, lane lines, intersections, crosswalks, speed limits, traffic signals, buildings, signs, real-time traffic information (updated as received from a remote computing device such as computing device 410 or other computing devices discussed below), pull-overs, vegetation, or other such objects and information.

[0036] Figure 2 is an example of map information 200 for a small road segment in a road. The map information 200 includes information identifying the shape, location, and other characteristics of lane markings or lane lines 210, 212, 214 (which define lanes 220, 222). The map information also identifies the shape, location, and other characteristics of a shoulder area 230 and a curb 232 adjacent to the shoulder area. In addition to the above features and information, the map information may also include information identifying the direction of traffic for each lane and information that allows the computing device 110 to determine whether the vehicle has the right of way to complete a particular maneuver (i.e., to complete a turn or to cross a lane of traffic or an intersection).

[0037] In addition to the aforementioned physical feature information, the map information can be configured as a road map, which includes a plurality of graphic nodes that together form a road network of the map information and an edge representing a road or lane segment. Each edge is defined by a starting graphic node with a specific geographic location (e.g., latitude, longitude, altitude, etc.), an ending graphic node with a specific geographic location (e.g., latitude, longitude, altitude, etc.), and a direction. The direction may refer to the direction in which the vehicle 100 must move in order to follow the edge (i.e., the direction of the traffic flow). The graphic nodes may be located at a fixed or variable distance. For example, the spacing of the graphic nodes may range from a few centimeters to a few meters, and may correspond to the speed limit of the road where the graphic node is located. In this regard, a faster speed may correspond to a greater distance between the graphic nodes. The edge may represent driving along the same lane or changing lanes. Each node and edge may have a unique identifier, such as the latitude and longitude position of the node or the starting and ending position or node of the edge. In addition to the nodes and edges, the map may also identify additional information, such as the type of maneuver required at different edges and which lanes are drivable.

[0038] For example, Figure 2 Also depicted are a plurality of nodes s, t, u, v, w, x, y, z, and edges 240, 242, 244, 246, 248, 250, 252 extending between such pairs of nodes. For example, edge 240 extends between node s (the starting node of edge 240) and node t (the ending node of edge 240), edge 242 extends between node t (the starting node of edge 242) and node u (the ending node of edge 242), and so on.

[0039] The routing system 166 can use the aforementioned map information to determine a route from a current location (e.g., the location of the current node) to a destination. Routes can be generated using a cost-based analysis that attempts to select a route to a destination at the lowest cost. Costs can be evaluated in any number of ways, such as time to destination, driving distance (each edge may be associated with a cost to traverse the edge), type of maneuver required, convenience to passengers or vehicles, etc. Each route can include a list of multiple nodes and edges that a vehicle can use to reach a destination. Routes can be recalculated periodically as a vehicle travels to a destination.

[0040] The map information used for route guidance (routing) can have the same or different maps as the maps used for planning the trajectory. For example, the map information used for planning the route requires not only information about the individual lanes, but also the nature of the lane boundaries (e.g., solid white line, dashed white line, solid yellow line, etc.) to determine the location where lane changes are allowed. However, unlike the map used for planning the trajectory, the map information used for route guidance does not need to include other details, such as the location of crosswalks, traffic lights, stop signs, etc., although some of this information may be useful for the purpose of route guidance. For example, between a route with a large number of intersections with traffic controls (such as stop signs or traffic lights) and a route with no or few traffic controls, the latter route may have a lower cost (e.g., because it is faster) and is therefore preferred.

[0041] The computing device 110 may use a positioning system 172 to determine the relative or absolute position of the vehicle on a map or on the earth. For example, the positioning system 172 may include a GPS receiver to determine the latitude, longitude, and / or altitude position of the device. Other location systems (such as laser-based positioning systems, inertial assisted GPS, or camera-based positioning) may also be used to identify the location of the vehicle. The location of the vehicle may include an absolute geographic location (such as latitude, longitude, and altitude), the location of a node or edge of a road map, and relative location information, such as relative to the location of other cars in its immediate surroundings, which can typically be determined with less noise than the absolute geographic location.

[0042] The positioning system 172 may also include other devices (such as accelerometers, gyroscopes, or another direction / speed detection device) in communication with the computing device 110 to determine the direction and speed of the vehicle or changes thereto. By way of example only, an acceleration device may determine its pitch, yaw, or roll (or changes thereto) relative to the direction of gravity or a plane perpendicular to the direction of gravity. The device may also track increases or decreases in speed and the direction of such changes. The provision of position and orientation data by a device as set forth herein may be automatically provided to the computing device 110, other computing devices, and combinations of the foregoing.

[0043] The perception system 174 also includes one or more components for detecting objects external to the vehicle, such as other vehicles, obstacles in the road, traffic signals, signs, trees, etc. For example, the perception system 174 may include LIDAR, sonar, radar, cameras, and / or any other detection device that records data that can be processed by the computing device 110. In the case where the vehicle is a passenger vehicle such as a minivan, the minivan may include lasers or other sensors mounted on the roof or other convenient location.

[0044] For example, Figure 3is an example external view of the vehicle 100. In this example, the roof-top housing 310 and the dome housing 312 can include LIDAR sensors and various cameras and radar units. In addition, the housing 320 located at the front end of the vehicle 100 and the housings 330, 332 on the driver's side and passenger side of the vehicle can each store a LIDAR sensor. For example, the housing 330 is located in front of the driver's door 360. The vehicle 100 also includes housings 340, 342 for radar units and / or cameras that are also located on the roof of the vehicle 100. Additional radar units and cameras (not shown) can be located at the front and rear ends of the vehicle 100 and / or at other locations along the roof or top housing 310.

[0045] The computing device 110 may be able to communicate with various components of the vehicle to control the movement of the vehicle 100 according to the primary vehicle control code of the memory of the computing device 110. For example, returning Figure 1 , the computing device 110 may include various computing devices that communicate with various systems of the vehicle 100, such as a deceleration system 160, an acceleration system 162, a steering system 164, a signal system 166, a planning system 168, a routing system 170, a positioning system 172, a perception system 174, a behavior modeling system 176 and a power system 178 (i.e., the engine or motor of the vehicle) to control the movement, speed, etc. of the vehicle 100 according to the instructions 132 of the memory 130.

[0046] Various systems of the vehicle can be operated using autonomous vehicle control software to determine how to control the vehicle and control the vehicle. As an example, the perception system software module of the perception system 174 can use sensor data generated by one or more sensors of the autonomous vehicle (such as cameras, LIDAR sensors, radar units, sonar units, etc.) to detect and identify objects and the characteristics of the objects. These characteristics may include location, type, direction, orientation, speed, acceleration, change in acceleration, size, shape, etc. In some instances, the characteristics can be input into the behavior prediction system software module of the behavior modeling system 176, which uses various behavior models based on the object type to output the predicted future behavior of the object for detection. In other instances, the characteristics can be input into one or more detection system software modules, such as a traffic light detection system software module configured to detect the state of a known traffic signal, a construction zone detection system software module configured to detect a construction zone from sensor data generated by one or more sensors of the vehicle, and an emergency vehicle detection system configured to detect an emergency vehicle from sensor data generated by the vehicle's sensors. Each of these detection system software modules can use various models to output the possibility that the construction zone or object is an emergency vehicle. Detected objects, predicted future behaviors, various possibilities from the detection system software module, map information identifying the vehicle environment, positioning information from the positioning system 172 identifying the position and orientation of the vehicle, the destination or node of the vehicle, and feedback from various other systems of the vehicle can be input into the planning system software module of the planning system 168. The planning system 168 can use this input to generate a trajectory for the vehicle to follow for a short period of time in the future based on the route generated by the routing module of the routing system 170. In this regard, the trajectory can define specific characteristics of acceleration, deceleration, speed, etc. to allow the vehicle to follow a route towards reaching the destination. The control system software module of the computing device 110 can be configured to control the movement of the vehicle (e.g., by controlling the braking, acceleration, and steering of the vehicle) in order to follow the trajectory.

[0047] The computing device 110 can control the vehicle in the autonomous driving mode by controlling various components. For example, as an example, the computing device 110 can use data from the detailed map information and the planning system 168 to fully autonomously navigate the vehicle to the destination location. The computing device 110 can use the positioning system 172 to determine the location of the vehicle, and use the perception system 172 to detect objects and respond to objects when necessary to safely reach the location. Similarly, to this end, the computing device 110 and / or the planning system 168 can generate trajectories and cause the vehicle to follow these trajectories, for example, by accelerating the vehicle (e.g., by supplying fuel or other energy to the engine or power system 178 by the acceleration system 162), decelerating (e.g., by reducing the fuel supplied to the engine or power system 178, shifting gears and / or applying brakes through the deceleration system 160), changing direction (e.g., turning the front wheels or rear wheels of the vehicle 100 through the steering system 164), and using the signal system 166 to signal such changes (e.g., by illuminating the turn signal). Thus, acceleration system 162 and deceleration system 160 may be part of a transmission system that includes various components between the vehicle's engine and the vehicle's wheels. Likewise, by controlling these systems, computing device 110 may also control the vehicle's transmission system to autonomously maneuver the vehicle.

[0048] The computing device 110 of the vehicle 100 may also receive information from or transmit information to other computing devices, such as those computing devices that are part of the transportation service and other computing devices. Figure 4 and Figure 5 4 and 460, respectively, are schematic and functional diagrams of an example system 400 that includes a plurality of computing devices 410, 420, 430, 440 and a storage system 450 connected via a network 460. The system 400 also includes a vehicle 100A and a vehicle 100B that may be configured the same or similar to the vehicle 100. Although only a few vehicles and computing devices are described for simplicity, a typical system may include significantly more vehicles and computing devices.

[0049] like Figure 5 As shown, each of computing devices 410, 420, 430, 440 may include one or more processors, memory, data, and instructions. Such processors, memory, data, and instructions may be configured similarly to one or more processors 120, memory 130, data 134, and instructions 132 of computing device 110.

[0050] The network 460 and the intermediate graph nodes may include a variety of configurations and protocols, including short-range communication protocols such as Bluetooth, Bluetooth LE, the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local area network, a private network using one or more company-proprietary communication protocols, Ethernet, WiFi, and HTTP, and various combinations of the foregoing. Such communications may be facilitated by any device capable of sending and receiving data to and from other computing devices, such as modems and wireless interfaces.

[0051] In one example, one or more computing devices 410 may include one or more server computing devices (e.g., a load balancing server cluster) having multiple computing devices, the server computing devices exchanging information with different nodes of the network for the purpose of receiving data from other computing devices, processing data, and sending data to other computing devices. For example, one or more computing devices 410 may include one or more server computing devices capable of communicating with computing device 110 of vehicle 100 or similar computing devices of vehicles 100A, 100B and computing devices 420, 430, 440 via network 460. For example, vehicles 100, 100A, 100B may be part of a fleet that can be dispatched to various locations by server computing devices. In this regard, server computing device 410 may be used as a fleet management system that can be used to dispatch vehicles such as vehicles 100, 100A, 100B to different locations to pick up and drop off passengers. In addition, computing device 410 may use network 460 to send information to users such as users 422, 432, 442 and present the information on displays such as displays 424, 434, 444 of computing devices 420, 430, 440. In this regard, computing devices 420, 430, 440 may be considered client computing devices.

[0052] like Figure 4 As shown, each of the client computing devices 420, 430 can be a personal computing device for use by a user 422, 432, and has all components commonly used in conjunction with a personal computing device, including one or more processors (e.g., a central processing unit (CPU)), memory (e.g., RAM and an internal hard drive) for storing data and instructions, a display such as a display 424, 434, 444 (e.g., a monitor with a screen, a touch screen, a projector, a television, or other device operable to display information), and a user input device 426, 436, 446 (e.g., a mouse, keyboard, touch screen, or microphone). The client computing device may also include a camera for recording a video stream, a speaker, a network interface device, and all components for connecting these elements to each other.

[0053] Although each of the client computing devices 420, 430 may include a full-size personal computing device, they may alternatively include a mobile computing device capable of wirelessly exchanging data with a server over a network such as the Internet. By way of example only, the client computing device 420 may be a mobile phone or a device capable of obtaining information via the Internet or other network, such as a wireless-enabled PDA, a tablet PC, a wearable computing device or system, or a netbook. In another example, the client computing device 430 may be a wearable computing system such as a Figure 4 As an example, the user can use a small keyboard, a keypad, a microphone, use visual signals with a camera, or a touch screen to enter information. As another example, the client computing device 440 can be a desktop computing system including a keyboard, a mouse, a camera, and other input devices.

[0054] Each client computing device may be a remote computing device used by a person (e.g., a human operator or user 422, 432, 442) to view and analyze sensor data and other information generated by a perception system of a vehicle (such as perception system 174 of vehicle 100). For example, user 442 may use client computing device 440 to view visualizations generated as discussed herein. Figure 4 and Figure 5 Only a few remote computing devices are shown, but any number of such workstations would be included in a typical system.

[0055] As with memory 130, storage system 450 may be any type of computerized storage device capable of storing information accessible by server computing device 410, such as a hard drive, memory card, ROM, RAM, DVD, CD-ROM, writable memory, and read-only memory. In addition, storage system 450 may include a distributed storage system in which data is stored on multiple different storage devices that may be physically located in the same or different geographic locations. Storage system 450 may be accessed through, for example, Figure 4 and Figure 5 The network 460 is shown connected to the computing devices, and / or may be directly connected to or incorporated into any of the computing devices 110, 410, 420, 430, 440, etc.

[0056] The storage system 450 may store various types of information as described in more detail below. The information may be retrieved or otherwise accessed by a server computing device such as one or more server computing devices 410 to perform some or all of the features described herein. For example, the storage system 450 may store log data. The log data may include data generated by various systems of a vehicle such as the vehicle 100 when the vehicle is operating in a manual driving mode or an autonomous driving mode. For example, the log data may include sensor data generated by a perception system such as the perception system 174 of the vehicle 100. As an example, the sensor data may include raw sensor data and data identifying defined characteristics of a perception object (such as objects such as vehicles, pedestrians, cyclists, vegetation, curbs, lane lines, sidewalks, crosswalks, buildings, etc., such as shape, position, orientation, speed, etc.).

[0057] As the autonomous vehicle drives around, various systems of the autonomous vehicle can detect and identify objects, such as parked or slow-moving vehicles. For example, when an object is classified as a vehicle, a parked vehicle classifier can be used to determine whether the vehicle is parked or temporarily stopped (e.g., due to a traffic condition such as a red light or stop sign) and add an appropriate tag. In this regard, the sensor data logged into the data can also identify observations of parked or stopped vehicles. Additionally or alternatively, the logged data can be viewed by a human operator who can verify the vehicle and / or add tags to the vehicle.

[0058] The log data may also include "event" data that identifies different types of events, such as collisions or near collisions with other objects, planned trajectories described as the geometry and / or speed of potential path plans for the vehicle, the actual position of the vehicle at different times, the actual heading / direction of the vehicle at different times, the actual speed, acceleration and deceleration of the vehicle at different times, classification of and responses to sensed objects, predictions of behavior of sensed objects, the state of various systems of the vehicle (such as acceleration, deceleration, perception, steering, signals, routing, planning, power, etc.) at different times including stored errors, inputs and outputs of various systems of the vehicle at different times, etc.

[0059] Thus, the events and sensor data can be used to "reconstruct" the vehicle environment, including the behavior of sensed objects and vehicles in the simulation. In some instances, the log data can be annotated with information identifying the behavior of the autonomous vehicle (such as passing, lane changes, merging, etc.) and information identifying the behavior of other agents or objects in the log data (such as passing or passing the autonomous vehicle, lane changes, merging, etc.). Additionally or alternatively, the "log data" can be simulated, that is, the "log data" can be created by a human operator rather than generated from a real vehicle driven in the world.

[0060] Server computing device 410 can use the log data to run simulations. These simulations can be log-based simulations that are run using actual log data or simulated log data collected by the vehicle over a short period of time as the vehicle approaches a destination. At the same time, the actual vehicle is replaced with a virtual or simulated autonomous vehicle that can make decisions using the autonomous vehicle control software. By using simulations, the autonomous vehicle control software can be rigorously evaluated.

[0061] The log data and / or the results of a simulation based on the actual or simulated log data may include or otherwise be associated with an identified pullover location selected by the autonomous vehicle control software. In this regard, storage system 450 may also store this information. In some instances, an actual or simulated vehicle may not actually be able to stop at an identified pullover location due to certain circumstances (such as obstructing objects in the pullover location, etc.). In such cases, the identified pullover location may be used for evaluation purposes even though it is not the "real" pullover location of the actual or simulated vehicle.

[0062] Example Method

[0063] In addition to the operations described above and shown in the figures, various operations will now be described. It should be understood that the following operations do not have to be performed in the exact order described below. Instead, the various steps may be processed in a different order or simultaneously, and steps may be added or omitted.

[0064] Fig.11 1 is an example flow chart 1100 for identifying a parking area, which may be executed by one or more processors of one or more computing devices, such as a processor of the server computing device 410 or a processor of any of the client computing devices 420, 430, 440. At block 1110, an observation of a parked vehicle is identified from the stored data.

[0065] As described above, when an autonomous vehicle, such as vehicles 100, 100A, 100B, drives around, various systems of the autonomous vehicle can detect and identify objects, such as parked or slow-moving vehicles. For example, when an object is classified as a vehicle, a parked vehicle classifier can be used to determine whether the vehicle is parked or temporarily stopped (e.g., due to a traffic condition such as a red light or stop sign) and add an appropriate tag. Additionally or alternatively, the logged data can be viewed by a human operator who can verify the vehicle and / or add a tag to the vehicle.

[0066] In some cases, the stored data may be analyzed to make certain determinations. For example, each time a detected object is identified or labeled as a parked vehicle, this may be considered an "observation" of a parked vehicle. Additionally, observations of empty lanes (i.e., lanes observed without parked vehicles) may also be used to be able to calculate the percentage of time a given lane is seen occupied versus empty.

[0067] Figure 6 An example of log data 600 that may have been captured by various systems of vehicle 100 in a geographic area corresponding to the geographic area of ​​map information 200 is shown. In this regard, map information 200 is also depicted for reference. In this example, log data 600 includes information identifying various detected objects and characteristics of those objects (e.g., location, type, direction, orientation, speed, acceleration, change in acceleration, size, shape, etc.). Therefore, log data 600 identifies objects 610, 620, 630, 640 and the above-mentioned characteristics of each of these objects and other information such as behavior prediction. In this example, each of these objects can be associated with a mark that identifies the object as a parked vehicle. Therefore, each of these objects can be considered to be an "observation" of a parked vehicle as described above. In addition, each of these objects can be represented by one or more bounding boxes generated by perception system 174.

[0068] In some instances, these bounding boxes may be adjusted or refined by the vehicle's computing device or in an offline process (e.g., at a server computing device) using various smoothing and filtering techniques (such as those described in U.S. Pat. No. 8,736,463, which is incorporated herein by reference), which involves attempting to increase or maximize the average density of noisy data points along the edges of the bounding box by adjusting the parameters of the bounding box (including at least one of the heading, location, orientation, and dimension). This may thereby improve the dimensions of the bounding box as well as the pose (location and orientation) of the parked vehicle. This may result in the identification of false positives and possible classification of false negatives of parked vehicles.

[0069] The sensor data may include LIDAR data points, camera images, radar images, and / or sensor data generated by other types of sensors. The log data may also identify additional detected objects corresponding to features of the map information, such as lane markings or lane lines 210, 212, 214, lanes 220, 222, shoulder areas 230, and curbs 232.

[0070] In some cases, server computing device 410 may discard repeated observations of the same parked vehicle by different autonomous vehicles at approximately the same time. For example, if both vehicle 100 and vehicle 100B will drive through an area of ​​map information 200 within a short period of time (such as a few minutes or more or less), then observations of objects 610, 620, 630, 640 for one of these vehicles may be discarded.

[0071] Back to Fig.11 In block 1120, observations are used to determine whether sub-portions of an edge of a road map correspond to a parking area, wherein the edge defines a drivable area in the road map. In order to determine whether a vehicle has been observed to park at or near various features, observations within a given geographic area may be analyzed to determine whether a vehicle has been observed to park at or near various features in the road map (e.g., an edge). The analysis may include the server computing device 410 reviewing areas adjacent to a sub-portion of each edge and classifying or labeling the areas. As an example, an edge may be subdivided into two or more sub-portions, such as 0.2 meters from the beginning of the edge, 0.2 to 0.4 meters from the beginning of the edge, 0.4 to 0.6 meters from the beginning of the edge, etc. Segmenting the edge may provide more useful granularity in identifying a parking area (e.g., not all edges may be adjacent to a completely parking area, and not all parking areas may be the same length).

[0072] Areas directly adjacent to subportions occupied by parked vehicles (e.g., vehicles detected and classified as parked) may be identified as parking areas. For example, for each observation of a parked vehicle, the server computing device may identify the nearest subportion of an edge. A limit or threshold may be placed on the distance between an observation of a parked vehicle and the nearest edge (or lane center), such as 50 meters or more or less. Above this threshold, observations of parked vehicles will not be associated with any lane or any subportion of an edge, and therefore may be discarded (possibly as a false positive), although this may not be necessary in more urban areas.

[0073] Figure 7 An example representation of sub-portions of an edge of map information 200 is provided. In this example, for simplicity and ease of understanding, each edge is divided into two sub-portions, although significantly more sub-portions may also be used in order to extract finer granularity in the analysis. Each edge in edge SZ is divided into sub-portions, resulting in sub-portions 701-715. As an example, edge 240 is subdivided into sub-portion 701 and sub-portion 702, edge 242 is subdivided into sub-portion 703 and sub-portion 704, and so on. The sub-portions are depicted as adjacent edges only for ease of representation.

[0074] Figure 8 The log data 600 depicts Figure 7 2-715 of the sub-portions. Using this example, the "left" and "right" areas of the sub-portions of the edge SZ can be analyzed by the server computing device 410. In this regard, the server computing device 410 can look for observations of parked vehicles in both the lane 222 and the shoulder area 230. In this example, observations of objects 610, 620, 630, 640 marked as parked vehicles are analyzed to determine that each of the sub-portions 703-713 was occupied (at least partially) by a parked vehicle at the time the log data 600 was captured, and is therefore "parkable" or more precisely, a parking area. Although the examples herein relate to analyzing observations from log data from only a single vehicle, observations from multiple vehicles can be aggregated and analyzed together.

[0075] Furthermore, while the examples herein describe analysis of log data using only edges in lane 220, a similar process may also occur for other edges in the map information, such as those in lane 222 (not depicted). In this regard, server computing device 410 may look for observations of parked vehicles in both lane 220 (below lane 222) and in the area above lane 222.

[0076] In addition, subsections adjacent to parking areas may be "grouped" to more specifically identify which areas are parking areas. In this regard, turning to the description of log data, map information, and additional data Fig. 9 , areas adjacent to sub-portions 703 - 715 may be grouped together into a single, larger parkable area 900 .

[0077] The server computing device 410 can further analyze the observations to determine details such as whether the observation is adjacent to the left or right of an edge, along an edge (e.g., in the center of a lane), or between two edges in opposite directions (e.g., between two lanes of traffic); the width of the parking area of ​​a given subsection (e.g., equal to or based on the width of the observed vehicle); the offset of the parking area of ​​a given subsection from the center of the lane (e.g., the nearest road map edge); the angle of the parked car relative to the edge (e.g., parallel or perpendicular); whether the vehicle is backed into position and the percentage of vehicles that are backed into position; the percentage of time that the parking area of ​​a given subsection is occupied or unoccupied (which can be described with further granularity by bucketizing this information into different time periods (e.g., different times of the day, weekdays versus weekends, etc.); whether the parking area of ​​a given subsection is designated for a particular type of vehicle (e.g., designated for buses or designated for taxis by looking for "taxi" signs, color characteristics, etc.); whether there are signs nearby that may describe parking restrictions; the color of the curb adjacent to the given subsection; dimensions (e.g., the length and width of the observed parked vehicle), etc.

[0078] Analysis of the observations can be used to determine additional details about the parking areas for a given subdivision. For a given geographic area, this can include the percentage of parking areas, the average offset of parking areas from the center of the lane, the angle and direction (e.g., whether the vehicle will be driving forward or reverse), etc. In addition to collecting this information from the observations, analysis of the observations can also be used to make predictions about parking areas, such as the percentage of parking areas that may be available at any given time.

[0079] Back to Fig.11, at block 1130, map information is generated based on determining whether the sub-portion of the edge corresponds to a parkable area. The classifications, details, and additional details described above can be used in any number of different ways. In some examples, server computing device 410 can use the details and additional details to add annotations to map information (e.g., a road map), and the annotated map information can be used for any number of different purposes as described below. Annotations can include details and additional details, such as, for example, the width of the parking area for a given sub-portion (e.g., equal to or determined based on the width of the observed vehicle); the offset of the parking area for a given sub-portion from the center of the lane (e.g., the nearest road map edge); the angle of the parked cars relative to the edge (e.g., parallel or perpendicular); whether the vehicle is backing in and the percentage of vehicles backing in; the percentage of time that the parking area for a given sub-portion is occupied or unoccupied (which can be described with further granularity by bucketing this information for different time periods (such as different times of day, weekdays vs. weekends, etc.); whether the parking area for a given sub-portion is designated for a particular type of vehicle (e.g., designated for buses or designated for taxis by looking for "taxi" signs, color characteristics, etc.); whether there are signs nearby that may describe parking restrictions; the color of the curb adjacent to the given sub-portion; dimensions (e.g., length and width of observed parked vehicles), percentage of parking area, average offset of the parking area from the center of the lane, angle and direction (e.g., whether the vehicle will be driving forward or backing in), etc.

[0080] The offset and width of the parking area can be used to determine the 2D polygon. Fig.10 is an example of annotated map information 200', including Figure 8 and Fig. 9 The observed area depicted in FIG. 2 corresponds to a 2D polygon of a parking area 250 .

[0081] In addition to or in lieu of annotating the map information, the observations, details, and additional details may be used to train a model. The model may be a machine learning model that takes an embedding of a road map or a vectorized representation of a portion of a road map and outputs a list of road map features (e.g., edges) and values ​​identifying the likelihood that these road map features are or are not adjacent to a parking area. The model may be used by a computing device of an autonomous vehicle, either offline or in real time, for any number of different purposes described further below.

[0082] The model can be trained using the above observations as training outputs and the corresponding portion of the road map as training input. In some instances, the results of the above analysis can also be used as training outputs. Thus, the model can be trained using high confidence historical data.

[0083] In one example, training can be performed using unsupervised techniques by making assumptions about parking areas. For example, if cars frequently park in a lane taking into account nearby lanes, then the lane may be considered "parkable". This can reduce the likelihood of detecting erroneous parked cars in turning lanes, middle lanes, intersections, etc. As another example, if there are vehicles parked in an area along the lane for no less than a certain percentage of the time (such as 10% or more or less), then the area can be considered the most parked part of the lane. This type of training may involve utilizing an unsupervised (hyperparameter-based) clustering algorithm involving segmentation, natural break optimization, or kernel density estimation with threshold-based classification.

[0084] In many instances, there may be few or no observations of parked vehicles at certain times of the day or certain days of the week. Various techniques can be used to infer parking areas in unseen areas or areas with few observations. As an example, the model can be trained on available data and then used to make predictions for missing data (for unobserved lanes and / or one of the days / hours).

[0085] The output of the aforementioned annotated map information and / or model may be used in various ways. In this regard, the output of the annotated map information and / or model may be sent to or otherwise downloaded to a memory of the autonomous vehicle for use by various systems of the autonomous vehicle. As an example, the annotated map information may be provided as a map update by server computing device 410 via network 460 to computing devices of vehicles in a fleet of autonomous vehicles, such as computing device 110 of vehicles 100, 100A, 100B.

[0086] For example, the output of the annotated map information and / or model can be used by the server computing device 410 and / or the computing device of the autonomous vehicle to identify parking areas in locations where there have been no or few observations. As an example, road map features can be input into the above model to provide a reasonable guess about the availability of parking in various areas. Knowing where possible parking areas are can assist the behavioral modeling system of the autonomous vehicle in generating behavioral predictions that identify whether the vehicle is parked, for example, as a check or supplement to the above-mentioned parking vehicle classifier to reduce false positive detections of parked or non-parked vehicles.

[0087] Similarly, this information may be useful to improve the determination by the autonomous vehicle's computing device via the autonomous vehicle's perception system, such as perception system 174 of vehicle 100, of whether another vehicle is engaged in the maneuver of parking. This may be particularly useful in parallel parking situations where there is another vehicle directly in front of vehicle 100. This, in turn, may improve the ability of vehicle 100 to react to a vehicle engaged in the maneuver of parking.

[0088] In other instances, this information can be used to make route guidance decisions by the autonomous vehicle's routing system. For example, routing system 170 can penalize routes where the vehicle will have to drive along roads that have parking on both sides (e.g., narrow roads). For example, if routing system 170 classifies narrow lanes based on lane width but knows that parallel parked cars are expected, the routing system can subtract the width of the potential parallel parked cars from the drivable lane width, and penalize (give a higher cost) edges in the road map that have a certain dimension or smaller width (e.g., where it would be difficult for two vehicles to pass at the same time). In addition, identifying potentially narrower lanes can be used to improve the estimated time to traverse such areas, as the autonomous vehicle will typically reduce its speed to improve safety.

[0089] The planning system of the autonomous vehicle can use this information when planning the trajectory. For example, the planning system 168 of the vehicle 100 can use this information to plan the geometry of the trajectory so as to position the vehicle for the occluded area. As an example, parked vehicles may block the area in front of them, which may cause the autonomous vehicle to be unable to detect whether other vehicles are parked there, or to detect oncoming traffic in narrow passages or around bends. In this regard, in such a situation, the planning system can plan a trajectory that avoids certain roads that may cause such a situation, or the routing system can plan a route that avoids certain roads that may cause such a situation. As another example, the planning system 168 can plan the trajectory by penalizing those vehicles driving in the lanes of adjacent parked cars, or the routing system 170 can plan the route by penalizing those vehicles driving in the lanes of adjacent parked cars.

[0090] Computing device 410 or other computing devices (e.g., computing devices of a dispatch system) may also use this information to make decisions regarding pickup and drop-off locations. For example, when suggesting trip locations, locations near areas where little parking availability is expected may be indicated as less desirable or unavailable. As another example, knowing that some areas that are not normally available for parking are available for parking at certain times (such as the examples described above) may better allow the autonomous vehicle to find a location to park when it is ready to pull over to pick up or drop off passengers or cargo, or simply wait between trips.

[0091] The features described herein can allow identification of parking areas that are not necessarily defined, but rather a person decides to park in such an area at a particular time. In other words, the features described herein allow identification of parking areas that are not necessarily defined by a city map, parking database, etc.

[0092] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the above-described features can be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be understood by way of illustration, rather than by limiting the subject matter defined by the claims. In addition, the provision of examples described herein, as well as clauses phrased as "such as," "including," etc., should not be interpreted as limiting the subject matter of the claims to specific examples; rather, these examples are intended to illustrate only one of a variety of possible embodiments. In addition, the same reference numerals in different figures may identify the same or similar elements.

Claims

1. A method for identifying a parking area, the method comprises: identifying, by one or more processors, an observation of a parked vehicle from stored data; determining, by one or more processors, based on the observation, whether a sub - portion of a road map edge not identified as a parking area in the road map is a parking area, wherein the edge includes a distance between two graph nodes of the road map and defines a drivable area in the road map; and generating, in response to determining that the sub - portion of the road map edge not identified as a parking area in the road map is a parking area, map information associated with the area identified as the parking area, by one or more processors.

2. The method according to claim 1, further comprising using the observation to determine whether a second sub - portion of the edge corresponds to a second parking area.

3. The method according to claim 1, further comprising further analyzing the observation to determine whether the parking area is to the left of the edge.

4. The method according to claim 1, further comprising further analyzing the observation to determine whether the parking area is along the edge.

5. The method according to claim 1, further comprising further analyzing the observation to determine whether the parking area is between the edge and a second edge of the road map.

6. The method according to claim 1, further comprising further analyzing the observation to determine the width of the parking area.

7. The method according to claim 1, further comprising further analyzing the observation to determine the percentage of time the parking area is occupied by a vehicle.

8. The method according to claim 1, further comprising further analyzing the observation to determine the likelihood that the parking area is available during multiple different time periods.

9. The method according to claim 1, further comprising using the observation to train a machine - learning model to provide the likelihood that the parking area will be occupied at a future point in time.

10. The method according to claim 9, further comprising further analyzing the observation to determine the percentage of time the parking area is occupied by a vehicle and using the percentage of time to train the model.

11. The method according to claim 9, further comprising providing the model to an autonomous vehicle so that the autonomous vehicle can use the model to make driving decisions.

12. The method according to claim 1, further comprising using the map information to identify potential locations for a vehicle to stop and pick up or drop off passengers or cargo.

13. A system for identifying a parking area, the system comprises: a memory storing stored data; and one or more processors configured to: identify an observation of a parked vehicle from the stored data; determine, based on the observation, whether a sub - portion of a road map edge not identified as a parking area in the road map is a parking area, wherein the edge includes a distance between two graph nodes of the road map and defines a drivable area in the road map; and generate, in response to determining that the sub - portion of the road map edge not identified as a parking area in the road map is a parking area, map information associated with the area identified as the parking area.

14. The system according to claim 13, wherein the one or more processors are further configured to use the observation to determine whether a second sub - portion of the edge corresponds to a second parking area.

15. The system according to claim 13, wherein, the one or more processors are further configured to further analyze the observations to determine whether the parkable area is between an edge and a second edge of the road map.

16. The system according to claim 13, wherein, the one or more processors are further configured to further analyze the observations to determine the percentage of time that the parkable area is occupied by a vehicle.

17. The system according to claim 13, wherein, the one or more processors are further configured to further analyze the observations to determine the likelihood that the parkable area is available during multiple different time periods.

18. The system according to claim 13, wherein, the one or more processors are further configured to provide the map information to the autonomous vehicle to enable the autonomous vehicle to use the map information to make driving decisions.

19. The system according to claim 13, wherein, the one or more processors are further configured to use the observations to train a machine learning model to provide the likelihood that the parkable area will be occupied at a future point in time.

20. The system according to claim 13, wherein, the one or more processors are further configured to use the map information to identify potential locations for the vehicle to stop and pick up or drop off passengers or cargo.

Images