Live parking space monitoring and prediction in automotive fleets

Abstract

The disclosed systems and technologies are intended for tracking and predicting the live availability of parking space resources by a fleet of vehicles. [Solution] The disclosed technology includes receiving a fleet's communication vehicle having, from the vehicle's sensing system, identification of an object(s) located at the edge of the driving environment and edge visibility data of the edge; updating a map of live parking space occupancy rates in the driving environment using the received identification information and edge visibility data; determining, based on the updated maps of live parking space occupancy rates and historical parking space availability, that the probability values ​​associated with parking spaces in the driving environment remain vacant within a target time; and guiding the fleet's vehicles to parking spaces, at least based on the probability values.

Description

Technical Field

[0001] This specification generally relates to autonomous vehicles. More specifically, this specification relates to the detection of shared public resources such as parking spaces in an automotive environment.

Background Art

[0002] Autonomous (fully or partially autonomous) vehicles (AVs) operate by sensing the external environment with various electromagnetic (e.g., radar, and optical), as well as non-electromagnetic (e.g., audio, and humidity) sensors. Some autonomous vehicles chart a driving route within the environment based on the sensed data. The driving route can be determined based on Global Navigation Satellite System (GNSS) data, and road map data. GNSS, and road map data can provide information about static aspects of the environment (buildings, street layout, road closures, etc.), while dynamic information (information about other vehicles, pedestrians, streetlights, etc.) is obtained from simultaneously collected sensed data. The accuracy and safety of the driving route, as well as the accuracy and safety of the speed regime selected by the autonomous vehicle, depend on the timely and accurate identification of various objects present in the external environment, as well as the ability of the driving algorithm to process information about the environment and provide correct instructions to vehicle control, and the drivetrain.

[0003] This disclosure is shown by way of example and not limitation, and can be more fully understood when considered in connection with the following detailed description, with reference to the figures.

Brief Description of the Drawings

[0004] [Figure 1] FIG. 1 is a diagram showing the components of an exemplary autonomous vehicle (AV) in which a system for predicting the presence of target objects in a shielded area of a driving environment can be deployed, according to some implementations of the present disclosure. [Figure 2]Figure 2 shows examples of operational data flows directed towards tracking and predicting the live availability of shared resources by a fleet of vehicles, as demonstrated by several implementations of this disclosure. [Figure 3] Figure 3 shows an exemplary driving scenario that may be perceived by one or more agents of a fleet of autonomous vehicles as part of live parking space availability tracking in some implementations of this disclosure. [Figure 4] Figure 4 illustrates exemplary methods of tracking and predicting the live availability of parking resources for a fleet of autonomous vehicles, as performed by a fleet server, according to several implementations of the present disclosure. [Figure 5] Figure 5 illustrates exemplary methods of tracking and predicting the live availability of parking resources, performed by autonomous vehicles, for a fleet of autonomous vehicles, according to several implementations of the present disclosure. [Figure 6] Figure 6 shows a block diagram of an exemplary computer device that can track and predict the live availability of parking resources, as implemented by a fleet server for autonomous vehicles, according to several implementations of the present disclosure. [Overview of the Initiative]

[0005] One implementation discloses a system including a memory device and a processing device communicatively coupled to the memory device. The processing device receives a plurality of communications, each of which is received from each autonomous vehicle of a fleet of autonomous vehicles and includes at least (i) identification of one or more objects located at the edge of a drivable portion of a driving environment, and (ii) edge visibility data from the sensing system of each autonomous vehicle, representing the visibility of the edge of the drivable portion. The processing device further updates a map of live parking space occupancy in the driving environment using the identification of at least one or more objects and the edge visibility data, and at least determines a first probability value associated with a first parking space in the driving environment that is not occupied within a target time, based on the updated map of live parking space occupancy in the driving environment and past parking space availability. The processing device further guides a first autonomous vehicle of the fleet to the first parking space based at least the first probability value.

[0006] Another implementation discloses an autonomous vehicle including a sensing system for collecting sensing data that characterizes the driving environment of the autonomous vehicle. The autonomous vehicle further includes a data processing system for using the sensing data to generate at least (i) identification of one or more objects located at the edge of the drivable portion of the driving environment, and (ii) edge visibility data representing the visibility of the edge of the drivable portion of the driving environment. The autonomous vehicle further includes a transceiver for communicating the identification of at least one or more objects and the edge visibility data to a fleet server and receiving from the fleet server an identification of a first parking space in the driving environment. The autonomous vehicle further includes a vehicle control system for moving the autonomous vehicle to the first parking space.

[0007] In yet another implementation, disclosed is a vehicle fleet comprising multiple vehicles, each of which uses its own sensing system to collect sensing data characterizing the driving environment of each vehicle, and uses its own vehicle's data processing system and sensing data to generate curb occupancy data. The curb occupancy data includes at least (i) identification of one or more objects located on the curb in the driving environment, and (ii) curb visibility data representing the visibility of the curb in the driving environment. The vehicle fleet further includes a processing device that updates a map of live parking space occupancy with the curb occupancy data generated by the multiple vehicles, and determines a first probability value associated with a first parking space that is not occupied within a target time, based on the updated map of live parking space occupancy in the driving environment and past parking space availability. The processing device further guides a first vehicle of the multiple vehicles to a first parking space based on at least the first probability value. The first vehicle of the multiple vehicles then moves to the first parking space. [Modes for carrying out the invention]

[0008] Autonomous vehicles (AVs), or vehicles deploying various advanced driver-assistance features, can use multiple sensor modalities to facilitate the detection and identification of objects in the driving environment and track the trajectories of such objects. Sensors may include radio detection and ranging (radar) sensors, optical detection and ranging (lidar) sensors, multiple digital cameras, ultrasonic sensors, position sensors, etc. Different types of sensors may offer different complementary benefits. For example, radar and lidar emit electromagnetic signals (radio signals or optical signals) that are reflected from objects and transmit information about the distance to the object (e.g., from the time of flight of the signal) and the velocity of the object (e.g., from the Doppler shift of the frequency of the reflected signal). Radar and lidar can scan the entire 360-degree field of view by using a series of continuous sensing frames. Sensing frames may include numerous reflections covering the external environment in a high-density grid of return points. Each return point may be associated with the distance to the corresponding reflective object and the radial velocity of the reflective object (a component of the velocity along the line of sight).

[0009] LiDAR possesses high spatial resolution due to its submicron optical wavelengths, which makes it easy to obtain many close-up return points from the same object. This enables accurate detection and tracking of objects once they enter the range of the LiDAR sensor. Depending on the specific LiDAR model, LiDAR has an operating range of 150 to 350 m, and higher ranges are typically achieved with more powerful and expensive systems.

[0010] Radar sensors are inexpensive, require less maintenance than lidar sensors, have a wider operating range, and are more resistant to adverse weather conditions. As a result of using much longer wavelengths (radio), the resolution of radar data is much lower than that of lidar. In particular, while radar can accurately determine the speed of moving objects (rather than those moving too slowly for the radar receiver), detecting the precise location of an object can often be problematic.

[0011] Cameras (e.g., still cameras or video cameras) can acquire high-resolution images at both close range (where LiDAR operates) and long range (where LiDAR does not reach). A camera captures a two-dimensional projection of the three-dimensional external space onto the image plane (or some other non-planar imaging surface). Cameras have a longer operating range than LiDAR, but have a higher error margin along the radial direction compared to the lateral direction when determining the position of an object.

[0012] Camera and LiDAR images (and, in some applications, radar images) can be processed by a variety of object detection models, including deep learning neural network models. Such models can determine the position and orientation of objects, as well as the evolution of their position and orientation over time. These models can further classify objects by type (e.g., trucks, cars, school buses, cyclists, pedestrians, and / or similar), manufacturer, model, and / or similar.

[0013] Autonomous vehicles, especially those used in urban environments, often need to be parked (sometimes called off-haul parking) between driving missions, for example, between passengers, to reduce driving costs, and / or to avoid contributing to traffic during non-operational times. Finding off-haul parking spaces can be difficult in busy urban environments, where available parking spaces at a given time may soon be occupied afterward. As a result, autonomous vehicles may need to travel additional distances, incurring costs in fuel, vehicle wear, and / or similar, while potentially contributing to traffic congestion.

[0014] The aspects and implementations of this disclosure address these and other challenges of existing autonomous driving technologies by providing systems and technologies that enable more efficient identification of unoccupied parking spaces. In particular, in some implementations, autonomous vehicles may be deployed as part of a larger fleet of vehicles, which can exchange live information about the driving environment among multiple vehicles in the fleet. The disclosed technologies are scalable to a large number of vehicles in such a fleet. More specifically, individual vehicles of a fleet and / or other objects (e.g., sensing devices) involved in collecting fleet data (hereinafter referred to as fleet agents, or simply agents) may acquire sensor data (e.g., lidar data, radar data, camera data, sonar data, and / or similar) representing the availability of parking, including (but not limited to) the location and type of an object parked or stopped near a road, or the side or edge of another road. Roads / road edges, regardless of whether such edges are connected to physical curbs or changes in surface type (e.g., transition from sidewalk to grass, gravel, and / or similar), are often referred to as curbs in this specification and are marked with painted markings, posts, and / or any other objects, displays, or semantics, and / or any combination thereof. Agents processing live sensing data with one or more object detection models (e.g., computer vision machine learning models) may generate one or more feeds of data or signals that are sent to a fleet server that collects and manages data received from multiple agents. In one embodiment, the first signal may include the boundary shape (e.g., bounding box) of a vehicle or other object (e.g., a cargo container, a garbage disposal unit, etc.) that is detected at the curb and indicates a curb parking space occupied. The second signal may include a segment of the curb that is directly visible to the agent. Direct visibility of the curb may indicate that a parking space (or a potential parking space, where legally permitted) at a particular location on the curb is not occupied.A third signal may indicate the legal status of a curb, for example, whether parking on the curb is permitted, prohibited, permitted at certain times, or permitted for certain types of vehicles (e.g., taxis, delivery vehicles, and / or similar). This third signal may be generated based on the detection of one or more traffic signs, road / curb markings, and / or similar by one or more computer vision models.

[0015] A fleet server can collect signals from various agents that collectively represent the state of various curb segments in the environment in which the agents are operating. The fleet server can use these signals to generate a dynamic parking space occupancy map. For example, different agents may travel along the same street(s) at different times (and / or directions) and collect information about various curb segments during those times. Each agent may observe some curb segments and not observe others that are occupied (e.g., by other vehicles) from the agent's field of view, but other vehicles can complement this information by observing the same and / or different curb segments, for example, at the same and / or different times or at different angles. Each observed curb segment may be identified via its segment ID in a dynamic occupancy map, the most recent observation time, the status of the most recent observation (e.g., occupied if an object is observed within the segment boundary), unoccupied (if the curb is visible across the length of the segment), or unknown (if no object is observed but the curb is unavailable), and / or other relevant information (e.g., legal status of the segment ID, e.g., temporarily illegal, legal after 6 p.m.). The dynamic occupancy map can be updated when one of the agents communicates data that includes data about previously unseen segments and / or data updates about previously perceived segments. In some implementations, agents can send updates to the fleet server at a rate at which new perception processing is performed by the agent, e.g., every 0.1 seconds, every 0.2 seconds, and / or similar. In some implementations, agents can send updates to the fleet server at a lower rate, over 5 seconds, 10 seconds, 20 seconds, and / or similar, while performing initial aggregation of perception data. In some implementations, the dynamic occupancy map may be generated using a static map of known parking spaces, which is preloaded into agent memory (and fleet server memory) before the start of a driving mission.In some implementations, the parking space map may be dynamic itself and may be updated using signals received from the agent based on perceptions (object recognition) performed live by the agent, such as data indicating that part of the parking space has been blocked by temporary signs, cones, physical barriers, etc., or data indicating that part of the parking space that was previously unavailable is now legal to occupy.

[0016] Parking space maps and live parking space occupancy maps can be used to predict the availability of parking spaces at various locations in a driving environment. For example, a predictive model running on a fleet server may use a live parking space occupancy map as its first input. A second input may include historical parking space occupancy detected during previous driving missions of vehicles in the fleet (historical occupancy data may be continuously updated as new data is sent to the fleet server). The second input indicates to the predictive model the likelihood that a currently available parking space will nevertheless be taken before a vehicle reaches that space. In some implementations, the predictive model may receive a third input, including historical success rates for parking spaces. The third input can further indicate to the predictive model whether it is likely that the parking space will remain unoccupied. In some implementations, historical occupancy rates and / or success rates may be subdivided into multiple time bins, such as 6am-8am, 8am-10am, 10am-2am, or days of the week (e.g., weekday vs. weekend), and occupancy / success rates are tracked individually for each time bin. In some implementations, occupancy rates and / or success rates may correlate with one or more events and / or similar events taking place in a nearby area at roughly the same time, such as a sports game or a concert. Once one or more vehicles begin to park (e.g., after the completion of a driving mission, with no immediate next mission scheduled), the predictive model can evaluate several metrics for various parking spaces, including, but not limited to, the aforementioned inputs, the distance from the fleet vehicle(s) to the parking space, the length of available curb segments (continuous or interrupted) near a given parking space, the density of traffic in the area, the likelihood of a starting area for the next driving mission, and / or similar information. The predictive model can select the optimal parking space for the vehicle(s) by weighting the proximity of the parking space to the fleet vehicle(s) current location and the likelihood of the parking space being occupied by the time the vehicle(s) arrive at that space, against the likelihood of the parking space being occupied by the time the vehicle(s) arrive at that space.The optimal parking space selected by the predictive model can then be communicated to the vehicle. The vehicle can drive to the parking space and attempt to park there. The vehicle then reports to the fleet server whether the parking attempt was successful or unsuccessful, and the fleet server can update its past success rate. If the parking attempt is unsuccessful, the fleet server can identify another optimal parking space for the vehicle, given the vehicle's new position. A vehicle responding to a command from the fleet server and driving to the indicated parking space can provide the fleet server with live occupancy data, as described above. In some implementations, when a vehicle follows instructions from the fleet server and attempts to park in an available parking space, the vehicle's own planning system can override the fleet server's instructions by abandoning the route to the parking space selected by the fleet server and instead parking in an observed available parking space.

[0017] Numerous other implementations are disclosed herein. Benefits of the disclosed technology and system include, but are not limited to, the rapid and more efficient identification of available parking spaces, as well as a reduction in driving time and distance traveled by autonomous vehicles in searching for off-haul parking spaces. The disclosed technology includes mechanisms for onboard observation of parking space availability, mechanisms for sharing these observations with the rest of the fleet with low latency, and mechanisms for using aggregated fleet-wide knowledge to improve parking decision-making. Implementations of the disclosed technology have been shown to result in savings of approximately 10–20 hours of driving time per vehicle per week (depending on the number of driving missions) in urban driving environments (e.g., the San Francisco environment).

[0018] In these cases where the implementation description refers to autonomous vehicles, it should be understood that similar technologies could be used in various driver assistance systems that do not reach the level of fully autonomous driving systems. More specifically, the disclosed technologies could be used in Level 2 driver assistance systems that implement steering, braking, acceleration, lane centering, adaptive cruise control, and other driver support. Similarly, the disclosed technologies could be used in Level 3 driver assistance systems that enable autonomous driving under limited conditions (e.g., highways). Such systems could share live parking data across a large fleet of vehicles and receive commands from a fleet server to inform the driver of the availability of nearby parking spaces, even when the driver retains final control over driving decisions.

[0019] Figure 1 shows components of an exemplary autonomous vehicle (AV) 100 that implements systems and technologies that can track and predict the live availability of shared resources by a fleet of autonomous vehicles, as implemented in several ways of this disclosure. Shared resources may include parking spaces, charging stations, and / or other shared roads or vehicle resources. Autonomous vehicles may include automobiles (such as cars, trucks, buses, motorcycles, all-terrain vehicles, recreational vehicles, any special agricultural or construction vehicles), or any other self-driving vehicles capable of operating in an autonomous driving mode (without or with reduced human input) (e.g., robots, factory or warehouse robotic vehicles, sidewalk delivery robotic vehicles). "Objects" may include any objects, items, devices, vehicle bodies, or articles (moving or stationary) located outside the autonomous vehicle, such as roads, buildings, trees, bushes, sidewalks, bridges, mountains, other vehicles, piers, dikes, runways, animals, birds, or other things. As used herein, “agent” refers to any mobile or stationary device involved in data sharing, such as uploading live parking space occupancy data to the fleet server. Agents may include vehicles in the fleet, but may also include other devices (e.g., stationary sensing devices) that can sense any part of the driving environment and live stream the collected sensing data to the fleet server. Some vehicles may be agents, while others may be able to receive commands from the fleet server but cannot stream data to the fleet server. In some implementations, the fleet server may be physically separated from either a vehicle or / or agent, for example, implemented as a stationary server. In some implementations, the fleet server may be located in one of the vehicles and / or agents, or distributed among multiple vehicles, agents, and / or stationary servers.

[0020] The driving environment 101 can include any (moving or stationary) object located outside the AV, such as roads, buildings, trees, bushes, sidewalks, bridges, mountains, other vehicles, pedestrians, etc. The driving environment 101 can be an urban area, a suburban area, a rural area, etc. In some implementations, the driving environment 101 can be an off-road environment (e.g., agriculture or farmland). In some implementations, the driving environment can be an indoor environment, such as the environment of an industrial plant, a shipping warehouse, a hazardous area of a building, etc. In some implementations, the driving environment 101 can be such that various objects are moving parallel to the surface (e.g., parallel to the surface of the earth) and are substantially flat. In other implementations, the driving environment can be three-dimensional and can include objects that can move along all three directions (e.g., balloons, leaves, etc.). Hereinafter, the term "driving environment" should be understood to include all environments in which autonomous operation of a self-driving vehicle can occur. The objects in the driving environment 101 can be located at any distance from the AV, from a short distance of a few feet (or less) to a distance of several miles (or more).

[0021] As described herein, in a semi-autonomous driving mode or a partial-autonomous driving mode, the vehicle assists with one or more driving operations (e.g., steering, braking, and / or accelerating to perform lane centering, adaptive cruise control, an advanced driver assistance system (ADAS), or emergency braking), but the human driver is expected to situationally perceive the surroundings of the vehicle and monitor the assisted driving operations. In such driving mode(s), the vehicle can perform all driving tasks in certain situations, but the human driver is expected to take responsibility for control as needed.

[0022] For simplicity and conciseness, various systems and methods are described hereinafter in connection with autonomous vehicles, but the same techniques can be used in various driver assistance systems that do not reach the level of a fully autonomous driving system. In the United States, the Society of Automotive Engineers (SAE) has defined different levels of automated driving operations to indicate how much or how little a vehicle controls the driving, but different organizations in the United States or other countries may classify the levels differently. More specifically, the disclosed systems and methods can be used in a Society of Automotive Engineers (SAE) Level 2 (L2) driver assistance system that implements steering, braking, acceleration, lane centering, adaptive cruise control, etc., and other driver support. The disclosed systems and methods can be used in a Society of Automotive Engineers (SAE) Level 3 (L3) driver assistance system that can drive autonomously under limited (e.g., highway) conditions. Similarly, the disclosed systems and methods can be used in vehicles that use a Society of Automotive Engineers (SAE) Level 4 (L4) automated driving system that operates autonomously in most normal driving situations and requires only occasional attention from a human operator. In all such driver assistance systems, an accurate assessment of the driving environment can be automatically performed without driver input or control (e.g., while the vehicle is moving), resulting in improved reliability of vehicle positioning and navigation, as well as overall safety of autonomous driving, semi-autonomous driving, and other driver assistance systems. As noted above, in addition to the way SAE classifies the levels of automated driving operations, other organizations in the United States or other countries may classify the levels of automated driving operations differently. Without limitation, the disclosed systems and methods herein can be used in driver assistance systems defined by the levels of automated driving operations of these other organizations.

[0023] An exemplary AV100 could be a fleet agent (not shown in Figure 1) that collects live data related to parking availability and streams the collected data to a fleet server. The exemplary AV100 may include a sensing system 110. The sensing system 110 may include various electromagnetic (e.g., optical) and non-electromagnetic (e.g., audio) sensing subsystems and / or devices. The sensing system 110 may include a radar 114 (or more radars 114), which may be any system that utilizes radio or microwave high-frequency signals to sense objects within the driving environment 101 of the AV100. The radar(s) 114 may be configured to sense both the spatial position of objects (including their spatial dimensions) and their velocity (e.g., using Doppler shift techniques). Hereinafter, “velocity” refers to both how fast an object is moving (object speed) and the direction of the object’s motion. The sensing system 110 may include a lidar 112, which may be a laser-based unit capable of determining the distance to an object and the velocity of an object within the driving environment 101. Each of the lidar 112 and radar 114 may include a coherent sensor, such as a frequency-modulated continuous-wave (FMCW) lidar or radar sensor. For example, radar 114 may use heterodyne detection for velocity determination. In some implementations, the functionality of ToF and coherent radar is combined into a radar unit capable of simultaneously determining both the distance to a reflective object and the radial velocity of the reflective object. Such a unit may be configured to operate in a non-coherent sensing mode (ToF mode) and / or a coherent sensing mode (e.g., a mode using heterodyne detection), or both modes simultaneously. In some implementations, multiple lidar 112s or radar 114s can be mounted on the AV 100.

[0024] Lidar 112 may include one or more light sources that generate and emit signals, and one or more detectors that receive signals reflected back from objects. In some implementations, Lidar 112 may perform a 360-degree scan in the horizontal direction. In some implementations, Lidar 112 may have the ability to perform spatial scans along both the horizontal and vertical directions. In some implementations, the field of view may be up to 90 degrees vertically (e.g., at least a portion of the area above the horizon is scanned by the radar signal). In some implementations, the field of view may be spherical (consisting of two hemispheres).

[0025] The sensing system 110 may further include one or more cameras 118 to capture images of the driving environment 101. The images may be two-dimensional projections of the driving environment 101 (or a portion of the driving environment 101) onto the projection surface (flat or non-flat) of the cameras. Some of the cameras 118 of the sensing system 110 may be video cameras configured to capture a continuous (or semi-continuous) streaming of images of the driving environment 101. The sensing system 110 may also include one or more infrared (IR) sensors 119. The sensing system 110 may further include one or more sound sensors 116, such as microphones, sonars, which in some implementations may be ultrasonic sonars and / or sound sensors. The sensing data acquired by the sensing system 110 may be processed by the data processing system 120 of the AV 100. For example, the data processing system 120 may include a perception and planning system 130. The perception and planning system 130 may be configured to detect and track objects in the driving environment 101 and to classify (recognize) the detected objects as, for example, vehicles, pedestrians, animals, and / or similar. For example, the perception and planning system 130 may be able to analyze images captured by the camera 118 and have the ability to detect traffic signals, road signs, road layout (e.g., traffic lane boundaries, intersection topology, parking space designations, etc.), the presence of obstacles, etc. The perception and planning system 130 may also further receive radar sensing data (Doppler data and ToF data) to determine the distance to various objects in the environment 101 and the velocity of such objects (radially and, in some implementations, laterally, as described below). In some implementations, the perception and planning system 130 may use radar data in combination with data captured by the camera(s) 118.

[0026] The perception and planning system 130 may also receive additional information from a positioning subsystem 122, which may include a GPS transceiver and / or an inertial measuring unit (IMU) configured to acquire information about the AV's position relative to the Earth and its surroundings. The positioning subsystem can use positioning data (e.g., GPS data and IMU data) in conjunction with the perception data to help accurately determine the AV's position relative to fixed objects in the driving environment 101 (e.g., roads, lane boundaries, intersections, sidewalks, crosswalks, road signs, curbs, surrounding buildings, etc.) whose positions may be provided by road graph information 124. In some implementations, the data processing system 120 may receive non-electromagnetic data such as audio data (e.g., ultrasonic sensor data or data from a microphone picking up an emergency vehicle siren), temperature sensor data, humidity sensor data, pressure sensor data, and weather data (e.g., wind speed and direction, precipitation data).

[0027] The perception and planning system 130 may perform any number of tasks related to the selection and tracking of the AV100's selected trajectory (driving path), including the detection and tracking of objects in the driving environment, identification of the state of traffic signals, traffic signs, and road / lane markings, selection of a safe and legal trajectory for the AV100, modification of the selected trajectory in consideration of changes in driving conditions, and / or the performance of various other tasks related to the operation of the AV100. Furthermore, the perception and planning system 130 may perform any number of tasks related to the collection and processing of live data related to parking availability in the area of ​​the driving environment 101 observable by the sensing system 110. In some implementations, the collection and processing of parking availability data may be performed by the same modules and processes that perform various trajectory-related tasks. In some implementations, the collection and processing of parking space availability data may be performed by dedicated modules and processes dedicated to such tasks. In some implementations, the perception and planning system 130 may include a perception module 132 that receives perception data from the sensing system 110 and generates curb occupancy data 135 indicating which parts of the curb are obstructed by detected objects, which parts of the curb are visible, which parts of the curb are legal / illegal for parking, and / or similar.

[0028] Preferably, curb occupancy data 135, annotated with a timestamp indicating the data collection time, may be provided to an agent server communication module 136 that communicates the data to a fleet server. The agent server communication module 136 may include a radio signal transmitter capable of transmitting modulated radio waves carrying (at least) the curb occupancy data 135, and a radio signal receiver capable of receiving modulated radio waves carrying (at least) parking commands 138, for example, using one or more preferred radio communication protocols. In some implementations, the transmission and / or reception of radio waves may be carried out by a transceiver that combines the functions of a transmitter and a receiver.

[0029] In some implementations, curb occupancy data 135 may be streamed at regular time intervals, for example, at a frequency of 5Hz, 10Hz, and / or similar. In cases where AV100 is a vehicle attempting to park (e.g., after completing a driving mission), the agent server communication module 136 may receive a parking command 138 from the fleet server by identifying a parking space where AV100 is likely to park successfully. The identification of the parking space (e.g., adjustment) may also be passed to the planner module 134 of the perception and planning system 130, which charts the driving trajectory of AV100 to the identified parking space. In some implementations, for example, if the perception module 132 observes an available and legal parking space near AV100, the perception module 132 may issue a parking command directly to the planner module 134 without waiting for a parking command 138 from the fleet server.

[0030] Various systems and subsystems of the data processing system 120 may have software stored in one or more system memory 126 devices. System memory 126 may include any volatile or non-volatile memory devices, such as read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), flash memory, flip-flop memory, or any other device capable of storing data. RAM may be static memory, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), or static random access memory (SRAM). In some implementations, system memory 126 may be on-chip memory.

[0031] The operation of the data processing system 120 may be carried out by one or more processors 128, which may include CPUs, GPUs, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc. As used herein, “processor” refers to a device capable of executing instructions that code arithmetic, logical, or I / O operations, for example, stored in the system memory 126. In some implementations, the processors 128 and the system memory 126 may be implemented as a single controller, for example, as an FPGA.

[0032] Data generated by the perception and planning system 130 and / or received via the agent server communication module 136 may be used by an autonomous driving system, such as a vehicle control system (VCS) 140. The VCS 140 may include one or more algorithms that control how the AV should behave in various driving situations and environments. For example, the VCS 140 may include a navigation system for determining a global driving route to a destination. The VCS 140 may also include a driving route selection system for selecting a specific route through the immediate driving environment, which may include selecting a traffic lane, navigating through traffic congestion, selecting a place to make a U-turn, and selecting a trajectory for parking maneuvers. The VCS 140 may also include an obstacle avoidance system for safely avoiding various obstacles in the AV's driving environment, such as stones, stalled vehicles, and pedestrians crossing in disregard of traffic rules and signals. An obstacle avoidance system may be configured to evaluate the size of an obstacle and its trajectory (if the obstacle is moving) and to select the optimal driving strategy (e.g., braking, steering, acceleration) to avoid the obstacle.

[0033] The VCS140 algorithm and modules can generate commands for various systems and components of a vehicle, including the powertrain, brakes, and steering 150, vehicle electronics 160, signaling 170, and other systems and components not explicitly shown in Figure 1. The powertrain, brakes, and steering 150 may include the engine (internal combustion engine, electric engine, etc.), transmission, differential, axles, wheels, steering mechanism, and other systems. The vehicle electronics 160 may include the onboard computer, engine management, ignition, communication systems, car computer, telematics, in-car entertainment system, and other systems and components. The signaling 170 may include high headlights, low headlights, stop lights, turn signals, and taillights, horn and alarms, interior lighting systems, dashboard notification systems, passenger notification systems, radio, and wireless network transmission systems. Some of the commands output by the VCS140 can be delivered directly to the powertrain, brakes, and steering 150 (or signaling 170), while other commands output by the VCS140 are first delivered to the vehicle electronics 160, which generates commands for the powertrain, brakes, and steering 150, and / or signaling 170.

[0034] In one example, a parking command 138 received from the fleet server via the agent server communication module 136 (and / or directly generated by the planner module 134) may include the coordinates of an identified available parking space, such as a curb parking space on a road. In some implementations, the coordinates may be specified in relation to a reference grid of the positioning subsystem 122 and may include any preferred GNSS coordinates. In some implementations, the command may include a unique parking space identifier PSID assigned to a given parking space on a parking space map shared among the vehicles in the fleet. After receiving the PSID and / or the coordinates of the parking space, the planner module 134 can find the corresponding parking space in the road graph information 124, for example, it can determine the specific road and side of the road on which the parking space is located, and further, it can identify the specific driving lane that provides access to the parking space. Next, the planner module 134 may chart a driving path from the current position to the parking space and provide the charted driving path, or path, to the VCS 140 by specifying, for example, various path points along the path, target speeds to be maintained between path points, and / or similar. The VCS 140 may output corresponding commands to the powertrain, brakes, and steering 150 (directly or via the vehicle electronics 160) to, for example, (1) control throttle settings, fuel flow to the engine, steering settings, and / or similar, to (2) brakes, downshift the transmission to a lower gear, and (3) pull forward and / or backward into the parking space, following the provided driving path. After the off-haul parking period, the VCS 140 may output commands to the powertrain, brakes, and steering 150, for example, based on new commands received from the fleet server, so that they may be charted by the planner module 134, to exit the parking space and follow a preferred trajectory associated with the new driving mission.

[0035] Figure 2 shows an example of the data flow of Operation 200, which, according to several implementations of the present disclosure, targets tracking and predicting the live availability of shared resources by a fleet of vehicles. Shared resources may include parking spaces, charging stations, and / or other shared roads or vehicle resources. The fleet schematically shown in Figure 2 may include one or more agents 202 and one or more fleet servers 250 (for simplicity, one fleet server is shown). As shown, Operation 200 may involve the use of sensing data 210 acquired by a sensing system (e.g., sensing system 110 in Figure 1) of an agent 202, which may be an autonomous vehicle and / or a driver-operated vehicle. The sensing data 210 can be collected by any suitable sensor, e.g., lidar 112, radar 114, camera(s) 118, and / or voice sensor 116 (see Figure 1), and can be received via a sensor data acquisition module 204 that receives data acquired by one or more sensors. More specifically, the sensor data acquisition module 204 may receive a series of camera images 202, for example, a two-dimensional projection of the operating environment (or a portion thereof) on an array of sensing detectors (e.g., charge-coupled elements, or CCD detectors, complementary metal-oxide-semiconductor, or CMOS detectors, and / or similar). Individual camera images may have pixels of varying intensities, one color (in the case of a grayscale image) or multiple colors (in the case of a color image). The camera images may be panoramic images or images showing a specific portion of the operating environment. The camera images may contain a large number of pixels. The number of pixels may depend on the resolution of the image. Each pixel may be characterized by one or more intensity values. Grayscale pixels may, for example, be characterized by a single intensity value representing the brightness of the pixel, where a value of 1 corresponds to a white pixel, a value of 0 to a black pixel (and vice versa). Intensity values ​​may be continuous (or discrete) values ​​between 0 and 1 (or between any other selected limits, e.g., 0 and 255).Similarly, a color pixel can be represented by two or more intensity values, for example, three intensity values ​​(e.g., when using the RGB color encoding scheme) or four intensity values ​​(when using the CMYK color encoding scheme). Camera images can be preprocessed, for example, by downscaling (combining multiple pixel intensity values ​​into a single pixel value), upsampling, filtering, and denoising. Camera images can be in any preferred digital format (JPEG, TIFF, GIG, BMP, CGM, SVG, etc.).

[0036] The sensor data acquisition module 204 can also receive lidar and / or radar images 204, which may include a set of return points (point clouds) corresponding to laser and / or lidar beam reflections from various objects in the driving environment. Each return point can be understood as a data unit (pixel), including the coordinates of the reflective surface, radial velocity data, intensity data, and / or similar. For example, the sensor data acquisition module 204 can receive lidar (radar) data including an intensity map I(R,θ,φ), where R, θ, and φ are sets of spherical coordinates. In some implementations, Cartesian, elliptic, parabolic, or other preferred coordinates may be used instead. The intensity map identifies the intensity of radar (lidar) reflections for various points in the field of view. The coordinates of an object (or surface of an object) reflecting a lidar (radar) signal can be determined from directional data (e.g., polar angle θ and azimuth angle φ angles in the lidar transmission direction) and distance data (e.g., radial distance R, determined from the time of flight of the signal). Lidar and / or radar images may further include velocity data of various reflecting objects, identified based on the detected Doppler shift of the reflected signals. Figure 2 shows an implementation where three data modalities are used, although some implementations may omit (or disable) one or more data modalities. For example, a camera data modality and a lidar (or radar) data modality may be used, but a radar data modality (or lidar data modality) may not be used.

[0037] Camera images, LiDAR data, and / or radar data can be a large image (a set of images) of the entire driving environment, or images of a more important part of the driving environment (e.g., camera images acquired by the front camera(s) of the vehicle's sensing system). The acquired camera, LiDAR, and / or radar images can be combined into sensing frames associated with a specific time of acquisition, for example, through a timestamp. In some implementations, different sensors can acquire images at different rates, for example, LiDAR images can be acquired at a rate of 50 Hz, camera images at a rate of 24 Hz, and radar images at a rate of 10 Hz. Therefore, in some implementations, a frame of sensing data 210 may have a minimum rate (e.g., 10 Hz) with higher-rate sensing data aggregated across multiple frames (e.g., five LiDAR images aggregated and combined with one radar image). In some implementations, the frame of sensing data 210 may have a highest rate (e.g., 50 Hz) with lower-rate sensing data used across multiple sensing frames (e.g., a single radar image combined with five consecutive lidar sensing frames).

[0038] Frames of the sensing data 210 may be processed by the perception module 132 (e.g., part of the perception and planning system 130 in Figure 1). The perception module 132 may include an object detection model, which may be a computer vision model (or a set of models) trained to identify individual objects in the sensing data 210, such as vehicles, curbs, pedestrians, animals, road signs, buildings, structures, overpasses, and / or similar. In some implementations, the object detection model may output bounding boxes or any other boundary shape (e.g., a convex hull) that encloses or otherwise indicates the location of various objects in the environment. The object detection model may further output the type (label) of the object, such as "pedestrian," "vehicle," "traffic signal," "road sign," and / or similar. The perception module 132 may also receive road graph information 124, which may include static (e.g., map) data of various drivable surfaces, such as roads, streets, lanes, driveways, and / or similar. For example, the road graph information 124 may include a map of polylines corresponding to various driving lanes, lane curves, lane divisions, lane merges, lane start points, lane end points, and / or similar. The perception module 132 may further receive input from the parking space map 212, which may be (or include) various known maps (e.g., from previous driving missions or associated city records).

[0039] The sensing module 132 may generate curb occupancy data 135, including a parked parking object detection signal 220 that characterizes the type and boundary shape (e.g., bounding box, convex hull, etc.) of a vehicle or other object located on a curb (e.g., any edge or boundary of a drivable surface). Figure 3 shows an exemplary driving scene 300 that may be sensed by one or more agents in a fleet of autonomous vehicles as part of tracking the live availability of parking spaces, according to some implementations of the present disclosure. The driving scene 300 includes an agent 310 that drives along a portion of an urban street and collects sensing data indicating the live occupancy of one or more parking spaces 320-1...320-12, indicated by rectangular boxes. The agent 310 may sense the driving scene using one or more active sensing beams 330, e.g., lidar sensing beams, radar sensing beams, and / or similar (e.g., ultrasonic sensing beams). In some implementations, parking spaces 320-1...320-12 may be identified in the parking space map 212 (see Figure 2) via, for example, a PSID, coordinates, and / or similar, or as curb segments where parking is legal. Furthermore, the parking space map may identify curb segments where parking is not permitted, as schematically indicated by crosses and arrows.

[0040] The parking object detection signal 220 may include the identification of boundary shapes of various detected objects located near curbs in areas where parking is legally permitted. As shown, Figure 3 shows schematic boundary shapes 341, 343, and 345 drawn around parked vehicles 340, 342, and 344, respectively. Figure 3 also shows a bounding box 347 drawn around a parked fire truck 346. The parking object detection signal 220 may identify a bounding box (or other boundary shape) by any preferred set of parameters, such as the coordinates of the box's center and the dimensions of the box (e.g., length, width, and height in some implementations) or any equivalent. The parking object detection signal 220 (see Figure 2) may further identify parking spaces blocked by parked objects, such as parking spaces 320-1, 320-3, 320-4, 320-6, 320-8, 320-10, and 320-11, as occupied spaces. While this specification refers to parking spaces blocked by parked objects, it should be understood that, in this context, “parked object” may also include stationary vehicles and / or any other stationary objects (objects with zero or near-zero velocity).

[0041] In some implementations (referring to Figure 2), the curb occupancy data 135 generated by the perception module 132 may include a curb visibility signal 230. The curb visibility signal 230 can identify segments of the curb that are directly visible to the agent's sensing system. For example, agent 310 in Figure 3 may directly observe the curb segment corresponding to parking space 320-9, and shorter segments of the curb corresponding to parking space 320-9. The curb visibility signal 230 can identify any visible continuous portion of the curb on either side of the road, even when those portions are part of a space where a vehicle can be parked. Thus, the curb visibility signal 230 identifies a curb segment as unoccupied and therefore potentially available as a parking space. Parts of the curb segment that are either occupied or not identified as unoccupied may be classified as unknown. For example, parking spaces 320-2, 320-5, and 320-12, where no blocking object is detected but the curb is not directly visible, could be classified in this way.

[0042] In some implementations (referring to Figure 2), the curb occupancy data 135 generated by the perception module 132 may include a parking space status detection signal 240. The parking space status detection signal 240 may indicate the legal status of various visible segments of the curb in a driving scene. For example, the perception module 132 may use a computer vision model to identify one or more road markings, curb markings, posted signs indicating whether parking at the curb is permitted or prohibited, permitted at a specific time, or permitted for a specific type of vehicle (e.g., taxi, delivery vehicle, etc.) and / or similar. In some cases, the parking space status detection signal 240 may determine that some parking spaces identified as legal by the parking space map 212 are no longer available (or are temporarily unavailable, for example, due to construction, an accident, etc.). This may be indicated by a “No Parking” sign, cone, physical block, and / or similar. Conversely, the parking space status detection signal 240 can determine that parking in some spaces previously identified as illegal by the parking space map 212 is now legal, for example, if a previous "No Parking" sign has been removed. In such cases, if there is a discrepancy between the signal from the parking space status detection signal 240 and the parking space map 212, the latter can be updated with live information within the signal.

[0043] The parking object detection signal 220, curb visibility signal 230, and / or parking space state detection signal 240 can be communicated to the fleet server 250. The fleet server 250 can collect signals from various agents via the data acquisition module 252 and generate live (dynamic) parking space occupancy 260, for example, by performing parking space occupancy aggregation 254 using data from various agents to determine the state of the overall driving environment in which multiple agents of the fleet operate. In some cases, different agents may be moving along the same street(s) at different times, and in such cases, information about various curb segments can be collected. For example, referring to Figure 3, agent 310 may not be able to directly observe parking spaces 320-2 and 320-5 and has a limited field of view of parking space 320-7, while another agent 350 may have a better field of view of those spaces.

[0044] The live parking space occupancy rate 260 may include entries for various parking spaces and / or curb segments defined in the saved parking space map 256, and may also include any additional parking spaces identified in live processing by the perception modules of various agents in the fleet. Individual curb segments may be identified by the segment's PSID, the time of the segment's last observation, the state of the last observation, e.g., occupied (if the segment is blocked by a stationary object), unoccupied (the segment has a curb visible to the agent), or unknown (no blocking object or visible curb). Furthermore, individual curb segments may be characterized by their legal status (e.g., temporarily unavailable, available after a specific time of day, and / or similar) and / or related information.

[0045] The live parking space occupancy rate 260 may be updated whenever one of the agents (e.g., agent 202) communicates new data, including data on previously unperceived segments and / or updates on previously perceived segments. In some implementations, agents may send updates to the fleet server 250 at the rate at which new data is generated by each agent's perception module, for example, every 0.1 seconds, every 0.2 seconds, and / or similar. In some implementations, the perceived data may first be aggregated on the agent side and communicated to the fleet server 250 at a less frequent frequency, for example, every 5 seconds, every 10 seconds, every 30 seconds, and / or similar, to save communication and data processing bandwidth for the fleet server. In some implementations, the signals communicated by agents may include data related to new parking spaces and / or curb segments perceived by the agent, as well as data related to previously perceived spaces / segments from which a different field of view has been acquired since the last update, for example, as a result of the agent's actions, but not data collected in previous time. In some implementations, in addition to performing parking space occupancy aggregation 254, the fleet server 250 may update the stored parking space map 256 using data received from one or more agents.

[0046] The parking space map 256 and the live parking space occupancy rate 260 are used by the parking space availability forecast 270 to predict parking availability at various locations in the driving environment. Additional inputs to the parking space availability forecast 270 may include, for example, historical parking space occupancy rates 262 detected during previous driving missions of the fleet's vehicles. As shown by the dashed arrows in Figure 2, the historical parking space occupancy rates 262 may be updated, for example, continuously or at regular time intervals, such as hourly, daily, and / or similar, based on the live parking space occupancy rate 260 and new data received by the fleet server 250. Further inputs to the parking space availability forecast 270 may include historical parking success rates 264 for various parking spaces and / or curb segments. Using either or both of the historical parking space occupancy rates 262 and / or historical parking success rates 264, it is possible to estimate the likelihood that a currently available parking space (or the same or similar nearby parking space) will remain available or be occupied by another vehicle when a fleet vehicle reaches that space. A typical fleet of vehicles can observe significantly more parking spaces together within any given time interval (e.g., one hour, one day) than the number of times the fleet's vehicles park during the same time interval. Therefore, historical parking space occupancy rates 262 can be expected to be statistically more representative of the probability that a given parking space remains available than historical parking success rates 264. Accordingly, historical parking space occupancy rate 262 data can be given more weight in the initial prediction of parking space availability. Over time, the fleet server 250 collects more statistics on many fleet vehicles attempting to park in various specific locations in the driving environment, so that past parking success rates 264 become progressively more accurate and can then be given more weight in the parking space availability prediction 270.

[0047] In some implementations, past parking space occupancy rates 262 and / or past parking success rates 264 can be subdivided into multiple subcategories associated with specific times, e.g., 6am-8am, 10am-1pm, 5pm-7pm, and / or similar, and specific days of the week (e.g., Monday-Friday, Saturday-Sunday, public holidays, and / or similar), and occupancy / success rates can be collected separately for such subcategories. In some implementations, occupancy / success rates can also be collected separately for instances where an event (e.g., a sports game, a concert, etc.) is taking place in the area, and instances where no event is taking place, and / or similar.

[0048] When one or more fleet vehicles are attempting to park, for example after the completion of a driving mission for which no immediate next mission is scheduled, the parking space availability forecast 270 may evaluate several indicators for selecting one or more potential parking spaces for the fleet vehicle(s). In one exemplary embodiment, the indicators may include the historical availability HA of the candidate parking space. For example, historical availability HA may include the historical success rate SR of parking in the expected parking space (e.g., measured in a percentage or some other preferred unit), and / or the historical occupancy rate O of the expected parking space, for example, the percentage of time the space was observed to be occupied (e.g., measured in a percentage). In some implementations, historical availability may be a combination of the historical success rate and the historical occupancy rate.

[0049]

number

[0050]

number

[0051]

number

[0052] One or more fleet vehicles 290 can move to a parking space selected by the fleet server 250 and attempt to park. The fleet vehicles can then report to the fleet server 250 whether the parking attempt was successful or unsuccessful, and the fleet server 250 may update the past parking success rate 264 for the parking space. If the parking attempt is unsuccessful, the fleet server 250 may identify another optimal parking space for the fleet vehicle 290, taking into account the vehicle's current location.

[0053] Any fleet vehicle 290 acting as a fleet agent, responding to commands from the fleet server 250 and driving to a selected parking space, may provide the fleet server 250 with any of the aforementioned signals, for example, the parking object detection signal 220, the curb visibility signal 230, the parking space status detection signal 240, and / or similar. In some implementations, when a fleet vehicle 290 follows a parking command from the fleet server 250, if it detects an available parking space different from the one selected by the fleet server 250, the vehicle's own planning module 134 (see Figure 1) may override the fleet server's command by abandoning the route to the parking space selected by the fleet server 250 and parking in the detected available parking space.

[0054] Calculations related to parking space occupancy aggregation 254, live parking space occupancy 260, parking space availability prediction 270, and / or other systems and modules of the fleet server 250 may be supported by a suitable processor 258 in response to instructions stored in system memory 257.

[0055] Figures 4–5 illustrate embodiments of Method 400–500, by a fleet of vehicles, of several implementations of the present disclosure, for tracking and predicting the live availability of parking resources. For specific purposes, the operation of Method 400–500 is illustrated in conjunction with parking resources, but similar operation may be used to track and predict the live availability of other shared resources, including but not limited to charging stations and / or similar. A processing device having one or more processing units (CPUs), one or more graphics processing units (GPUs), one or more parallel processing devices (PPUs), and memory devices communicably coupled to the CPUs, GPUs, and / or PPUs may perform each of Method 400 and / or Method 500 and / or their individual functions, routines, subroutines, or operations. Method 400 and / or Method 500 may apply to vehicle systems and components. In some implementations, the vehicle may be an autonomous vehicle. In some implementations, the vehicle may be a driver-operated vehicle equipped with driver assistance systems, e.g., Level 2 or Level 3 driver assistance systems, that provide limited assistance through specific vehicle functions (e.g., systems such as steering, braking, and acceleration) or under limited driving conditions (e.g., highway driving). Methods 400 and / or 500 may be performed by a preferred processing device or a number of processing devices. In a particular implementation, a single processing thread may perform Methods 400 and / or 500. Alternatively, two or more processing threads may perform Methods 400 and / or 500, each thread performing one or more individual functions, routines, subroutines, or operations of the method. In exemplary embodiments, the processing threads performing Methods 400 and / or 500 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads performing Methods 400 and / or 500 may be executed asynchronously with respect to each other.Some operations of Method 400 and / or Method 500 may be performed in an order different from that shown in Figures 4-5. Some operations of Method 400 and / or Method 500 may be performed simultaneously with other operations. Some operations may be optional.

[0056] Figure 4 shows an embodiment of Method 400, which tracks and predicts the live availability of parking resources, as performed by a fleet server of an autonomous vehicle fleet, according to several implementations of the present disclosure. In some implementations, the operation of Method 400 may be performed using the system memory 257 and processor 258 of the fleet server 250.

[0057] In block 410, method 400 may receive multiple communications (e.g., curb occupancy data 135 in Figure 1). Each of the multiple communications may be received from each autonomous vehicle in a fleet of autonomous vehicles and may include identification of one or more objects located at the edge of the drivable portion of the driving environment (e.g., boundary shapes 341, 343, 345, etc. in Figure 3) (e.g., as part of the parking object detection signal 220 in Figure 2). In some implementations, each of the multiple communications may further include edge visibility data (e.g., curb visibility signal 230 in Figure 2) representing the visibility of the edge of the drivable portion of the driving environment from the sensing system of each autonomous vehicle (e.g., visibility of parking space 320-9 from the sensing system of agent 310). In some implementations, the sensing system of each autonomous vehicle may include one or more lidar sensors, one or more radar sensors, and / or one or more camera sensors.

[0058] In block 420, method 400 may include identifying at least one object and updating a map of live parking space occupancy rates in the driving environment (e.g., live parking space occupancy rate 260 in Figure 2) using edge visibility data. In some implementations, updating the map of live parking space occupancy rates may include combining edge visibility data that represent the same segment of edges received from multiple autonomous vehicles in the fleet (e.g., as part of parking space occupancy rate aggregate 254, see Figure 2), as shown in upper callout block 422.

[0059] In block 430, method 400 may continue to determine a first probability value (e.g., a probability function value as disclosed in conjunction with Figure 2) associated with a first parking space in the driving environment that is not occupied within the target time, based on an updated map of live parking space occupancy rates in the driving environment and past parking space availability. In some implementations, the target time may be the travel time between the current location of the autonomous vehicle and the first parking space. In some implementations, past parking space availability may include past parking space occupancy rates (e.g., past parking space occupancy rate 262 in Figure 2) for multiple parking spaces, including the first parking space. In some implementations, past parking space availability may include past parking success rates associated with the first parking space (e.g., parking success rate 264 in Figure 2).

[0060] In block 440, method 400 may include guiding a first autonomous vehicle in the fleet to a first parking space based on at least a first probability value. In some implementations, guiding a first autonomous vehicle to a first parking space may include operations in the lower callout portion of Figure 3. More specifically, in block 442, method 400 may include determining a second probability value associated with a second parking space in the driving environment that remains vacant within a target time, based on at least an updated map of live parking space occupancy rates and historical parking space availability. In block 444, method 400 may include selecting a first parking space based on the difference between the first and second probability values. In some implementations, the selection of a first parking space may further be based on the difference between a first driving distance to the first parking space and a second driving distance to the second parking space, or the difference between a first driving time to the first parking space and a second driving time to the second parking space. For example, a parking space characterized by a higher probability value may be selected. In some implementations, guiding a first autonomous vehicle to a first parking space may be further based on a number of alternative parking spaces within at least one of a target distance from the first parking space or a target driving time (e.g., some maximum driving distance from the first parking space or within the maximum driving time, such as several other known available parking spaces, the length of a known available curb segment large enough to park the vehicle).

[0061] In some implementations, block 450 may include method 400 receiving an indication from the first autonomous vehicle that the first autonomous vehicle was unable to park in the first parking space. Block 460 may continue to determine a second probability value associated with the second parking space in the driving environment that remains vacant during the driving time between the first and second parking spaces. Block 470 may continue to guide the first autonomous vehicle to the second parking space based on at least the second probability value. Block 480 may include receiving an indication from the first autonomous vehicle whether the first autonomous vehicle was able to park in the first parking space, and in block 490 updating the past parking space availability in the driving environment based on the received indication.

[0062] In some implementations, any of the actions in blocks 440-490 may be performed to identify parking spaces for multiple autonomous vehicles (e.g., a second autonomous vehicle, a third autonomous vehicle, etc.) and guide those autonomous vehicles to their respective identified parking spaces.

[0063] Figure 5 shows an exemplary method 500 for tracking and predicting the live availability of parking resources, implemented by autonomous vehicles in a fleet of automated vehicles, according to several implementations of the present disclosure. The operation of method 400 can be implemented using various components and modules of AV100 shown in Figure 1.

[0064] In block 510, method 500 may include collecting sensing data that characterizes the operating environment of the AV (e.g., AV100 in Figure 1) using a sensing system (e.g., sensing system 110 in Figure 1) of the AV.

[0065] In block 520, method 500 may include using an AV data processing system (e.g., data processing system 120 in Figure 1) to use sensing data to generate identification of one or more objects located at the edge of the drivable portion of the driving environment (e.g., parking object detection signal 220 in Figure 2).

[0066] In block 530, method 500 may include using an AV data processing system to generate edge visibility data (e.g., curb visibility signal 230 in Figure 2) that represents the visibility of the edges of the drivable portion of the driving environment.

[0067] In block 540, method 500 may continue to communicate the identification of at least one object and edge visibility data to a fleet server (e.g., fleet server 250 in Figure 2) via the AV transceiver.

[0068] In block 550, method 500 may include receiving, via transceiver, the identification of a first parking space in the driving environment from a fleet server.

[0069] In block 560, method 500 may continue to move the AV (e.g., VCS140 in Figure 1) to the first parking space using the vehicle control system of the AV.

[0070] In some implementations, method 500 may include the operations shown in blocks 570-578. More specifically, in block 570, method 500 may include collecting additional sensing data for a first parking space. In block 572, method 500 may include using the AV's sensing system to determine, based on the additional sensing data, that the first parking space is occupied. In block 574, method 500 may include using the AV's transceiver to communicate to the fleet server an indication that the first parking space is occupied. In block 576, method 500 may include receiving, via the AV's transceiver, an identification of a second parking space in the driving environment from the fleet server. In block 578, method 500 may include using the AV's vehicle control system to move the AV to the second parking space.

[0071] In some implementations, method 500 may include the operations shown in blocks 580-584. More specifically, in block 580, method 500 may include using the AV's sensing system to collect additional sensing data characterizing the autonomous vehicle's driving environment. In block 582, method 500 may continue to use a data processing system to determine, based on the additional sensing data, that a second parking space is available, and that the second parking space is located closer to the autonomous vehicle than the first parking space. In block 584, method 500 may include using the AV's vehicle control system to park the AV in the second parking space.

[0072] In some implementations, the techniques disclosed herein may also be used to track and verify the availability of parking spaces, for example, in a depot of an autonomous vehicle fleet. In such implementations, a fleet server may know the locations of various vehicles in the fleet, but such knowledge may not guarantee knowledge of which parking spaces in the depot are occupied and which are available, because non-vehicles may be parked in some of the spaces in the depot. In such cases, the disclosed techniques may be used to collect sensory data from autonomous vehicles driving around the depot, for example, before the start of a first mission, returning to the depot between missions, and / or similar. The collected sensory data may include identification of objects (e.g., other vehicles) parked in various parking spaces, and / or curb visibility data, in which case “curb” in this context should be understood as any physical object(s) that (individually or collectively) indicates the boundaries of parking spaces, markings, signs, and / or other indications of parking spaces in the depot. Identifying available parking spaces for parking one or more vehicles of a fleet can be carried out substantially as disclosed above, in conjunction with methods 400 and 500 shown in Figures 4 and 5, for example.

[0073] Figure 6 shows a block diagram of an exemplary computer device 600 that can track and predict the live availability of parking resources, run by a fleet server of an autonomous vehicle fleet, according to several implementations of this disclosure. Embodiments of computer device 600 may be connected to other computer devices in a LAN, intranet, extranet, and / or the Internet. Computer device 600 may operate as a server in a client-server network environment. Computer device 600 may be a personal computer (PC), set-top box (STB), server, network router, switch or bridge, or any device capable of executing a set of instructions (sequentially or otherwise) that specify actions to be taken by such device. Furthermore, although only a single embodiment of a computer device is shown, the term “computer” shall also be considered to include any group of computers that individually or collectively execute a set of instructions (or sets of instructions) to implement any one or more of the methods discussed herein.

[0074] Embodiments of computer device 600 may include a processing device 602 (also called a processor or CPU), main memory 604 (e.g., read-only memory (ROM), flash memory, synchronous DRAM (SDRAM), dynamic random access memory (DRAM), etc.), static memory 606 (e.g., flash memory, static random access memory (SRAM), etc.), and secondary memory (e.g., data storage device 618), which may communicate with each other via bus 630. In some implementations, the processing device 602 may be the processor 128 in Figures 1 and 2, or the processor 258 in Figure 2, or may include them, and the main memory 604 may include the system memory 126 in Figures 1 and 2, or the system memory 257 in Figure 2.

[0075] The processing device 602 (which may include a logic processor 603) represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or similar. More specifically, the processing device 602 may be a composite instruction set computer (CISC) microprocessor, a reduced instruction set computer (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor that executes other instruction sets, or a processor that executes combinations of instruction sets. The processing device 602 may also be one or more application-specific processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or similar. According to one or more aspects of this disclosure, the processing device 602 may be configured to execute instructions that perform methods 400-500 for tracking and predicting the live availability of parking resources by a fleet of vehicles.

[0076] An embodiment of the computer device 600 may further include a network interface device 608 that can be communicatively coupled to a network 620. An embodiment of the computer device 600 may further include a video display 610 (e.g., a liquid crystal display (LCD), a touchscreen, or a cathode ray tube (CRT)), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and an audio signal generation device 616 (e.g., a speaker).

[0077] The data storage device 618 may include a computer-readable storage medium (or more specifically, a non-temporary computer-readable storage medium) 628 on which one or more sets of executable instructions 622 are stored. According to one or more aspects of the present disclosure, the executable instructions 622 may include executable instructions that implement methods 400-500 for tracking and predicting the live availability of parking resources by a fleet of vehicles.

[0078] The executable instructions 622 may also reside, for example, entirely or at least partially, in main memory 604 and / or processing device 602 during their execution in computer device 600, and main memory 604 and processing device 602 also constitute computer-readable storage media. The executable instructions 622 may also be further transmitted or received over a network via network interface device 608.

[0079] Although the computer-readable storage medium 628 is shown as a single medium in Figure 6, the term “computer-readable storage medium” should be understood to include a single medium or multiple media (e.g., centralized or distributed databases, and / or associated caches and servers) that store one or more sets of operational instructions. The term “computer-readable storage medium” should also be understood to include any medium capable of storing or encoding a set of instructions for machine execution that causes a machine to perform any one or more of the methods described herein. Thus, the term “computer-readable storage medium” should be understood to include, but not be limited to, solid-state memory, as well as optical media and magnetic media.

[0080] Some parts of the detailed description above are presented in terms of algorithms of operation and symbolic representations of data bits in computer memory. These algorithmic descriptions and representations are means used by those skilled in the field of data processing to most effectively communicate the content of the work to others skilled in the art. An algorithm is understood here, and generally, as a step-self-consistent sequence that produces a desired result. A step is one that requires the physical manipulation of a physical quantity. Usually, but not always, these quantities take the form of electrical or magnetic signals that can be stored, moved, combined, compared, and otherwise manipulated. Referring to these signals as bits, values, elements, symbols, characters, terms, numbers, or similar has sometimes proven convenient, primarily for reasons of general use.

[0081] However, it should be noted that all these terms, and similar terms, are associated with appropriate physical quantities and are merely convenient labels applied to those quantities. Otherwise, unless specifically stated, as will be evident from the following considerations, any considerations throughout the explanation using terms such as “specify,” “determine,” “store,” “adjust,” “bring forth,” “return,” “compare,” “generate,” “stop,” “load,” “copy,” “insert,” “replace,” “implement,” or similar terms will be understood to refer to the operation and processes of a computer system or similar electronic computing device that manipulates and converts data represented as physical (electronic) quantities in the registers and memory of a computer system into other data similarly represented as physical quantities in computer system memory or registers or other such information storage, transmission, or display devices.

[0082] Examples of this disclosure also relate to apparatus for carrying out the methods described herein. Such apparatus may be a general-purpose computer system, which may be specifically constructed for a required purpose or selectively programmed by computer programs stored within the computer system. Such computer programs may be stored on computer-readable storage media, including, but not limited to, any type of disk, including optical disks, CD-ROMs, and magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic disk storage media, optical storage media, flash memory devices, other types of machine-accessible storage media, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

[0083] The methods and representations presented herein are not inherently related to any particular computer or other device. Various general-purpose systems may be used with the programs taught herein, or it may be convenient to construct more specialized devices to carry out the necessary method steps. The necessary structures for various such systems will appear as described below. In addition, the scope of this disclosure is not limited to any particular programming language. It will be understood that various programming languages ​​can be used to carry out the teachings of this disclosure.

[0084] It should be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will become apparent to those skilled in the art upon reading and understanding the above description. While this disclosure describes specific embodiments, it will be recognized that the systems and methods of this disclosure are not limited to the examples described herein and can be modified and implemented within the scope of the appended claims. Therefore, the specification and drawings should be considered illustrative, not restrictive. Accordingly, the scope of this disclosure should be determined by reference to the appended claims, along with the entire scope of equivalents to which such claims are entitled.

Claims

1. It is a system, Memory devices and, A processing device that is communicatively coupled to the memory device, Receiving multiple communications, each of which is received from each autonomous vehicle in a fleet of autonomous vehicles, and each of which is at least Identification of one or more objects located at the edge of the drivable portion of the driving environment, The system includes at least one of the following: edge visibility data representing the visibility of the edges of the drivable portion of the driving environment from the sensing system of each of the autonomous vehicles; The map of live parking space occupancy in the aforementioned driving environment is updated with the identification of at least one or more objects and the edge visibility data. At a minimum, based on an updated map of live parking space occupancy and past parking space availability in the driving environment, it is determined that the first probability value associated with the first parking space in the driving environment will remain unoccupied within the target time. Based at least the first probability value, the first autonomous vehicle of the fleet is guided to the first parking space, A processing device that performs this, A system equipped with these features.

2. The sensing system of each of the aforementioned autonomous vehicles One or more lidar sensors, One or more radar sensors, and One or more camera sensors, The system according to claim 1, comprising at least one of the following.

3. In order to update the map of live parking space occupancy, the processing device, The system according to claim 1, which combines edge visibility data received from multiple autonomous vehicles of the fleet, each representing the same segment of the edge.

4. The aforementioned past availability of parking spaces, The past parking space occupancy rates of multiple parking spaces, including the aforementioned first parking space, and The past parking success rate associated with the aforementioned first parking space, The system according to claim 1, comprising at least one of the following.

5. In order to guide the first autonomous vehicle to the first parking space, the processing device, To determine a second probability value associated with a second parking space within the driving environment that remains unoccupied within the aforementioned target time, The first parking space is selected as described above. The difference between the first probability value and the second probability value, The difference between the first driving distance to the first parking space and the second driving distance to the second parking space, and At least one of the differences between the first driving time to the first parking space and the second driving time to the second parking space, The system according to claim 1, which makes a selection and performs a selection based on the above.

6. The system according to claim 1, wherein the processing device guides the first autonomous vehicle to the first parking space based on an amount of alternative parking, which is at least one of a target distance from the first parking space and a target driving time.

7. The processing device, The first autonomous vehicle receives a message indicating that it was unable to park in the first parking space. To determine a second probability value associated with the second parking space that is not occupied during the driving time, between the first parking space and the second parking space, The system according to claim 1, wherein the first autonomous vehicle is guided to the second parking space based at least on the second probability value.

8. The processing device, At a minimum, based on map updates of the live parking space occupancy rate and the past parking space availability in the aforementioned driving environment, a second possibility associated with a second parking space that is not occupied within the second target time is determined. The system according to claim 1, further comprising guiding a second autonomous vehicle of the fleet to a second parking space based on at least a second probability value.

9. The processing device, The first autonomous vehicle receives an indication from the first autonomous vehicle indicating whether it was able to park in the first parking space. Based on the received display, the past availability of parking spaces in the driving environment is updated, The system according to claim 1, further comprising the following steps.

10. It is an autonomous vehicle, A sensing system for collecting sensing data that characterizes the driving environment of the autonomous vehicle, A data processing system that uses the sensing data to at least, Identification of one or more objects located at the edge of the drivable portion of the aforementioned operating environment, A data processing system that generates edge visibility data representing the visibility of the edges of the drivable portion of the aforementioned operating environment, It is a transceiver, The fleet server communicates the identification of at least one of the aforementioned objects and the edge visibility data, The fleet server receives identification of the first parking space within the driving environment, A transceiver and A vehicle control system for moving the autonomous vehicle to the first parking space, An autonomous vehicle equipped with [the following features].

11. The sensing system, One or more lidar sensors, One or more radar sensors, and One or more camera sensors, The autonomous vehicle according to claim 10, comprising at least one of the following:

12. The sensing system, Further, additional sensing data is collected for the first parking space. The aforementioned data processing system Based on the aforementioned additional sensing data, it is determined that the first parking space is occupied. The aforementioned transceiver, The fleet server is informed that the first parking space is occupied, The fleet server receives identification of a second parking space within the driving environment, Further, The aforementioned vehicle control system The autonomous vehicle according to claim 10, further comprising moving the autonomous vehicle to the second parking space.

13. The sensing system, Further, the following steps are taken to collect additional sensory data that characterizes the driving environment of the autonomous vehicle: The aforementioned data processing system Based on the additional sensing data, it is determined that a second parking space is available, and that the second parking space is located closer to the autonomous vehicle than the first parking space. The aforementioned vehicle control system The autonomous vehicle according to claim 10, further comprising parking the autonomous vehicle in the second parking space.

14. A fleet of vehicles, Multiple vehicles, each of the multiple vehicles is Using the sensing system of each of the aforementioned multiple vehicles, sensing data is collected to characterize the driving environment of each of the aforementioned vehicles. Using the data processing system of each of the aforementioned vehicles and the sensing data, the curb occupancy data is generated, and the curb occupancy data is Identification of one or more objects located on the curb in the aforementioned driving environment, and Curb visibility data representing the visibility of the curb in the aforementioned driving environment, It must include at least one of the following: Multiple vehicles will perform the task, A processing device, Updating the live parking space occupancy map using curb occupancy data generated by multiple vehicles, At a minimum, determine a first probability value associated with a first parking space that remains vacant within the target time, based on an updated map of live parking space occupancy rates and historical parking space availability in the driving environment. To guide a first vehicle out of several vehicles to a first parking space based on at least a first probability value, A processing device that performs this, Includes, The first of several vehicles, The fleet of vehicles moves to the first parking space.

15. The sensing system of each of the aforementioned vehicles One or more lidar sensors, One or more radar sensors, and One or more camera sensors, A fleet of vehicles according to claim 14, comprising at least one of the following:

16. In order to update the map of live parking space occupancy rates, the processing device, The vehicle fleet according to claim 14, which combines curb occupancy data received from multiple vehicles of the multiple vehicles, each representing the same segment of the curb.

17. The aforementioned past availability of parking spaces, The past parking space occupancy rates of multiple parking spaces, including the aforementioned first parking space, and The past parking success rate associated with the aforementioned first parking space, A fleet of vehicles according to claim 14, comprising at least one of the following.

18. In order to guide the first vehicle to the first parking space, the processing device, To determine a second probability value associated with the second parking space, within the driving environment, that remains unoccupied during the aforementioned target time, The first parking space is selected as described above. The difference between the first probability value and the second probability value, With respect to the first vehicle, the difference between the first driving distance to the first parking space and the second driving distance to the second parking space, and With respect to the first vehicle, at least one of the differences between the first driving time to the first parking space and the second driving time to the first parking space, Based on that, to make a choice, A fleet of vehicles according to claim 14, which performs the following.

19. The aforementioned first vehicle, Using the sensing system of the first vehicle, additional sensing data for the first parking space is collected. The first vehicle's data processing system is used to process the additional sensing data, thereby determining that the first parking space is occupied. Further, The processing device Receiving a message from the first vehicle indicating that the first parking space is occupied, To determine a second probability value associated with the second parking space that is not occupied during the driving time, between the first parking space and the second parking space, At a minimum, guide the first vehicle to the second parking space based on the second probability value, A fleet of vehicles according to claim 14, which performs the following.

20. The aforementioned first vehicle, Collecting additional sensing data using the sensing system of the first vehicle, Determining that a second parking space is available by processing the additional sensing data using the data processing system of the first vehicle, wherein the second parking space is located closer to the first vehicle than the first parking space, Moving to the second parking space, The fleet of vehicles according to claim 14, further comprising the above.

Images