Fleet vehicle operation method and system

Abstract

The method for operating a fleet of vehicles includes collecting a set of inputs, processing the set of inputs to determine a set of actions related to the vehicles and / or sites, and triggering the set of actions. Additionally or alternatively, the method may include aggregating any or all of the set of inputs, and / or any other suitable processing. A system for operating a fleet of vehicles may include and / or interface with any or all of a computing subsystem, a set of management subsystems, a set of user interfaces, a set of sensors, a set of fleet vehicles, a set of non-fleet vehicles, and / or other components.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 253,866, filed October 8, 2021, which is incorporated by this reference in its entirety.

[0002] The present invention relates generally to the field of autonomous vehicles, and more particularly to novel and useful systems and methods for operating fleet vehicles in the field of autonomous vehicles. [Background technology]

[0003] Autonomous vehicle technology has made significant advancements in recent years, greatly expanding the potential use cases for these technologies. One major use case is autonomous trucking, with the industry having a strong desire to automate the delivery of goods.

[0004] The inventors discovered that implementing autonomous vehicles in this use case poses several challenges, as not having a human on board the autonomous vehicle can lead to many inefficiencies at the vehicle loading and delivery site. For example, when a vehicle arrives at a customer site, site personnel may be elsewhere, engaged in other tasks, have already begun loading or unloading another vehicle, or be otherwise occupied. Without a human operator on board the vehicle to initiate these processes or check in, operational efficiency can be significantly reduced.

[0005] Therefore, there is a need in the field of autonomous vehicles to create improved and useful systems and methods for operating fleet vehicles. [Brief explanation of the drawings]

[0006] This patent or application contains one or more color drawings. Copies of any color drawing(s) in this patent or patent application publication will be provided by the Office upon request and payment of the necessary fee. [Figure 1] FIG. 1 is a schematic diagram of how a fleet of vehicles is operated. [Figure 2] FIG. 2 is a schematic diagram of a vehicle operation system. [Figure 3] 3A-3B are schematic diagrams of an example use of the system and / or method implemented in a delivery and associated fleet. [Figure 4] 4A-4B show an example of a user interface for vehicle tracking, insight, and / or management. [Figure 5] 5A-5B show an example of a user interface for interacting with a vehicle. [Figure 6] FIG. 6 shows a schematic example of information flow and / or triggered actions along a vehicle's delivery route. [Figure 7] 7A-7B show schematic variations of the management of a set of vehicles interacting with a site. [Figure 8] FIG. 8 is a schematic diagram illustrating an example of information flow between components of a vehicle operation system. [Figure 9] FIG. 9 shows a schematic example of the journey of a set of vehicles to a set of sites. [Figure 10] 10A-10C show a schematic example of the management of a set of vehicles interacting with a site. DETAILED DESCRIPTION OF THE INVENTION

[0007] The following description of preferred embodiments of the invention is not intended to limit the invention to those preferred embodiments, but rather to enable any person skilled in the art to make and use the invention.

[0008] 1. Overview As shown in FIG. 1 , fleet vehicle operation method 100 includes step S110 of collecting a set of inputs, step S120 of processing the set of inputs to determine a set of actions related to the vehicles and / or sites, and step S130 of triggering the set of actions. Additionally or alternatively, method 100 may include step S115 of aggregating any or all of the set of inputs and / or any other suitable processing. Additionally or alternatively, method 100 may include and / or interface with any or all of the methods, processes, embodiments, and / or examples for operating an ego agent, such as those described in any or all of U.S. Application No. 17 / 116,810 (filed December 9, 2020), U.S. Application No. 17 / 125,668 (filed December 17, 2020), and U.S. Application No. 17 / 127,599 (filed December 18, 2020), which are incorporated herein by reference in their entireties.

[0009] The method 100 is preferably performed on a system 200 described below, but may additionally or alternatively be performed on other systems.

[0010] 2 , the system 200 for operating a fleet of vehicles may include and / or interface with any or all of a computing subsystem, a set of management subsystems, a set of user interfaces, a set of sensors, a set of fleet vehicles, a set of non-fleet vehicles, and / or other components. Additionally or alternatively, the system may include or all of the components described in U.S. application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. application Ser. No. 17 / 125,668, filed December 17, 2020, and U.S. application Ser. No. 17 / 127,599, filed December 18, 2020, each of which is incorporated herein by reference in its entirety.

[0011] In a preferred set of variations (e.g., as shown in FIGS. 3A-3B), the method and / or system are configured for use in short-haul logistics applications, such as using autonomous vehicles to deliver items between destinations. The autonomous vehicles preferably make deliveries along fixed routes, but may additionally or alternatively follow dynamic routes or be otherwise operated.

[0012] Additionally or alternatively, the method and / or system may be utilized in any other use case (e.g., ride sharing, people transportation, long-distance logistics, etc.).

[0013] 2.Effects A fleet vehicle operating system and method can provide several advantages over current systems and methods.

[0014] In a first variation, the technology provides the advantage of automatically improving the efficiency (e.g., operational efficiency) of loading and / or unloading of autonomous vehicles at a site, thereby preventing the accumulation of excess vehicles at the site, prioritizing and / or optimizing the order in which vehicles are engaged (e.g., for loading and unloading at a loading dock), providing instructions to autonomous vehicles at the site, and / or performing any other action to improve efficiency.

[0015] Additionally or alternatively, the technology may provide benefits that increase operational efficiencies associated with a site through any or all of the dynamic updating of any or all management subsystems associated with the site and / or fleet, inter-vehicle communications (e.g., between fleets, between fleet and non-fleet vehicles, etc.) to transmit updated and / or dynamic information, automatic adjustments to the routing and / or sequencing of vehicle destinations, and / or other actions or outputs.

[0016] In a second variation, in addition to or instead of the first variation, the present technology advantageously leverages sensors onboard autonomous vehicles to provide awareness of the site situation (e.g., how good is the site, how many other vehicles are on the site, whether there are weather conditions issues at the site, etc.) and uses this awareness to initiate one or more actions for fleet management, optimizing the efficiency of the ego agent's route set, updating the site's environmental representation, triggering any appropriate action to increase efficiency and / or safety, and / or triggering any other action.

[0017] In one set of embodiments, for example, the system and / or method may advantageously detect when a site has limited or no additional vehicle capacity and adjust the routes of other vehicles en route to the site accordingly (e.g., by diverting to another site, waiting at the previous site, etc.).

[0018] In another example, in addition to or instead of the above examples, the system and / or method may provide the advantage of dynamically updating and / or maintaining a site-related environmental representation (e.g., a map) that is more accurate than an environmental representation maintained by the site itself (e.g., which requires manually entered data from non-AV vehicles and does not take into account the locations of non-vehicle objects such as humans), thereby enabling deliveries to be optimized for fleet and / or non-fleet vehicles.

[0019] In another example, in addition to or in place of the above, the system and / or method may provide an advantage of increasing safety associated with a site where the site is overcrowded or associated with outdated knowledge of the site management subsystem. In certain implementations, for example, a dynamic environmental representation maintained by the fleet management subsystem may provide warnings and / or other actionable insights based on the user's location, the location of objects that may be in the blind spots of other vehicles, and / or any other information.

[0020] In a third variation, in addition to or as an alternative to the above variations, the technology provides the advantage of collecting and aggregating a corpus of data that can help customers optimize their site (e.g., site layout) and / or site operational protocols, which can be used to determine, for example, how many workers to best staff each shift, what site layout is best to prevent congestion, how many loading docks a site should have, which features of a site are causing operational inefficiencies, the average wait time for vehicles to be loaded and / or unloaded after arriving at a site, and / or other information.

[0021] Additionally or alternatively, the systems and methods may provide other benefits.

[0022] 3. System 2 , the system 200 for operating a fleet of vehicles may include and / or interface with any or all of a computing subsystem, a set of management subsystems, a set of user interfaces, a set of sensors, a set of fleet vehicles, a set of non-fleet vehicles, and / or other components. Additionally or alternatively, the system may include any or all of the components described in U.S. application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. application Ser. No. 17 / 125,668, filed December 17, 2020, and U.S. application Ser. No. 17 / 127,599, filed December 18, 2020, each of which is incorporated herein by reference in its entirety.

[0023] The system is preferably configured for implementation using a set of autonomous vehicles (equivalently referred to herein as autonomous agents, ego agents, etc.), each of which is preferably fully autonomous and / or capable of operating as a fully autonomous vehicle, but may additionally or alternatively be any semi-autonomous or fully autonomous vehicle, remotely operated vehicle, and / or any other suitable vehicle. The autonomous vehicles are preferably motor vehicles (e.g., passenger cars, unmanned vehicles, buses, shuttles, taxis, ride-share vehicles, trucks, semi-trucks, etc.), but may alternatively include personal watercraft (e.g., boats, water taxis, etc.), aircraft (e.g., airplanes, helicopters, drones, etc.), land vehicles (e.g., motorcycles, bicycles, motorcycles, scooters, etc.), and / or other suitable vehicles and / or transportation devices, autonomous machines, autonomous devices, autonomous robots, and / or other suitable devices.

[0024] The system is further preferably configured for implementation in a fleet of autonomous vehicles, where the autonomous vehicles in the fleet are managed by a fleet management subsystem (e.g., as described below), which may, for example, enable inter-vehicle communication from one vehicle in the fleet to another vehicle in the fleet (e.g., directly, indirectly via the fleet management subsystem), control and manage the vehicles in the fleet to optimize the overall operational efficiency of the fleet, enable the vehicles in the fleet to combine sensor information to create a complete or most complete environmental representation of their surroundings (e.g., a site), and / or achieve any other output or result.

[0025] Additionally or alternatively, the fleet may include non-autonomous (e.g., manual) vehicles, autonomous vehicles may be controlled outside the fleet, any or all of the valid results and / or outputs may occur between fleet and non-fleet vehicles, and / or the system may be suitably implemented in any other manner.

[0026] The system preferably includes and / or interfaces with a set of one or more computing subsystems that individually and / or collectively function to process any or all of the sets of inputs received to implement method 100, as described below. Additionally or alternatively, the computing subsystems may function to perform actions (e.g., route planning, trajectory planning, obstacle detection, perception, prediction, etc.) for the operation of the vehicle (e.g., in combination with the vehicle's control subsystems (e.g., set of controllers) and / or actuation subsystems (e.g., drive-by-wire subsystems)). Further, additionally or alternatively, the computing subsystems may perform any other suitable functions.

[0027] The computing subsystems (e.g., a set of computers, a set of processors, etc.) may include any or all of a set of one or more on-board computing subsystems aboard each fleet of vehicles, a set of one or more remote computing subsystems, and / or any combination of on-board and remote computing subsystems. Additionally or alternatively, the computing subsystems may include and / or interface with a computing subsystem aboard a user device (e.g., an edge device, a tablet, a laptop, a mobile phone, etc.) and / or any other computing subsystem.

[0028] In a preferred variation, the computing subsystems include an on-board computer installed in each of the set of ego agents and a remote computing subsystem (e.g., a cloud computing subsystem) that communicates with the on-board computing computers, but may additionally or alternatively include subsets of these, other computing subsystems, and / or any combination.

[0029] The system further preferably includes and / or interfaces with a set of user interfaces that may be provided to and / or accessed by any or all of site personnel (e.g., as described below), vehicle drivers (e.g., safety drivers), remote operators (e.g., teleoperators), fleet management entities, and / or any other entities. The user interfaces are preferably in the form of client applications executable on a device (e.g., user device, mobile device, tablet, laptop, computer, smartphone, etc.) or other suitable computing subsystem and / or any associated output device (e.g., display, speaker, etc.), but may additionally or alternatively include any other user interface.

[0030] The client may be a native application, a browser application, an operating system application, or any other suitable application or executable.

[0031] For example, in some variations, the user interface is utilized by site personnel (equivalently referred to herein as customers, site personnel, associates, etc.) and is configured to enable any or all of the following: optimizing interactions between site personnel and vehicles within the fleet; optimizing interactions between site personnel and other vehicles (e.g., vehicles outside the fleet, AVs, non-AVs, etc.); securing the load (e.g., ensuring the load is secure, ensuring only designated individuals are interacting with the AV, etc., via authorized access procedures implemented in the fleet's user interface); tracking load information before, during, and after the loading and unloading process; enabling authorized users to control basic functions of the AV, communicate with the AV when it is safe to depart, request assistance if needed, or otherwise interact with the AV; providing information to the AV (e.g., informing the AV that it is safe to depart); providing information to the AV (e.g., informing the AV that it is safe to depart the dock and / or site, indicating to the AV why it is not yet safe to depart and / or why a delay has occurred, etc.), and / or conveying other information or actions.

[0032] Examples of user devices include tablets, smartphones, mobile phones, laptops, watches, wearable devices (such as glasses), or other suitable user devices. In some variations, for example, site (e.g., yard) and / or warehouse personnel can interact with user devices (e.g., tablets, computers, mobile phones, etc.) that function to allow the user to provide input and / or receive information to / from a display or other output of the user device.

[0033] The user device may include a power storage device (e.g., a battery), a processing system (e.g., a CPU, GPU, memory, etc.), a user output (e.g., a display, a speaker, a vibration mechanism, etc.), a user input (e.g., a keyboard, a touchscreen, a microphone, etc.), a location system (e.g., a GPS system), sensors (e.g., a light sensor, a light sensor such as a camera, an orientation sensor such as an accelerometer, a gyroscope, an altimeter, an audio sensor such as a microphone, etc.), a data communication system (e.g., a WiFi module, a BLE, a cellular module, etc.), or other suitable components.

[0034] The output of the user device may include any or all of a display (such as an LED display, OLED display, LCD, etc.), an audio speaker, a light (such as an LED), a haptic output (such as a ticel system, vibration motor, etc.), or other suitable output.

[0035] The user device input may include any or all of a touchscreen (capacitive, resistive, etc.), a mouse, a keyboard, a motion sensor, a microphone, a biometric input, a camera, or other suitable input.

[0036] Examples of the user interface are shown in FIGS. 4A-4B and 5A-5B.

[0037] The system further preferably includes and / or interfaces with a set of sensors onboard the ego agent, such as, but not limited to, any or all of sensors coupled to the vehicle's exterior, sensors coupled to the vehicle's interior, sensors of the vehicle's auxiliary systems, diagnostic sensors associated with the vehicle (e.g., as part of an on-board diagnostic (OBD, OBD-I, OBD-II, etc.) subsystem), and / or any other sensors. Additionally or alternatively, sensors may be located off-board the ego agent (e.g., in the ego agent's environment, at the site, on a user device, etc.), or in other suitable locations.

[0038] The sensors may include, but are not limited to, cameras (visible range, multispectral, hyperspectral, IR, stereoscopic, etc.), light detection and ranging (lidar) sensors, radio detection and ranging (radar) sensors, orientation sensors (accelerometers, gyroscopes, altimeters, etc.), acoustic sensors (microphones, etc.), optical sensors (photodiodes, etc.), temperature sensors, pressure sensors, flow sensors, vibration sensors, proximity sensors, chemical sensors, electromagnetic sensors, force sensors, sensors that communicate with an OBD port or other original equipment manufacturer (OEM) systems, telematic sensors, and / or other types of sensors.

[0039] The system may optionally include and / or interface with a set of one or more communication subsystems, which may function to enable any or all of the following: communication between vehicles (e.g., fleet vehicles, fleet vehicles and non-fleet vehicles), communication between management subsystems (e.g., as described below), communication between management subsystems and vehicles, communication between management subsystems and user interfaces (UIs), communication between vehicles and user interfaces, and / or any other type of communication.

[0040] The communications subsystem may include one or more radios or other suitable components. The communications subsystem may be a long-range communications system, a short-range communications system, or any other suitable communications system. The communications system may implement wired and / or wireless communications. Examples of communications systems may include 802.11x, Wi-Fi, Wi-Max, WLAN, NFC, RFID, Bluetooth, low-energy Bluetooth, long-range BLE, ZigBee, cellular communications (2G, 3G, 4G, LTE, etc.), radio (RF), microwave, IR, audio, optical, wired connections (USB, etc.), or other suitable communications modules or combinations thereof.

[0041] The system preferably includes and / or interfaces with a set of one or more management subsystems (e.g., a software platform, a centralized software platform, management software, etc.), which function to implement, integrate, coordinate, interface, and / or trigger (e.g., initiate) any or all of the processes of method 100. Additionally or alternatively, the set of management subsystems may function to enable operation of the set of vehicles (e.g., driving along a route) and / or perform other functions.

[0042] The set of one or more management subsystems preferably communicates with (e.g., forms a part of, includes, interfaces with, etc.) the set of computing subsystems, e.g., such that actions of the management subsystems can be determined and / or triggered based on processing performed in and / or with the computing subsystems. In a preferred variation, for example, the set of management subsystems is at least partially cloud-based (e.g., implemented in a remote computing subsystem). Additionally or alternatively, the set of management subsystems can interface with and / or include a local computing subsystem (e.g., an on-board computing subsystem, a site-based computing subsystem, etc.) or a combination of computing subsystems. The set of management subsystems can further interface with any or all of user interfaces and / or user devices (e.g., to effectuate and initiate the provision of notifications and / or alerts to users and / or vehicles), vehicles (e.g., fleet vehicles, non-fleet vehicles, etc.), databases (e.g., for use in fleet optimization, for use in data aggregation, etc.), and / or other components.

[0043] The set of one or more management subsystems may interface with any number of users and / or teams of users, which may include, for example, teams associated with one or more customer sites (e.g., loading yards and / or docks and / or bays, customer parking lots, etc.) (which may be referred to as any or all of customer operations teams, receiving teams, site / yard crews, and / or other teams), teams associated with one or more fleet sites (e.g., crews located at fleet depots where fleet vehicles are stored and / or maintained (e.g., during transfers between customer sites)), teams associated with the operation of the fleet (e.g., human drivers, fleet managers and / or schedulers, fleet maintenance and / or cleaning and / or repair teams, etc.), which may be referred to as fleet operations teams, remote operations teams, and / or other teams and / or combinations of teams.

[0044] A customer site, as used herein, preferably refers to a location and / or area (e.g., yard, lot, bay, parcel of land, etc.) associated with a fleet customer, where fleet vehicles and optionally any number of non-fleet vehicles receive and / or load / unload items (e.g., loads, deliveries, etc.). These may include and / or be associated with, for example, any or all of retail stores, distribution centers, fulfillment and / or sortation centers (e.g., micro-fulfillment centers), storage centers, warehouses, manufacturing sites, private residences, and / or other locations. In a preferred use case, for example, fleet vehicles are utilized to autonomously and / or semi-autonomously transport goods between customer sites (e.g., pick up from a first set of customer sites and drop off at a second set of customer sites). Additionally or alternatively, fleet vehicles may be used to transport passengers, transport goods and / or passengers between any number and / or types of sites, and / or in any other suitable manner.

[0045] A fleet site, as used herein, preferably refers to a location and / or area (e.g., lot, yard, depot, etc.) where fleet vehicles undergo any or all of storage (e.g., when not at a customer site and / or between customer sites, overnight stays, etc.), maintenance, repair, refueling, and / or other disposition. Additionally or alternatively, non-fleet vehicles may interface with a fleet site, fleet vehicles may load and / or unload items at a fleet site, fleet vehicles may be stored and / or maintained and / or repaired and / or refueled at a customer site and / or other locations, fleet vehicles may operate without a fleet site, and / or fleet vehicles may operate as otherwise suitable.

[0046] Additionally or alternatively, the fleet vehicles may interact with any other suitable site for any suitable functionality.

[0047] The users may be human users, automated assistants and / or robots, and / or any combination of human and non-human users.

[0048] The set of management subsystems may function individually or collectively to perform and / or be involved in any or all of shipment scheduling, route planning and / or route optimization, vehicle monitoring (e.g., live fleet monitoring), data collection (e.g., for analytics, fleet optimization, site optimization, etc.), authentication and / or security of goods collection, site safety, vehicle acceptance and / or release, and / or other functions.

[0049] The set of management subsystems may optionally include a fleet management subsystem (e.g., a fleet management platform, a fleet management software platform, etc.), which may function individually and collectively to perform some or all of the following site management functions, for example, shipment scheduling, route planning (e.g., high-level route planning, site sequencing, etc.), vehicle dispatch scheduling and / or initiation, delivery and / or pickup optimization, live fleet monitoring, data collection and / or aggregation, fleet-wide optimization, vehicle-to-vehicle interactions, fleet vehicle rerouting (e.g., changing the order of sites that fleet vehicles travel through in a day), and / or other functions.

[0050] The Fleet Management Subsystem preferably receives as input sensor data (e.g., camera data, LIDAR data, RADER data, location data, etc.) from fleet vehicles, but may additionally or alternatively receive and / or collect any or all of the following: information from the Site Management Subsystem (described below) (e.g., scheduling information, requests, notifications, etc.), information from one or more databases (e.g., aggregated historical fleet data, traffic database data, weather database data, etc.), information from user devices and / or user interfaces, and / or other information.

[0051] In a preferred variation, the Fleet Management Subsystem is involved in dispatch scheduling of the fleet vehicles and / or executes dispatch schedules for the fleet vehicles (e.g., received from the Site Management Subsystem). In one set of examples, for example, based on high-level scheduling instructions received from the Site Management Subsystem (e.g., times and locations of available docks), the Fleet Management Subsystem can initiate, adjust, optimize, and / or integrate schedules for associated fleet vehicles. In a variation, for example, a high-level set of available times and locations associated with customer sites is provided to the Fleet Management Subsystem (e.g., from the Site Management Subsystem) and used by the Fleet Management Subsystem to assign and manage time and destination assignments to individual fleet vehicles (e.g., based on schedules of other fleet vehicles, based on fleet vehicle availability, based on other customer sites to which the fleet vehicles need to travel, based on optimization of the types and / or weights of goods that the fleet vehicles can hold, based on fuel efficiency considerations, based on optimized transport consolidation, etc.). In one set of examples, for example, the fleet management subsystem receives high-level scheduling information from the site management subsystem, consolidates the information, optimizes individual vehicle schedules, assigns schedules to vehicles, and creates a single source of truth schedule for the fleet. In particular examples, the fleet management subsystem may further adjust the schedules of the fleet vehicles (e.g., adjust the order of sites to which the fleet vehicles travel, reroute the vehicles to an alternative location, change the timing of loading and / or unloading of the fleet vehicles at a site, etc.) in response to any or all of the processes of method 100 in response to a determination that a site is currently unavailable and / or not predicted to be available at the original time (e.g., based on sensor data collected by another fleet vehicle at that site). This may optionally conflict with information associated with the site management subsystem, for example, thereby allowing the fleet vehicles to operate based on the most recent, dynamically determined information.Additionally or alternatively, scheduling of shipments may be accomplished in any other suitable manner.

[0052] The fleet management subsystem may optionally, additionally or alternatively, perform live (e.g., real-time, near real-time, substantially real-time, less than 5 second delay, less than 1 second delay, less than 500 millisecond delay, less than 100 millisecond delay, less than 50 millisecond delay, etc.) monitoring and / or data collection of fleet vehicles, which may function to provide up-to-date information to the site (e.g., current location, estimated time of arrival (ETA), information regarding loads being carried by the vehicle and / or loads to be picked up at the site, etc.) so that site personnel can optimally (e.g., efficiently) prepare the vehicle for arrival, provide up-to-date information regarding vehicle status (e.g., fuel level, diagnostic trouble codes, tire pressure, engine temperature, etc.) to fleet management entities (e.g., fleet management subsystem, depot personnel, fleet maintenance teams, etc.) so that the fleet can be maintained efficiently and / or optimally, optimize overall fleet efficiency, enable efficient re-routing of vehicles in the event of delays or unexpected events (e.g., based on precise known locations and / or estimated times of arrival), and / or perform other functions.

[0053] In one set of variations (e.g., as shown in FIG. 6 ), for example, the fleet management subsystem may collect and process location data of a fleet vehicle during its travel (e.g., upon leaving a geofenced area such as a customer site and / or depot, while traveling to a customer site, any time during travel, etc.) to determine an ETA for the vehicle to arrive at its destination (e.g., a customer site, a depot, a maintenance location, etc.). For example, in some examples, additional information (e.g., collected by the vehicle, received from a database or third-party subsystem, etc.) may be used and / or processed (e.g., in conjunction with the vehicle's location, independent of the vehicle's location, etc.) to determine the most accurate ETA for the vehicle, which may be determined based on and / or adjusted for some or all of traffic conditions (e.g., detected by other vehicles on the road, received from a traffic database and / or map estimates, etc.), weather conditions (e.g., road conditions based on current and / or forecasted weather, forecasted road conditions, etc.), vehicle conditions (e.g., current fuel level, current maintenance status, etc.), and / or other conditions.

[0054] This supplemental information (traffic conditions, road conditions, etc.) may be determined based on any or all of the following: information collected from other vehicles on the road (fleet vehicles, non-fleet vehicles, etc.) (e.g., determined based on processed sensor data (e.g., camera data) from other fleet vehicles passing through the route; database and / or third party information (e.g., traffic databases, traffic reports, map applications, live weather updates, etc.); predicted (e.g., using a set of models, such as a set of trained (machine learning, deep learning, etc.) models, based on aggregated and / or historical information); received from a user (e.g., site workers, depot workers, etc.); determined based on a combination of information; and / or determined in any other suitable manner.

[0055] In one set of examples (e.g., as shown in Figures 4A-4B), a user interface is provided to one or more users (e.g., customer site personnel, depot personnel, vehicle operators, etc.) that provides live monitoring of the vehicle's location and, optionally, additional information such as, but not limited to, the ETA for the vehicle to arrive at its destination, the vehicle's sub-destinations (e.g., which loading dock and / or parking lot it is assigned to), the vehicle's driving history and / or health information (e.g., sensor information, diagnostic information, fuel level, etc.), cargo information, and / or other information.

[0056] The fleet management subsystem may optionally, additionally or alternatively, collect and / or process environmental information collected from the fleet vehicles (e.g., with a set of sensors), which may function to maintain a dynamic understanding of the site, enable more optimized scheduling for the fleet based on the most up-to-date information, increase the available types of information associated with the site (e.g., people locations), reduce vehicle idling time at the site, prevent chaos and / or overcrowding at the site, improve safety associated with the site, and / or perform any other function.

[0057] In a preferred set of variations, for example, dynamic data is collected from sensors in each fleet vehicle (e.g., at the site, en route to the site, at the depot, etc.) and can be used to determine, create, and / or supplement a dynamic representation of that vehicle's environment, which can be used to optimize and / or coordinate schedules and / or routes of other vehicles, facilitate sequencing and / or minimize congestion relative to the site, improve vehicle and human safety at the site, optimize movement within the site (e.g., informing where vehicles should arrive and / or wait if the site is congested, informing vehicle trajectories within the site, etc.), and / or perform other functions.

[0058] The dynamic data is preferably collected from at least one set of optical sensors, such as camera data and / or LIDAR data (e.g., from cameras and / or LIDAR sensors mounted inside and / or outside the vehicle), but may additionally or alternatively include RADAR data and / or other sensor data. The sensor data may then be processed (e.g., using a series of models and / or algorithms (e.g., object detection models and / or algorithms, computer vision models and / or algorithms, etc.)) to determine any or all of the following: the presence of an object (e.g., a vehicle, a human, a non-living object, etc.), the location of the object (e.g., relative to a loading dock, relative to a parking spot, etc.), the size of the object, parameters related to the object (e.g., direction of movement, potential direction of movement, speed, status of the vehicle's turn signal, etc.), site information (e.g., the number of loading docks, whether there are waiting areas for vehicles and / or their occupants, the number and location of site personnel, etc.), site conditions (e.g., whether the road is wet, slippery, icy, snowy, whether there are potholes or obstacles, whether there are spilled cargo, etc.), whether there are objects present that may be obscured by other vehicles (e.g., in the vehicle's blind spot) so that other vehicles can use this information, and / or other information.

[0059] In one series of variations, the vehicles of the fleet are configured as and / or effectively function as mobile sensor units to collectively maintain dynamic awareness of sites, such as customer sites and / or sites managed by the fleet (e.g., depots). Additionally or alternatively, the fleet vehicles may monitor other environments (e.g., roads along their routes, etc.).

[0060] For example, in a preferred variation, vehicles will naturally be located in multiple different locations throughout the day as they complete pickups and deliveries. Because sensor information is collected throughout the day, a dynamic environmental representation of the site can be identified and maintained (e.g., as long as at least one vehicle is present at each site, as long as there are at least enough vehicles present at each site to collectively have a view of the entire site, etc.). This can have many advantages over traditional and / or current management subsystems, such as site management subsystems (e.g., described below) and / or other management subsystems, which identify only a subset of this information (e.g., only the location of vehicles at loading docks) and do not dynamically update this information (e.g., relying on manual input from site workers and / or human drivers), which can lead to outdated information and / or otherwise not include and / or provide accurate information. For example, in one set of examples of conventional site management subsystems, the site management subsystem requires at least some manually entered data (e.g., from site workers, from non-AV drivers, etc.), which can lead to delay issues, inaccurate and / or infeasible schedule propagation, long waiting times for vehicles at the site, potential collisions or other unsafe conditions for vehicles at chaotic and / or overcrowded sites, and / or other consequences.

[0061] Examples of the difference between environmental representations generated based on information from vehicle sensors and data collected by the site management subsystem are shown in Figures 10B and 10C, respectively.

[0062] Additionally or alternatively, the fleet management subsystem may update and / or replace the representations of the site management subsystem, the representations may be combined and / or aggregated, the fleet management subsystem may be used in the absence of a site management subsystem, and / or any other configuration of the management subsystem may be implemented.

[0063] The outputs of the fleet management subsystem (and / or other management subsystems) may include, without limitation, any or all of notifications (e.g., to site personnel, to a human driver, to a fleet operator, to a teleoperator, etc.) or other alerts, information provided in a user interface, environmental representations and / or updates to environmental representations, automated scheduling, other triggered actions (e.g., as described below), and / or any other outputs.

[0064] Example outputs include, for example, notifications providing customer stakeholders (e.g., site workers) with accurate vehicle ETAs for preparation for unloading (see Customer / Site Interface), notifications to users (e.g., customers) regarding vehicle delivery delays and / or issues, notifications to other vehicles, fleet or non-fleet, regarding updates and / or adjustments to schedules, site conditions, potential site outages, and / or other information, determining and / or adjusting vehicle schedules and / or routes to reduce vehicle idle time, maximize fleet utilization, and / or achieve other results, providing visibility to users (e.g., to a user interface) across the fleet's delivery network, and / or other outputs.

[0065] The set of management subsystems may optionally additionally or alternatively include and / or interface with one or more site management subsystems (e.g., per site, per customer, multiple sites, etc.), which may function to perform scheduling of vehicles (e.g., fleet vehicles, non-fleet vehicles, etc.) at a site, provide inputs used in scheduling (e.g., general loading dock and / or site availability), and / or in any other suitable manner.

[0066] The Site Management Subsystem (also referred to herein equivalently as the Customer Management Subsystem) preferably communicates with the Fleet Management Subsystem, but may additionally or alternatively communicate directly with one or more vehicles (e.g., via the vehicle's computing and / or control subsystems, with the vehicle driver via a user interface, etc.), directly with one or more users (e.g., site personnel, fleet operations team members, manual vehicle drivers, via a user interface, etc.), not communicate with the Fleet Management Subsystem, and / or communicate with other entities, or not communicate with other entities.

[0067] In a first set of variations, the fleet management subsystem is integrated with a site management subsystem for each customer and / or each site of each customer. In a second set of variations, each customer is integrated with a fleet management subsystem of a set of vehicles.

[0068] Examples of site management subsystems include, but are not limited to, any or all of a transportation management system (TMS), a warehouse management system (WMS), a yard management system (YMS), an enterprise resource planning (ERP) system, other software platforms, and / or any combination of software platforms.

[0069] Information from the site management subsystem may include, but is not limited to, any or all of scheduling information (e.g., appointment time, location where load is to be and / or needs to be dropped off, time load is to be and / or needs to be dropped off, type of load that needs to be picked up, etc.), dock availability and / or assigned docks (e.g., planned and / or current status of dock doors as occupied or unoccupied), and / or other information.

[0070] In one set of variations, for example, the site management subsystem provides scheduling information to the fleet management subsystem, such as a set of reservation times for fleet vehicles, a set of locations where fleet vehicles are to load and / or unload (e.g., dock door assignments), etc. Additionally or alternatively, other information may be provided and / or information may be provided to other entities (e.g., to site personnel via a user interface, directly to vehicles, fleet vehicles, non-fleet vehicles, etc.).

[0071] In one set of examples, the fleet management subsystem collects, determines, and / or supplements this information using any or all of information (e.g., presence, location, etc.) about non-vehicle objects (e.g., pedestrian location, such as detecting a person being positioned within a loading dock), information about non-fleet vehicles (e.g., not manually entered and / or not yet entered into the site management subsystem), information about off-dock and / or off-site areas (e.g., occupancy of parking lots, waiting areas, roads leading to a site, etc., detecting congestion leading to a site, detecting that there are "x" vehicles waiting to enter a site and / or dock, etc.), and / or other information.

[0072] Additionally or alternatively, the site administration subsystem may receive and / or provide this information and / or any or all of the information described above. Additionally or alternatively, the system may include and / or interface with other management subsystems, the management subsystems may have other functionality, the functions of the fleet management subsystem may be performed by other and / or multiple management subsystems, the functions of the site management subsystem may be performed by other and / or multiple management subsystems, and / or the management subsystems may be in other configurations.

[0073] Additionally or alternatively, the system may include and / or interface with any other suitable components.

[0074] 4. Method As shown in FIG. 1 , fleet vehicle operation method 100 includes step S110 of collecting a set of inputs, step S120 of processing the set of inputs to determine a set of actions related to the vehicles and / or sites, and step S130 of triggering the set of actions. Additionally or alternatively, method 100 may include step S115 of aggregating any or all of the set of inputs and / or any other suitable processing. Additionally or alternatively, method 100 may include and / or interface with any or all of the methods, processes, embodiments, and / or examples for operating an ego agent, such as those described in any or all of U.S. Application No. 17 / 116,810 (filed December 9, 2020), U.S. Application No. 17 / 125,668 (filed December 17, 2020), and U.S. Application No. 17 / 127,599 (filed December 18, 2020), which are incorporated herein by reference in their entireties.

[0075] The method 100 is preferably performed in the system 200 described above, but may additionally or alternatively be performed in any other suitable system.

[0076] Method 100 functions to optimize interactions between a set of vehicles (e.g., fleet vehicles, non-fleet vehicles, etc.) and a site, including increasing the speed and / or efficiency with which vehicles can load and / or unload materials, reducing vehicle congestion at a site, managing multiple vehicles at a site, optimizing schedules and / or routes of vehicles and / or fleets of vehicles based on conditions associated with one or more sites (e.g., diverting ego agents away from congested and / or delayed sites) and / or conditions leading to one or more sites, updating and / or maintaining a dynamic representation of the site that is utilized by and / or propagated to other entities (e.g., other fleet vehicles, non-fleet vehicles, users, etc.), and / or optimizing for any other parameters or use cases.

[0077] 4.1 Method—Collecting a Set of Inputs Step S110 The method 100 includes step S210 of receiving a set of inputs, which functions to receive information for evaluating a set of one or more sites associated with the ego agent, which can be used in subsequent steps of the method to determine a schedule for a vehicle fleet, optimize the operation of one or more fleet vehicles, optimize one or more processes for the sites, and / or otherwise improve fleet and / or site management.

[0078] The set of inputs is preferably at least partially collected at and / or received from the fleet vehicles (equivalently referred to herein as ego agents), but may additionally or alternatively be received from non-fleet vehicles (e.g., via user interfaces associated with the vehicle drivers, via management subsystems associated with those vehicles, etc.). The inputs may additionally or alternatively be received from any or all of a set of devices (e.g., driver and / or safety driver mobile devices, teleoperator devices, site worker devices, devices and / or systems onboard the ego agents, etc.), a set of user interfaces (e.g., client applications running on mobile devices), third-party information sources, any number of management subsystems (e.g., site management subsystem, fleet management subsystem, etc.), and / or any other information source.

[0079] The set of inputs preferably includes sensor data (e.g., telematic information, camera data, LIDAR data, etc.) associated with the set of ego agents. In some variations, for example, at least a portion of the sensor data is in the form of telematics information (e.g., location information) received from the ego agent's Original Equipment Manufacturer (OEM) components via an on-board diagnostics (OBD) port (e.g., an OBD-ii port, a Wifi OBD port, etc.), and another portion of the sensor data is optical data (e.g., camera data, LIDAR data, etc.) or other data (e.g., location data collected by a GPS sensor) collected by the vehicle's sensor stack. Additionally or alternatively, other types of sensor data can also be received from other components.

[0080] The sensor data (e.g., telematic information) may include and / or be used to identify (e.g., derive, calculate, etc.) any or all of the following information related to any or all of the set of ego agents: location information (e.g., real-time tracking of the vehicle's location, where the vehicle has been idling for the longest time, etc.), speed, gear shift information (e.g., which gear shift the vehicle was in, which gear shift the vehicle has changed, etc.), vehicle idle time (e.g., the amount of time the vehicle idled at a loading / unloading site, the ratio of idle time to travel time, etc.), acceleration information (e.g., acceleration magnitude, acceleration change, maximum acceleration, maximum deceleration, etc.), braking information (braking magnitude, braking change, time to full stop while braking, maximum braking, etc.), fuel information (fuel consumption, fuel level, etc.), maintenance information (time since last maintenance, type of maintenance performed on the vehicle, etc.), environmental information (what objects are around the vehicle and where, how congested the site is, what vehicles are located at the site, etc.), and / or other information.

[0081] The sensor data may additionally or alternatively include a set of sensor inputs received from the vehicle's sensor stack (e.g., a non-OEM sensor stack), which may include, for example, some or all of: cameras (e.g., 360-degree coverage cameras, ultra-high resolution cameras, etc.), light detection and ranging (LiDAR) sensors, radio detection and ranging (RADAR) sensors, motion sensors (e.g., accelerometers, gyroscopes, inertial measurement units (IMUs), speedometers, etc.), position sensors (e.g., global navigation satellite system (GNSS) sensors, inertial navigation system (INS) sensors, global positioning system (GPS) sensors, any combination, etc.), ultrasonic sensors, and / or any suitable sensors.

[0082] In some variations, for example, the set of sensor inputs includes sensor streams from at least one set of camera and / or LIDAR sensors that can be used to determine one or more features associated with the site. These features may include, for example, the number of other ego agents detected at the site, the total number of loading and / or unloading locations (e.g., loading docks, loading dock spots, etc.), the total number of available (e.g., empty) loading / unloading locations, the number of personnel at the site, the number of available personnel at the site, the number and / or type of vehicles at the site (e.g., truck class, truck size, truck type, etc.), the size of the site, the location of any or all of these objects, the degree of congestion at the site (e.g., the number of available waiting areas / spots), and / or other information.

[0083] In one set of examples, sensor inputs (e.g., from a set of cameras and / or LIDAR mounted on the ego agents) are processed with a set of computer vision algorithms / models to identify how congested a site is and / or the availability associated with the site to accommodate one or more ego agents, which can then be used to determine and trigger a set of actions (e.g., as described below), which can be determined based on any or all of the number of vehicles at the site, the number of workers at the site, the number of available loading / unloading spots at the site, the number of available waiting spots at the site, and / or other information.

[0084] In another particular set of examples, sensor inputs (e.g., from a set of cameras and / or LIDAR mounted on the ego agent) are processed (e.g., with a set of computer vision algorithms / models, with other algorithms / models, etc.) and compared with a set of safety codes associated with the site, and one or more actions may be determined and triggered in S230 if a safety code is detected to have been violated and / or attempted to be violated.

[0085] The set of inputs can optionally be included and / or used to determine (e.g., derive, calculate, etc.) information related to the functionality of the ego agent (e.g., providing selective access to the vehicle, automatically or partially loading and unloading items, etc.) and / or auxiliary systems (e.g., retrofit systems) configured for the particular use case of loading and unloading items from an autonomous vehicle. In some variations, for example, the set of inputs includes information related to the ego agent's robotic system (e.g., for automatic loading and / or unloading), ego agent's door information (e.g., lock status of doors configured to automatically close and open), ego agent's lift gate information (e.g., automatic lift gate lift status, automatic lift gate lift height, etc.), inventory information, and / or any other information.

[0086] The set of inputs may optionally be included and / or used to determine (e.g., derive, calculate, etc.) information related to an operator (e.g., driver, safety driver, teleoperator, etc.) associated with the ego agent. This may include, for example, any or all of an identifier associated with the driver and / or teleoperator operating the ego agent, disengagement information associated with the driver and / or teleoperator, and / or other information. In specific examples where a driver (e.g., safety driver, manual driver, etc.) resides on the ego agent, information related to the driver (e.g., driver identification, driver behavior, driver rating, etc.) may be collected and / or exchanged via a user interface (e.g., an application programming interface (API) associated with a client application running on a device (e.g., a mobile device, a device affixed to the ego agent, etc.) associated with the driver and communicating with a remote computing system. The API may further enable the driver's client application to communicate (e.g., via a site management subsystem, directly, etc.) and / or facilitate other communications with site personnel's client applications.

[0087] The set of inputs may optionally be included and / or used to determine (e.g., derive, calculate, etc.) information related to site personnel, such as personnel facilitating loading and unloading, facilitating movement of vehicles within the site, and / or monitoring or performing other tasks. This may include information detected based on sensors of the ego agent (e.g., the number of personnel present on the site based on video streams collected by cameras of the ego agent), information received directly from the personnel (e.g., information received at a client application associated with the personnel and / or the site, as described below), and / or other inputs. In certain examples, the system may enable the ego agent to request authentication of the personnel before triggering one or more actions of S230, such as unlocking the ego agent, operating a lift gate, initiating loading and / or unloading of the ego agent, and / or other actions.

[0088] S110 may additionally or alternatively include receiving other inputs and / or performing other processing.

[0089] In a first set of variations, the data collected in S110 includes any or all of the sensor data from the set of fleet vehicles, including a first subset of sensor data collected from vehicles located at and / or proximate to the site (e.g., sensor data identifying that a vehicle has passed a geofence associated with the site, optical data visually assessing and / or characterizing the site, the vehicle's location within the site, etc.), and a second subset of sensor data collected from vehicles not located at the customer site (e.g., between sites, while traveling to a site, at a depot, etc.) (e.g., location information, derived data such as ETA, detection of a vehicle crossing a geofence defining departure from a depot or site, etc.), scheduling information from the site management subsystem (e.g., assigned locations and / or time slots of fleet vehicles), information from user interfaces (e.g., site personnel, fleet operators, non-fleet vehicles), third-party information (e.g., traffic data, weather data, etc.), and / or other information.

[0090] In a particular set of examples (e.g., shown in FIG. 10A ), a first subset of sensor data includes optical data from the field of view of a vehicle's optical sensor at the site, which can be used, for example, to construct an environmental representation of the site in S120 (see figure).

[0091] 4.2 Method - Input Set Aggregation and / or Preprocessing Step S115 Method 100 may optionally include step S115 of aggregating any or all of the set of inputs, which may function to determine an actionable analysis for performing any or all of the remaining processes of method 100. Additionally or alternatively, S220 may function to reduce latency associated with transmitting large amounts of information, reduce the computational burden of processing the set of inputs in S230, and / or perform other suitable functions.

[0092] The step of aggregating the set of inputs may include aggregating various inputs from the ego agent before further processing at S120.

[0093] The step of aggregating the set of inputs may additionally or alternatively include aggregating optical data from multiple ego agents at a site so that a representation of the environment can be constructed.

[0094] The step of aggregating the set of inputs may additionally or alternatively include aggregating data from multiple management subsystems to determine a schedule for the fleet vehicles.

[0095] Additionally or alternatively, S110 may include any other process. In a preferred set of variations, S115 includes collecting sensor data from a set of sensors onboard the ego agent, using an onboard computing system and a set of algorithms and / or models to process the sensor data to determine high-level insights (because sending raw sensor data would result in too large data packets that would result in high computational requirements and / or latency) (wherein the high-level insights are equivalently referred to herein as the first layer of lean data), combining the first layer of lean data with telematic information (e.g., OEM data) and / or other sensor data, and transmitting this combined data through the ego agent's modem / router to a remote computing system for processing in S120.

[0096] S115 may optionally additionally or alternatively include aggregating input from multiple sources (e.g., ego agent input and site input, ego agent input and remote operator input, etc.) and / or any other process.

[0097] 4.3 Method—Step S120 of Processing a Set of Inputs to Determine a Set of Actions Related to an Ego Agent and / or Site Method 100 may include step S120 of processing the set of inputs to determine a set of actions associated with the ego agent and / or the site, which actions function to determine which actions will optimize the operation of the ego agent and / or the site in which the ego agent is operating. Additionally or alternatively, S120 may function to determine actions that can be taken to increase safety associated with the site (e.g., preventing collisions due to site congestion, preventing collisions due to the presence of unidentified objects in a vehicle's blind spot, etc.), and / or perform any other function.

[0098] The set of inputs and / or aggregated set of inputs are preferably at least partially processed in a computing subsystem, and more preferably in a remote computing subsystem (e.g., a cloud computing subsystem, a remote computing subsystem in communication with and / or hosting the fleet management subsystem, etc.), which processes the inputs with a set of algorithms and / or models and / or other tools (e.g., decision trees, lookup tables, rule-based models, logic, etc.). The output of this processing can be used to determine and trigger one or more actions in any or all of the ego agent and / or its auxiliary systems, another ego agent in the fleet, the site (e.g., through a client application running on a site worker's mobile device), the remote operator's device, and / or other components and / or users.

[0099] Additionally or alternatively, S120 may be performed using on-board and / or local computing subsystems, a combination of computing subsystems, and / or any other components. The set of algorithms and / or models may include, without limitation, any or all of trained algorithms and / or models (e.g., machine learning models, deep learning models, neural networks, etc.), programmed and / or rule-based algorithms, and / or any combination. Additionally or alternatively, the set of inputs may be processed using decision trees, lookup tables, and / or any other tools.

[0100] In some variations, a first set of algorithms is used to process data from the ego agent (e.g., from S110, from S115), a second set of algorithms processes data received from an application (e.g., a client application running on a user device associated with a site worker), and insights from these sets of algorithms can be used to trigger actions and / or communicate information between the ego agent and the site (e.g., directly, via the site management subsystem, via the fleet management subsystem, etc.). Additionally or alternatively, actions can be triggered and / or communicated between vehicles, between users of the site, between the site and an operator (e.g., a fleet vehicle operator, a remote fleet vehicle operator, a non-fleet vehicle operator, etc.), between operators and vehicles, and / or between any other entities.

[0101] Additionally or alternatively, the information may be processed and / or handled in any other suitable location, using any suitable components, and / or in combination with any suitable management subsystem and / or other software platform.

[0102] Actions may include, but are not limited to, sending information (e.g., via notifications, alerts, visual displays, audio alerts, etc.) to any suitable endpoint (e.g., vehicle, user interface, user, user device, management subsystem, etc.), generating and / or updating (dynamically, in real time, etc.) an environmental representation (e.g., map) of the current state of a site or other area, updating information and / or its output (e.g., schedules, vehicle and / or driver notifications, etc.) of the management subsystem based on collected data (e.g., sensor data) and / or processed sensor data, triggering notifications and / or sending of information (e.g., triggering a notification and / or visual display in a customer interface when a vehicle passes a geofence of a current site with an updated location when moving to a new site, triggering a notification from a customer interface to a vehicle and / or fleet management subsystem in response to a customer indicating that a loading / unloading process is complete and / or that the vehicle is ready to depart, etc.), site personnel (e.g., as used herein, the vehicle's location and / or the vehicle's location (commonly referred to as the customer) and / or triggering a series of checks to be performed before the vehicle leaves the site (e.g., ensuring the load has been unloaded, ensuring the doors are secured, ensuring the vehicle has checked its surroundings, etc.); triggering an authentication process for site personnel before providing them with access to the vehicle load (e.g., automatically / autonomously unlocking the vehicle doors); triggering a set of vehicle-to-vehicle communications (e.g., when a first vehicle detects that an object may be located in the blind spot of a second vehicle, through the Fleet Management Subsystem, through a dynamically updated environmental representation, involving the current availability of docks and / or waiting areas, involving the location of potential obstacles and / or people on the site, involving weather conditions on the site, indicating that another vehicle that has not yet reached and / or arrived at the site should be re-routed to another site, etc.); managing and / or updating the control of the fleet vehicles based on sensor data and / or processed sensor data and / or dynamic environmental representations (e.g., updating the schedules of the fleet vehicles,updating the order of site locations through which the fleet vehicles will pass, updating waiting locations for fleet vehicles at busy sites, etc.), and / or any number of other actions.

[0103] An example of information exchanged between the vehicle and / or user and / or system components is shown in FIG.

[0104] In a first set of variations, S220 includes detecting that the ego agent has reached and / or passed through a geofence associated with the site, and using this detection to determine and trigger an action to notify site personnel (e.g., via a client application running on the worker's tablet device) that the ego agent has arrived at the site and / or will soon be arriving (e.g., at an estimated time). This can serve to help the personnel prepare the ego agent by reserving space on a loading dock, by assigning a site location to the arriving ego agent, by ensuring that personnel are present and / or available to facilitate loading and / or unloading of the ego agent, and / or through any other action.

[0105] Notifying the site and / or site personnel may further prompt a set of inputs to be received from the personnel, including, but not limited to, any or all of a location assignment for the ego agent (e.g., a specific parking spot, a specific loading dock, a waiting area, etc.), an estimated time until the ego agent is loaded and / or unloaded (e.g., if this time exceeds a threshold, the ego agent may be directed to another site first), and / or any other information.

[0106] In one set of examples, for example, detecting that an ego agent has passed through a geofence corresponding to a boundary of a site (e.g., based on telematic information, based on sensor information, based on aggregated telematic information and sensor information as shown in FIG. 7A , as shown in FIG. 10A , etc.) can trigger a notification to a user interface (e.g., a client application) of a user device (e.g., a tablet, laptop, smartphone, etc.) of a site worker, which can communicate information to the ego agent so that the worker can facilitate the ego agent's arrival and / or follow instructions from the worker about where to park and / or whether the site is ready for the ego agent to load and / or unload a vehicle. Additionally or alternatively, geofences can be placed at the ego agent's previous site (e.g., to communicate to a second site that the ego agent has left the first site), along the ego agent's route, and / or at any other location.

[0107] In this particular set of examples, when an ego agent arrives at a site, a worker may be notified and / or processed (e.g., by a predictive algorithm) asking if the ego agent can occupy an active loading spot at the site. The worker and / or algorithm may then reply with instructions to either occupy the loading spot or a temporary spot until a notification is sent that the ego agent is ready to receive the ego agent.

[0108] In a second set of variations, S220 includes detecting that the ego agent has breached a geofence associated with the ego agent's fixed route, which may potentially indicate any or all of the following: that the vehicle has encountered an emergency and must deviate from the fixed route (e.g., this may trigger a remote operation of the vehicle), that the safety driver aboard the ego agent may be malicious and stealing the cargo of the newly loaded vehicle (e.g., this may trigger an alarm system), and / or other scenarios.

[0109] In a third set of variations, S220 includes detecting that an ego agent has parked at the site (e.g., based on the ego agent's location information and zero speed, based on the ego agent's camera and / or LIDAR sensors, etc.) and, in response, automatically notifying site personnel of the ego agent's arrival and where the ego agent has parked (e.g., coordinates, specific parking spot number, etc.). In use cases where the ego agents are unmanned (and therefore a human cannot directly communicate this information to the site personnel) and / or where the site is congested with a large number of ego agents, this is useful to help site personnel prioritize and deal with the ego agents in a timely manner.

[0110] In a fourth set of variations, S220 includes detecting (e.g., based on sensors in the ego agent, based on input from the worker's client application, etc.) that a worker (e.g., a human worker, a robotic worker, etc.) is ready to load and / or unload the ego agent, which can trigger unlocking the ego agent, moving the ego agent's lift gate, initiating the robotic loading and / or unloading process, and / or any other action. In response, the client application can trigger an authentication process for the worker (e.g., verifying the worker's ID, confirming that the correct worker is present, etc.), and in response to this authentication, the ego agent provides access to the ego agent and / or its cargo. For example, in a particular set of example embodiments, the authentication process is performed in a backend, where the worker has the client application on a device / kiosk and prompts the worker for a password or other form of authentication. In response to the authentication request, the backend can authenticate the worker and trigger any or all of the actions described in the ego agent. Another particular set of examples may detect that a worker who has initiated the unloading process has left before finishing (e.g., to accommodate another vehicle) and, in response, trigger the closure of the lift gate and / or the closing / locking of the vehicle doors to prevent the vehicle's merchandise from being stolen or exposed to the elements (e.g., heat in the case of frozen merchandise).

[0111] In a fifth set of variations, S220 includes detecting a violation of a safety code based on sensor inputs (e.g., as described above) and / or environmental representations, which may then trigger any or all of the following actions: contacting site management, contacting safety and / or regulatory agencies associated with the site, rerouting ego agents currently en route to the site, and / or any other action.

[0112] In a sixth set of variations, S220 includes detecting potential hazards and / or obstacles associated with the site, which can trigger notifications to site managers and / or workers. As a specific example, an ego agent may detect via a camera that ice is forming on a portion of a road at the site, which could lead to a vehicle collision. This can automatically send a notification to the site to alert them to the hazard and encourage remediation.

[0113] In a seventh set of variations, S220 includes detecting, based on a first set of one or more ego agents at the site (e.g., based on sensor data collected at the ego agents, based on an updated environmental representation, etc.), that the site is congested and / or has limited availability, and in response, rerouting a second ego agent to another site so that the second ego agent can avoid the congested site and return when availability is higher.

[0114] Examples of detected and / or triggered actions are shown in Figures 7A-7B.

[0115] Additionally or alternatively, S220 may include any other suitable process.

[0116] 4.4 Method - Triggering a Set of Actions Step S130 The method 100 may include a step S130 of triggering a set of actions that function to perform one or more actions that improve and / or optimize the operation of the ego agent and / or the site.

[0117] Actions may include any or all of the following: sending notifications (e.g., as described above) and / or other information to the site (e.g., site personnel, site management, etc.), sending instructions to the ego agent's on-board computing system (e.g., prompting a re-routing of the second ego agent, prompting a location for the ego agent to occupy, prompting a trajectory determined for the ego agent, etc.), manipulating (e.g., activating any or all of the ego agent's auxiliary systems (e.g., unlocking the ego agent, locking the ego agent, activating a lift gate, initiating robotic unloading of the ego agent, initiating robotic loading of the ego agent, etc.), and / or other actions.

[0118] Actions may be triggered in any or all of the computing subsystem, the site management subsystem, the user interface, the user device, the vehicle control subsystem, the vehicle communication subsystem, and / or other components of the system and / or components independent of the system.

[0119] Additionally or alternatively, S130 may include any other suitable process.

[0120] In a first exemplary embodiment of method 100 (e.g., as shown in FIGS. 8A-8B ), various information exchanged between the vehicle and site personnel (e.g., via a user interface, via a management subsystem, etc.) can be used to optimize the loading and / or unloading of vehicle payloads.

[0121] Additionally or alternatively, communication may occur between vehicles (e.g., between fleet vehicles, between fleet and non-fleet vehicles, via management subsystems, etc.), between management subsystems, between any permutation or combination of entities, and / or may trigger any other action.

[0122] In a second exemplary embodiment of method 100 (e.g., as shown in FIG. 9 ), in addition to or alternatively to the first embodiment, data (e.g., sensor data) is collected and actions are triggered to update vehicle schedules and / or routes based on updated information collected by vehicles located at the site. As shown in FIG. 9 , for example, information collected by one or more vehicles at a first site (e.g., V_1, V_1 and V_3, etc.) and / or an associated environmental representation created based on that information may indicate that the first site is fully occupied (e.g., this may be in contrast to an understanding of the site management subsystem, which may be used to update the understanding of the site management subsystem, etc.), and therefore may be used to reroute another vehicle (e.g., en route to the first site, scheduled to depart the first site, etc.) to the second site.

[0123] In a third exemplary embodiment of method 100 (e.g., as shown in FIGS. 10A-10C ), additionally or alternatively to those described above, vehicles deployed at a site may function as a mobile sensor stack and may be used to create an environmental representation (e.g., a map) of the site, which map may then be used to identify when the site is fully occupied and / or over-occupied (e.g., as indicated by a site being full and / or a queue of vehicles waiting to access the site, recognition that is not part of the site management subsystem's understanding, identification of delays in the site management subsystem due to non-fleet vehicles and / or waiting for manual information entry by site personnel, etc.), determine a most efficient schedule and / or updated schedule (e.g., determine a most efficient schedule and / or updated schedule for other vehicles (e.g., in the fleet, outside the fleet, etc.) (e.g., detouring to other sites until the first site is no longer congested), display to the vehicles objects that may be in the blind spots of other vehicles, and The environmental information may be used for any or all of the following: a warning (e.g., detection of site worker 2 by V_3 communicated to V_2 before departing loading dock 2), instructions or warnings to site workers (e.g., a warning that they may be in a non-fleet vehicle's blind spot, an instruction that a vehicle arriving at the site should instead head to another site first, etc.), and / or other actions. In the particular example shown in FIGS. 10A-10C , for example, the environmental representation in FIG. 10C is determined at least in part based on sensor information collected by vehicles (e.g., fleet vehicles) at Site 1, and is an up-to-date representation of the site in FIG. 10A , as opposed to an older representation (e.g., in the site management subsystem) that relies on manually entered information (e.g., from site workers, from non-AVs, from non-fleet vehicles) and / or non-exhaustive information (e.g., information that excludes non-vehicle information). The representation in FIG. 10C may optionally be used to update the representation in FIG. 10B , communicate updated information to vehicles and / or site workers, and / or in other ways.

[0124] Although omitted for simplicity, preferred embodiments include all combinations and permutations of the various system components and various method processes, which may be performed in any suitable order, sequentially or simultaneously.

[0125] Embodiments of the systems and / or methods may include any combination and permutation of the various system components and various method processes, and one or more instances of the methods and / or processes described herein may be performed asynchronously (e.g., serially), simultaneously (e.g., simultaneously, in parallel, etc.), or in any other suitable order by and / or using one or more instances of the systems, elements, and / or entities described herein. The following system and / or method components and / or processes may additionally, alternatively, or otherwise be integrated with all or a portion of the systems and / or methods disclosed in the above-referenced applications, each of which is incorporated by this reference in its entirety.

[0126] In additional or alternative embodiments, the methods and / or processing modules described above are implemented in a non-transitory computer-readable medium storing computer-readable instructions. The instructions may be executed by computer-executable components integrated with the computer-readable medium and / or processing system. The computer-readable medium may include any suitable computer-readable medium, such as RAM, ROM, flash memory, EEPROM, optical devices (CD or DVD), hard drives, floppy drives, non-transitory computer-readable media, or any suitable device. The computer-executable components may include a computing system and / or processing system (e.g., including one or more co-located or distributed, remote or local processors) connected to the non-transitory computer-readable medium, such as a CPU, GPU, TPUS, microprocessor, or ASIC, although the instructions may alternatively or additionally be executed by any suitable dedicated hardware device.

[0127] Those skilled in the art will recognize from the foregoing detailed description and drawings and the appended claims that modifications and variations can be made to the preferred embodiments of the invention without departing from the scope of the invention as defined in the appended claims.

Claims

1. 1. A method for controlling a fleet of autonomous vehicles associated with a set of customer sites, comprising: receiving scheduling information from a site management subsystem associated with the set of customer sites; controlling a first autonomous vehicle of the fleet of autonomous vehicles along a route to a first site based on the scheduling information; collecting, at the first autonomous vehicle, a set of sensor information including at least camera data and LIDAR data while the first autonomous vehicle is located at the first site; determining an environmental representation of at least a portion of the first site based at least in part on the set of sensor information, the environmental representation comprising: a set of locations of a set of vehicles at the first site, the set of vehicles including a fleet of autonomous vehicles and a subset of vehicles separate from the fleet of autonomous vehicles; a set of positions of a set of humans at the first site; transmitting the environmental representation to a fleet management subsystem, the fleet management subsystem in communication with the site management subsystem and the fleet of autonomous vehicles; determining an updated set of control instructions for a second autonomous vehicle in the fleet of autonomous vehicles based on the environmental representation, and upon identifying a mismatch between the updated set of control instructions and a previous set of control instructions determined based on the scheduling information, preventing control of the second autonomous vehicle based on the previous set of control instructions; and controlling a second autonomous vehicle in accordance with the updated set of control instructions.

2. 10. The method of claim 1, wherein the updated set of control instructions automatically initiates a change in the order of the first site relative to other sites within an ordered list of customer sites planned for the second autonomous vehicle.

3. The environmental representation may be: each set of loading docks at the first site is occupied; and The method of claim 2 , further comprising indicating when the first site's waiting spots are at least partially filled.

4. The method of claim 3 , wherein the altering the order includes delaying the first site relative to at least one of the other sites.

5. 2. The method of claim 1, wherein the step of identifying the environmental representation further includes identifying the locations of a set of loading dock spots and a set of parking spots at the first site, thereby enabling the locations of the vehicles and people to be identified relative to the locations of the set of loading dock spots and the set of parking spots.

6. The method of claim 1 , further comprising transmitting at least a subset of the environment representation to the site management subsystem.

7. 7. The method of claim 6, further comprising updating the scheduling information in the site management subsystem, wherein updating the scheduling information triggers transmission of an updated schedule to non-fleet vehicles associated with the first site.

8. The method of claim 1 , wherein the subset of vehicles includes non-autonomous vehicles.

9. 10. The method of claim 1, further comprising, for a third autonomous vehicle located at the first site concurrently with the first autonomous vehicle, using the set of sensor information to update a second set of control instructions associated with the third autonomous vehicle.

10. 10. The method of claim 9, further comprising detecting that each of the first autonomous vehicle and the third autonomous vehicle is located at the first site based on a geofence defining the first site.

11. 10. The method of claim 9, wherein the second set of updated control instructions is configured to prevent the third autonomous vehicle from exiting a loading dock at the first site, and the sensor information indicates an object is present in a blind spot of a sensor of the third autonomous vehicle.

12. 12. The method of claim 11, wherein controlling the third autonomous vehicle according to a second set of updated control instructions is triggered in response to receiving an instruction from a user interface associated with a site worker that the third autonomous vehicle is free to leave the loading dock in response to completion of at least one of a loading process and an unloading process.

13. The method of claim 9 , wherein the environmental representation identifies a number of vehicles waiting to enter the first site.

14. 14. The method of claim 13, wherein the second set of updated control instructions is operable to reroute the third vehicle to a different area within the first site than originally planned.

15. The method of claim 1 , further comprising, after the step of identifying the environmental representation, sending a notification operable to alter a trajectory of a non-fleet vehicle.

16. The method of claim 15 , wherein the notification is sent to an application running on a user device associated with a human driver of the non-fleet vehicle.

17. 1. A system for controlling a fleet of autonomous vehicles associated with a set of customer sites, comprising: a Fleet Management Subsystem in communication with the fleet of autonomous vehicles and a Site Management Subsystem, the Site Management Subsystem operable to provide scheduling information for a set of vehicles comprising the fleet of autonomous vehicles at at least a first site of the set of customer sites; a set of sensor subsystems, each sensor subsystem of the set being onboard an autonomous vehicle of the fleet; a set of processing subsystems associated with the fleet of autonomous vehicles and in communication with the fleet management subsystem, the set of processing subsystems including a first processing subsystem associated with a first autonomous vehicle of the fleet of autonomous vehicles, the first processing subsystem in communication with a first sensor subsystem of the set of sensor subsystems, the first processing subsystem comprising: providing a first set of control instructions operable to control traveling of the first autonomous vehicle along a route to the first site based on the scheduling information; collecting sensor information including at least one of camera data and LIDAR data from the first sensor subsystem while the first autonomous vehicle is located at the first site; determining an environmental representation of at least a portion of the first site based at least in part on the set of sensor information, wherein the environmental representation comprises: a set of locations of a subset of the set of vehicles located at the first site, the subset of vehicles including vehicles separate from the fleet of autonomous vehicles; and a set of positions for a set of site personnel at the first site; transmitting the environmental representation to the fleet management subsystem; determining an updated set of control instructions for a second autonomous vehicle in the fleet of autonomous vehicles based on the environmental representation, wherein upon identifying a mismatch between the updated set of control instructions and a previous set of control instructions determined based on the scheduling information, preventing control of the second autonomous vehicle based on the previous set of control instructions; configured to initiate control of a second autonomous vehicle in accordance with the updated set of control instructions; A system comprising a set of user interfaces associated with a set of site personnel located at the first site.

18. 20. The system of claim 17, wherein sensors of the first sensor subsystem are fixed relative to the first autonomous vehicle and movable relative to the first site.

19. 20. The system of claim 17, wherein the first processing subsystem is further configured to receive input from the set of site personnel via the set of user interfaces.

20. The system of claim 19 , wherein the updated set of control instructions is further determined based on the input.

Images