587results about "Autonomous decision making process" patented technology

A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and / or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and / or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and / or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

Systems and techniques are described for implementing autonomous control of earth-moving construction and / or mining vehicles, including to automatically determine and control autonomous movement of part or all of one or more such vehicles (e.g., a vehicle's arm(s) and / or attachment(s), such as a digging bucket, claw, hammer, blade, etc.) to move materials or perform other actions in a manner that is based at least in part on data from simulated operation of the vehicle(s). For example, the systems / techniques may include using data from simulated operation of the earth-moving vehicle(s) in various manners, such as for use in training one or more machine learning models that are used in implementing the autonomous operations, determining optimal or otherwise preferred hardware component configurations to use, determining optimal or otherwise preferred implementation plans to use for one or more tasks and / or multi-task jobs, enabling user what-if experimentation activities, etc.

Disclosed are systems and methods that can be used for adjusting the field of view of one or more sensors of an autonomous vehicle. In the systems and methods, each sensor of the one or more sensors is configured to operate in accordance with a field of view volume up to a maximum field of view volume. The systems and methods include determining an operating environment of an autonomous vehicle. The systems and methods also include based on the determined operating environment of the autonomous vehicle, adjusting a field of view volume of at least one sensor of the one or more sensors from a first field of view volume to an adjusted field of view volume different from the first field of view volume. Additionally, the systems and methods include controlling the autonomous vehicle to operate using the at least one sensor having the adjusted field of view volume.

A method for generating intermediate waypoints for a navigation system of a robot includes receiving a navigation route. The navigation route includes a series of high-level waypoints that begin at a starting location and end at a destination location and is based on high-level navigation data. The high-level navigation data is representative of locations of static obstacles in an area the robot is to navigate. The method also includes receiving image data of an environment about the robot from an image sensor and generating at least one intermediate waypoint based on the image data. The method also includes adding the at least one intermediate waypoint to the series of high-level waypoints of the navigation route and navigating the robot from the starting location along the series of high-level waypoints and the at least one intermediate waypoint toward the destination location.

A system for mapping road segment free spaces for use in autonomous vehicle navigation. The system includes at least one processor programmed to: receive from a first vehicle one or more location identifiers associated with a lateral region of free space adjacent to a road segment; update an autonomous vehicle road navigation model for the road segment to include a mapped representation of the lateral region of free space based on the received one or more location identifiers; and distribute the updated autonomous vehicle road navigation model to a plurality of autonomous vehicles.

A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and / or other sensor data may be stitched together, stacked, and / or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and / or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and / or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.

Evaluating the performance of an autonomous vehicle includes determining a plurality of simulation scenarios, determining a set of features correlated to a performance metric of interest for the autonomous vehicle, executing a simulation for each simulation scenario in the plurality of simulation scenarios, determining, by a machine learning model, a weight for each simulation scenario in the set of simulation scenarios subject to a constraint that a simulated expected value of each feature in the plurality of simulation scenarios falls within a threshold range of an observed expected value of each feature in an operation design domain of interest of the autonomous vehicle, and estimating an expected value of the performance metric of interest of the autonomous vehicle based on the determined weight and the execution of the simulation for each simulation scenario in the plurality of simulation scenarios.

Navigation instructions are determined using visual data or other sensory information. Individual frames can be extracted from video data, captured from passes through an environment, to generate a sequence of image frames. The frames are processed using a feature extractor to generate frame-specific feature vectors. Image triplets are generated, including a representative image frame (or corresponding feature vector), a similar image frame adjacent in the sequence, and a disparate image frame that is separated by a number of frames in the sequence. The embedding network is trained using the triplets. Image data for a current position and a target destination can then be provided as input to the trained embedding model, which outputs a navigation vector indicating a direction and distance over which the vehicle is to be navigated in the physical environment.

A power equipment machine with operator selectable autonomous mode and manual mode is provided. By way of example, the power equipment device can provide user-assisted semi-autonomous steering along user-defined paths of a geographic area. Manual steering controls and autonomous guidance controls can be positioned on one or more movable armrests to enhance comfort and minimize operator fatigue. The manual steering controls can include a jog wheel and encoder mechanism located on a first armrest, eliminating conventional mechanical shaft and wheel steering devices and dual lap bar steering devices. Guidance and computer settings can be accessed through a touchscreen display mounted in front of an operator position on the power equipment device.

Systems, methods, and apparatus related to sensor data processing for a vehicle to improve operation when sunlight or other bright light enters a sensor of the vehicle. In one approach, adjustable filtering is configured for a sensor of a vehicle. In one example, an optical filter is positioned on the path of light that reaches an image sensor of a camera. For example, the filtering improves ability to stay in adaptive cruise control when driving into direct sunlight at sunset. The optical filters can have controllable properties such as polarization. In one example, a controller of the vehicle is configured to automatically adjust the properties of the optical filter to improve image quality to improve object recognition. In another example, a camera is configured with composite vision that uses sensors in different radiation spectrums (e.g., visible light, and infrared light). The composite vision can provide enhanced vision capability for an autonomous vehicle that is driving in the direction of the sun.

Various embodiments enable a robot, or other autonomous or semi-autonomous device or system, to receive data involving the performance of a task in the physical world. The data can be provided as input to a perception network to infer a set of percepts about the task, which can correspond to relationships between objects observed during the performance. The percepts can be provided as input to a plan generation network, which can infer a set of actions as part of a plan. Each action can correspond to one of the observed relationships. The plan can be reviewed and any corrections made, either manually or through another demonstration of the task. Once the plan is verified as correct, the plan (and any related data) can be provided as input to an execution network that can infer instructions to cause the robot, and / or another robot, to perform the task.

Sensors measure information about actors or other objects near an object. Sensor data is used to determine a sequence of possible actions for the maneuverable object to achieve a determined goal. For each possible action to be considered, one or more probable reactions of the nearby actors or objects are determined. This can take the form of a decision tree in some embodiments, with alternative levels of nodes corresponding to possible actions of the present object and probable reactive actions of one or more other vehicles or actors. Machine learning can be used to determine the probabilities, as well as to project out the options along the paths of the decision tree including the sequences. A value function is used to generate a value for each considered sequence, or path, and a path having a highest value is selected for use in determining how to navigate the object.

A computing system can include one or more processors and one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the computing system to perform operations including obtaining model structure data indicative of a plurality of parameters of a machine-learned model; determining a codebook comprising a plurality of centroids, the plurality of centroids having a respective index of a plurality of indices indicative of an ordering of the codebook; determining a plurality of codes respective to the plurality of parameters, the plurality of codes respectively comprising a code index of the plurality of indices corresponding to a closest centroid of the plurality of centroids to a respective parameter of the plurality of parameters; and providing encoded data as an encoded representation of the plurality of parameters of the machine-learned model, the encoded data comprising the codebook and the plurality of codes.

Vehicles may have autonomous capabilities that permit the vehicle to navigate according to varying levels of participation by a human operator. A human operator may be associated with a driving capability profile and the environment surrounding a vehicle may be associated with an environmental risk profile. A vehicle, or another party in communication with a vehicle, may determine whether, based on the human operator's driving capability profile and the environmental risk profile, it is likely that the vehicle will be navigated more safely if there is a change to the vehicle's autonomous capabilities and a corresponding change to the human operator's driving responsibilities. If it is determined that a change in autonomous vehicle capability is likely to increase safe navigation of the vehicle, an insurer may offer an insurance premium cost reduction if the human operator agrees to the proposed change in autonomous capabilities.

The present disclosure relates to driving decision-making methods, apparatuses, and chips. One example method includes constructing a Monte Carlo tree based on a current driving environment state, where the Monte Carlo tree includes a root node and N−1 non-root nodes, each node represents one driving environment state, and a driving environment state represented by any non-root node is predicted by a stochastic model of driving environments. Based on at least a value function of each node in the Monte Carlo tree, a node sequence that starts from the root node and ends at a leaf node is determined. A driving action sequence is determined based on a driving action corresponding to each node in the node sequence.

A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and / or other sensor data may be stitched together, stacked, and / or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and / or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and / or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.

Evaluating a simulation of an autonomous vehicle may be performed by using one or more processors to receive log data collected for a given area, generate environment data for the given area using the log data, run the set of simulations using an autonomous vehicle software, extract one or more metrics from the set of simulations, and evaluate the set of simulations using the one or more metrics. The set of simulations includes one or more of a selection simulation comprising a selection of a location related to the particular maneuver in the given area, a decision process simulation comprising a decision process playthrough for the particular maneuver in the given area, a maneuver simulation comprising a maneuver playthrough of the particular maneuver in the given area, and a replay simulation comprising a replay of the particular maneuver in a run from the log data.

In particular embodiments, a computing system may determine a predicted amount of ride requests for a plurality of collectively-managed vehicles and determine an availability of the collectively-managed vehicles to satisfy the predicted amount of ride requests. Subsequent to determining that the availability fails to satisfy one or more predetermined criteria for servicing the predicted amount of ride requests, the system may determine status information associated with the collectively-managed vehicles and determine, based on at least the status information, one or more minimum services for servicing one or more vehicles among the plurality of collectively-managed vehicles at one or more service centers such that the availability satisfies the one or more predetermined criteria. The system may instruct the one or more vehicles that are to receive the one or more minimum services to travel to the one or more service centers to be serviced.

A mud treatment system that can be used during emergencies is disclosed. The system includes an autonomous vehicle carrying a mud treatment device. In response to information about a mud-related disaster at a specific location, the system will instruct the vehicle to travel to the location, enter a building, and dispense one or more applications of mud softener in the interior space of the building. The vehicle can navigate from room to room and determine whether moisture content levels are below a specified threshold. In response, the vehicle can direct its mud or dirt softening efforts to these specific zones.

Techniques are provided which may be implemented using various methods and / or apparatuses in a vehicle to utilize vehicle external sensor data, vehicle internal sensor data, vehicle capabilities and external CV2X input to determine, send, receive and utilize data elements to determine inter-vehicle spacing, intersection priority, lane change behavior and spacing and other autonomous vehicle behavior.

A navigation path can be determined for an object using one or more neural networks. In various embodiments, image data is obtained that is representative of an environment in which the object is to be navigated. Relevant features are identified from the image, and a curve fit to those features. Loss values for the potential paths are scaled based at least in part upon the distance of those features in the real world. This can include, in at least some embodiments, performing the scaling as a function of the curvature of the curve fit to the features. Temporal smoothing can be performed with respect to prior path predictions in order to prevent sudden changes in the predicted path. The paths are analyzed to select a path with a highest confidence value that also at least satisfies a minimum confidence criterion. The path can be converted into three-dimensional navigation information.

In one embodiment, a transportation system may receive a ride request of a requestor from a requestor computing device. In response to receiving the ride request, the transportation system may present an option to include an autonomous vehicle among available vehicle types for fulfilling the ride request. The transportation system may determine whether the option to include the autonomous vehicle was selected and match the ride request to a vehicle based on determining whether the option to include the autonomous vehicle was selected.

An autonomous driving system includes a sensing module, an intervention module, and a vehicle control module. The sensing module detects an upcoming situation to be experienced by a vehicle. The intervention module: determines that there is a low level of confidence in a driving decision to be made for the upcoming situation; implements an intervention of an autonomous driving mode and indicates information related to the upcoming situation to a driver of the vehicle via an interface; requests assistance from the driver based on the upcoming situation by (i) indicating available options for the situation via the interface or (ii) requesting information from the driver to aid in making a driving decision; and determines whether input has been received from the driver via the interface, the input indicating (i) a selected one of the available options or (ii) the requested information. The vehicle control module autonomously drives the vehicle based on whether the input has been received.

Systems and methods are disclosed for determining the efficacy of an autonomous vehicle driving system in exhibiting safe driving behavior during test track scenarios. Testing parameters eliciting autonomous vehicle driving system behavior may be chosen for the test track scenario. The autonomous vehicle driving system may be tested in the test track and autonomous vehicle driving system behavior may be observed in response to the presentations of testing parameters. A safe driving score may be calculated for the autonomous vehicle driving system based on measured autonomous vehicle performance and operational data generated responsive to the presentation of testing parameters in the test track. A benchmark insurance premium may be calculated for an autonomous vehicle driving system based on the calculated safe driving score.

An autonomous vehicle control system includes one or more sensors configured for coupling with an agricultural vehicle, the one or more sensors configured to determine kinematics of the agricultural vehicle relative to a crop row. The system includes a guidance control module configured to coordinate steering of one or more steering mechanisms of the agricultural vehicle. The guidance control module includes a sensor input configured to receive kinematics of the agricultural vehicle, a vehicle kinematics comparator configured to determine one or more error values using the received vehicle kinematics, a crop curvature generator configured to determine crop row curvature using the one or more error values, and a steering interface configured to provide instructions to a vehicle steering controller to guide the agricultural vehicle using the crop row curvature.

An intelligent vehicle charging system for charging a fleet of autonomous vehicles throughout a network of charging stations dispersed throughout a geographic area. The intelligent vehicle charging system includes a remote control system that is in operative communication with each of the autonomous vehicles in the fleet and each of the charging stations in the network. When an autonomous vehicle is in need of a power charge, or as directed by the remote control system, the remote control system will identify an available charging station, guide the autonomous vehicle to the charging station, verify that the autonomous vehicle has arrived at the charging station, initiate the power charging process, account and bill appropriate fees for the charging process, and log all associated activity. The remote control system is also capable of remotely and instantaneously terminating the power charging process to dynamically return a vehicle back to service.

Techniques are directed to controlling a mower. Such techniques involve initiating an automated mowing task that directs the mower to operate in a geographic area defined by a virtual boundary. Such techniques further involve electronically detecting presence of a device within the geographic area defined by the virtual boundary. Such techniques further involve suspending the automated mowing task in response to detecting the presence of the device within the geographic area defined by the virtual boundary.

Aspects of the disclosure relate to generating scouting objectives in order to update map information used to control a fleet of vehicles in an autonomous driving mode. For instance, a notification from a vehicle of the fleet identifying a feature and a location of the feature may be received. A first bound for a scouting area may be identified based on the location of the feature. A second bound for the scouting area may be identified based on a lane closest to the feature. A scouting objective may be generated for the feature based on the first bound and the second bound.