Center-Based Detection and Tracking

Abstract

Techniques for detecting and tracking objects in an environment are described herein. For example, the techniques may include detecting a center point of a block of pixels associated with the object. A unimodal (e.g., Gaussian) confidence value may be determined for a group of pixels associated with the object. A center point of a proposed detection box may be determined based on the Gaussian confidence values ​​of the pixels, and an output detection box may be determined using filtering and / or suppression techniques. Furthermore, a machine learning model may be trained by determining a focal loss based on parameters of the center pixel of the detection box and the unimodal confidence value, which may then be back-propagated to other pixels of the detection.

Description

[Technical field]

[0001] The present invention relates to center-based detection and tracking. [Background technology]

[0002] This application claims priority to U.S. Application No. 17 / 537,920, filed November 30, 2021, and entitled "Center-Based Detection and Tracking," which is incorporated by reference herein in its entirety.

[0003] Various systems and techniques are utilized by autonomous vehicles to perform detection and tracking of objects, such as vehicles, pedestrians, and bicycles, within an environment. Detection and tracking techniques may use sensors to capture data within the environment. The captured sensor data can be processed for use in detecting objects within the environment and generating bounding boxes that represent such objects. These bounding boxes can then be used to navigate the autonomous vehicle within the environment. Generating bounding boxes that accurately represent objects present within an environment based on sensor data can sometimes present challenges. [Brief description of the drawings]

[0004] The detailed description will be set forth with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears, and the use of the same reference number in different drawings indicates similar or identical components or features. [Figure 1] FIG. 1 is a pictorial flow diagram of an exemplary process for generating detection boxes of objects in an environment using center-based detection, in accordance with an example of this disclosure. [Figure 2A] FIG. 2A is a schematic representation of data and objects associated with a process for generating a detection box of an object in an environment using center-based detection, according to an example of the present disclosure. [Figure 2B]FIG. 2B is a schematic representation of data and objects associated with a process for generating detection boxes of objects in an environment using center-based detection, according to an example of the present disclosure. [Figure 2C] FIG. 2C is a schematic representation of data and objects associated with a process for generating detection boxes of objects in an environment using center-based detection, according to an example of the present disclosure. [Figure 2D] FIG. 2D is a schematic representation of data and objects associated with a process for generating a detection box of an object in an environment using center-based detection, according to an example of the present disclosure. [Figure 2E] FIG. 2E is a schematic representation of data and objects associated with a process for generating detection boxes of objects in an environment using center-based detection, according to an example of the present disclosure. [Figure 2F] FIG. 2F is a schematic representation of data and objects associated with a process for generating detection boxes of objects in an environment using center-based detection, according to an example of the present disclosure. [Figure 2G] FIG. 2G is a schematic representation of data and objects associated with a process for generating detection boxes of objects in an environment using center-based detection, according to an example of the present disclosure. [Diagram 3] FIG. 3 is a schematic diagram of an exemplary curve representing unimodal confidence values, according to an example of the present disclosure. [Figure 4] FIG. 4 is a pictorial flow diagram of an exemplary process for generating and inputting detection box pixels having one or more center pixel parameters in accordance with an example of this disclosure. [Diagram 5] FIG. 5 illustrates a block diagram of an exemplary system for implementing the methods described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Techniques for detecting and tracking objects in an environment are described herein. For example, the techniques may include using a machine learning model (e.g., in inference mode) to determine candidate bounding boxes of an object by determining points or pixels of a group of pixels associated with the object that are associated with a unimodal confidence value that meets or exceeds a threshold (e.g., based on a unimodal distribution of the associated confidence values). The bounding boxes of the determined points or pixels may be decoded based on object data associated with such points or pixels, and a detection box of the object may be determined based on those bounding boxes (e.g., using non-maximal suppression). In training mode, the machine learning model may be trained by propagating one or more object data parameters of the determined object's center point or pixel across the remaining points or pixels in the group of pixels associated with the object. A loss may then be determined for each point or pixel in the group of pixels and back-propagated as the model is trained.

[0006] In certain techniques, the center points may be determined using a binary mask on the pixels to determine pixels that have sufficient confidence values ​​(e.g., non-zero confidence values ​​or confidence values ​​above a minimum threshold) to be associated with the object. In such techniques, using a binary mask in training mode gives equal weight to the object data values ​​of all points or pixels potentially associated with the object, even if those points or pixels further from the object's center are less likely to represent accurate object detection information. Points or pixels further from the object's center are less likely to represent accurate object detection information because they may be more susceptible to noise and therefore have less reliable object information. The use of a binary mask in inference mode is resource intensive, as it requires the determination of more candidate bounding boxes in the process of determining the object's detection box than when a unimodal mask is used as described herein. Because the unimodal confidence values ​​and masking techniques described herein may result in more accurate determination of candidate center points or pixels, the techniques described herein allow for the processing of fewer candidate bounding boxes (e.g., the use of non-maximal suppression) to determine the object's detection box, thereby increasing system speed and efficiency. The unimodal confidence values ​​and associated object data determined in accordance with the systems and techniques described herein may also enable other systems and processes (e.g., prediction, planning, etc.) to gain better insight into the environment, thereby generating more accurate data for controlling the vehicle.

[0007] In an example, a system and technique for improved detection of center points of a block of pixels associated with an object is disclosed. A machine learning model may determine that a group of pixels is associated with an object (e.g., as represented in sensor data). The model may determine unimodal confidence values ​​(e.g., Gaussian confidence values) for individual pixels within the group of pixels, and then generate a curve that represents the unimodal distribution of confidence values ​​for the group of pixels. Using this curve, pixels having confidence values ​​above a threshold may be determined as proposed detection box center points. Candidate detection boxes associated with these proposed center points are generated (e.g., decoded based on pixel data associated with the proposed center points), and the model may then determine an output detection box based on the candidate detection boxes (e.g., using one or more various techniques, such as non-maximum suppression (NMS)).

[0008] The machine learning model may be trained by propagating one or more parameters (e.g., speed, center coordinate and / or offset value, length, width, direction, stationary, etc.) of the determined center pixel of the output detection box to other pixels of the output detection box to ensure consistent parameters throughout the box. A loss may then be determined for other pixels of the output detection box based on the unimodal confidence value of the associated pixel and back-propagated as the model is trained.

[0009] The systems and techniques described herein may leverage sensor data and pixel data to enable vehicles, such as autonomous vehicles, to more accurately identify and locate objects in an environment and therefore navigate the environment more safely by avoiding such objects. The systems and techniques described herein may utilize unimodal functions to more accurately determine potential center points of output detection boxes determined by machine learning models for objects in an environment, and thus facilitate the generation of more accurate detection boxes for the objects. The systems and techniques described herein may also increase the efficiency of output detection box determination operations, since fewer candidate detection boxes may be determined (e.g., due to fewer proposed center points being determined) using the unimodal operations described herein, thereby reducing the processing required to determine an output detection box from among the candidate detection boxes.

[0010] The systems and techniques described herein may also increase the accuracy of machine learning models by training such models to determine more accurate object data by inputting one or more parameters, such as a speed parameter, from a central pixel determined according to the systems and techniques described herein to other pixels in a detection box and backpropagating a loss associated with such pixel based on the pixel's confidence value. This may improve the accuracy of object detection and improve safety outcomes, as the central pixel of a detection box determined as described herein may be a more accurate representation of a detected object than a central pixel determined using conventional techniques. Thus, the examples described herein may result in increased certainty of object detection, thereby enabling an autonomous vehicle to generate more accurate and / or safer trajectories and traverse an environment. For example, the techniques described herein may be faster and / or more robust than conventional techniques, as they may increase the reliability of determining dimensions and other physical parameters of objects in an environment and reduce the likelihood of failing to or misidentifying an object. That is, the techniques described herein provide technical improvements over existing object detection, localization, tracking, and / or navigation techniques. In addition to the increased accuracy with which objects can be determined and such objects precisely located using the upscaled data structures, the systems and techniques described herein can provide a smoother ride and improve safety outcomes, for example, by more accurately providing a safe passage to an intended destination.

[0011] The systems and techniques described herein can be implemented in a number of ways. Examples are presented below with reference to the figures. Although described in the context of an autonomous vehicle, the techniques described herein can be applied to a variety of systems (e.g., sensor systems, or robotic platforms) and are not limited to autonomous vehicles. For example, the techniques described herein can be applied to semi-autonomous and / or manually operated vehicles. In another example, the techniques may be utilized in an aviation or nautical context, or in any system that includes objects or entities having dimensions and / or other physical parameters that may be unknown to the system. Furthermore, while described in the context of originating from a particular type of sensor data and processing with a particular type of component, the data and data structures described herein can include any two-dimensional, three-dimensional, or multi-dimensional data, such as other types of image data (e.g., stereo cameras, time-of-flight data, etc.), radar data, sonar data, etc. Furthermore, the techniques described herein can be used with real data (e.g., captured with a sensor), simulated data (e.g., generated by a simulator), or any combination of the two. Additionally, although specific examples are described herein using Gaussian confidence values ​​and distributions, other techniques using other types of unimodal confidence values ​​and distributions are contemplated.

[0012] FIG. 1 is a pictorial flow diagram illustrating an example process 100 for generating detection boxes for use in object detection and tracking. In an example, one or more operations of the process 100 may be implemented by a vehicle computing system, such as using one or more of the components and systems shown in FIG. 5 and described below. For example, the one or more components and systems may include components and systems associated with the deep tracking network 532 shown in FIG. 5. In an example, one or more operations of the process 100 may be performed by a remote system in communication with the vehicle, such as the deep tracking network 550 shown in FIG. 5. In yet other examples, one or more operations of the process 100 may be performed by a combination of a remote system and a vehicle computing system. However, the process 100 is not limited to being performed by such components and systems, and the components and systems of FIG. 5 are not limited to performing the process 100.

[0013] One or more operations of process 100, as well as any other operations, systems, and techniques described herein, may be aimed at detecting any type and number of objects, including one or more vehicles, pedestrians, and bicycles. For example, process 100 and / or any other process described herein may implement one or more object detection networks (e.g., may be arranged into a deep tracking network). Individual networks of such object detection networks may be implemented (e.g., in parallel) for a particular object. For example, a tracking network may implement a vehicle detection network, a pedestrian detection network, and / or a bicycle detection network in parallel (or otherwise). In another example, a tracking network may implement one or more coarse-grained vehicle detection networks, one or more fine-grained vehicle detection networks, one or more coarse-grained pedestrian and / or bicycle detection networks, and one or more fine-grained pedestrian and / or bicycle detection networks in parallel (or otherwise). Any other suitable detection network and / or detection system configuration may incorporate one or more of the systems and techniques described herein.

[0014] In operation 102, one or more sensors configured on the autonomous vehicle may detect data in an environment using the one or more sensors and generate sensor data based on such detection. In various examples, the sensors may include one or more lidar sensors, camera sensors, radar sensors, sonar sensors, audio sensors, ultrasonic transducers, and / or any other suitable sensors. This data may be collected (or received), stored, and / or processed by a vehicle computing device to perform one or more operations as described herein. In an example, the sensor data collected in operation 102 may be provided to a deep tracking network (DTN) for processing, as described in more detail below.

[0015] Example 104 illustrates a top view of an environment including a vehicle 106 that may be configured with sensors to detect objects in the environment, such as object 108. A vehicle computing device of vehicle 106 may be configured to receive sensor data from sensors of vehicle 106 and use the sensor data as described herein to generate a detection box for object 108, for example.

[0016] In operation 110, a vehicle computing system (e.g., a DTN configured therein executing a machine learning model) may receive pixel data for pixels based on the sensor data. For example, the vehicle computing device may determine pixel data for pixels associated with objects in the environment based on the sensor data collected in operation 102 (e.g., generated and / or detected by one or more sensor modalities configured on the vehicle) and provide the data to a DTN configured on or interoperating with the vehicle computing device. In examples, other data may be used and / or processed in this operation and elsewhere in process 100. For example, one or more discretized values ​​of any form may be received in operation 110.

[0017] Example 112 illustrates pixel data and / or discretized values ​​associated with region 113 and object 108 of example 104. The pixels or values ​​in example data group 114 may be pixels or values ​​determined to be associated with the object based on sensor data captured from the environment of example 104 and for which the vehicle computing device has generated the pixel data. In the example, the vehicle computing device may determine a unimodal (e.g., Gaussian) confidence value for each pixel of data group 114, as well as a center offset value and / or other parameters such as coordinates, length, width, orientation, stationarity, yaw, and / or velocity.

[0018] In operation 116, the vehicle computing system (e.g., in a DTN running the machine learning model) may determine a unimodal confidence value (e.g., a Gaussian confidence value) for each pixel associated with the object. The vehicle computing device (e.g., a DTN) may also, or instead, generate other object and / or pixel data for each pixel associated with the object. This object and / or pixel data may include center offset coordinates and / or values, length, width, direction, stationarity, yaw, and / or velocity, and the determined unimodal confidence value. The vehicle computing device may represent this object and / or pixel data as a multi-channel data structure, with particular layers of the data structure corresponding to particular types of data, as described in more detail herein. In an example, this multi-channel data structure may be generated as multi-channel output data by a DTN (e.g., running the machine learning model) operating at or in conjunction with the vehicle computing system, and may include a channel or layer that includes the determined unimodal confidence value for each pixel. Further, in operation 116, the vehicle computing system may determine a representation of a unimodal distribution (e.g., a Gaussian curve) of pixel confidence values ​​associated with individual pixels associated with the object. For example, the vehicle computing system may use a two-dimensional Gaussian function to generate a Gaussian curve that represents the confidence values ​​associated with the pixels. In an example, the result of the unimodal distribution representation of confidence values ​​may indicate pixels having relatively high and low confidence values.

[0019] Example 118 shows a top view of a unimodal representation 120 that represents unimodal confidence values ​​determined for data group 114. In this example, thicker lines indicate higher confidence values ​​and thinner lines represent lower confidence values. As shown in this example, the unimodal confidence values ​​of pixels closer to the center of a detected object may be relatively higher than pixels farther from the center of the object.

[0020] In operation 122, the vehicle computing system (e.g., executing a machine learning model) may determine candidate detection box center points or pixels for the object based on the unimodal distribution generated in operation 116. For example, the vehicle computing system may determine that those along the unimodal distribution with unimodal confidence values ​​above a certain threshold may be better candidates for candidate detection box center points than those along the unimodal distribution associated with points below the threshold. The threshold may be a machine learning threshold generated using a machine learning (ML) model trained to output a machine learning threshold based on initializing the ML model with one or more parameters and an initial pre-defined threshold, and then process a training set of pixel data using variations of the initial pre-defined threshold until a best quality machine learning threshold is found. In various examples, the vehicle computing system may determine an averaged or aggregated unimodal confidence value for more than one pixel and may determine a unimodal distribution of confidence values ​​based on the aggregated or averaged unimodal confidence value. The vehicle computing system may then determine those pixels that may be candidates for the proposed detection box center points based on the distribution based on the aggregated or averaged unimodal confidence values.

[0021] Example 124 illustrates a side view of unimodal representation 120 representing a unimodal distribution determined based on unimodal confidence values ​​determined for data group 114 as input of operation 116. This side view of unimodal representation 120 may represent a two-dimensional cross-section of unimodal representation 120 at its highest point (e.g., peak). Threshold line 126 may represent a threshold that may be used by the vehicle computing system to determine which pixels to use in generating a candidate detection box. Those pixels associated with the portion of unimodal distribution 120 above threshold line 126 may be determined to be candidates for the center point of the candidate detection box, while those pixels associated with the portion of unimodal distribution 120 below threshold line 126 may be determined to not be candidates for the center point of the candidate detection box.

[0022] At operation 128, the vehicle computing system may generate candidate detection boxes based on the pixels determined in operation 122. For example, the vehicle computing system may generate candidate detection boxes for the pixels determined in operation 122 by decoding one or more of various parameters associated with such pixels and / or represented in the object and / or pixel data structures. Parameters that may be associated with the pixels and / or used to generate the detection boxes may include, but are not limited to, center coordinates and / or offset values, length, width, orientation, stationarity, yaw, and / or velocity.

[0023] Example 130 shows a top view of a candidate detection box 132 that may be generated based on proposed center pixels 134. In this example, each proposed center pixel 134 may have a unimodal confidence value that meets or exceeds a threshold configured at the vehicle computing device. The vehicle computing device may then use one or more parameters associated with the proposed center pixels 134 to generate the candidate detection box 132.

[0024] In operation 136, the vehicle computing system may determine an output detection box. In an example, the vehicle computing system may perform a non-maximal suppression operation using the candidate detection boxes as inputs to determine an output detection box from among the candidate detection boxes to use for navigation and / or other functions of the vehicle. Alternatively, or in addition, the vehicle computing system may suppress candidate detection boxes that meet one or more criteria (e.g., have a central pixel with a unimodal confidence value below a threshold) and / or filter the candidate detection boxes using one or more techniques until an output detection box is determined. This output detection box may be the output of the DTN and / or machine learning model executed by the DTN to determine the output detection box.

[0025] In operation 138, the vehicle computing system may generate or otherwise provide object detection based on the output detection box determined in operation 136, and / or may navigate the vehicle based on the output detection box, and / or perform other functions.

[0026] Example 140 again illustrates a top view of an exemplary environment including vehicle 106 and object 108. As shown in this example, the vehicle computing system has determined an output detection box 142 from among candidate detection boxes 132 for use in representing object 108. Output detection box 142 may be generated based on center pixels 144 that were among proposed center pixels 134.

[0027] In some examples, the confidence values ​​determined as described herein may be used by downstream processes or received by other components along with the object data. For example, a prediction system may receive the object data and the confidence values ​​associated with the data to make predictions on the data. In some examples, a planning system may receive the object data and the associated confidence values ​​and plan a trajectory based on the object data and / or the confidence values. In some examples, the object data and the confidence values ​​may be used to populate an occupancy grid that may weight the occupancy status of individual regions of an occupancy grid or otherwise be determined based on the confidence values, as described herein. Additional examples are contemplated and are not limited in scope to the explicit examples discussed herein.

[0028] 2A-G show schematic representations of various data captured and / or generated by a process for generating detection boxes of objects in an environment using center-based detection, e.g., as performed by a machine learning model running in inference mode. FIG. 2A shows a top view of an example area 210 that includes objects 211 and 212. Area 210 may be a section or area in an environment in which an autonomous vehicle is operating. To safely operate and navigate within the environment, the autonomous vehicle may be configured to detect objects such as objects 211 and 212 in the environment using sensors and one or more other systems, such as a vehicle computing system. As part of this object detection process, the vehicle computing system may capture and / or generate sensor, object, and / or pixel data as described herein.

[0029] 2B illustrates sensor data 220 generated by a vehicle's computing system, for example as described herein. Data 220 may include pixels 223 associated with an area and representing sensor data generated by one or more sensors and / or associated systems configured on the vehicle. In FIG. 2B, groups of pixels 221 and 222 may be associated with potential objects in the environment. Groups of pixels 221 and 222 are expanded in this figure for illustrative purposes and are subsets of pixels 223.

[0030] FIG. 2C illustrates pixel data 230 including data for groups of pixels 221 and 222. The vehicle computing system may determine pixel data for pixels potentially associated with an object based on the sensor data. For example, the vehicle computing device may generate unimodal confidence values ​​(e.g., Gaussian confidence values) for individual pixels within pixel groups 221 and 222. The vehicle computing device may also, or instead, generate other data for pixels potentially associated with an object, including pixel data for individual pixels within pixel groups 221 and 222. This pixel data may include center coordinates and / or offset values, length, width, orientation, stationary degree, yaw, and / or velocity. Such pixel and / or object data may be represented as a multi-channel data structure and / or multi-channel input data, with specific layers representing specific types of data, as described in more detail herein.

[0031] The determined unimodal confidence values ​​of the pixels in groups 221 and 222 may be used to determine a unimodal distribution. For example, as shown in example 240 of FIG. 2D, a unimodal distribution of confidence values ​​for pixel group 221 is shown by top view representation 241A and side view representation 241B. Also shown in FIG. 2D is a unimodal distribution of confidence values ​​for pixel group 222, shown by top view representation 242A and side view representation 242B. In the topographical maps of example top view representations 241A and 242A, thicker lines indicate higher unimodal confidence values, and thinner lines represent lower unimodal confidence values. In the example side views of representations 241B and 242B, higher points on the illustrated curves correspond to portions of the distribution associated with higher unimodal confidence values, while lower points on the curves correspond to portions of the distribution associated with lower unimodal confidence values. As shown in this example, the portion of the distribution associated with the confidence values ​​of pixels closer to the center of the detected object may be relatively higher than those further from the object center.

[0032] The vehicle computing system may determine candidate detection box center points or pixels of the object based on the unimodal distribution associated with the unimodal confidence values ​​determined for the pixels. For example, the pixels may be determined based on whether they are associated with a portion of the unimodal distribution that meets or exceeds a threshold. Now referring to example 250 of FIG. 2E, using the data associated with representations 241 and 242 of FIG. 2D, the vehicle computing system may determine that a subset of the group of pixels 221 and 222 are associated with portions of their respective associated unimodal distributions that exceed a threshold. As noted above, those pixels associated with portions of the unimodal distribution that are above a particular threshold may be better candidates for a proposed detection box center point than pixels associated with portions of the unimodal distribution that are below the threshold. In example 250, the subset of pixels 251 and 252 may be determined to be pixels associated with portions of the unimodal distribution that meet or exceed a threshold and / or have unimodal confidence values ​​that meet or exceed a threshold.

[0033] These subsets of pixels may be used as center pixels to determine candidate detection boxes of objects in the environment. As shown in example 260 of FIG. 2F, candidate detection boxes 261 and 262 may be determined based on the subsets of pixels 251 and 252, respectively. In an example, candidate detection boxes 261 and 262 may be generated by decoding parameters associated with the subsets of pixels 251 and 252. For example, candidate detection boxes 261 and 262 may be determined based on one or more of center coordinates and / or offset values, length, width, orientation, stationary degree, yaw, and / or velocity associated with center pixels 251 and 252, respectively, of the proposed detection boxes.

[0034] 2G , to determine output detection boxes to use for objects 211 and 212 detected in the environment, one or more non-maximal suppression operations may be performed using candidate detection boxes 261 and 262 as inputs to determine output detection boxes 271 and 272, respectively, from among the candidate detection boxes. Also, based on the output detection boxes, or other associated multi-channel output data may be determined (e.g., for use in operations associated with controlling a vehicle).

[0035] FIG. 3 illustrates a three-dimensional plot of an exemplary unimodal (e.g., Gaussian) distribution of unimodal pixel confidence values. The distribution 310 may represent a set of unimodal confidence values ​​associated with a set of pixels associated with sensor data. The range of the distribution 310 may be measured relative to a z-axis scale 320. In this example, the range of the distribution 310 may be between 0.0 and 1.0. A threshold 330 of 0.8 is also shown. In the example, those pixels associated with the portion of the distribution 310 that is equal to or greater than the threshold 330 may be used as center pixels of a candidate detection box, as described herein. Those pixels associated with the portion of the distribution 310 that is below the threshold 330 may not be used as center pixels of a candidate detection box.

[0036] FIG. 4 is a pictorial flow diagram illustrating an example process 400 that may be performed as part of training a machine learning model as described herein. In process 400, the machine learning model may be trained to output object data and a unimodal confidence value associated with the object data. In an example, one or more operations of process 400 may be implemented during a training mode of a machine learning model (e.g., performed by a DTN) used by a vehicle computing system, such as by using one or more of the components and systems shown in FIG. 5 and described below. For example, the one or more components and systems may include components and systems associated with the deep tracking network 532 shown in FIG. 5. In an example, one or more operations of process 400 may be performed by a remote system in communication with the vehicle, such as the deep tracking network 550 shown in FIG. 5. In yet another example, one or more operations of process 400 may be performed by a combination of a remote system and a vehicle computing system. However, process 400 is not limited to being performed by such components and systems, and the components and systems of FIG. 5 are not limited to performing process 400.

[0037] In operation 402, the machine learning model running in training mode may determine object data that, when decoded and output as a detection box (e.g., output detection box), can be compared to ground truth data representing known bounding boxes and other object parameters, e.g., using unimodal (e.g., Gaussian) confidence values ​​and / or candidate detection box filtering and / or suppression techniques (e.g., NMS) as described herein. In operation 404, the model may determine a center pixel or center discretized value parameter of the detection box. In an example, the pixel and / or discretized value parameters may be represented in layers of a multi-channel or multi-layer data structure (e.g., a tensor data structure). For example, individual layers of a multi-layer data structure may include data associated with a particular type of pixel data. Each column of such a data structure may be associated with a particular pixel.

[0038] For example, referring now to example 420, center pixel 430 may be a center pixel of a detection box. Pixel 440 may be another pixel (e.g., a ground truth pixel) associated with the detection box. Center pixel 430 may be associated with one or more of a confidence value (e.g., a unimodal confidence value, a Gaussian confidence value, etc.), a center offset X value, a center offset Y value, a length, a width, a yaw value, an orientation, a stationary or moving state, a velocity X value, and a velocity Y value. These individual parameters may be associated with a particular individual layer of a multi-layer data structure and a particular column within the data structure associated with the center pixel. Similarly, pixel 440 may also have parameters associated with a particular layer based on the type of parameter and within the particular column associated with the respective pixel.

[0039] In operation 406, the model may be configured to apply a binary mask to the data output by the model to determine a loss based on a central pixel of the output data.

[0040] In operation 408, the model may determine and back-propagate losses for one or more parameters of individual pixels associated with the detection box. For example, for various parameters of pixel 440, the model may back-propagate regression losses (e.g., focus loss, propagation loss, classification loss, etc.) of one or more parameters of pixel 440. In an example, a focus loss may be determined for a unimodal confidence value of pixel 440. This focus loss may then be back-propagated to pixel 440 as the model is trained. In this manner, data having more accurate object information (e.g., associated with pixels closer to the center of the object) may contribute more to training the model than data that may have less accurate information (e.g., associated with pixels farther from the center of the object).

[0041] 5 shows a block diagram of an example system 500 for implementing the techniques described herein. In at least one example, the system 500 can include a vehicle 502. The vehicle 502 can include a vehicle computing device 504 that can function as a vehicle controller for the vehicle 502 and / or perform the functions of a vehicle controller. The vehicle 502 can also include one or more sensor systems 506, one or more emitters 508, one or more communication connections 510, at least one direct connection 512, and one or more drive systems 514.

[0042] The vehicle computing device 504 may include one or more processors 516 and a memory 518 communicatively coupled to the one or more processors 516. In the illustrated example, the vehicle 502 is an autonomous vehicle, although the vehicle 502 may be any other type of vehicle. In the illustrated example, the memory 518 of the vehicle computing device 504 stores a localization component 520, a perception component 522, a planning component 524, one or more system controllers 526, one or more maps 528, a prediction component 530, and a deep tracking network component 532. While shown in FIG. 5 as residing in the memory 518 for illustrative purposes, it is contemplated that each of the localization component 520, the perception component 522, the planning component 524, the one or more system controllers 526, the one or more maps 528, the prediction component 522, and the deep tracking network component 532 may additionally or alternatively be accessible to the vehicle 502 (e.g., stored remotely).

[0043] In at least one example, the localization component 520 can include functionality to receive data from the sensor system 506 to determine a position and / or orientation (e.g., one or more x, y, z positions, roll, pitch, or yaw) of the vehicle 502. For example, the localization component 520 can include and / or request / receive a map of the environment and can continually determine the position and / or orientation of the autonomous vehicle within the map. In some examples, the localization component 520 can receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., utilizing simultaneous localization and mapping (SLAM), calibration, localization and mapping, simultaneously (CLAMS), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc., to accurately determine the position of the autonomous vehicle. In some examples, as discussed herein, the localization component 520 can provide data to various components of the vehicle 502 to determine an initial position of the autonomous vehicle for generating a trajectory and / or for generating map data.

[0044] In some examples, the perception component 522 may include functionality for performing object detection, segmentation, and / or classification. In some examples, the perception component 522 may provide processed sensor data indicative of the presence of an entity proximate to the vehicle 502 and / or a classification of the entity as an entity type (e.g., automobile, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, traffic signal, traffic light, car light, brake light, unknown, etc.). In additional or alternative examples, the perception component 522 may provide processed sensor data indicative of one or more characteristics associated with the detected entity (e.g., tracked object) and / or the environment in which the entity is located. The perception component 522 may generate the processed sensor data using a multi-channel data structure described herein, such as a multi-channel data structure generated by the deconvolution process described. In some examples, characteristics associated with an entity or object may include, but are not limited to, x-position (global and / or local position), y-position (global and / or local position), z-position (global and / or local position), orientation (e.g., roll, pitch, yaw), entity type (e.g., classification), entity velocity, entity acceleration, entity range (size), etc. Such entity characteristics may be represented in a multi-channel data structure as described herein (e.g., a multi-channel data structure generated as the output of one or more deconvolution layers using learned upsampling factors). Characteristics associated with an environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, time of day, day of the week, season, weather conditions, darkness / light indications, etc.

[0045] In general, the planning component 524 can determine a path that the vehicle 502 should follow to traverse an environment. In examples, the planning component 524 can determine various paths and trajectories as well as various levels of detail. For example, the planning component 524 can determine a route (e.g., a planned route) to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the route can be a series of waypoints for travel between the two locations. As non-limiting examples, the waypoints include roads, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 524 can generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 524 can determine how to guide the autonomous vehicle from a first waypoint in the series of waypoints to a second waypoint in the series of waypoints. In some examples, the instructions can be a trajectory or a portion of a trajectory. In some examples, multiple trajectories can be generated substantially simultaneously (e.g., within technical tolerances) according to receding horizon techniques, and one of the multiple trajectories is selected for vehicle 502 to travel.

[0046] In at least one example, the vehicle computing device 504 can include one or more system controllers 526, which can be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of the vehicle 502. These system controllers 526 can communicate with and / or control systems corresponding to the drive system 514 and / or other components of the vehicle 502.

[0047] The memory 518 further includes one or more maps 528 that can be used by the vehicle 502 to navigate within the environment. For purposes of illustration, a map can be any number of data structures modeled in two, three, or N dimensions that can provide information about the environment, such as, but not limited to, topology (e.g., intersections), roadways, mountain ranges, roads, terrain, and the general environment. In some examples, the map can include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), invisible light information (near-infrared light information, infrared light information, etc.), intensity information (e.g., lidar information, radar information, near-infrared light intensity information, infrared light intensity information, etc.), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), and reflectance information (e.g., specularity information, retroreflectance information, BRDF information, BSSRDF information, etc.). In one example, the map can include a three-dimensional mesh of the environment. In some examples, as described herein, Thus, the map can be stored in a tiled format and loaded into the working memory as needed, such that individual tiles of the map represent separate portions of the environment. In at least one example, the one or more maps 528 can include at least one map (e.g., an image and / or a mesh). In some examples, the vehicle 502 can be controlled based at least in part on the map 528. That is, the map 528 can be used in conjunction with the localization component 520, the perception component 522, and / or the planning component 524 to determine a position of the vehicle 502, identify objects in the environment, and / or generate a route and / or trajectory to navigate through the environment.

[0048] In some examples, one or more maps 528 may be stored on a remote computing device (such as computing device 542) accessible via network 540. In some examples, multiple maps 528 may be stored, for example, based on a characteristic (e.g., type of entity, time of day, day of the week, season, etc.). Storing multiple maps 528 keeps memory requirements the same but increases the speed at which the data in the maps can be accessed.

[0049] In general, the prediction component 530 can generate predicted trajectories for objects in the environment. For example, the prediction component 530 can generate one or more predicted trajectories for vehicles, pedestrians, animals, etc. within a threshold distance from the vehicle 502. In some examples, the prediction component 530 can measure the trajectories of the objects and generate the trajectories of the objects based on the observed and predicted behavior.

[0050] In some cases, some or all aspects of the components discussed herein may include any model, algorithm, and / or machine learning algorithm. For example, in some examples, the components in memory 518 (and memory 546 described below) may be implemented as a neural network. For example, memory 518 may include a deep tracking network 532 that may be configured as a convolutional neural network (CNN) 534. The CNN 534 may include one or more convolutional and / or deconvolutional layers.

[0051] As described herein, an exemplary neural network is an algorithm that passes input data through a series of connected layers to generate an output. Each layer in a neural network can also include another neural network, or can include any number of layers, each of which may be a convolutional, deconvolutional, or other type of layer. As can be understood in the context of the present disclosure, a neural network can utilize machine learning, which can refer to a broad class of such algorithms in which an output is generated based on learned parameters.

[0052] Although described in the context of neural networks, any type of machine learning can be used in accordance with the present disclosure, for example, to determine the learned upsampling factor. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least-angle regression (LARS)), decision tree algorithms (e.g., classification and regression trees (CART), iterative dichotomiser 3 (ID3), chi-squared automated interaction detection (CHAID), decision stump, conditional decision tree), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average one-dependence (AODE), etc.), and can be used to determine the upsampling factor. estimators), Bayesian belief networks (BNNs), Bayesian networks), clustering algorithms (e.g., k-means, k-medians, expectation maximization (EM), hierarchical clustering), association rule learning algorithms (e.g., perceptrons, backpropagation, Hopfield networks, RBFNs (Radial Basis FunctionNetwork), deep learning algorithms (e.g., Deep Boltzmann Machines (DBM), Deep Belief Networks (DBN), Convolutional Neural Networks (CNN), stacked autoencoders), dimensionality reduction algorithms (e.g., Principal Component Analysis (PCA), Principal Component Regression (PCR), Partial Least Squares Regression (PLSR), Sammon Mapping, Multidimensional Scaling (MDS), Projection Pursuit, Linear Discriminant Analysis (LDA), Mixture Discriminant Analysis (MDA), Quadratic Discriminant Analysis (QDA), Flexible Discriminant Analysis (FDA)), ensemble algorithms (e.g., Boosting, Bootstrap Aggregation (Bagging), AdaBoost, Stacked Generalization (Blending), Gradient Boosting Machines (GBM), Gradient Boosted Regression Trees (GBRT), Random Forests), SVM (Support Vector Machines), supervised learning, unsupervised learning, semi-supervised learning, etc. Further examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, PointNet, etc.

[0053] In at least one example, the sensor system 506 can include radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, etc.), time-of-flight sensors, microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), and the like. The sensor system 506 can include multiple instances of each of these or other types of sensors. For example, a camera sensor can include multiple cameras positioned at various locations on the exterior and / or interior of the vehicle 502. The sensor system 506 can provide input to the vehicle computing device 504. Additionally and / or alternatively, the sensor system 506 can transmit sensor data to one or more computing devices at a particular frequency, such as in near real-time, after a predetermined period of time has elapsed, and / or over one or more networks 436.

[0054] The vehicle 502 may also include one or more emitters 508 for emitting light (visible and / or non-visible) and / or sound. The emitters 508 in this example include interior audio and visual emitters to communicate with occupants of the vehicle 502. By way of example, and not by way of limitation, the interior emitters may include speakers, lights, symbols, display screens, touch screens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. The emitters 508 in this example also include exterior emitters. By way of example, and not by way of limitation, the exterior emitters in this example include lights for signaling a direction of travel or other indicators of the operation of the vehicle (e.g., indicator lights, signs, light arrays, etc.), and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for audibly communicating with one or more pedestrians or other nearby vehicles with acoustic beam steering technology. The external emitter in this example may also, or instead, include an infrared emitter, a near-infrared emitter, and / or a non-visible light emitter, such as a lidar emitter.

[0055] The vehicle 502 may also include one or more communication connections 510 that enable communication between the vehicle 502 and one or more other local or remote computing devices. For example, the communication connections 510 may facilitate communication with other local computing devices on the vehicle 502 and / or the drive system 514. The communication connections 510 may also enable the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). The communication connections 510 may also enable the vehicle 502 to communicate with remotely operated computing devices or other remote services.

[0056] The communications connection 510 may include physical and / or logical interfaces for connecting the vehicle computing device 504 to another computing device or network, such as the network 540. For example, the communications connection 510 may enable Wi-Fi-based communications, such as over frequencies defined by the IEEE 80211 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communications protocol that enables each computing device to interface with other computing devices.

[0057] In at least one example, the vehicle 502 can include one or more drive systems 514. In some examples, the vehicle 502 can have a single drive system 514. In at least one example, when the vehicle 502 has multiple drive systems 514, the individual drive systems 514 can be located at either end of the vehicle 502 (e.g., the front and the rear, etc.). In at least one example, the drive system 514 can include one or more sensor systems to detect conditions surrounding the drive system 514 and / or the vehicle 502. By way of example and not limitation, the sensor systems can include one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of the drive system, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive system, cameras or other imaging sensors, ultrasonic sensors for acoustically detecting objects in the environment surrounding the drive system, lidar sensors, lattice sensors, etc. Some sensors, such as wheel encoders, can be of the drive system 514. In some cases, the sensor systems on the drive system 514 may overlap or supplement corresponding systems on the vehicle 502 (e.g., the sensor system 506).

[0058] The drive system 514 can include many vehicle systems including a high voltage battery, a motor to propel the vehicle, an inverter to convert direct current from the battery to alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which can be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for braking force distribution to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights that illuminate the vehicle's exterior environment), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC / DC converters, high voltage junctions, high voltage cables, charging systems, charging ports, etc.). Additionally, the drive system 514 can include a drive system controller that can receive and pre-process data from the sensor systems and control the operation of various vehicle systems. In some examples, the drive system controller can include one or more processors and a memory communicatively coupled to the one or more processors. The memory can store one or more components and perform various functions of the drive system 514. Additionally, drive system 514 may also include one or more communication connections that enable each drive system to communicate with one or more other local or remote computing devices.

[0059] In at least one example, the direct connection 512 can provide a physical interface to couple one or more drive systems 514 with the body of the vehicle 502. For example, the direct connection 512 can enable the transfer of energy, fluid, air, data, etc. between the drive systems 514 and the vehicle. In some examples, the direct connection 512 can also removably secure the drive systems 514 to the body of the vehicle 502.

[0060] In some examples, the vehicle 502 can transmit sensor data to one or more computing devices 542 over the network 540. In some examples, the vehicle 502 can transmit raw sensor data to the computing device 542. In other examples, the vehicle 502 can transmit processed sensor data and / or a representation of the sensor data (e.g., a multi-channel data structure representing the sensor data) to the computing device 542. In some examples, the vehicle 502 can transmit sensor data to the computing device 542 at a particular frequency, after a predetermined period of time, in near real-time, or in other cases. In some cases, the vehicle 502 can transmit the sensor data (raw or processed) to the computing device 542 as one or more log files.

[0061] The computing device 542 may include a processor 544 and a memory 546 that stores a deep tracking network component 550 and / or a perception component 548. In some examples, the deep tracking network component 550 may substantially correspond to the deep tracking network component 532 and may include substantially similar functionality. For example, the deep tracking network component 550 may include a convolutional neural network (CNN) 552 that may be composed of one or more convolutional and / or deconvolutional layers. In some examples, the perception component 548 may substantially correspond to the perception component 522 and may include substantially similar functionality.

[0062] The processor 516 of the vehicle 502 and the processor 544 of the computing device 542 may be any suitable processor capable of executing instructions, processing data, and performing operations as described herein. By way of example and not limitation, the processors 516 and 544 may comprise one or more central processing units (CPUs), graphics processing units (GPUs), and / or any other device or portion of a device that processes electronic data and converts the electronic data into registers and / or other electronic data that may be stored in memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to implement encoded instructions.

[0063] The memories 518 and 546 are examples of non-transitory computer-readable media. The memories 518 and 546 can store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods and functions attributed to the various systems described herein. In various implementations, the memories can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein can include many other logical, programmatic, and physical components, the ones illustrated in the accompanying drawings of which are merely examples relevant to the description herein.

[0064] 5 is shown as a distributed system, it should be noted that in alternative examples, components of the vehicle 502 may be associated with the computing device 542 and / or components of the computing device 542 may be associated with the vehicle 502. That is, the vehicle 502 may perform one or more of the functions associated with the computing device 542 and vice versa. Example clauses A: A system comprising one or more processors and one or more non-transitory computer-readable media storing instructions executable by the one or more processors, the instructions, when executed, causing the system to perform operations including receiving sensor data from a sensor associated with an autonomous vehicle in an environment; determining first multi-channel data based at least in part on the sensor data; inputting the first multi-channel data to a machine learning model; determining, by the machine learning model, Gaussian confidence values ​​for pixels associated with an object in the environment; determining a plurality of proposed central pixels from the pixels associated with the object based at least in part on the Gaussian confidence values; determining a candidate detection box for each pixel of the plurality of proposed central pixels; determining an output detection box based at least in part on the candidate detection box based at least in part on non-maximum suppression; generating second multi-channel data representing the object based at least in part on the central pixel associated with the output detection box; tracking the object based at least in part on the second multi-channel data as a tracked object; and controlling the autonomous vehicle based at least in part on the tracked object.

[0065] B: The system of paragraph A, wherein the machine learning model is trained to determine a Gaussian confidence value for a pixel based on the focus loss.

[0066] C: The system of paragraphs A or B, wherein the channel of the second multi-channel data includes one or more of a center offset value, a classification, a dimension, a yaw, a velocity, or a motion state.

[0067] D: The system of any of paragraphs A to C, wherein a channel of the second multi-channel data includes Gaussian confidence values ​​for pixels associated with the output detection box.

[0068] E: The system of any of paragraphs A-D, wherein determining the plurality of proposed central pixels includes determining, based at least in part on the Gaussian confidence values, that Gaussian confidence values ​​of individual pixels of the plurality of proposed central pixels meet or exceed a threshold value.

[0069] F: The system of any of paragraphs A-E, wherein the output detection box is associated with one or more object detections of a vehicle, bicycle, or pedestrian.

[0070] G: A method comprising: receiving sensor data from a sensor associated with a vehicle in an environment; determining input data based at least in part on the sensor data; inputting the input data to a machine learning model; determining a unimodal confidence value for a discretized value associated with an object represented in the input data by the machine learning model; determining a proposed centroid value from the discretized values ​​associated with the object based at least in part on the unimodal confidence value; determining a candidate detection box representing the object based at least in part on the proposed centroid value; determining an output detection box based at least in part on the candidate detection box; and controlling the vehicle based at least in part on the output detection box.

[0071] H: The method of paragraph G, wherein the machine learning model is trained to determine a unimodal confidence value for the discretized values ​​based on the focal loss.

[0072] I: The method of paragraph G or H, further comprising generating multi-channel output data representing the object based at least in part on a center value associated with the output detection box.

[0073] J: The method of paragraph I, wherein a channel of the multi-channel output data includes a unimodal confidence value for a discretized value associated with an output detection box.

[0074] K: The method of paragraph I, wherein the channels of the multi-channel output data include one or more of a center offset value, a classification, a dimension, a yaw, a velocity, or a motion state.

[0075] L: The method of any of paragraphs G-K, wherein determining the output detection box includes performing a non-maximum suppression operation based at least in part on the candidate detection boxes.

[0076] M: The method of any of paragraphs G-L, wherein determining the proposed central value includes determining, based at least in part on the unimodal confidence value, that the unimodal confidence value of the proposed central value meets or exceeds a threshold value.

[0077] N: The method of any of paragraphs G-M, wherein the input data represents a top-down perspective view of the environment.

[0078] O: One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, perform operations including receiving sensor data from sensors associated with a vehicle in an environment; determining input data based at least in part on the sensor data; inputting the input data to a machine learning model; determining a unimodal confidence value for a discretized value associated with an object represented in the input data by the machine learning model; determining a proposed center value from the discretized values ​​associated with the object based at least in part on the unimodal confidence value; determining a candidate detection box based at least in part on the proposed center value; determining an output detection box based at least in part on the candidate detection box; and controlling the vehicle based at least in part on the output detection box.

[0079] P: The one or more non-transitory computer-readable media of paragraph O, wherein the machine learning model is trained to determine a unimodal confidence value for the discretized values ​​based on a focal loss.

[0080] Q: The one or more non-transitory computer-readable media of paragraphs O or P, wherein determining the proposed central value includes determining that a unimodal confidence value of the proposed central value meets or exceeds a threshold based at least in part on the unimodal confidence value.

[0081] R: The one or more non-transitory computer-readable media of paragraph Q, wherein the threshold is a machine learning threshold.

[0082] S: One or more non-transitory computer-readable media of any of paragraphs O-R, wherein determining the output detection box includes performing a non-maximum suppression operation based at least in part on the candidate detection boxes.

[0083] T: One or more non-transitory computer-readable media described in any of paragraphs O-S, wherein determining a unimodal confidence value for the discretized values ​​associated with the object includes determining a Gaussian confidence value for the discretized values ​​associated with the object.

[0084] Although the exemplary provisions above are described with respect to one particular implementation, it should be understood in the context of this specification that the contents of the exemplary provisions can be implemented via a method, device, system and / or computer readable medium, and / or other implementations. Furthermore, any of the embodiments A-T can be implemented alone or in combination with any other one or more of any of the embodiments A-T. summary One or more examples of the technology described herein have been described; however, various modifications, additions, permutations, and equivalents thereof fall within the scope of the technology described herein.

[0085] In the description of the examples, reference is made to the accompanying drawings, which form a part of this specification, and which show by way of example specific examples of the claimed subject matter. It is understood that other examples can be used and modifications or substitutions, such as structural changes, can be made. Such examples, modifications or substitutions do not necessarily depart from the scope of the intended claimed subject matter. Although the steps in this specification can be presented in a particular order, in some cases the order can be changed, such as providing certain inputs at different times or in a different order, without changing the functionality of the described systems and methods. The disclosed procedures can also be performed in different orders. Furthermore, the various calculations in this specification need not be performed in the order disclosed, and other examples using alternative orders of calculations can be easily implemented. In addition to reordering, calculations can also be decomposed into sub-calculations with the same results.

Claims

1. receiving sensor data from sensors associated with a vehicle in an environment; determining input data based at least in part on the sensor data; inputting the input data into a machine learning model; determining, by the machine learning model, a unimodal confidence value for a discretized value associated with an object represented in the input data; determining a proposed center value from the discretized values ​​associated with the object based at least in part on the unimodal confidence value; determining a candidate detection box representing the object based at least in part on the proposed center value; determining an output detection box based at least in part on the candidate detection boxes; controlling the vehicle based at least in part on the output detection box; A method comprising:

2. The method of claim 1 , wherein the machine learning model is trained to determine the unimodal confidence values ​​of the discretized values ​​based on a focal loss.

3. The method of claim 1 or 2, further comprising generating multi-channel output data representing the object based at least in part on a center value associated with the output detection box.

4. The method of claim 3 , wherein a channel of the multi-channel output data comprises a unimodal confidence value for a discretized value associated with the output detection box.

5. The channels of the multi-channel output data include: Center offset value, classification, size, Yo, speed, or Movement state The method of claim 3, comprising one or more of:

6. The method of claim 1 , wherein determining the output detection box comprises performing a non-maximum suppression operation based at least in part on the candidate detection boxes.

7. 2. The method of claim 1 , wherein determining the proposed center value comprises determining, based at least in part on the unimodal confidence value, that the unimodal confidence value of the proposed center value meets or exceeds a threshold.

8. The method of claim 1 , wherein the input data represents a top perspective view of the environment.

9. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause one or more computing devices to perform the method of claim 1 or 2.

10. one or more processors; one or more non-transitory computer-readable media storing instructions executable by the one or more processors, the instructions, when executed, causing the system to: receiving sensor data from sensors associated with a vehicle in an environment; determining input data based at least in part on the sensor data; inputting the input data into a machine learning model; determining, by the machine learning model, a unimodal confidence value for a discretized value associated with an object represented in the input data; determining a proposed center value from the discretized values ​​associated with the object based at least in part on the unimodal confidence value; determining a candidate detection box based at least in part on the proposed center value; determining an output detection box based at least in part on the candidate detection boxes; controlling the vehicle based at least in part on the output detection box; A system for causing an operator to perform an operation including:

11. The system of claim 10 , wherein the machine learning model is trained to determine the unimodal confidence value of the discretized value based on a focal loss.

12. 12. The system of claim 10 or 11, wherein determining the proposed central value comprises determining, based at least in part on the unimodal confidence value, that the unimodal confidence value of the proposed central value meets or exceeds a threshold.

13. The system of claim 12 , wherein the threshold is a machine learning threshold.

14. The system of claim 10 , wherein determining the output detection box comprises performing a non-maximum suppression operation based at least in part on the candidate detection boxes.

15. The system of claim 10 , wherein determining a unimodal confidence value for the discretized value associated with the object comprises determining a Gaussian confidence value for the discretized value associated with the object.

Images