SYSTEMS AND METHODS FOR PREDICTING VEHICLE CAPACITY DIMENSIONING

Abstract

Prediction methods for vehicle dimensions, including: providing a set of known vehicle dimensions; generating a set of simulated observed dimensions based on the set of known vehicle dimensions; Training a machine learning module based on the set of simulated observed dimensions and the set of known vehicle dimensions; the acquisition of sensor data associated with a first vehicle being observed by a sensor system of an autonomous vehicle; Determining a multitude of observed dimensions of the first vehicle based on the captured sensor data; Determining, with a processor, a multitude of predicted actual dimensions of the first vehicle by applying the multitude of observed dimensions to the machine learning model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to autonomous vehicles, and more particularly to systems and methods for predicting vehicle dimensions observed by various sensors of an autonomous vehicle. BACKGROUND

[0002] An autonomous vehicle (AV) is a vehicle capable of sensing its environment and navigating with little or no user input. This is achieved through the use of sensor devices such as radar, lidar, image sensors, and the like. Autonomous vehicles also utilize information from global positioning systems (GPS), navigation systems, vehicle-to-vehicle communications, vehicle infrastructure technologies, and / or wired systems to navigate the vehicle.

[0003] While significant advances have been made in AVs in recent years, such systems could still be improved in several aspects. For example, it would be advantageous for AVs to be able to determine the actual dimensions of vehicles they encounter during operation. That is, while AVs are equipped with advanced sensor systems that can characterize the observed dimensions of vehicles and other objects in the environment (e.g., vehicles viewed from the side, front, etc.) in a number of ways, the actual dimensions of these objects are not always inferable, partly due to the varying orientation of the observed vehicle from the viewpoint of the sensors.

[0004] Accordingly, it is desirable to provide systems and methods capable of more accurately determining the actual dimensions of a vehicle observed by sensor systems. Furthermore, other desirable features and characteristics of the present invention will become apparent from the following detailed description and appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.

[0005] The document US 2016 / 0358475 A1 discloses that observed dimensions of vehicles are determined based on sensor data.

[0006] In the document US 2013 / 0197736 A1, sensor data is processed using a machine learning model, whereby an object type is determined.

[0007] The publication DE 10 2015 109 832 A1 discloses an object classification for vehicle radar systems in which sensor data is processed using a machine learning model. SUMMARY

[0008] Systems and methods are provided for controlling an autonomous vehicle. In one embodiment, a vehicle dimension prediction method includes generating a set of simulated observed dimensions based on a set of known vehicle dimensions and training a machine learning module based on the set of simulated observed dimensions and the set of known vehicle dimensions. The method further includes acquiring sensor data associated with a first vehicle observed by a sensor system of an autonomous vehicle, determining observed dimensions of the first vehicle based on the acquired sensor data, and determining, with a processor, predicted actual dimensions of the first vehicle by applying the plurality of observed dimensions to the machine learning model.

[0009] In one embodiment, a system for controlling a vehicle includes a vehicle dimension prediction module configured to acquire sensor data associated with a first vehicle observed by a sensor system of an autonomous vehicle and determine a plurality of observed dimensions of the first vehicle based on the acquired sensor data. The vehicle dimension prediction module is further configured to determine a plurality of predicted actual dimensions of the first vehicle by applying the plurality of observed dimensions to a machine learning model, wherein the machine learning model is trained by generating a set of simulated observed dimensions based on the set of known vehicle dimensions and training the machine learning module based on a set of simulated observed dimensions and a set of known vehicle dimensions. DESCRIPTION OF THE DRAWINGS

[0010] The exemplary embodiments are described below in conjunction with the following drawings, wherein like reference numerals designate like elements, and wherein: Fig. 1 is a functional block diagram illustrating an autonomous vehicle with a control system according to various embodiments; Fig. 2 is a functional block diagram illustrating a transport system with one or more autonomous vehicles from Fig. 1 according to various embodiments; Fig. 3 is a functional block diagram illustrating an autonomous driving system (ADS) in conjunction with an autonomous vehicle according to various embodiments; Fig. 4 is a data flow diagram illustrating a vehicle dimension prediction system according to various embodiments; Fig. 5 is a conceptual block diagram of a machine language (ML) model according to various embodiments; Fig. 6 is a flowchart illustrating a control method for dimension prediction according to various embodiments; Fig. 7 illustrates exemplary vehicle dimensions according to various embodiments; Fig. 8 is a conceptual illustration of a known database of vehicle dimensions according to various embodiments; Fig. 9 illustrates a dimension-based decision tree according to various embodiments; and Fig. 10 illustrates the simulation of a lidar point cloud according to various embodiments. DETAILED DESCRIPTION

[0011] The following detailed description is provided for illustrative purposes only and is not intended to limit the application and uses in any way. Furthermore, there is no intention to be bound by any theory expressed or implied in the preceding technical field, background, brief summary, or the following detailed description.The term "module" as used herein refers to any hardware, software, firmware product, electronic control component, processing logic, and / or processor device, individually or in any combination, including, but not limited to, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), an electronic circuit, a processor (shared, dedicated, or group processor), and memory that executes one or more software or firmware programs, combinational logic circuitry, and / or other suitable components that provide the described functionality.

[0012] Embodiments of the present disclosure may be described as functional and / or logical block components and various method steps herein. It should be noted that such block components may be constructed from any number of hardware, software, and / or firmware components configured to perform the required functions. For example, an embodiment of the present disclosure may employ various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, look-up tables, or the like, capable of performing multiple functions under the control of one or more microprocessors or other control devices.Furthermore, those skilled in the art will recognize that embodiments of the present disclosure may be used in connection with any number of systems, and that the system described herein is merely an exemplary embodiment of the present disclosure.

[0013] For the sake of brevity, conventional techniques related to signal processing, data transmission, signaling, control, autonomous vehicles, machine learning, image analysis, neural networks, lidar, analytic geometry, and other functional aspects of the systems (and the individual operating components of the systems) are not described in detail herein. Furthermore, the connecting lines depicted in the various figures are intended to represent exemplary functional relationships and / or physical connections between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in an embodiment of the present disclosure.

[0014] With reference to Fig. 1 illustrates a vehicle dimension prediction system (or simply "system") 100 associated with an autonomous vehicle (AV) 10 according to various embodiments. Generally, the vehicle dimension prediction system 100 includes a machine learning (ML) model (e.g., a neural network or a decision tree) capable of predicting the actual dimensions of a vehicle in the vicinity of the AV 10. Using a database of known vehicle dimensions for a large number of vehicle models, an ML model is trained based on simulated sensor data associated with those known dimensions. The resulting ML model takes as input the observed dimensions of a vehicle (e.g., the apparent height, width, and length as viewed by the AV 10) and produces as output a predicted actual value of those dimensions.

[0015] Referring to Fig. 1 includes an autonomous vehicle ("AV" or simply "vehicle") 10 generally including a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is disposed on the chassis 12 and substantially encloses the other components of the vehicle 10. The body 14 and the chassis 12 may together form a frame. The wheels 16-18 are each rotatably connected to the chassis 12 near a respective corner of the body 14.

[0016] In various embodiments, the vehicle 10 is an autonomous vehicle, and the vehicle dimension prediction system 100 is incorporated into the autonomous vehicle 10. The autonomous vehicle 10 is, for example, a vehicle that is automatically controlled to transport passengers from one location to another. The vehicle 10 is depicted as a passenger car in the illustrated embodiment, but it should be noted that any other vehicle, including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), vessels, aircraft, etc., may be used.

[0017] In an exemplary embodiment, the autonomous vehicle 10 corresponds to a level four or level five automation system according to the Society of Automotive Engineers (SAE) standard taxonomy of automated driving levels, "J3016." Using this terminology, a level four system indicates "high automation," referring to a driving mode in which the automated driving system assumes all aspects of the dynamic driving task, even if a human driver does not adequately respond to a request for intervention. A level five system, on the other hand, exhibits "full automation," referring to a driving mode in which the automated driving system performs all aspects of the dynamic driving task under all road and environmental conditions that a human driver can handle.It will be appreciated that the embodiments according to the present subject matter are not limited to any particular taxonomy or rubric of automation categories. Furthermore, systems and methods according to the present embodiment may be used in connection with any autonomous vehicle that utilizes a navigation system to provide route guidance.

[0018] As illustrated, the autonomous vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, at least one controller 34, and a communication system 36. The propulsion system 20, in various embodiments, may include an internal combustion engine, an electric motor, such as a traction motor, and / or a fuel cell propulsion system. The transmission system 22 is configured to transfer power from the propulsion system 20 to the vehicle wheels 16 and 18 according to selectable gear ratios. According to various embodiments, the transmission system 22 may include a stepped-ratio automatic transmission, a continuously variable transmission, or other suitable transmission.

[0019] The braking system 26 is configured to provide braking torque to the vehicle wheels 16 and 18. In various embodiments, the braking system 26 may include friction brakes, brake-by-wire, a regenerative braking system such as an electric motor, and / or other suitable braking systems.

[0020] The steering system 24 influences a position of the vehicle wheels 16 and / or 18. While illustrated as a steering wheel 25 for illustrative purposes in some embodiments within the scope of the present disclosure, the steering system 24 may not include a steering wheel.

[0021] The sensor system 28 includes one or more sensor devices 40a-40n that detect observable conditions of the external environment and / or the internal environment of the autonomous vehicle 10. The sensor devices 40a-40n may include, but are not limited to, radars, lidars, global positioning systems, optical cameras, thermal imaging cameras, ultrasonic sensors, and / or other sensors. The actuator system 30 includes one or more actuator devices 42a-42n that control one or more vehicle features, such as, but are not limited to, the propulsion system 20, the transmission system 22, the steering system 24, and the braking system 26. In various embodiments, the autonomous vehicle 10 may also include vehicle interior and / or exterior features not included in Fig. 1, such as various doors, trunk and cabin features, such as air, music, lighting, touchscreen display components (such as those used in conjunction with navigation systems) and the like.

[0022] The data storage device 32 stores data for use in automatically controlling the autonomous vehicle 10. In various embodiments, the data storage device 32 stores defined maps of the navigable environment. In various embodiments, the defined maps are predefined and retrieved from a remote system (in further detail with respect to Fig. 2). For example, the defined maps may be assembled by the remote system and communicated to the autonomous vehicle 10 (wirelessly and / or wired) and stored in the data storage device 32. Route information may also be stored in the data device 32—i.e., a series of road segments (geographically linked to one or more of the defined maps) that together define a route the user may take from a starting location (e.g., the user's current location) to a destination location. As can be appreciated, the data storage device 32 may be part of the controller 34, separate from the controller 34, or part of the controller 34 and part of a separate system.

[0023] The controller 34 includes at least one processor 44 and a computer-readable storage device or media 46. The processor 44 may be a custom-made or off-the-shelf processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among multiple processors connected to the controller 34, a semiconductor-based microprocessor (in the form of a microchip or chipset), a combination thereof, or generally any device for executing instructions. The computer-readable storage device or media 46 may include volatile and non-volatile memory in read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is persistent or non-volatile memory that can be used to store various operating variables while the processor 44 is powered off.The computer-readable storage device or media 46 may be implemented using any of a number of known storage devices, such as PROMs (programmable read-only memory), EPROMs (electrical PROMs), EEPROMs (electrically erasable PROMs), flash memory, or any other electrical, magnetic, optical, or combination storage devices capable of storing data, some of which represent executable instructions used by the controller 34 in controlling the autonomous vehicle 10.

[0024] The instructions may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. The instructions, when executed by processor 44, receive and process signals from sensor system 28, perform logic, calculations, methods, and / or algorithms to automatically control the components of autonomous vehicle 10, and generate control signals that are transmitted to actuator system 30 to automatically control the components of autonomous vehicle 10 based on the logic, calculations, methods, and / or algorithms. Although in Fig. 1, only one controller 34 is depicted, embodiments of the autonomous vehicle 10 may include any number of controllers 34 communicating and cooperating over any suitable communication medium or combination of communication media to process the sensor signals, perform logic, calculations, methods, and / or algorithms, and generate control signals to automatically control the functions of the autonomous vehicle 10. In one embodiment, as discussed in detail below, the controller 34 is configured to classify objects in the environment using an ML model that has been previously trained based on the type of boundary curve associated with such objects.

[0025] The communication system 36 is configured to wirelessly transmit information to and from other units 48, such as, but not limited to, other vehicles (“V2V” communication), infrastructure (“V2I” communication), remote transportation systems, and / or user devices (relating to Fig. 2). In an exemplary embodiment, the communication system 36 is a wireless communication system configured to communicate over a wireless local area network (WLAN) using IEEE 802.11 standards or using cellular data communication. However, additional or alternative communication methods, such as a dedicated short-range communication (DSRC) channel, are also contemplated within the scope of the present disclosure. DSRC channels refer to unidirectional or bidirectional short- to medium-range wireless communication channels specifically designed for use in vehicles, and a corresponding set of protocols and standards.

[0026] With reference now to Fig. 2, in various embodiments, the autonomous vehicle 10 described with reference to Fig. 1, may be suitable for use in connection with a taxi or shuttle system in a specific geographic area (e.g., a city, a school or business complex, a shopping center, an amusement park, an event center, or the like), or may simply be managed by a remote system. For example, the autonomous vehicle 10 may be associated with an autonomous vehicle-based remote transportation system. Fig. Figure 2 illustrates an exemplary embodiment of an operating environment, shown generally at 50, including an autonomous vehicle-based remote transportation system (or simply “remote transportation system”) 52, which, as described with reference to Fig. 1, is assigned to one or more autonomous vehicles 10a-10n. In various embodiments, the operating environment 50 (which may in whole or in part correspond to the Fig. 1) further comprises one or more user devices 54 that communicate with the autonomous vehicle 10 and / or the remote transportation system 52 via a communications network 56.

[0027] The communications network 56 supports communication as needed between devices, systems, and components supported by the operating environment 50 (e.g., via physical communication links and / or wireless communication links). For example, the communications network 56 may include a wireless carrier system 60, such as a cellular phone system, that includes a plurality of cellular towers (not shown), one or more mobile switching centers (MSCs) (not shown), and any other network components necessary to connect the wireless carrier system 60 to the fixed network communications system. Each cellular tower includes transmit and receive antennas and a base station, with the base stations of various cellular towers connected to the MSCs, either directly or through intermediate devices, such as a base station controller.The wireless carrier system 60 may implement any suitable communications technology, for example, digital technologies such as CDMA (e.g., CDMA2000), LTE (e.g., 4G LTE or 5G LTE), GSM / GPRS, or other current or emerging wireless technologies. Other cell tower / base station / MSC arrangements are possible and could be used with the wireless carrier system 60. For example, the base station and cell tower could be co-located or remote from each other, each base station could serve a single cell tower, a single base station could serve multiple cell towers, or multiple base stations could be connected to a single MSC, to name a few of the possible arrangements.

[0028] In addition to using the wireless carrier system 60, a second wireless carrier system in the form of a satellite communications system 64 may be used to provide unidirectional or bidirectional communications with the autonomous vehicles 10a-10n. This may be accomplished using one or more communications satellites (not shown) and an uplink transmitting station (not shown). Unidirectional communications may include, for example, satellite radio services, wherein programmed content data (news, music, etc.) is received from the transmitting station, packaged for upload, and then transmitted to the satellite, which broadcasts the programming to subscribers. Bidirectional communications may include, for example, satellite telephone services that utilize the satellite to relay telephone communications between the vehicle 10 and the station.Satellite telephony can be used either in addition to or instead of the wireless carrier system 60.

[0029] A fixed-line communication system 62 may further include a conventional fixed-line telecommunications network connected to one or more landline telephones and connecting the wireless carrier system 60 to the remote transport system 52. For example, the fixed-line communication system 62 may be a public switched telephone network (PSTN) such as that used to provide wired telephony, packet-switched data communications, and the Internet infrastructure. One or more segments of the fixed-line communication system 62 could be implemented using a standard wired network, a fiber optic or other optical network, a cable network, power lines, other wireless networks such as wireless local area networks (WLANs) or networks providing wireless broadband access (BWA), or any combination thereof.Furthermore, the remote transport system 52 need not be connected via the fixed network communication system 62, but could include radiotelephone equipment so that it can communicate directly with a wireless network, such as the wireless carrier system 60.

[0030] Although in Fig. 2 illustrates only one user device 54, embodiments of the operating environment 50 may support any number of user devices 54, including multiple user devices 54 owned, operated, or otherwise used by a person. Each user device 54 supported by the operating environment 50 may be implemented using any suitable hardware platform. In this regard, the user device 54 may be implemented in a common form factor, including: a desktop computer; a mobile computer (e.g., a tablet computer, a laptop computer, or a netbook computer); a smartphone; a video game device; a digital media player; a component of a home entertainment device; a digital camera or video camera; a wearable computing device (e.g., a smart watch, smart glasses, smart clothing); or the like.Each user device 54 supported by operating environment 50 is embodied as a computer-implemented or computer-based device having the hardware, software, firmware, and / or processing logic necessary to perform the various techniques and methods described herein. For example, user device 54 includes a microprocessor in the form of a programmable device that includes one or more instructions stored in an internal memory structure and is operable to receive binary inputs and generate binary outputs. In some embodiments, user device 54 includes a GPS module that can receive GPS satellite signals and generate GPS coordinates based on those signals.In further embodiments, user device 54 includes cellular communication functionality such that the device performs voice and / or data communications over communication network 56 using one or more cellular communication protocols, as explained herein. In various embodiments, user device 54 includes a visual display, such as a graphical touchscreen display or other display.

[0031] The remote transportation system 52 includes one or more backend server systems (not shown), which may be cloud-based, network-based, or resident at the particular campus or geographic location served by the remote transportation system 52. The remote transportation system 52 may be staffed with a live advisor, an automated advisor, an artificial intelligence system, or a combination thereof. The remote transportation system 52 may communicate with the user devices 54 and the autonomous vehicles 10a-10n to schedule trips, dispatch autonomous vehicles 10a-10n, and the like. In various embodiments, the remote transportation system 52 stores account information, such as subscriber authentication data, vehicle license plates, profile records, biometric data, behavior patterns, and other related subscriber information.In one embodiment, as described in more detail below, the remote transportation system 52 includes a route database 53 that stores information related to navigation system routes.

[0032] According to a typical use case workflow, a registered user of the remote transportation system 52 may create a ride request via the user device 54. The ride request typically specifies the passenger's desired pickup location (or current GPS location), the desired destination (which may identify a predefined vehicle stop and / or a user-defined passenger destination), and a pickup time. The remote transportation system 52 receives the ride request, processes the request, and dispatches a selected one of the autonomous vehicles 10a-10n (when and if available) to pick up the passenger at the designated pickup location and time. The transportation system 52 may also generate and send an appropriately configured confirmation message or notification to the user device 54 to notify the passenger that a vehicle is en route.

[0033] As can be seen, the subject matter disclosed herein provides certain enhanced features and functionality for what may be considered a standard or baseline autonomous vehicle 10 and / or an autonomous vehicle-based transportation system 52. To this end, an autonomous vehicle-based transportation system may be modified, enhanced, or otherwise supplemented to provide the additional functionality described in more detail below.

[0034] According to various embodiments, the controller 34 implements an autonomous drive system (ADS) 70, as shown in Fig. 3. That is, suitable software and / or hardware components of the controller 34 (e.g., the processor 44 and the computer-readable storage device 46) are used to provide an autonomous propulsion system 70 used in conjunction with the vehicle 10.

[0035] In various embodiments, the instructions of the autonomous driving system 70 may be structured according to function or system. For example, the autonomous driving system 70 may, as shown in Fig. 3, include a sensor fusion system 74, a positioning system 76, a guidance system 78, and a vehicle control system 80. As can be appreciated, in various embodiments, the instructions may be divided into any number of systems (e.g., combined, further subdivided, etc.), as the disclosure is not limited to the present examples.

[0036] In various embodiments, the sensor fusion system 74 synthesizes and processes sensor data and predicts the presence, location, classification, and / or history of objects and features of the environment of the vehicle 10. In various embodiments, the sensor fusion system 74 may include information from multiple sensors, including, but not limited to, cameras, lidars, radars, and / or any number of other types of sensors.

[0037] The positioning system 76 processes sensor data along with other data to determine a position (e.g., a local position relative to a map, an exact position relative to the lane of a road, vehicle direction, speed, etc.) of the vehicle 10 relative to its surroundings. The guidance system 78 processes sensor data along with other data to determine a route for the vehicle 10 to follow. The vehicle control system 80 generates control signals for controlling the vehicle 10 according to the determined route.

[0038] In various embodiments, the controller 34 implements machine learning techniques to support the functionality of the controller 34, such as feature detection / classification, obstacle mitigation, route traversal, mapping, sensor integration, ground truth determination, and the like.

[0039] According to various embodiments, the controller 34 implements an autonomous drive system (ADS) as described in Fig. 3. That is, suitable software and / or hardware components of the controller 34 (e.g., the processor 44 and the computer-readable storage device 46) are used to provide an ADS 70 used in conjunction with the vehicle 10.

[0040] In various embodiments, the instructions of the autonomous driving system 70 may be structured according to function or system. For example, the autonomous driving system 70 may, as shown in Fig. 3, include a sensor fusion system 74, a positioning system 76, a guidance system 78, and a vehicle control system 80. As can be appreciated, in various embodiments, the instructions may be divided into any number of systems (e.g., combined, further subdivided, etc.), as the disclosure is not limited to the present examples.

[0041] In various embodiments, the sensor fusion system 74 synthesizes and processes sensor data and predicts the presence, location, classification, and / or history of objects and features of the environment of the vehicle 10. In various embodiments, the sensor fusion system 74 may include information from multiple sensors, including, but not limited to, cameras, lidars, radars, and / or any number of other types of sensors.

[0042] The positioning system 76 processes sensor data along with other data to determine a position (e.g., a local position relative to a map, an exact position relative to the lane of a road, vehicle direction, speed, etc.) of the vehicle 10 relative to its surroundings. The guidance system 78 processes sensor data along with other data to determine a route for the vehicle 10 to follow. The vehicle control system 80 generates control signals for controlling the vehicle 10 according to the determined route.

[0043] In various embodiments, the controller 34 implements machine learning techniques to support the functionality of the controller 34, such as feature detection / classification, obstacle mitigation, route traversal, mapping, sensor integration, ground truth determination, and the like.

[0044] As briefly mentioned above, the system is 100 of Fig. 1 is capable of predicting the actual dimensions of another vehicle in the vicinity of AV 10 based on a previously trained ML model. In various embodiments, the ML model is trained using simulated sensor readings (e.g., simulated lidar clouds) derived from a database of known dimensions for actual vehicle models that AV is likely to encounter during actual operation.

[0045] Fig. 4 is a data flow diagram illustrating various embodiments of the system 100 that may be embedded in the controller 34. Referring to Fig. 4, an exemplary system 400 generally includes a vehicle dimension prediction module 420 that receives sensor data 402 relating to observed dimensions of a vehicle encountered in the environment (e.g., camera images, lidar data, or other sensor data received from sensor system 28) and has as output 403 the predicted actual dimensions of that vehicle. In various embodiments, module 420 implements an ML model that has been previously trained using a corpus of example images generated by simulated sensor data based on a database of known vehicle dimensions.

[0046] It is understood that various embodiments of the system 100 according to the present disclosure may include any number of sub-modules embedded in the controller 34. As can be seen, all modules / sub-modules included in Fig. 4, may be combined and / or further subdivided to similarly perform the various methods described herein. Inputs to the system 100 may be received from the sensor system 28, received from other control modules (not shown) associated with the autonomous vehicle 10, received from the communication system 36, and / or by other submodules (not shown) within the controller 34 of Fig. 1. Module 420 can be determined / modeled, for example, in one of the various Fig. 3 illustrated modules must be implemented.

[0047] The vehicle dimension prediction module 420 may implement a variety of machine learning models that are trained using a set of images and other data that have been previously acquired and stored (e.g., in server 53 of Fig. 2). In this respect, Fig. 5 a conceptual overview of an ML model 500 that could be used in conjunction with various embodiments.

[0048] The ML model 500 includes a number of inputs 501 (e.g., 501a-501c) and a number of outputs 504 (e.g., 504a-504c). In the illustrated embodiment, the inputs 501a, 501b, and 501c correspond, respectively, to the observed height, width, and length of the vehicle observed by AV 10 (e.g., real numbers representing distance in meters or other suitable units), and the outputs 504a, 504b, and 504c correspond, respectively, to the predicted actual height, width, and length of the vehicle observed by AV 10. In this regard, depending on the particular sensor data used to measure these attributes, the observed height, width, and length may be based on a simple linear measure determined by AV 10 or may be based on a more detailed measure that takes into account depth information derived from the sensor data.For example, in the case where lidar data (which includes distance information) is used to determine the observed length 501c, and the observed vehicle is facing AV 10 at an oblique angle, the observed length could be estimated by first estimating the angle of the oncoming car and then taking this angle into account when calculating the observed length 501c. The observed width 501b and the observed height 501a can be calculated similarly, although it should be noted that due to the generally lateral movement of vehicles on a road, the observed height 501a is typically substantially the same as the predicted actual output of the length 501c (except in cases where the observed vehicle is on an incline, etc.).

[0049] In various embodiments, the ML model 500 is "trained" using a learning rule that allows for effective learning by example. Such learning may be supervised (using known examples as input), unsupervised (using uncategorized examples as input), or involve reinforcement learning (where some "reward" is provided during training). Once the ML model 500 is "trained" to a satisfactory level, it can be used as an analytical tool to make predictions during normal operation of AV 10.

[0050] As described in detail below, the ML model 500 can Fig. 5 can be used to classify the classification module 420 of Fig. 4 by accepting inputs 501 corresponding to observed dimensions of a vehicle in the vicinity of AV 10 and generating predicted actual values ​​of those dimensions as an output 504.

[0051] The ML model 500 may be implemented using a variety of machine learning techniques known in the art. In one embodiment, the ML model 500 is implemented using an artificial network, such as a recurrent neural network (RNN). In another embodiment, the ML model 500 is implemented using a decision tree model. In other embodiments, the ML model is implemented using a random forest model, Bayesian classifiers (e.g., naive Bayes), a principal component analysis (PCA) model, a cluster model, a support vector machine model, a linear discriminant analysis model, a regression model, an ensemble model, or any other model that can be effective in generating numerical values ​​corresponding to predicted dimensions based on numerical values ​​corresponding to observed dimensions.

[0052] Fig. 6 is a flowchart illustrating a control method for predicting vehicle dimensions according to various embodiments and will now be described in connection with Fig. 1-5 and Fig. 7-10. In general, and as described in detail below, Fig. 7 exemplary dimensions of a known vehicle and Fig. Figure 8 conceptually illustrates an exemplary database of known vehicle dimensions. Fig. 9 shows a dimension-based decision tree according to various embodiments and Fig. 10 shows the generation of simulated lidar data according to various embodiments.

[0053] Referring to Fig. 6, a control method 600 may be performed by the system 100 of Fig. 1 in conjunction with the vehicle dimension prediction module 420 of Fig. 4. As will be appreciated in light of the disclosure, the order of operation within method 600 is not limited to sequential execution as described in Fig. 6, but may be performed in one or more different orders as applicable and in accordance with the present disclosure.

[0054] Control method 600 begins at 601 with the compilation of a database of known vehicle dimensions. Such a database is conceptually Fig. 8, which shows a database 800 with a number of fields (801-804) that characterize the set of vehicle dimensions considered by the vehicle dimension prediction module 420. In Fig. 8, for example, the database 800 could include a vehicle identification field 801 (e.g., a unique ID relating to the specific year, make, and model of the actual vehicle, such as “2018 CHEVY BOLT”), as well as an altitude field 802, a latitude field 803, and a longitude field 804. To briefly refer to Fig. 7, these values ​​are illustrated as a height 702, a length 703, and a width 704 of an exemplary vehicle 700, respectively. In some embodiments, more or fewer dimensions are used. For example, in various embodiments, only the vehicle width 803 and the length 804 are combined.

[0055] The database 800 may be filled or otherwise compiled in a number of ways and may be stored in any suitable manner (e.g., in the database 53 of Fig. 2). Database 800 may, for example, be derived from a publicly available database, server, or website containing information regarding the dimensions of new and / or used cars. In other embodiments, database 800 is populated in an automated and / or manual manner through direct observation and measurement of vehicles in the surrounding area.

[0056] Next, at 602, a corpus (or set) of simulated observed dimensions is generated based on a database of known vehicle dimensions 800. This corpus of training data could include, for example, a large set of images containing sensor data corresponding to vehicles that the AV 10 is likely to encounter (e.g., at various orientations relative to the AV 10), along with known actual dimensions corresponding to these images (e.g., fields 802-804 of Fig. 8) are assigned.

[0057] In one embodiment, the simulated observed dimensions are generated as shown in the Fig. 9 and Fig. 10 by first optimizing (or otherwise training) a decision tree 900 to effectively represent the knowledge gained via the database 800 and then randomly (or systematically) generating exemplary vehicles and corresponding simulated sensor data based on the decision tree 900.

[0058] The dimension-based decision tree 900 generally includes a set of nodes (e.g., 901-905) and branches (e.g., 910-913) between those nodes. Nodes 901-905 represent decision points (and, consequently, sets of vehicles corresponding to specific ranges of dimensions), and branches (910-913) correspond to criteria associated with nodes located lower in the tree. The "leaf nodes" (i.e., endpoints of the tree that have no child nodes, such as nodes 903, 904, and 905) correspond to individual vehicle IDs 801 (or logically grouped IDs of similar vehicles) from database 800. For example, branch 910 could correspond to the criterion "length < 4.5 meters," branch 901 could correspond to the criterion "length >= 4.5 meters and length < 5.2 meters," and so on. Each branch (910 - 913) can contain any number of conditions based on the various dimensions compiled in database 800.It is thus apparent that representative actual vehicle dimensions can be generated by simply traversing the tree 900 from top to bottom, either randomly or systematically (e.g., to generate exemplary vehicles that correspond to the size ratios typically encountered in the real world). In another embodiment, instead of using a decision tree 900, exemplary vehicles are randomly or systematically selected directly from the database 800.

[0059] Regardless of how vehicle dimensions are selected for simulation, appropriate software is then used to simulate sensor data corresponding to a vehicle with the selected dimensions and spatial orientation. This approach is described, for example, in Fig. 10, which shows an example of an oncoming vehicle 1000 oriented at an angle of approximately 45 degrees with respect to the plane of the drawing (also assuming that this is the viewpoint of the AV 10 observing the vehicle 1000). The system then simulates the sensor data that might result from observing the vehicle 1000. For example, in the illustrated embodiment, a lidar point cloud 1010 is shown, although any other type of sensor data may be simulated (e.g., optical data, radar data, etc.). Using known techniques, the observed length (1013), observed width (1012), and observed height (1011) may then be used to generate a set of simulated observed dimensions that are used as input(s) 501 of Fig. 5 can be used.

[0060] The above simulation procedure can be carried out, for example, via a server and / or suitable software located within the system 52 of Fig. 2. Systems and software for simulating the presence of a vehicle in a virtual 3D space and for determining the lidar and / or other sensor data that might be generated by such a simulated vehicle are known in the art and need not be described in detail herein.

[0061] With continued reference to Fig. 6, at 503, the simulated observed dimensions from 603 are used in conjunction with the known vehicle dimensions (as simulated based on decision tree 900 or otherwise derived from database 800) to train ML model 500. For example, in the embodiment where ML model 500 is a neural network, the simulated observed dimensions are applied to input nodes 501a-c, while the known vehicle dimensions are applied to output nodes 504a-c. Backpropagation or other techniques are then used to train ML model 500. In the embodiment where ML model 500 is a decision tree, a suitable tree training / optimization technique is used to generate the desired ML model 500.

[0062] Next, at 604, sensor data is collected from the AV 10 during normal operation. As mentioned above, the sensor data may include any available data collected by the sensor system 28 during operation of the AV 10. In various embodiments, lidar data from an observed vehicle is of particular interest in the vehicle dimension prediction process. However, the embodiments described herein are not so limited, and any available sensor data may be used.

[0063] Next, at 605, the observed dimensions of the vehicle are determined based on the sensor data collected at 604. As described above in connection with Fig. 5, in one embodiment, this classification generally involves feeding the extracted observed dimensions (e.g., height, width, and length) into the previously trained ML model 500 as inputs 501.

[0064] According to various embodiments, other data (such as mapping data stored within or external to AV 10) is used as input to the trained ML model 500 or in a subsequent process to further assist in estimating the actual dimensions. For example, mapping data near AV 10 might indicate that a wall or other predetermined structure (whether stationary or mobile) is in such a position relative to an observed vehicle that it limits the possible dimensions of that observed vehicle. In other words, the system might generally assume that an observed vehicle cannot spatially intersect a nearby wall. In this way, additional information regarding the vehicle's dimensions (e.g., width, length, etc.) can be derived.

[0065] Finally, at 606, the predicted actual dimensions are determined via outputs 504 of the ML model 500. These predicted actual dimensions can then be used by the various modules of AV 10 (e.g., any of the modules described in Fig. 3) to more accurately perform their respective functions. For example, the path planning module 90 could use the predicted actual dimensions 504 to plan a path that avoids intersecting very long vehicles (i.e., a vehicle whose length is not obvious based solely on the available sensor data).

[0066] While at least one exemplary embodiment has been presented in the foregoing detailed description, it should be understood that numerous variations exist. It should further be understood that the exemplary embodiment or embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of this disclosure in any way. Rather, the foregoing detailed description provides those skilled in the art with a convenient roadmap for implementing the exemplary embodiment or embodiments. It should be understood that various changes in the function and arrangement of elements may be made without departing from the scope of the disclosure as recited in the appended claims and their legal equivalents.

Claims

[1] Vehicle dimension prediction method comprising: providing a set of known vehicle dimensions; generating a set of simulated observed dimensions based on the set of known vehicle dimensions; training a machine learning module based on the set of simulated observed dimensions and the set of known vehicle dimensions; collecting sensor data associated with a first vehicle observed by a sensor system of an autonomous vehicle; determining a plurality of observed dimensions of the first vehicle based on the collected sensor data; determining, with a processor, a plurality of predicted actual dimensions of the first vehicle by applying the plurality of observed dimensions to the machine learning model. [2] The method of claim 1, wherein the machine learning model is an artificial neural network model. [3] The method of claim 1, wherein the machine learning model is a decision tree model. [4] The method of claim 1, wherein generating the set of simulated observed dimensions includes training a dimension-based decision tree based on the set of known vehicle dimensions and generating simulated vehicles based on the dimension-based decision tree. [5] The method of claim 4, wherein generating the set of simulated observed dimensions includes generating simulated sensor data associated with the simulated vehicles. [6] The method of claim 1, wherein the set of known vehicle dimensions includes at least a vehicle type identifier, a length of a vehicle corresponding to the vehicle type identifier, and a width of a vehicle corresponding to the vehicle type identifier. [7] System for controlling an autonomous vehicle, comprising: a vehicle dimension prediction module, including a processor, configured to: collecting sensor data associated with a first vehicle observed by a sensor system of an autonomous vehicle; determining a plurality of observed dimensions of the first vehicle based on the acquired sensor data; and determining a plurality of predicted actual dimensions of the first vehicle by applying the plurality of observed dimensions to a machine learning model, wherein the machine learning model is trained by generating a set of simulated observed dimensions based on the set of known vehicle dimensions and the machine learning module is trained based on a set of simulated observed dimensions and a set of known vehicle dimensions. [8] The system of claim 7, wherein the machine learning model is an artificial neural network model. [9] The system of claim 8, wherein the machine learning model is a decision tree model. [10] The system of claim 9, wherein the set of simulated observed dimensions is generated by training a dimension-based decision tree based on the set of known vehicle dimensions and generating simulated vehicles based on the dimension-based decision tree.

Images