Unsupervised Metadata Generation for Vehicle Data Logs

Abstract

A method for performing teacherless metadata generation for vehicle data includes the steps of receiving vehicle data collected during driving by a vehicle, where the vehicle data includes position data, speed data, and timestamps of the position data and the speed data; using the vehicle data to define a map route corresponding to the driving within map data; using the map route to determine metadata about the driving; and annotating the vehicle data using the determined metadata.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims priority to U.S. Provisional Patent Application No. 63 / 366,729, filed on June 21, 2022, entitled "UNSUPERVISED METADATA GENERATION FOR VEHICLE DATA LOGS", the disclosure of which is hereby incorporated by reference in its entirety.

[0002] This document relates to unsupervised metadata generation for vehicle data logs.

Background Art

[0003] Some of the vehicles manufactured today are equipped with one or more types of systems that can at least partially process actions related to the operation of the vehicle. Some of these aids include the ability to automatically survey the area around the vehicle and take actions regarding detected vehicles, pedestrians, or objects. Such systems are developed, in part, by collecting a significant amount of data from the moving vehicle. The data collected can initially be characterized as raw data in that it does not include any metadata that would explain the type of driving or any of the events that may have occurred during which the data was collected.

[0004] One approach for generating metadata for autonomous driving data sets is manual review by human annotators. A human reviewer examines the content of a data log, such as a video recording, and flags it if relevant criteria are met. This approach is slow and / or requires significant resources. Additionally, the results may be biased by the subjectivity of the person performing the review.

[0005] As another example, metadata about a dataset has been created using a trained computer vision model. In this supervised approach, a model trained with respect to a set of specific tasks such as detecting toll booths or motorcycles can be run against a recorded video feed. Detected objects can be recorded for each input video frame. This approach involves first performing advanced training of a machine learning model, which also requires access to a large set of annotated vehicle data. Thus, the process of generating metadata about the vehicle dataset is not avoided by using the model.

SUMMARY OF THE INVENTION

[0006] In one aspect, a method for performing unsupervised metadata generation for vehicle data includes receiving vehicle data collected during driving by a vehicle, the vehicle data including position data, speed data, and timestamps of the position data and the speed data; determining a map route corresponding to the driving in map data using the vehicle data; determining metadata about the driving using the map route; and annotating the vehicle data using the determined metadata.

[0007] The implementation may include any or all of the following features. The method may further comprise the step of downsampling the position data of the received vehicle data, and the downsampled position data is used when determining the map route. The method may further comprise the step of filtering the map data before determining the map route, and only the remaining map data is used when determining the map route. The filtering step is based on a fixed distance from the vehicle during travel, and the fixed distance is substantially perpendicular to the direction of travel. The method may further comprise the step of upsampling the map data before determining the map route, and the map route is determined within the upsampled map data. The step of determining the metadata includes the step of reading the metadata from the map data. The step of determining the metadata includes the step of calculating the metadata from the map data. The metadata includes at least one selected from the group consisting of the number of lanes, the presence or absence of highway ramps, the presence or absence of toll booths, road curvature, traffic data, the presence or absence of bridges, the presence or absence of tunnels, or road surface materials. The method may further comprise the step of presenting at least a portion of the determined metadata to a graphical user interface. The portion of the determined metadata is presented to a person performing annotation of the vehicle data, and presenting the portion of the determined metadata includes storing the portion of the determined metadata in an input control within the graphical user interface. The method may further comprise the step of using the determined metadata to determine whether a person should perform annotation of the vehicle data. The method may further comprise the step of making the annotated determined metadata available for selection by a person developing an advanced driver assistance system for the vehicle.

Brief Description of the Drawings

[0008]

Figure 1

[0009]

Figure 2

[0010]

Figure 3

[0011]

Figure 4

Figure 5

Figure 6

Figure 7

[0012]

Figure 8

[0013]

Figure 9

[0014] Like reference numerals in the various drawings indicate like elements.

DETAILED DESCRIPTION OF THE INVENTION

[0015] This document describes examples of systems and techniques that can provide probabilistic map matching for teacherless metadata generation. This subject can apply map matching to annotate autonomous driving data or, if not, assisted driving data. Thereby, a means can be provided to determine which data from a significant amount of vehicle data collected during driving is targeted for the purpose of developing an advanced driver assistance system (ADAS). A scene selector tool may be provided that enables developers to select from among various driving situations reflected in a large-scale vehicle data set. ADAS developers with access to a large amount of vehicle data may need to investigate issues such as where in the ADAS algorithm under development is functioning well or where improvements may be needed; whether good performance by the ADAS algorithm on the vehicle data set indicates that the feature requirements are met; or where the ADAS developer should collect more vehicle driving data (e.g., which types of traffic scenes are missing or sparsely represented in the current data set).

[0016] The present subject matter may be associated with one or more of the following advantages. In some implementations, the present subject matter can: (i) automatically review a data log collected by a vehicle and determine where a target event occurred; (ii) determine metadata about a vehicle data log more quickly than existing techniques that are human annotation and avoid complex training of models associated with supervised techniques; (iii) enable a developer to select a particular scenario from a database and determine whether individual records should be reviewed by a human annotator and / or analyze algorithm performance under various road conditions and environmental factors; (iv) curate a large dataset collected for autonomous driving development; (v) more efficiently identify potential logs with rare scenarios; or (vi) when working with camera sensors, not be affected by common environmental challenges such as headlight glare or low light conditions.

[0017] The present subject matter may operate on a vehicle data log that includes vehicle position data. As used herein, position data includes, but is not limited to, satellite-based position coordinates such as Global Positioning System (GPS) coordinates. The system can match the GPS coordinates with the most likely route that the vehicle follows, where the route data is extracted from a road network database. The map matching process may consider probabilistic factors including GPS sensor noise, inaccuracies in the road network database, and road connectivity information. The system does not need to know the destination of the vehicle or receive direct input from the user. Corresponding metadata can be extracted from the most likely route to determine attributes such as the number of driving lanes, the speed limit of the road, and the type of paved road material. Additionally, using the original map attributes and GPS coordinates, additional metadata fields such as local road curvature, traffic conditions, and when the vehicle passes toll booths, bridges, and highway ramps can be calculated.

[0018] The scene selector tool according to this subject matter can create metadata for raw vehicle data by matching the vehicle position with map data (e.g., open source maps). The metadata can be automatically extracted or determined based on the matching. Such metadata can include, but is not limited to, for example, the presence or absence of tunnels, bridges, toll plazas, or ramps; road curvature or number of lanes; traffic conditions, paved road materials, or speed limits; or dawn / dusk timing, geographical area, or road type. According to this subject matter, map matching can be performed using a probabilistic method that determines the most likely sequence of map waypoints that match the vehicle's GPS measurements or other position data. The matching process can consider one or more relevant elements including, but not limited to, the connectivity of the road network or the position and direction of travel of the vehicle relative to the road. Once the most accurate route is determined with respect to the map data, the attributes of the metadata can be extracted from the map and / or calculated from the data.

[0019] In contrast to this subject matter, existing consumer navigation applications do not extract and derive metadata information along the route being traversed. For example, this subject matter can consider and record when a vehicle passes through a highway ramp. The highway ramp does not have to be part of the route being traveled, but the presence of the ramp can suggest interesting road geometries relevant to cognitive applications for autonomous driving.

[0020] Also, this subject matter is at least different from the road horizon approach in that the metadata extracted by the system can generate more information than the underlying map database. For example, traffic conditions can be estimated based on the vehicle speed relative to the speed limit, or potential low light conditions can be noted based on the sunset time at the input GPS coordinates.

[0021] Examples in this specification refer to vehicles. A vehicle is a machine that transports passengers or cargo, or both. A vehicle can have one or more motors that use at least one type of fuel or other energy source (e.g., electricity). Examples of vehicles include, but are not limited to, cars, trucks, and buses. The number of wheels can vary between vehicle types, and one or more (e.g., all) of the wheels can be used for vehicle propulsion, or the vehicle may not have power (e.g., when a trailer is attached to another vehicle). A vehicle can include a passenger compartment that accommodates one or more persons.

[0022] Examples in this specification refer to ADAS. In some implementations, ADAS can perform assisted driving and / or autonomous driving. ADAS can at least partially automate one or more dynamic driving tasks. ADAS can operate, at least in part, based on the output of one or more sensors that are typically positioned on, under, or within a vehicle (sometimes referred to as the "ego vehicle"). ADAS can plan one or more trajectories of the vehicle before and / or while controlling the movement of the vehicle. The planned trajectory can define a path for the vehicle to travel. Thus, propelling the vehicle according to the planned trajectory can correspond to controlling one or more aspects of the vehicle's behavior of operation, such as, but not limited to, the vehicle's steering angle, gear (e.g., forward or reverse), speed, acceleration, and / or braking.

[0023] An autonomous vehicle is an example of an ADAS, but not all ADASs are designed to provide fully autonomous vehicles. Multiple levels of driving automation have been defined by SAE International, and are commonly referred to as levels 0, 1, 2, 3, 4, and 5, respectively. For example, a level 0 system or driving mode may not include continuous vehicle control by the system. For example, a level 1 system or driving mode may include adaptive cruise control, emergency brake assist, automatic emergency brake assist, lane keeping, and / or lane centering. For example, a level 2 system or driving mode may include highway assist, autonomous obstacle avoidance, and / or autonomous parking. For example, a level 3 or 4 system or driving mode may include incremental control of the vehicle by an assistive driving system. For example, a level 5 system or driving mode may not require human intervention in the assistive driving system.

[0024] FIG. 1 shows an example of a graphical user interface (GUI) 100 that presents metadata determined about vehicle data. The GUI 100 may be used with one or more other examples described elsewhere in this specification. The GUI 100 can include a video area 102. In some implementations, the video area 102 can present still or moving images of a drive that the vehicle made while the vehicle was collecting a data log. For example, the GUI 100 may be used by a human reviewing the vehicle data log (e.g., to perform metadata annotation of the vehicle data). Here, the video area 102 shows a landscape captured in the driving direction indicating that the reporting vehicle is currently on a highway (e.g., a road that is part of the U.S. Interstate Highway System).

[0025] The GUI 100 includes a metadata area 104 that can present one or more types of metadata related to at least vehicle data logs. Such metadata may be determined at least partially automatically using vehicle data logs and other information according to this subject matter. As another example, a human annotator may enter some metadata or edit any of the metadata provided by the system. The metadata area 104 includes metadata packaged for user consumption. Such packaged metadata can have any of multiple levels of granularity. For example, frame-level metadata (e.g., related to one or more specific frames (e.g., image frames) of vehicle data) can provide fine-grained labels about when an event or transition occurred during the driving of the reported vehicle. As another example, session-level metadata (e.g., related to an entire driving session) can provide schematic tags suitable for overview data filtering.

[0026] Any type of metadata related to the driving of the vehicle can be presented in the metadata area 104. Here, the metadata includes the following exemplary types of metadata applicable at the moment of the vehicle driving currently presented by the GUI 100: Vehicle position data (e.g., GPS coordinates) Number of lanes of the road Speed of the reported vehicle Whether a highway ramp is nearby State in which the reported vehicle is driving Amount of curvature on the road Speed limit of the road Whether the reported vehicle is inside a tunnel Name of the road Curvature type of the road Surface type of the road Whether the reported vehicle is on a bridge Traffic flow parameters of the road, or Whether the reported vehicle is at a toll booth on the road.

[0027] Other types of metadata related to driving may be used additionally or alternatively.

[0028] The metadata presented in metadata area 104 regarding the driving of the reporting vehicle may be determined based at least on vehicle data collected during driving and map data. In summary, the most appropriate map route corresponding to the driving may be determined based on map matching and may be used when extracting metadata from the map data or, if not, when calculating or determining it.

[0029] Figure 2 shows an example of a GUI 200 of an annotation tool that may use the metadata determined for vehicle data. The GUI 200 may be used together with one or more other examples described elsewhere in this specification. For example, the GUI 200 may be used to input and / or edit metadata related to any or all frames of a vehicle data log related to a driving session.

[0030] The GUI 200 may include at least one metadata description 202. The metadata description 202 may correspond to any metadata of the subject matter, including but not limited to any of the types of metadata mentioned above. For example, the metadata description 202 here indicates that the metadata is related to whether the reporting vehicle is currently located inside a tunnel.

[0031] The GUI 200 may include at least one input control 204 for metadata description 202. Any of a plurality of types of input controls that are compatible with the GUI 200 may be used. For example, the input control 204 provides for input by selecting from an existing list (sometimes called a drop-down list). The input control 204 indicates a metadata value 206 representing the current input. For example, the metadata value 206 indicates that the metadata about whether the vehicle is currently located inside a tunnel currently has the value False (i.e., the reporting vehicle is not currently inside the tunnel). Other values, or types of values, may be used.

[0032] That is, the GUI 200 may include a significant number of instances of the input control 204 (e.g., at least one instance for each type of metadata mentioned above). If a user utilizes the GUI 200 to annotate a vehicle data log according to a method used in the past (i.e., without the advantages of the present subject matter), the user may need to enter values into each of the input controls 204. This can be extremely time-consuming and may be susceptible to human error. Here, in contrast, at least one of the input controls 204 may store a metadata value 206 automatically determined according to the present subject matter. For example, the value "False" may be automatically selected (or entered if not) within the input control 204. This may enable the user to quickly review each metadata value being entered and make changes only if the user believes a different value should be used.

[0033] FIG. 3 shows an example of a GUI 300 of a scene selection tool that may enable a user to select from vehicle data based on metadata about the vehicle data. The GUI 300 may be used with one or more other examples described elsewhere in this specification. One purpose of the scene selection tool is to enable an ADAS developer to select some aspects or characteristics of vehicle data from a vast amount of vehicle logs.

[0034] The GUI 300 may include at least one metadata definition 302. The metadata definition 302 may correspond to any metadata of the present subject matter, including but not limited to any of the types of metadata mentioned above. For example, the metadata definition 302 here defines the metadata that the reporting vehicle is currently located inside a tunnel.

[0035] The GUI 300 may include at least one input control 304 for the metadata definition 302. Any of a plurality of types of input controls that are compatible with the GUI 300 may be used. In some implementations, the input control 304 provides for input by specifying (e.g., by typing or selecting) a percentage of the amount of vehicle log data to be characterized by the metadata definition 302, and the remainder of the vehicle log data is specified by another metadata definition or left unspecified. For example, if an ADAS developer uses the input control 304 to specify a value 306, the data retrieved from the vehicle data log will have an amount corresponding to the value 306 among the vehicle log data corresponding to the metadata definition 302. Other parameters other than the percentage may be used instead or additionally. For example, the user may specify the amount of data to retrieve (e.g., by driving time or number of miles).

[0036] Figures 4-7 show exemplary diagrams 400, 500, 600, and 700 illustrating unsupervised metadata generation for vehicle data logs. FIG. 8 schematically shows an example 800 of unsupervised metadata generation for vehicle data logs. Each of these examples may be used in conjunction with one or more other examples described elsewhere in this specification.

[0037] Each of diagrams 400, 500, 600, and 700 shows a map position with respect to a vertical axis labeled northward and a horizontal axis labeled eastward. Other position coordinates may be used instead or additionally. For example, a conversion between the northward / eastward and latitude and longitude coordinates of a GPS system may be performed.

[0038] The map data of diagrams 400, 500, 600, and 700 may be any of a plurality of types of maps. In some implementations, the map includes standard accuracy map data. For example, such map data can represent a multi-lane highway by a single curve in each direction of travel. One advantage of using relatively low-resolution map data is that this data may be available almost globally for all roads in the world, thereby enabling ADAS development to consider virtually all locales where a vehicle may be expected to travel. In some implementations, the map includes high-precision map data. In either approach, the map data can include waypoints and relatively approximate GPS coordinates, and optionally can also include some metadata about the map waypoints or other structures.

[0039] The original (unfiltered) map content may cover the entire area on the Earth, but in practice, the reporting vehicle usually only travels over a part of that map area. In the filtering operation 806, the original map data 802 can be filtered based on the position data to generate relevant map data 804. Here, the map content currently shown in diagram 400 is only the area 402 of diagram 400. That is, the map content 404, which is part of the original map data, has been removed by filtering and thus is not shown in diagram 400, and the area 402 is the remaining map data that will be used when defining the map route.

[0040] From the filtered map, a road network structure and a connectivity graph are created. Such created information may relate to how some or all of the waypoints of the relevant map data 804 are related to each other. In some implementations, the relevant map data 804 includes the definition of a road as a series of map waypoints, and names are given to the start and end points of that series. However, such map data may not convey any intuitive meaning about which roads of the map data are connected to each other (such as in the case of a highway exit lane attached somewhere along the highway length). Therefore, the map data can be processed to define a structure and / or graph regarding the nature of the roads shown by the map. This information can be useful because connectivity is also important when the reporting vehicle crosses between different roads and highways and regarding perception in the context of ADAS development. For example, lane detection or object detection implemented by ADAS may depend on the perception algorithm.

[0041] That is, the present subject matter can decode a data log to extract position data, a timestamp, and the speed of the reporting vehicle. By filtering out all of the map content except for region 402, a selection of map content relevant to the particular vehicle data being analyzed can be represented. Thus, region 402 can be the target area created along the track followed by the vehicle. Such a track can be defined using the position data of the vehicle data log. The filtering can be based on a fixed distance from the reporting vehicle while in motion. For example, the fixed distance 406 can be defined as being substantially perpendicular to the direction of travel of the reporting vehicle. The fixed distance 406 can extend to both sides of the position of the reporting vehicle. Then, the area traversed by the fixed distance 406 can define region 402. For example, region 402 can be defined using two-dimensional coordinates (e.g., as a polygon).

[0042] Here, the track 502 of diagram 500 represents the position data (e.g., GPS coordinates) where the reporting vehicle has traveled. In some implementations, the track 502 results from downsampling high-frequency position data to a lower frequency. The downsampling operation 812 can leave the shape of the reporting vehicle's route intact while reducing the computational cost. The original position data within the vehicle data 808 having a frequency of about 20 per second can be downsampled by the downsampling operation 812 to position data within the downsampled vehicle data 810 having a frequency of about 1 per second.

[0043] The waypoints of the map data (e.g., in the associated map data 804) may be upsampled. In some implementations, interpolation may be performed in operation 812 to generate upsampled map data 814. For example, this can strive to ensure that the minimum spacing guarantee is met. Diagram 600 shows a trajectory 502 along with an area 402' that includes upsampled map content. In some implementations, the content of diagram 600 represents an input to a process that attempts to identify the most likely map waypoints for a given route of vehicle position data.

[0044] One or more operations 816 may be performed using the downsampled vehicle data 810 and the upsampled map data 814 to determine the most appropriate route 818 of the reporting vehicle. In some implementations, probabilistic map matching may be performed. A joint probability distribution between the position data of the reporting vehicle and the associated map data may be calculated. The road network structure and / or connectivity graph defined for the map data may be considered. For example, the probability that the position coordinates of a route point match a particular waypoint of the map can be maximized. A probabilistic algorithm can be applied to select a map waypoint that is considered to be the most likely map waypoint when given the GPS coordinates or other position data of the reporting vehicle. That is, the most appropriate route 818 represents the route along which the reporting vehicle is most likely to be traveling with respect to the waypoints originally obtained from the map data 802. In diagram 700, the most appropriate route 702 is shown, and the most appropriate route 702 is defined with respect to map waypoints as opposed to position data (e.g., GPS coordinates obtained from a vehicle log).

[0045] The following is a specific example regarding operation 816. The map matching algorithm can consider the best map waypoint associated with each downsampled position data (e.g., GPS measurement). First, the matching process can remove low-probability matchings such as highly unlikely far-apart matchings or roads on which the reported vehicle is traveling far above its speed limit. For example, if a waypoint is associated with a speed limit of 25 mph (40 kph) and the reported vehicle is currently traveling at 80 mph (129 kph), this tends to make it less likely that the waypoint matches the position data. Next, the map matching algorithm can assign high probabilities to measurements and map waypoints that are close to each other. The closeness metric can include multiple heuristics. For example, waypoints above and below an overpass are spatially close to each other but far apart in terms of height, road direction of travel, and navigation distance. A sequence of the most likely road waypoints given the measurements can be generated from the algorithm. To confirm that the algorithm has produced a valid result, the final solution may be checked by a series of validity checks.

[0046] The present subject matter can apply probabilistic map matching to annotate autonomous driving data or, if not, assisted driving data. In conventional approaches, probabilistic map matching has been used in consumer navigation applications. These applications are typically run from a smartphone or similar device, where the user requests navigation directions to an entered destination. The application then has to determine on which road the user is located and the corresponding route from the starting point to the entered destination. As another example, probabilistic map matching has been used in mapping and position estimation applications for autonomous driving. One common architecture includes a system that determines on which road a vehicle is traveling and the outlook of the coming road on which the vehicle is likely to travel. Information about the user's ultimate destination is not required but can assist the system. These systems report the most likely route and the fields known at the time the road was mapped.

[0047] The original map attributes and the most likely route are associated with each other. This provides basic metadata information such as the name of the road the vehicle is following. The most likely route, the road network, and the map database may be further processed for more complex metadata fields. A criteria search around the driving path can be applied to check whether toll booths, bridges, highway ramps, and / or other features are present. The traffic situation can be appraised by comparing the vehicle speed with the speed limit at the associated waypoint. The curvature of local roads can be calculated from the shape of the most likely route. Dawn, dusk, and other time-of-day categories can be generated according to the log timestamp and solar events for the associated geographical location. Weather data can be obtained based on the position data and the timestamp. Other techniques for calculating or otherwise determining the metadata applicable to the most appropriate route 818 may be additionally or alternatively used.

[0048] One or more operations 820 are shown for determining metadata 822 for the most appropriate route 818. In some implementations, operations 820 include reading metadata from map data according to the waypoints of the most appropriate route 818. In some implementations, operations 820 include calculating metadata from one or more available information sources, as mentioned above for example.

[0049] Metadata 822 may be used for one or more purposes. In some implementations, the metadata may be provided for use in ADAS development. For example, the GUI 300 of FIG. 3 may enable a developer to select aspects of a vehicle data log based on portions of the associated metadata. In some implementations, the metadata may be used in determining whether to perform human annotation of vehicle data logs. For example, if an ADAS developer is seeking more vehicle log data for driving in a tunnel or across a bridge, metadata 822 may be used to determine whether to have humans work on the collected vehicle data, such as to label other vehicles with bounding boxes and assign lane markers. In some implementations, metadata 822 may be used to classify data or store entries in tools used by humans when entering metadata. For example, one or more values may be automatically entered into the GUI 200 (FIG. 2) to simplify and speed up work by the user.

[0050] FIG. 9 shows an exemplary architecture of a computing device 900 that may be used to implement aspects of the present disclosure, including any of the systems, apparatuses, and / or techniques described herein, or any other systems, apparatuses, and / or techniques that may be utilized in various possible embodiments.

[0051] The computing device shown in FIG. 9 can be used to execute the operating system, application programs, and / or software modules (including software engines) described herein.

[0052] In some embodiments, computing device 900 includes at least one processing device 902 (e.g., a processor), such as a central processing unit (CPU). Various processing devices are available from various manufacturers, such as Intel or Advanced Micro Devices. In this example, computing device 900 also includes a system memory 904 and a system bus 906 that couples various system components including system memory 904 to processing device 902. System bus 906 is one of any number of types of bus structures that can be used, including, but not limited to, a memory bus or memory controller using any of a variety of bus architectures; a peripheral bus; and a local bus.

[0053] Examples of computing devices that can be implemented using computing device 900 include desktop computers, laptop computers, tablet computers, mobile computing devices (such as smartphones, touchpad mobile digital devices, or other mobile devices), or other devices configured to process digital instructions.

[0054] System memory 904 includes read-only memory 908 and random access memory 910. A basic input / output system 912, including basic routines that function to transfer information within computing device 900 during startup and the like, can be stored in read-only memory 908.

[0055] In some embodiments, computing device 900 also includes a secondary storage device 914, such as a hard disk drive, for storing digital data. The secondary storage device 914 is connected to the system bus 906 by a secondary storage interface 916. The secondary storage device 914 and its associated computer-readable medium provide non-volatile and non-transitory storage of computer-readable instructions (including application programs and program modules), data structures, and other data for the computing device 900 (e.g., a non-transitory computer-readable storage medium).

[0056] Although a hard disk drive is used as the secondary storage device in the exemplary environment described herein, in other embodiments other types of computer-readable storage media are used. Examples of these other types of computer-readable storage media include magnetic cassettes, flash memory cards, solid state drives (SSDs), digital video disks, Bernoulli cartridges, compact disk read-only memories, digital versatile disk read-only memories, random access memories or read-only memories. Some embodiments include non-transitory media. For example, a computer program product may be tangibly embodied in a non-transitory storage medium. Additionally, such computer-readable storage media may include local storage or cloud-based storage.

[0057] Multiple program modules may be stored in system memory 904, which includes secondary storage device 914, and / or an operating system 918, one or more application programs 920, other program modules 922 (such as software engines described herein) and program data 924. The computing device 900 may utilize any suitable operating system.

[0058] In some embodiments, a user provides input to computing device 900 through one or more input devices 926. Examples of input devices 926 include keyboard 928, mouse 930, microphone 932 (for voice and / or other audio input), touch sensor 934 (such as a touch pad or touch-sensitive display), and gesture sensor 935 (for gesture input, for example). In some implementations, input device 926 provides detection based on presence, proximity, and / or movement. Other embodiments include other input devices 926. The input device can be connected to processing device 902 through an input / output interface 936 coupled to system bus 906. These input devices 926 can be connected by any number of input / output interfaces (such as a parallel port, serial port, game port, or universal serial bus). In some possible embodiments, wireless communication between input device 926 and input / output interface 936 is also possible, including, to name only a few examples, infrared, BLUETOOTH® wireless technology, 802.11a / b / g / n, cellular, ultra-wideband (UWB), ZigBee®, or other radio frequency communication systems.

[0059] In this exemplary embodiment, a display device 938, such as a monitor, liquid crystal display device, light emitting diode display device, projector, or touch-sensitive display device, is also connected to system bus 906 via an interface such as video adapter 940. Computing device 900 can include various other peripheral devices (not shown), such as speakers or printers, in addition to display device 938.

[0060] Computing device 900 may be connected to one or more networks through network interface 942. Network interface 942 can provide wired and / or wireless communication. In some implementations, network interface 942 may include one or more antennas for transmitting and / or receiving wireless signals. When used in a local area networking environment or a wide area networking environment such as the Internet, network interface 942 may include an Ethernet interface. Other possible embodiments use other communication devices. For example, some embodiments of computing device 900 include a modem for communicating over a network.

[0061] Computing device 900 may include at least some form of computer-readable medium. Computer-readable medium includes any available medium that can be accessed by computing device 900. By way of example, computer-readable medium includes computer-readable storage medium and computer-readable communication medium.

[0062] Computer-readable storage medium is implemented in any device configured to store information such as computer-readable instructions, data structures, program modules, or other data, including volatile and non-volatile, removable and non-removable media. Computer-readable storage medium includes random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory, or other memory technologies, compact disc read-only memory, digital versatile disc or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be accessed by computing device 900 and used to store desired information, but is not limited thereto.

[0063] A computer-readable communication medium typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transmission mechanism, and includes any information transmission medium. The term "modulated data signal" refers to a signal whose one or more of its characteristics are set or changed in such a way as to encode information in the signal. By way of example, a computer-readable communication medium includes wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, radio frequency, infrared, and other wireless media. Any combination of the foregoing is also included within the scope of computer-readable media.

[0064] The computing device shown in FIG. 9 is also an example of a programmable electronic device that may include one or more such computing devices, and when multiple computing devices are included, these computing devices may be collectively coupled to a suitable data communication network to collectively implement the various functions, methods, or operations disclosed herein.

[0065] In some implementations, the computing device 900 may be characterized as an ADAS computer. For example, the computing device 900 may include one or more components that may be used to process tasks arising in the field of artificial intelligence (AI). The computing device 900 then includes sufficient processing power and the necessary support architecture for ADAS or general AI requirements. For example, the processing device 902 may include a multi-core architecture. As another example, the computing device 900 may include one or more coprocessors in addition to, or as part of, the processing device 902. In some implementations, at least one hardware accelerator may be coupled to the system bus 906. For example, a graphics processing unit may be used. In some implementations, the computing device 900 may implement neural network-specific hardware to process one or more ADAS tasks.

[0066] As used throughout this specification, the terms "substantially" and "about" are used to account for and consider minor variations, such as those due to variations in processing. For example, they can refer to less than or equal to ±5%, such as less than or equal to ±2%, such as less than or equal to ±1%, such as less than or equal to ±0.5%, such as less than or equal to ±0.2%, such as less than or equal to ±0.1%, such as less than or equal to ±0.05%. Also, as used herein, indefinite articles such as "a" or "an" mean "at least one".

[0067] It is to be understood that all combinations of the above concepts and additional concepts discussed in more detail below (subject to these concepts not being mutually inconsistent) are contemplated as being part of the subject matter of the invention disclosed herein. In particular, all combinations of the claimed subject matter that appear at the end of this disclosure are contemplated as being part of the subject matter of the invention disclosed herein.

[0068] Multiple implementations have been described. Nevertheless, it will be understood that various modifications can be made without departing from the spirit and scope of this specification.

[0069] Additionally, the logical flow shown in the figures does not require the particular order shown, or sequential order, to achieve the desired result. Additionally, other processes may be provided, or processes may be eliminated from the flow described, other components may be added to the systems described, or other components may be removed from the systems described. Accordingly, other implementations are within the scope of the following claims.

[0070] Some features of the described implementations have been shown as described herein, but now many modifications, substitutions, changes, and equivalents will occur to those skilled in the art. Accordingly, it should be understood that the appended claims are intended to cover all such modifications and changes that fall within the scope of the implementations. They are presented by way of example only and not limitation, and it should be understood that various changes in form and detail may be made. Except for mutually exclusive combinations, any part of the devices and / or methods described herein may be combined in any combination. The implementations described herein may include various combinations and / or sub-combinations of the functions, components, and / or features of the various implementations described.

Claims

1. A method for performing teacherless metadata generation for vehicle data, comprising: Receiving vehicle data collected during driving by a vehicle, the vehicle data including position data, speed data, and timestamps of the position data and the speed data; Determining a map route corresponding to the driving within map data using the vehicle data; Determining metadata about the driving using the map route; and Annotating the vehicle data using the determined metadata. A method comprising the above steps.

2. The method according to claim 1, further comprising downsampling the position data of the received vehicle data, wherein the downsampled position data is used when determining the map route.

3. The method according to claim 1 or 2, further comprising filtering the map data before determining the map route, wherein only the remaining map data is used when determining the map route.

4. The method according to claim 3, wherein the filtering step is based on a fixed distance from the vehicle during the driving, and the fixed distance is substantially perpendicular to the direction of the driving.

5. The method according to claim 1 or 2, further comprising upsampling the map data before determining the map route, wherein the map route is determined within the upsampled map data.

6. The method according to claim 1 or 2, wherein the step of determining the metadata includes reading the metadata from the map data.

7. The method according to claim 1 or 2, wherein the step of determining the metadata includes calculating the metadata from the map data.

8. The method according to claim 1 or 2, wherein the metadata includes at least one selected from the group consisting of the number of lanes, the presence or absence of highway ramps, the presence or absence of toll booths, road curvature, traffic data, the presence or absence of bridges, the presence or absence of tunnels, or road surface materials.

9. The method according to claim 1 or 2, further comprising presenting at least a portion of the determined metadata to a graphical user interface.

10. The portion of the determined metadata is presented to a human who performs annotation of the vehicle data, and the step of presenting the portion of the determined metadata includes storing the portion of the determined metadata in an input control within the graphical user interface. The method according to claim 9

11. The method according to claim 1 or 2, further comprising determining, using the determined metadata, whether a human should perform annotation of the vehicle data

12. The method according to claim 1 or 2, further comprising making the annotated determined metadata available for selection by a person developing an advanced driver assistance system for the vehicle

13. To at least one processor, A procedure for receiving vehicle data collected while the vehicle is in motion, the vehicle data including position data, speed data, and time stamps of the position data and the speed data; A procedure for determining a map route corresponding to the travel within map data using the vehicle data; A procedure for determining metadata about the travel using the map route; and A procedure for annotating the vehicle data using the determined metadata A program for causing the operations to be executed

14. The operations further include a procedure for filtering the map data before determining the map route, and only the remaining map data is used when determining the map route. The program according to claim 13

15. The filtering procedure is based on a fixed distance from the vehicle during the travel, and the fixed distance is substantially perpendicular to the direction of the travel. The program according to claim 14

16. The procedure for determining the metadata includes a procedure for reading the metadata from the map data. The program according to claim 13 or 14

17. The procedure for determining the metadata includes a procedure for calculating the metadata from the map data. The program according to claim 13 or 14

18. The program according to claim 13 or 14, wherein the metadata includes at least one selected from the group consisting of the number of lanes, the presence or absence of a highway ramp, the presence or absence of a toll gate, road curvature, traffic data, the presence or absence of a bridge, the presence or absence of a tunnel, or road surface material.

19. The program according to claim 13 or 14, wherein the operation further includes a procedure of presenting at least a part of the determined metadata to a graphical user interface.

20. The program according to claim 19, wherein the part of the determined metadata is presented to a person who performs annotation of the vehicle data, and the procedure of presenting the part of the determined metadata includes a procedure of storing the part of the determined metadata in an input control within the graphical user interface.

Images