Method for Selecting a Route for Recording a Vehicle

Abstract

The present disclosure provides a method for selecting a route for recording a vehicle. The computer-implemented method, performed by a vehicle data recording device (140), includes: downloading a recording target from a host data collection system (200); determining a plurality of possible routes for the vehicle; generating, for each route, a route code that encodes information about a predicted value of the route as a numerical value for a plurality of metrics, the metrics being functions of assigning values representing an amount of progress toward achieving a basic recording target to data segments; providing the route code and additional environmental information independent of the route to a reinforcement learning agent, the reinforcement learning agent selecting one of the routes to optimize a reward; recording data from an in-vehicle data source (110) over time as the vehicle travels along the selected route; uploading at least a portion of the recorded data to the host data collection system, and receiving a reward in return for the reinforcement learning agent.

Description

Technical Field

[0001] The present disclosure relates to the field of data recording by a group of data-logging vehicles and uploading the recorded data to a host data collection system. The recorded data recorded by the recording vehicles while traveling along a route can include raw data captured by sensors, data derived from the raw data (e.g., object detection data), and / or external data (e.g., GPS data, weather data, etc.). In the automotive industry, such recorded data from the vehicles is collected for algorithm development and / or validation, such as for training neural networks, particularly in advanced driver assistance systems (ADAS) and in the field of autonomous driving. Background Art

[0002] Known methods for recording data for ADAS algorithm development and validation use a single vehicle or a group of vehicles equipped with the required sensor hardware, typically including radar, lidar, and cameras. The vehicle's driver then drives around the location to provide valuable and useful data for development and validation. As each vehicle drives, the sensor data is recorded onto a hard drive installed in the vehicle using a vehicle data recording device, such as disclosed in US Pat. No. 6,073,063A. After recording, the recording vehicle's hard drive is returned to a collection site for further distribution and storage.

[0003] The aforementioned known methods for collecting data from vehicles have numerous drawbacks. One drawback is that drivers often drive randomly or use their own judgment about where to drive and record interesting data. Consequently, the value of the recorded data is uncertain and may not be as useful as desired. Furthermore, some of the recorded data must be selected after recording, and a significant amount of uninteresting recorded data is discarded.

[0004] Therefore, there is a need to increase the amount of high-value data recorded along the route. Summary of the Invention

[0005] The present disclosure relates to a computer-implemented method performed by a vehicle data recording device in a vehicle, the method comprising the following steps:

[0006] downloading a recording target for recording data along a route from a host data collection system, determining a plurality of routes that the vehicle can take at a current time t0;

[0007] generating, for each route, a route code that encodes information about a predicted value of the route as a numerical value for a plurality of metrics that are functions of assigning values representing an amount of progress toward achieving a basic recording goal to data segments;

[0008] providing the route encoding and additional environmental information independent of the route Ri to a reinforcement learning agent, the reinforcement learning agent selecting one of the routes to optimize the reward;

[0009] recording data from a data source in the vehicle over time as the vehicle travels along a selected route;

[0010] uploading at least a portion of the recorded data to the host data collection system and receiving a reward in return from the host data collection system;

[0011] The reward is provided to the reinforcement learning agent.

[0012] This method allows a vehicle that has received a recording target to drive and record data along a route that is expected to optimize the reward for successfully uploading the recorded data to the host data collection system, rather than driving randomly. The reward is generated by the host collection system with the recording target in mind.

[0013] In an embodiment, the step of generating a route code for each route includes the following steps:

[0014] dividing the route into route segments, and

[0015] A feature vector is generated for each route segment, wherein environmental features associated with the route segment and related to the metric are encoded as numerical values to form components of the feature vector.

[0016] In an embodiment, the step of generating a feature vector for each route segment includes the step of encoding past information about content occurring just before the current time t0 into a numerical value to form another component of the feature vector.

[0017] The method may further include the following steps: for each route segment, inputting the corresponding feature vector into a machine learning system for predicting the values of the multiple metrics, and providing a metric value prediction vector at the output of the machine learning system, the metric value prediction vector including the predicted values of the route segment for the multiple metrics.

[0018] The metric value prediction vector for each route segment may also include uncertainty information.

[0019] In an embodiment, the metric value prediction vectors of route segments of respective routes are grouped into high-dimensional route codes, the high-dimensional route codes of the plurality of routes are converted into lower-dimensional fixed-length route codes, and the fixed-length route codes are provided as input to the reinforcement learning agent.

[0020] A selector may pre-select a short list of the most promising route encodings and may provide only the short list of route encodings as input to the reinforcement learning agent.

[0021] In an embodiment, the method further comprises, in a preliminary step:

[0022] Retrieving historical data associated with a plurality of records performed while driving along a given past route;

[0023] dividing each past route into route segments; and evaluating a value of each route segment for the plurality of metrics based on the historical data;

[0024] The machine learning system is trained to calculate evaluation values associated with a plurality of metrics for corresponding route segments.

[0025] The plurality of routes may be determined by using a tree expansion algorithm.

[0026] The present disclosure further relates to a vehicle data recording device, comprising:

[0027] at least one interface that receives logged data from a data source in the vehicle and sends the logged data for upload to a host data collection system;

[0028] A storage module, the storage module stores the recorded data; a processor, the processor controls the execution of the steps of the above method.

[0029] The present invention also relates to a computer program comprising instructions for causing the above-mentioned vehicle data recording device to execute the steps of the above-mentioned method.

[0030] The present disclosure also relates to a computer readable medium having stored thereon the computer program defined above. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the accompanying drawings.

[0032] Figure 1 A distributed system for collecting data from a recording vehicle into a host data collection system is shown according to an embodiment.

[0033] Figure 2 A functional block diagram of an in-vehicle data recording device according to an embodiment is shown.

[0034] Figure 3 A flow chart illustrating a method performed by an in-vehicle data logging device to log data from a vehicle and upload at least a portion of the logged data to a host data collection system, according to an embodiment. DETAILED DESCRIPTION

[0035] The present disclosure relates to a computer-implemented method performed by an in-vehicle data recording device 140 in a recording vehicle 100 for recording data while the vehicle 100 travels along a selected route and uploading at least a portion of the recorded data to a host data collection system 200. As will be explained in greater detail later, the route along which the data is recorded is selected among a plurality of routes to optimize rewards from the host data collection system 200 after and / or during the uploading of the recorded data.

[0036] Figure 1 A distributed system for collecting data from a group or multiple recording vehicles 100 by a host data collection system 200 is shown.

[0037] In an embodiment, the recording vehicle 100 may communicate wirelessly with the host data collection system 200, for example, via a mobile communication network 300 such as a 5G network.

[0038] The data from each recording vehicle 100 may include different types of data, such as raw data, derived data, and / or external data.The data collected from the vehicles 100 is recorded over time, typically as the vehicle travels along a route.

[0039] Raw data (also referred to as basic data or source data) is unprocessed data. Raw data includes, in a non-limiting manner, sensor data from sensors installed in the vehicle 100 (such as radar, lidar, and cameras), GPS data, speed data, yaw rate data, etc.

[0040] Derived data includes data derived from raw data through data processing. Data processing can be performed by hardware devices and / or software devices in the vehicle 100 that execute processing operations and / or algorithms (e.g., machine learning algorithms or any other type of algorithms). Some in-vehicle hardware and software components can perform object detection based on radar, lidar, or camera data (with or without sensor fusion), semantic segmentation on camera data, encoding of signatures describing various aspects of environmental information (in the vehicle environment), calculation of data structures, and / or statistical information output from object detection modules or other algorithms.

[0041] External data includes data from sources external to the vehicle 100. Such external data may be obtained by calling an API (for Application Programming Interface) of a third-party data provider via the mobile communication network 300. Non-limiting examples of such external data include traffic data (e.g., traffic flow, construction areas, etc.), weather service data, time-related data (e.g., national holidays, weekdays, weekends, etc.), map service data (e.g., road layout information, satellite imagery, mobile network coverage, etc.), current transmission rates through the mobile communication network 300, data from environmental sources (e.g., stores, schools, public transportation stations, parks, etc.), data from routing services, and data from the host data collection system 200.

[0042] The host data collection system 200 is responsible for providing a recording (collection) target T0, downloading (transmitting) it to a group of recording vehicles 100, and in return, collecting the recorded data collected by each vehicle in the group of vehicles 100. The host data collection system 200 is also responsible for sending reward information to each vehicle 100 after and / or when the vehicle 100 uploads the recorded data to the host data collection system 200.

[0043] Record object T0 defines one or more basic or atomic record (collection) objects c k , k = 0, ..., m, and includes the basic recording object c k One or more metrics related to (also called data value metrics). In this disclosure, metrics is a function that assigns a data value (in other words: a number) to a data segment corresponding to a given point in time i or corresponding to a given time slot, said data value representing the achievement of the corresponding basic recording objective c k In other words, each metric Allows measurement towards the base recording target c k Basic record keeping in Objective C k It can be a quantitative target defined by a given target amount or number. The following are illustrative and non-limiting examples of basic recording targets:

[0044] - kilometers on specific roads (i.e. motorway, city, ...);

[0045] - Specific recorded time spent in specific scenarios (i.e., X number of hours spent in traffic jams, daytime, twilight, nighttime, weekends, rush hour, ...);

[0046] - the number of unique objects of a specific class (i.e., bicycles, trucks, pedestrians, ...);

[0047] -Specific maneuvers (i.e. overtaking, lane change, etc.);

[0048] - the number of specific road scenarios (i.e. detours, specific intersection constellations, passenger intersections, construction areas, tunnels, bridges, ...);

[0049] - the number of scenarios selected by active learning methods or predictive models;

[0050] - Number of rarely detected scenes.

[0051] Optionally, the base record objective c can be constrained in some way k To diversify the data collection. For example, a constraint may be to perform data collection across multiple regions (countries, cities, ...). Another constraint may be to limit data collection along roads that have been recorded in the past.

[0052] Basic logging objective c k It may be in the form of code that is executed by a computer or processor.

[0053] The recording object T0 may include m+1 different basic recording objects c k The m+1 metrics involved (Forming the basic record object c k The group C: C={c0,...,c m}), where m≥0 and the metric index j ranges from 0 to m. In this case, the recording target T0 may also include a definition of how to record different basic recording targets c in the recording target T0. k The collection strategy P is weighted. The collection strategy P defines how to combine multiple basic measurements or metrics. In an embodiment, the collection strategy P will weight the corresponding Assign different measures In other words, the collection strategy P targets m+1 basic record targets c j The m+1 metrics involved Define m+1 corresponding weights Where 0≤j≤m. The collection strategy can be expressed as: Preferably, the set of weights Make

[0054] As explained previously, each metric is a function that assigns a data value to a recorded data point for time i (in other words, to a data segment for a given point in time i) in a record R. A record R is a set of data captured by the recording vehicle 100 (e.g., on a given route or during a given time period) and contains raw data provided by sensors in the recording vehicle 100 and / or data from other data sources in the recording vehicle 100 (including derived data and / or external data).

[0055] In addition, according to the collection strategy P, at all time points {t0,...,t n} has basically recorded the target {c0,...,c m The value V(RC) of the record RC of the data aggregated on} can be expressed as follows:

[0056]

[0057] In an embodiment, the metric Generates values within a normalized range (ie, [0, .., 1.]). In this case, the data value assigned to the recorded data point at time i is comprised between 0 and 1 (including the extreme values 0 and 1).

[0058] In the first illustrative example, the primary recording target is the number of overtaking maneuvers, and the value metric assigns a value of 1 when an overtaking maneuver occurs and a value of 0 when no overtaking maneuvers occur. In the second illustrative example, the primary recording target is the number of bicycles, and the value metric assigns a value representing the number of bicycles in the scene. This value is normalized between 0 and 1 using a mapping function, for example, based on the maximum number of bicycles or, alternatively, the average number. Normalization facilitates combining or aggregating data values calculated by different value metrics, as explained later.

[0059] The collection strategy P is used to aggregate records R in time, each of which consists of m+1 metrics The resulting m+1 sets of data values allow for generating a set of aggregated data values over time for a record R.

[0060] Reference Figure 1 , each recording vehicle 100 has a plurality of data sources (raw data, derived data and / or external data) 110 , a radio transmitter-receiver 120 , a central processing unit 130 and a data recording device 140 .

[0061] The data source 110 includes hardware and software components that provide raw data and / or derived data and optionally external data. The data source 110 can be of different types, for example:

[0062] - sensors (e.g., radar, lidar, and / or camera) that provide raw data (e.g., radar data, lidar data, and / or camera data),

[0063] - a calculator for processing data, receiving input data such as raw data and outputting derived data,

[0064] - a machine learning or algorithmic module that performs one or more tasks (e.g., object detection), receives input data (e.g., raw data), and outputs derived data,

[0065] - Functional modules for receiving external data via the network 300, etc.

[0066] The radio transmitter-receiver 120 is responsible for sending and receiving radio signals (including information and / or messages and / or data) via the mobile communication network 300 .

[0067] All elements 110 , 120 , 140 of the recording vehicle 100 are connected to a central processing unit 130 which controls their operation.

[0068] The in-vehicle data recording device 140 is responsible for:

[0069] - Based on the logging goals provided by the host data collection system 200, predicting and selecting a route that is expected to maximize (optimize) the reward for uploading logged data collected along the selected route;

[0070] - recording data from a data source 110 in the vehicle 100 as the vehicle 100 travels along the selected route;

[0071] - Upload (in other words: wirelessly transmit at least a portion of the recorded data to the host data collection system 200 via the mobile communication network 300);

[0072] - Receive a reward in return from the host data collection system 200.

[0073] The in-vehicle data recording device 140 has hardware units and software units (in other words: hardware components and software components) for implementing the method described below for recording data from the recording vehicle 100 and uploading at least a portion of the recorded data to the host data collection system 200 over the air.

[0074] In an embodiment, communication between the data logging device 140 and the host data collection system 200 is performed via the in-vehicle radio transmitter-receiver 120. Alternatively, the data logging device 140 may include a radio transmitter-receiver that communicates directly with the host data collection system 200.

[0075] like Figure 2 As shown, in an embodiment, the in-vehicle data recording device 140 has an input / output interface 141 that interfaces with the vehicle 100, a storage module 142, a route planning module 143, a data recorder 147, a recorded data uploader 148 and a central processing unit (CPU) 149.

[0076] The input / output interface 141 interfaces with the vehicle 100. It allows for the reception of data from the in-vehicle data source 110, messages received from the host data collection system 200 via the transmitter-receiver 120 of the vehicle 100, and the transmission of data to be uploaded to the host data collection system 200 via the transmitter-receiver of the vehicle 100, as will be explained later.

[0077] The storage module 142 stores the logged data collected from the vehicle 100 before uploading at least a portion of the logged data to the host data collection system 200 .

[0078] The route planning module 143 is responsible for planning candidate routes for the vehicle 100 according to the recording target T0. The planned candidate routes are expected to provide high-value recording data for the recording target T0. The route planning module 143 has two components: the first component is a route encoder 144, and the second component is a route selector 146.

[0079] Optionally, the route planning module 143 includes a route preselector 145 that selects a short list of routes encoded by the first component 144 to provide only the short list to the route selector 146 .

[0080] The first component (Route Encoder) 144 is responsible for computing the route code. The route code is a representation of the route. It can be represented by a numeric vector that represents the route for a number of metrics. The value information of the route is encoded, as explained in more detail later. In an embodiment, the route encoder 144 includes a vehicle navigation system 1440 , a segmentation block 1441 , a feature encoder 1442 , a value estimator 1443 , a concatenator 1444 and a vector converter 1445 .

[0081] The second component (route selector) 146 is responsible for selecting candidate routes from the list of routes associated with the route code determined by the route encoder 144 to optimize (in other words, maximize) the reward received by the host data collection system 200 during or after the upload of the recorded data. In an embodiment, the route selector 146 can be implemented by a reinforcement learning agent to perform the task of selecting candidate routes. The route selector 146 can consider additional environmental information independent of the route. The term "independent" here means independent of static route characteristics compared to dynamic characteristics such as weather. The reinforcement learning agent that is rewarded for uploading information may be very interested in having 5G band available to optimize its function.

[0082] Environmental factors may include 5G band availability, how much gas is available in the vehicle, etc. The reinforcement learning agent is given some metrics, information about what is available or not available (e.g. how much gas is in the vehicle, mobile network bandwidth availability, etc.).

[0083] The route preselector 145 has the function of preselecting the most promising route subset (short list) among all possible routes Ri encoded by the route encoder 144 .

[0084] In an embodiment, the route encoder 144 , the route preselector 145 , and the route selector 146 comprise software or program instructions to cause the vehicle data recorder 140 to perform the steps of the method now described. The route encoder 144 and the route selector 146 run on a processor 149 .

[0085] Now refer to Figure 3 A computer-implemented method for recording data by the in-vehicle data recording device 140 is described according to an embodiment.

[0086] In an initial step S1, the host data collection system 200 provides a logging target T0 for logging data by a logging vehicle. In an embodiment, the logging target T0 includes m+1 data value metrics for assigning data values to the logged data over time. Thus, the measurement is performed on the realization of m+1 corresponding elementary (atomic) recording targets c k The progress of k ranges from 0 to m. A set of m+1 corresponding basic record targets {c0,...,c m} is marked as C. In this embodiment, the data value metric Normalized by a mapping function that assigns values in the range [0, ..., 1]. In addition, the recording target T0 includes a collection strategy P ,That The corresponding weight Assigning data values to metrics m+1 weights The sum is equal to 1.

[0087] In step S2, the host data collection system 200 wirelessly transmits the recording target T0 to each recording vehicle 100 of a group (or multiple vehicles) (not necessarily to all vehicles 100 simultaneously) via the mobile communication network 300. In an embodiment, the recording target T0 is transmitted to each recording vehicle 100 in response to a request from the vehicle 100. Alternatively, the recording target T0 is pushed toward the recording vehicle 100.

[0088] In step S3, in the vehicle 100, the vehicle data recording device 140 downloads (in other words, receives) the recording target T0 from the host data collection system 200 via the mobile network 300. In this embodiment, the recording target T0 is received via the radio transmitter-receiver 120 of the vehicle 100 and the I / O interface 141 of the recording device 140. The recording target T0 is provided as input to the path planning module 143.

[0089] Then, the vehicle data recording device 140 performs a path planning task for the vehicle 100. The path planning operation includes the following steps S4 to S11 performed by the path planning module 143. Steps S4 to S11 will be described with respect to one vehicle 100, but steps S4 to S11 can be performed by each recording vehicle 100 after receiving the recording target T0.

[0090] The route encoder 144 first calculates the route codes for the plurality of routes Ri in steps S4 to S9. As mentioned above, the route code of a given route Ri is a combination of the route codes of the plurality of metrics of the route Ri. The information of the predicted value is encoded into a numerical value.

[0091] In step S4, at the current time t0, the vehicle navigation system 1440 determines a plurality of routes Ri that the vehicle 100 could take. In this embodiment, the determination of the routes Ri is performed by executing a tree expansion algorithm. The root of the tree can advantageously be the current location of the vehicle 100. The vehicle navigation system explores all possible routes and constructs a tree of possible routes Ri for the vehicle 100. Each branch of the tree represents a specific route Ri. At some point, the algorithm stops expanding the tree based on a given stopping criterion (e.g., distance, estimated driving time, or any other appropriate criterion).

[0092] In an embodiment, in step S5, the segmentation block 1441 divides each route Ri determined in step S4 into route segments RSij. The segmentation of the route Ri is performed by following predetermined semantic boundaries (such as intersections, environmental features, distance, time, navigation system level, or a combination thereof). Such segmentation allows the characteristics of the route Ri to be encoded at a finer level.

[0093] In step S6, the feature encoder 1442 generates a feature vector Vij for each route segment RSij of each route Ri. In other words, each route segment RSij is described as a feature vector Vij. A feature vector can also be considered a feature point in a multidimensional space, where each dimension corresponds to a given feature class or category used to describe the route Ri. The feature vector Vij of a route segment RSij is a numerical representation (in other words: a description) of what can be observed along the route segment RSij. In an embodiment, step S6 includes three sub-steps S60 to S62, as described below.

[0094] In step S60, feature encoder 1442 encodes past information about the content of the recorded vehicle 100 prior to the current time t0 into a numerical value to form a first component Aij of feature vector Vij. In practice, at time t0, data recording device 140 has already recorded past data from vehicle 100 within the last few minutes and therefore knows the past data. The past information may include a summary of the recorded data collected by the in-vehicle data recording device 140 during the past period up to the current time t0. For example, the recorded data may include at least a portion of the following information:

[0095] a) Environmental information, including:

[0096] - Information derived from the vehicle's GPS location, such as features of the neighborhood (e.g., dining and drinking, groceries, shopping, trading, parks, schools, cultural and recreational areas, etc.), transportation-related information (e.g., train stations, public transportation, trams, other), bicycle facilities, map-derived features (e.g., encoding of satellite imagery from GPS location, encoding of street map imagery from GPS location, etc.);

[0097] Environmental context information, such as time of day, weather information, object presence and distribution (e.g., construction areas, potholes, debris, pedestrians, motor vehicles, trucks, motorcycles, bicycles, etc.), and ambient sound information (e.g., sirens, accidents, ambient sound profiles, etc.);

[0098] - Environmental context, such as urban, highway, or rural, road conditions (dry, wet, snowy, other; asphalt, other; missing line markings, yellow line markings, etc.), driving behavior information (e.g., lane changing, braking, acceleration, wheel turning, etc.);

[0099] b) Vehicle information (e.g., speed, yaw rate, remaining fuel level, signals related to feature functions, such as lane change warning, etc.).

[0100] The purpose of encoding past recorded data (or a summary of past recorded data) collected just before the current time point t0 is to capture specific conditions and / or environmental characteristics relevant to the current time period, such as weather information, the type of route driven by vehicle 100, the number of pedestrians on the street, or any other relevant past information recently detected, which may influence the value estimator 1443's near-term prediction or estimate of route Ri. The recorded data collected during the past time period up to time t0 can be aggregated, for example, by adding some values to obtain a summary of the recorded data and limit the amount of data in the components Aij of vector Vij. The information about the past encoded in the first component Aij of vector Vij can be very informative to the machine learning system (value estimator) 1443. For example, suppose that at time t0, it is nighttime and raining, and vehicle 100 is driving in a city near a store. Typically, in a city, there are many pedestrians near a store. However, because the machine learning module 1443 knows from the last few minutes that it is raining and nighttime, it may not expect as many pedestrians. If the metric is associated with a pedestrian, the machine learning model 1443 uses the pieces of information from the summary of what has happened in the last few minutes that have been encoded in the components Aij of the feature vector Vij to better evaluate the value of the segment for the metric associated with the pedestrian.

[0101] In step S61, the feature encoder 1442 associates the route segment RSij with the metric The relevant environmental features are encoded into numerical values to form the second component Bij of the vector Vij. The encoded environmental features of the route segment RSij include any type of features that may be related to the metric (in other words: any type of features that can be related to the metric). For example, the environmental features include at least a portion of the environmental information previously described. It may include the type of environment (e.g., city, highway, country, other), the number of nearby stores, the presence of nearby schools, intersections, weather information (e.g., from a weather forecast service or current weather), etc. As an illustrative example, let us consider the following metric, which is a function for determining the number of pedestrians in the surrounding environment. In this case, features such as the number of nearby stores, the presence of nearby schools, and any other segment features that may be related to the presence of pedestrians can be encoded in the second component Bij of the vector Vij.

[0102] Finally, in step S62 , the first component Aij and the second component Bij are concatenated to form a feature vector Vij of the route segment RSij.

[0103] Then, in step S7, the feature vectors Vij of each route segment RSij of each route Ri are input into the value estimator 1443. The value estimator 1443 may include a machine learning system that is trained to evaluate the value of a plurality of predetermined metrics. (In other words: for a given set of metrics ) evaluates the corresponding value of the route segment RSij described by the feature vector. The metric of the value estimation is performed by the value estimator 1443 The value estimator 1443 outputs a metric value prediction vector for each route segment RSij, which includes the metric value prediction vector for the route segment RSij for multiple predetermined metrics. The predicted values are advantageously normalized over the segment distance to obtain the data value density.

[0104] Optionally, the value estimator 1443 may also output uncertainty information about the predicted value. The uncertainty information may include the uncertainty of the predicted value for all metrics.

[0105] In step S8, at the output of the value estimator 1443, the data value densities (and optionally uncertainty information) of all route segments RSij of each route Ri are concatenated by a concatenator 1444 to form a high-dimensional route encoding ENCi for each route Ri. At this stage, the route encoding ENCi for each route Ri has many dimensions because, for each segment RSij of the route Ri, it includes Furthermore, since routes Ri can have different numbers of route segments RSij, the sizes of route codes ENCi can be different from each other.

[0106] In an embodiment, in step S9, the high-dimensional route encoding ENCi is converted (in other words, transformed) by the transformer 1445 into a lower-dimensional fixed-length route encoding ENCi'. The transformer 1445 decomposes the high-dimensional route encoding ENCi of each route Ri into a denser encoding ENCi', forming a fixed-length description of the entire route Ri in a normalized manner. This allows the encoding of each route Ri to be transformed into an encoding that can be used by the reinforcement learning agent 146 for route selection. The length of the route encoding ENCi' corresponds to the fixed input dimensionality of the reinforcement learning agent 146. In an embodiment, the encoding transformation aggregates the segments to form a route encoding consisting of three components: a short-term component, a medium-term component, and a long-term component, corresponding to three distance intervals (short, medium, and long) from the current position of the vehicle 100. For example, the short-term component can correspond to a distance of approximately 100 meters, the medium-term component can correspond to a distance of approximately 1000 meters, and the long-term component can correspond to a distance greater than 1000 meters. However, any other distance values can be used. Any other aggregation of route encodings ENCi can be used to find an appropriate representation for the reinforcement learning agent.

[0107] Optionally, in step S10, the route preselector 145 preselects a subset of candidate routes. Forming the route encoding into a normalized representation can integrate expected rewards and route characteristics for short-term, medium-term, and long-term scenarios. Therefore, the route preselector 145 can use the normalized route encoding to select candidate routes. For example, the route preselector 145 can only select the most valuable routes. Alternatively, the best candidate route for each recording target can be selected. The uncertainty of the predicted recording value may affect the preselected candidates. As an illustrative and non-limiting example, only the five most promising routes Ri can be selected.

[0108] In step S11, the fixed-length route encoding ENCi' of the route Ri preselected in step S10 (or the fixed-length route encoding ENCi' of all routes Ri if step S10 is not performed) and additional context information independent of the route Ri are provided as input to the reinforcement learning agent or route selector 146. This additional context information is information independent of the features encoded to generate the feature vector Vij of the route segment RSij. For example, it may include information about the availability of 5G bandwidth determined by the vehicle 100, information about the remaining gas in the vehicle 100, information about the remaining recording time (e.g., given by the driver), information about the recording buffer status (i.e., the remaining recording space in the buffer), or any other information independent of (different from) the encoded route features and relevant to selecting the route that the vehicle 100 will take to record data and upload the recorded data to the host data collection system 200. The reinforcement learning agent 146 then uses the fixed-length route code ENCi' and additional environmental information as input to select a route Rs from the pre-selected routes Ri that is expected to maximize or optimize the reward from the host data collection system 200 for uploading the recorded data collected along the selected route Rs in response to the received recording target T0. In one embodiment, the reinforcement learning agent has the option to stay on the current route. In other words, the reinforcement learning agent can choose to stay on the current route or switch to one of the proposed new routes.

[0109] In step S12 , when the reinforcement learning agent has decided to change the current route, the path planning module provides the selected route Rs to the vehicle 100 through the interface 141 .

[0110] Then, in step S12 , the vehicle 100 travels along the selected route Rs, and the data recording device 140 records data along the route Rs. As previously described, the recorded data is received from the vehicle 100 and may include different types, such as raw data, derived data, and / or external data.

[0111] In step S13, the recorded data collected (recorded) in step S12 is sent to the host data collection system 200. In an embodiment, only a portion of the recorded data is uploaded. If only bad data (low values) are recorded, it may be decided not to upload the collected recorded data.

[0112] In step S14 , the host data collection system 200 receives the successfully uploaded recording data from the recording vehicle 100 , and the host data collection system 200 stores the received recording data in the storage facility 400 .

[0113] Furthermore, in step S15, the master data collection system 200 generates reward information as a reward for successfully uploading the recorded data from the vehicle 100, and transmits the reward information to the vehicle 100 (more specifically, to the data recording device 140). The reward information may include different types of rewards to reward different types of actions, such as the action of continuously uploading recorded data, the action of completely successfully uploading records, the action of recording and uploading high-value recorded data for the recording target T0, etc.

[0114] The vehicle data recorder 140 receives the reward information (here via the transmitter-receiver 120 of the vehicle 100) and provides it to the reinforcement learning agent 146 to improve route selection.

[0115] In an embodiment, the host data collection system 200 does not need to give a reward. After a successful upload, it is sufficient to simulate the reward behavior in the vehicle 100.

[0116] The machine learning system 1443 is trained in a preliminary step by using historical (past) record data collected by the recording vehicles traveling along the route. In an embodiment, first, the historical records relate to a plurality of records performed in the past by one or more recording vehicles traveling along different routes. Training data is generated based on the historical data. To this end, each route traveled in the past is divided into segments (as in step S5), feature vectors for each route segment are generated (as in step S6), and the historical record data is used to analyze the training data for a plurality of predetermined metrics. Calculate the value of each route segment. The training data set includes the feature vectors of all route segments obtained from the routes traveled in the past as input training data, and the measurement The corresponding segment aggregate average is used as the output training data. The dataset of training data is provided to the machine learning system 1443 for training to calculate calculated values related to the corresponding route segments for multiple metrics. After being trained, the machine learning system 1443 is able to estimate (predict) the recorded value density of each route segment for each metric.

[0117] The machine learning system 1443 may be an aggregation model that is responsible for calculating all metrics for each route segment. Alternatively, the machine learning system 1443 includes N models to calculate corresponding N metrics.

[0118] The reinforcement learning agent of route selector 146 is trained to decide which of multiple suggested routes should be selected. The training of the reinforcement learning agent of route selector 146 can use data as described above. The input training data for the reinforcement learning agent includes route codes determined based on simulation data, preferably normalized route codes, and the output training data includes a score or value indicating how much the reinforcement learning agent wants to choose a particular route suggestion from the presented candidate routes.

Claims

1. A computer-implemented method performed by a vehicle data recording device (140) in a vehicle (100), the method comprising the following steps: downloading (S3) a recording target (T0) for recording data along the route from a host data collection system (200), determining (S4) a plurality of routes (Ri) that the vehicle (100) can take at a current time t0; For each route (Ri), a route code (ENCi) is generated, which is based on multiple metrics The information about the predicted value of the route (Ri) is encoded as a numerical value, the metric is a function that assigns to a data segment a value representing the amount of progress toward the basic recording goal; providing the route encoding (ENCi') and additional environmental information independent of the routes (Ri) to a reinforcement learning agent (146), which selects one of the routes (Ri) to optimize reward; recording (S12) data from a data source (110) in the vehicle (100) over time as the vehicle travels along a selected route (Rs); uploading (S13) at least a portion of the recorded data to the host data collection system (200), and receiving a reward from the host data collection system (200) in return; The reward is provided (S16) to the reinforcement learning agent (146).

2. The method according to claim 1, wherein The steps of generating a route code (ENCi) for each route (Ri) include the following steps: dividing (S5) the route into route segments (RSij), and For each route segment (RSij), a feature vector (Vij) is generated, wherein the feature vector (Vij) is associated with the route segment (RSij) and is related to the metric The relevant environmental features are encoded as numerical values to form the components (Bij) of the feature vector (Vij).

3. The method according to claim 2, wherein: The step of generating a feature vector (Vij) for each route segment (RSij) comprises the step of encoding past information about what happened just before said current time t0 into a numerical value to form another component (Aij) of said feature vector (Vij).

4. The method according to claim 2 or 3, further comprising the following step (S7): for each route segment (RSij), inputting the corresponding feature vector (Vij) into the in a machine learning system (1443) for predicting the values of the route segments (RSij), and providing a metric value prediction vector at the output of the machine learning system (1443), the metric value prediction vector comprising predicted values of the route segments (RSij) for the plurality of metrics.

5. The method according to claim 4, wherein The metric value prediction vector for each route segment (RSij) also includes uncertainty information.

6. The method according to claim 4, wherein: The metric value prediction vectors of the route segments (RSij) of the respective routes (Ri) are grouped into high-dimensional route codes (ENCi), the high-dimensional route codes (ENCi) of the plurality of routes (Ri) are converted into lower-dimensional fixed-length route codes (ENCi'), and the fixed-length route codes (ENCi') are provided as input to the reinforcement learning agent (146).

7. The method according to claim 1, wherein A selector (145) preselects (S10) a short list of the most promising route encodings (ENCi') and provides only said short list of said route encodings (ENC') as input to said reinforcement learning agent (146).

8. The method according to claim 4, further comprising: In the preliminary steps: Retrieving historical data associated with a plurality of records performed while driving along a given past route; Divide each past route into route segments (RSij); and Based on the historical data for the multiple metrics Evaluate the value of each route segment (RSij); The machine learning system (1443) is trained to calculate evaluation values associated with a plurality of metrics for corresponding route segments.

9. The method according to claim 1, wherein The plurality of routes (Ri) are determined by using a tree expansion algorithm.

10. A vehicle data recording device, comprising: at least one interface that receives logged data from a data source in the vehicle and sends the logged data for upload to a host data collection system; a storage module, wherein the storage module stores the recorded data; A processor controlling the execution of the steps of the method according to any one of claims 1 to 9.

11. A computer-readable medium storing a computer program which, when executed by a computer or a processor, causes a vehicle data recording device to perform the method according to any one of claims 1 to 9. 12 . A vehicle, comprising the vehicle data recording device according to claim 10 .

Images