Methods for training a machine learning model, methods for controlling a vehicle function, methods for manufacturing a motor vehicle, and motor vehicle

Abstract

Embodiments of the present disclosure provide a method for training a machine learning (ML) model to determine the position of a user device relative to a vehicle of a predetermined vehicle model as output. The position is determined based on a characteristic measured for the user device as input. The method comprises retraining an ML model, pre-trained on initial training data, using a second set of training data. The initial training data represents training measurements of the characteristic at a multitude of positions relative to the vehicle for different vehicle models. The second training data represents training measurements of the characteristic only for the predetermined vehicle model. Furthermore, the second training data represents the training measurements of the characteristic at a multitude of positions of the predetermined vehicle model.

Description

[0001] Embodiments of the present disclosure relate to a method for training a machine learning model, a method for controlling a vehicle function, a method for manufacturing a motor vehicle, and a motor vehicle.

[0002] Digital keys are vehicle keys stored on a mobile device (or user device). For example, a user device can be used to unlock a car, start an engine, or perform similar functions. This can be achieved by measuring an NFC (Near Field Communication) signal, a UWB (Ultra-Wideband) signal, or similar technology within the vehicle to determine whether the user device is inside or outside (or near) the vehicle.

[0003] There is a need to provide an improved method for recognizing a digital key. The methods and the vehicle address this need according to the independent claims.

[0004] One aspect concerns a method for training a machine learning (ML) model to determine the position of a user device relative to a vehicle of a predetermined model as output. The position is determined based on a characteristic measured for the user device as input. The method involves retraining an ML model, pre-trained on initial training data, using a second set of training data. The initial training data represents training measurements of the characteristic at a multitude of positions relative to the vehicle for various vehicle models. The second set of training data represents training measurements of the characteristic specifically for the predetermined vehicle model. Furthermore, the second set of training data represents the training measurements of the characteristic at a multitude of positions within the predetermined vehicle model.This allows for a more accurate model to be provided for determining the position of a user device.

[0005] In some examples, the pre-trained machine learning model is retrained based on the second set of training data to improve the determination of the user device's position relative to the vehicle of the predetermined vehicle model in a boundary region of the predetermined vehicle model. This allows for the provision of a model that provides more accurate predictions in the boundary region. Furthermore, such a model can be trained specifically for the boundary region with less effort and fewer data points.

[0006] In some examples, the characteristic is signal strength. This allows existing antennas to be used to receive a signal for position determination.

[0007] In some examples, the position of the user device relative to the vehicle indicates whether the user device is inside or outside the vehicle. This allows for a simple determination of whether, for example, a vehicle function should be activated that depends on whether the user device is inside or outside, such as starting the vehicle.

[0008] In some examples, the second training data represents the training measurements of the characteristic at a multitude of positions within a boundary region of the predetermined vehicle model. This boundary region defines an interior area of ​​the vehicle. The second data can thus be acquired in a concentrated manner for this boundary region.

[0009] In some examples, retraining involves fine-tuning the pre-trained machine learning model. This allows the use of an existing model and minimizes the retraining effort. Furthermore, retraining can focus on specific areas of the vehicle where the pre-trained machine learning model is inaccurate.

[0010] In some examples, retraining involves using metadata as input data before using the second training data, thus at least partially improving outputs with insufficient accuracy from the pre-trained ML model. This can further reduce the amount of data required for retraining.

[0011] In some examples, the machine learning model is pre-trained based on a regression about a coordinate origin relative to the vehicle, so that the position of the user device relative to the vehicle indicates the position of the user device about the coordinate origin. This avoids classification based on continuous data, which can be inaccurate.

[0012] A second aspect concerns a method for controlling a vehicle function. This method involves determining a first position relative to the vehicle based on a trained machine learning model that uses a characteristic measured for the user device as input. The method further includes applying a Kalman filter to determine a second position if, based on the first position, it is determined that the user device is outside the vehicle. The method also includes controlling the vehicle function based on the first position if it is determined that the user device is inside the vehicle, or based on the second position if it is determined that the user device is outside the vehicle. Such a method allows for more precise control of the vehicle function.

[0013] In some examples, the second position is determined for at least two consecutive points in time to identify whether the user device is moving towards or away from the vehicle. This allows the system to determine which vehicle function should be activated or controlled.

[0014] In some examples, the vehicle function includes at least one of the following: unlocking the vehicle and starting the vehicle. Thus, different vehicle functions can be considered and selected depending on the outcome of the procedure.

[0015] In some examples, the machine learning model used in the method for controlling the vehicle function was trained according to a method for training a machine learning model as disclosed herein. Thus, the machine learning model trained according to the first aspect can be used to further increase the accuracy of the position determination.

[0016] A third aspect concerns a method for manufacturing a vehicle. This method involves inserting into a vehicle of a predetermined vehicle model a data set representing a machine learning (ML) model trained using a method for training an ML model according to the present disclosure. The method further includes configuring the vehicle to determine the position of a user device relative to the vehicle using the trained ML model. This allows the same effects to be achieved as with the methods of the present disclosure.

[0017] A fourth aspect concerns a non-volatile, machine-readable medium on which a data set representing a machine learning model is stored. This model has been trained according to a method for training a machine learning model as disclosed herein. The machine learning model can thus be hard-coded into the vehicle. Furthermore, the same effects as those achieved with the methods disclosed herein can be obtained.

[0018] A fifth aspect concerns a vehicle comprising a non-volatile, machine-readable medium according to the present disclosure. Thus, a vehicle can be provided with an ML model according to the present disclosure. Furthermore, the same effects can be achieved as with the methods of the present disclosure.

[0019] Examples of implementation are explained in more detail below with reference to the accompanying figures. These show: Fig. 1 a schematic representation of a method for training a machine learning model according to the present disclosure; Fig. 2 a flowchart of a method for controlling a vehicle function according to the present disclosure; Fig. 3 a flowchart of a process for manufacturing a vehicle according to the present disclosure; and Fig. 4 a schematic representation of a vehicle according to the present disclosure.

[0020] Several embodiments are now described in more detail with reference to the accompanying drawings, in which some of these embodiments are illustrated. For the sake of clarity, the thickness dimensions of lines, layers, and / or regions may be exaggerated in the figures.

[0021] The Fig. Figure 1 illustrates a schematic representation of a method 1 for training a machine learning model according to the present disclosure. The embodiments described herein mainly describe the case where a pre-trained model already exists, which is then retrained with specific data to improve the pre-trained machine learning model. However, the present disclosure also explicitly covers the case of pre-training the model based on generic data, as discussed below.

[0022] A machine learning (ML) model can refer to a data structure and / or a set of rules representing a statistical model used to determine the position of a user device relative to a vehicle, as disclosed herein. The data structure and / or set of rules represents learned knowledge (e.g., based on training performed by an ML algorithm, as described herein). In machine learning, instead of a rule-based transformation of data, a transformation derived from an analysis of training data can be used.

[0023] The machine learning (ML) model can be trained based on an ML algorithm. The term ML algorithm refers to a set of instructions used to create, train, or use an ML model. By training the ML model with a large set of training data and associated information, the ML model learns to determine the position of the user device relative to the vehicle.

[0024] The machine learning (ML) model can be trained using training input data. For example, the ML model can be trained using supervised learning. In supervised learning, the ML model is trained with a variety of input samples, where each sample contains a variety of input data values ​​and a variety of output values; that is, each sample is associated with the desired output value. This allows the ML model to learn which output value to generate based on a given input value.

[0025] Alternatively or additionally, semi-supervised learning can be used. In semi-supervised learning, some output values ​​are missing. (Semi-)supervised learning can be based on a specific algorithm (e.g., a classification algorithm, a discriminative learning algorithm, a similarity learning algorithm, or the like). A classification algorithm can be used when the desired output of the model should be limited to a specific set of values ​​(e.g., category variables); that is, the input is classified into a limited set of values. Similarity learning algorithms are similar to classification algorithms but are based on learning from examples that use a similarity function, which measures how similar two objects are.

[0026] As an alternative or in addition to supervised or semi-supervised learning, unsupervised learning can be used. Here, only input data is provided, and an unsupervised learning algorithm is used to find a structure in the input data (e.g., by grouping or clustering). Clustering refers to the assignment of input data into subsets (clusters) so that input values ​​within the same cluster are similar, and predefined similarity criteria may exist.

[0027] Reinforcement learning represents another group of machine learning algorithms. Here, so-called software agents are trained to perform an action in a given environment. A reward is determined based on the action taken. Reinforcement learning relies on training the software agents to choose actions in a way that maximizes cumulative rewards, resulting in software agents that become increasingly better at the task at hand.

[0028] Additional techniques can be applied to the machine learning algorithms. For example, feature learning can be used, at least partially. Such algorithms can preserve the information at the input but transform it so that it can be used by another algorithm (e.g., for classification).

[0029] The machine learning model can be based on an artificial neural network (ANN). ANNs are systems inspired by biological networks, such as those found in the retina or brain. ANNs contain a multitude of interconnected nodes and a multitude of connections (edges) between the nodes. The number of nodes can vary, for example, three: input nodes (which receive the input data), hidden nodes (which are only connected to other nodes), and output nodes (which output the values). Each node can represent an artificial neuron. Each edge can transmit information from one node to another. The output of a node can be represented as a (non-linear) function of its inputs. The inputs of a node can be incorporated into the function as the weight of the node or the edge.The weight of the nodes / edges can be adjusted during the learning process. In other words, training an ANN can involve adjusting the weights to produce a desired output.

[0030] Alternatively, the ML model can be a different structure, such as a support vector machine, a random forest model, a gradient boosting model, or the like. Alternatively, the ML model can be based on a genetic algorithm that mimics natural selection.

[0031] The ML model can also be a combination of different aspects of the examples discussed here. For example, the ML model can be pre-trained with a first ML algorithm, and retraining takes place with a second ML algorithm. In other examples, the ML model is pre- and re-trained with the same ML algorithm.

[0032] As stated above, the ML model is set up to determine the position of a user device relative to a vehicle of a predetermined vehicle model as output.

[0033] The position can, for example, include a distance to the vehicle (or to a specific point within the vehicle). Alternatively or additionally, the position can include coordinates (two- or three-dimensional) of the user device with respect to a coordinate origin inside the vehicle. Alternatively or additionally, the position can include a binary statement about whether the user device is inside or outside the vehicle.

[0034] Position detection can be used, for example, to activate a vehicle function. If it is detected that the user device is inside the vehicle, the vehicle function can include starting the vehicle. If it is detected that the user device is outside the vehicle and moving away from it, the vehicle function can include locking and / or switching off the vehicle. If it is detected that the user device is outside the vehicle and moving towards it, the vehicle function can include unlocking the vehicle. However, the present disclosure is not limited to a specific vehicle function.

[0035] The user device includes, for example, a portable device that stores or accesses a digital vehicle key and is worn by the user, such as a mobile phone (e.g., smartphone), a smartwatch, smart glasses, smart headphones, smart contact lenses, or the like. The user device may also include a (physical) key. This disclosure is not limited to a specific type of user device, as long as it is configured to exhibit a characteristic that can be measured by sensors in or on the vehicle.

[0036] Such a characteristic is representative, for example, of a radio signal emitted by the user device, such as a UWB signal, a Bluetooth signal, an NFC signal, or the like. In some examples, the characteristic is representative of an infrared or near-infrared signal. A vehicle accordingly has one or more antennas capable of receiving such a signal. In the case of UWB, a vehicle might have, for example, 7 to 10 antennas configured to communicate with the user device in real time (e.g., 3 frames per second) or to receive signals from the user device.

[0037] The ML model uses such a characteristic measured for the user device as input and, based on this, outputs the position of the user device relative to the vehicle.

[0038] The vehicle includes, for example, a motor vehicle such as a passenger car, a truck, a van, a bus, or the like. However, the present disclosure is not limited to this case. For example, the vehicle may include a watercraft, an aircraft, or a non-motorized land vehicle.

[0039] The method according to the present disclosure comprises a retraining, 2, of a ML model pre-trained on the basis of first training data on the basis of second training data, as already discussed above.

[0040] The initial training data can represent a generic dataset collected for different vehicle models. This initial training data represents training measurements of the characteristic for the different vehicle models, where the characteristic was obtained at a multitude of positions relative to the various vehicles of the respective models. For example, a multitude of user devices can be used, each emitting the characteristic for the different vehicle models at a multitude of positions around the respective vehicle. This allows for the training of an initial (rough) machine learning model to predict the position of user devices relative to the respective vehicle models.

[0041] However, it was recognized that such a coarse model can be inaccurate in individual cases, as different vehicle models may have different geometries, different materials, a different number of antennas for receiving the signal, or similar characteristics (a similar situation can arise for different user devices that can emit different characteristics, such as signal strengths). One way to obtain a more accurate model would be, for example, to create a separate machine learning model for each vehicle model. However, this would mean that a large number of vehicles would have to be used for training each vehicle model, which would be very time-consuming and resource-intensive.

[0042] Accordingly, the present disclosure proposes to retrain a generic model (i.e., a machine learning model pre-trained on initial training data) based on a second set of training data. The second set of training data can then be obtained for a few vehicles (e.g., only one vehicle) of the vehicle model.

[0043] Accordingly, the second training data represents training measurements of the characteristic only for the predetermined vehicle model.

[0044] Furthermore, the second training data represent the training measurements of the characteristic at a variety of positions of the predetermined vehicle model, such as inside the vehicle of the predetermined vehicle model, outside the vehicle of the predetermined vehicle model, in a border region, or the like.

[0045] As discussed above, this type of retraining allows you to obtain a machine learning model for the predetermined vehicle model by using only a few (e.g., one) vehicles of that model for training. Furthermore, the data for retraining can be collected only at specific points within the vehicle model. This significantly reduces the dataset required for the second training dataset compared to conventional methods, saving (personnel) resources, data collection time, and costs. For example, conventional methods can require more than ten thousand data samples per vehicle model in just one UWB channel. This necessitates a large number of people dedicated solely to collecting a multitude of labeled data sets from various user devices for the different vehicle models.

[0046] In some examples, the pre-trained ML model is retrained based on the second training data to improve the determination of the user device's position relative to the vehicle of the predetermined vehicle model in a boundary region of the predetermined vehicle model.

[0047] The boundary region, for example, defines the interior of the vehicle. For instance, the boundary region includes all vehicle components that separate the interior from the outside world, as well as all points that are less than 50 cm, 40 cm, 30 cm, 20 cm, or 10 cm away from these components (and are located, for example, within the interior). For example, the boundary region includes an area of ​​a vehicle door. If the user device is located in the area of ​​the vehicle door (for example, in a door pocket or door compartment), the pre-trained machine learning model may not be able to determine whether the user device is inside or outside the vehicle.If the user device is located near a windshield, in a trunk, in a sunglasses compartment, or the like, the pre-trained ML model may also be unable to detect whether the user device is inside or outside the vehicle.

[0048] Accordingly, the determination of the user device's position relative to the vehicle in the boundary region is improved for the predetermined vehicle model. Here, the second training data can exclusively represent user device positions in the boundary region, further reducing the amount of data required for the second training data. On the other hand, the focus can be placed on the boundary region, and more data can be provided for this area. For example, if the user device is located near windows or the vehicle roof, it can happen, especially with UWB signals, that a large portion of the signal reaches the exterior of the vehicle because it is not reflected back into the interior by the windows or roof.In the pre-trained model, it can happen that the position of the user device is determined to be outside the vehicle, even though the user is inside the vehicle.

[0049] In some examples, the characteristic is a signal strength. However, the present disclosure is not limited to this. For example, the characteristic could also be the distance between the user device and a sensor. Such a distance could be determined, for example, based on a time-of-flight measurement. Furthermore, the characteristic could be the angle of incidence of the signal. In some embodiments, the characteristic comprises combinations of signal strength, distance, and angle.

[0050] In some examples, the position of the user device relative to the vehicle indicates whether the user device is located inside or outside the vehicle, as already discussed herein.

[0051] For example, the model can be trained (pre- and post-trained) to make this statement. In other examples, the model determines a precise position, which is evaluated by a (different) algorithm to determine whether the user device is inside or outside the designated area.

[0052] In some examples, the second set of training data represents the training measurements of the characteristic at a multitude of positions in a (or the) boundary region of the predetermined vehicle model. The boundary region, as previously discussed, defines the interior of the vehicle.

[0053] The boundary region used for training can correspond to the boundary region discussed above. However, depending on the use case, these can also differ. For example, even within the same vehicle model, the vehicle geometries can vary slightly, which is why the boundary region of the training data can differ from the boundary region used for (final) position determination.

[0054] In some examples, retraining can involve fine-tuning the pre-trained machine learning (ML) model. Model fine-tuning can utilize transfer learning to reduce the amount of data used for training. If a pre-trained model is used that represents a pool of existing data from a variety of different vehicle models, the model fine-tuning can be performed based on a comparatively small dataset. As discussed above, sufficient data may be collected for only one vehicle of the predetermined vehicle model to improve the determination of the user device's position. Accordingly, it is sufficient to collect data for the boundary region and fine-tune the pre-trained ML model for this boundary region.

[0055] The same applies to the number of user devices that need to be used for training: it may be sufficient to use only one user device of a predetermined type of user device to collect enough data to retrain the ML model.

[0056] If a neural network is used to implement the ML model, fine-tuning can involve updating all pre-trained model parameters (full fine-tuning) or updating a subset of the layers of the neural network (partial fine-tuning).

[0057] For example, the machine learning model can be pre-trained with a first vehicle and fine-tuned with a second vehicle (the target vehicle). Alternatively, the machine learning model can be pre-trained with a variety of vehicles (e.g., including the target vehicle) and fine-tuned with the target vehicle.

[0058] In some examples, retraining involves using metadata as input data before using the second training dataset. Using metadata can at least partially improve outputs that are insufficiently accurate in the pre-trained machine learning model.

[0059] For example, especially in borderline regions, the pre-trained model might produce a blurry prediction, meaning outputs with an uncertainty above a predetermined threshold. To improve these blurry predictions, the pre-trained model can be specialized based on metadata. Such metadata includes, for example, sensor positions (e.g., from FBD) in the vehicle relative to the borderline region, the vehicle material (or the vehicle body and / or trim and / or roof), or similar information. Alternatively or additionally, the metadata could include the positions, dimensions, and / or material of vehicle windows. Alternatively or additionally, the metadata could include the vehicle type, manufacturer, brand, or similar information.Alternatively or additionally, the metadata can include a vehicle type, such as coupé, station wagon, SUV (street utility vehicle), convertible, off-road vehicle, sedan, or the like. Alternatively or additionally, the metadata can include the number of doors and / or windows.

[0060] In some examples, the ML model is pre-trained based on a regression about a coordinate origin relative to the vehicle, so that the position of the user device relative to the vehicle indicates a position of the user device about the coordinate origin.

[0061] As discussed above, the position can be determined based on coordinates. In such examples, a regression algorithm can be used instead of a classification algorithm (which makes the statement "inside or outside"). Classification with hard boundaries based on continuous input data can cause difficulties for machine learning, resulting in the boundary region not being correctly (or very vaguely) identified.

[0062] In a regression algorithm, the origin of the coordinate system can be positioned, for example, in the center of the vehicle, and the user device can be predicted around this origin. This allows the pre-trained model to make a more accurate prediction. Furthermore, based on regression, it is possible to weight misclassifications or incorrect determinations of the user device differently in a loss function. Additionally, based on the regression, the user can be given the final decision as to whether the given position is considered inside or outside the vehicle. Accordingly, the user's decision can be taken into account for future position determinations.

[0063] Fig. Figure 2 shows a flowchart of a procedure 10 for controlling a vehicle function of a vehicle.

[0064] The procedure 10 includes determining, 11, a first position relative to the vehicle based on a trained ML model which uses as input a characteristic measured for the user device.

[0065] The first position can be determined by a machine learning model, as discussed above. However, the present disclosure is not limited to this. Based on the first position, it is determined whether the user device is located inside or outside the vehicle, as already discussed.

[0066] If it is determined that the user device is located outside the vehicle, the procedure further includes applying, 12, a Kalman filter to determine a second position. In principle, the first and second positions can be the same (or within a threshold value compared to the first position). In other words, the position is determined again, but based on a Kalman filter rather than the machine learning model. However, the second position can also differ from the first position, allowing, for example, the determination of whether the user is moving towards or away from the vehicle with the user device.

[0067] The Kalman filter can also use the measured characteristic as input. This allows for real-time tracking of the user device with an accuracy of up to 30 cm, even when positioned outside the vehicle.

[0068] Method 10 further includes controlling, 13, the vehicle function based on the first position when it is determined that the user device is inside the vehicle. For example, the vehicle function then includes starting the vehicle. Alternatively, Method 10 includes controlling, 13, based on the second position when it is determined that the user device is outside the vehicle. The vehicle function then includes, for example, locking or unlocking the vehicle, depending on whether, based on the second position, it is determined that the user device is moving away from the vehicle (locking) or toward the vehicle (unlocking). If the user device is moving toward the vehicle, the vehicle function may alternatively or additionally include starting the vehicle.

[0069] For example, the speed of the user device can also be taken into consideration when several second positions are determined consecutively (at least two) based on the Kalman filter.

[0070] In some examples, the Kalman filter is already combined with the ML model to improve position determination.

[0071] In some examples, the second position is determined for at least two consecutive points in time to determine whether the user device is moving towards or away from the vehicle, as previously discussed.

[0072] In some examples, the vehicle function includes at least one of unlocking the vehicle and starting the vehicle, as already discussed.

[0073] In some examples, the ML model for determining the first position is as described with reference to the Fig. 1 described trained, however, the present disclosure is not limited to that.

[0074] Fig. Figure 3 shows a method 20 for manufacturing a vehicle. The method 20 comprises inserting, 21, into a vehicle of the predetermined vehicle model, a data set representing a machine learning model trained as described in the Fig. 1 was described.

[0075] The data set can, for example, be implemented as software on a processing circuit integrated into the vehicle. The vehicle thus manufactured applies a machine learning model according to the present disclosure and can determine the position of a user device relative to itself (to the vehicle or to a predetermined point or area within the vehicle).

[0076] A processing circuit according to the present disclosure can comprise a single dedicated processor, a single shared processor, or a plurality of individual processors, some or all of which can be shared. In some examples, the processing circuit (alternatively or additionally) comprises at least one digital signal processor (DSP), an application-specific integrated circuit (ASIC), a system-on-a-chip (SoC), a neuromorphic processor, and a field-programmable gate array (FPGA). The processing circuit 11 can optionally be coupled, for example, with a memory such as a read-only memory (ROM) for storing software, a random-access memory (RAM), and / or a non-volatile memory (or medium). For example, a vehicle (such as the vehicle 30 described with reference to the Fig. 4) contain a memory configured to store instructions which, when executed by the processing circuit, cause the processing circuit to perform the steps and procedures described herein.

[0077] Procedure 20 further comprises configuring, 22, the vehicle to determine the position of a user device relative to the vehicle using the trained ML model. The configuration can, as described above, be performed by applying the data set through the processing circuit. The configuration can further include defining a coordinate origin for the vehicle to determine the position. The configuration can also include configuring a Kalman filter to implement a procedure according to the Fig. 2 to execute.

[0078] Fig. Figure 4 shows a vehicle 30 according to the present disclosure, comprising a non-volatile machine-readable medium on which a data set 31 is stored, containing an ML model as described in the Fig. 1 described, trained, represented.

[0079] The vehicle 30 may further comprise a data set containing instructions which, when executed (e.g., by a processing circuit as described above), establish a procedure for controlling a vehicle function (as referred to in the Fig. 2 as described).

[0080] As discussed above, the vehicle can be a motor vehicle, a watercraft, an aircraft, a non-motorized vehicle, or the like. Reference symbol list 1. Method for training a machine learning model 2. Retraining a pre-trained ML model 10 methods for controlling a vehicle function 11 Determining a first position 12. Applying a Kalman filter to determine a second position 13 Controlling a vehicle function 20 methods for manufacturing a vehicle 21. Importing a data set 22 Configuring the vehicle to determine the position of a user device 30 vehicles 31 Non-volatile machine-readable medium

Images