CONVERSION OF LATENT LAYERS OF SIGNALS BETWEEN SOFTWARE VERSIONS

Abstract

The use of universal signals to determine vehicle metrics is provided. Based on vehicle signals received from one or more vehicles, a latent space model is trained, incorporating an encoder and a decoder. The encoder generates universal signals as a latent representation of the vehicle signals, while the decoder reconstructs vehicle signals from these universal signals. The encoder is sent to the one or more vehicles. Universal signals are received by the one or more vehicles, generated by the encoder on the vehicle signals. The universal signals are applied to an analysis model to determine metrics related to the operation of the one or more vehicles from the latent representation.

Description

AREA OF TECHNOLOGY

[0001] Aspects of the disclosure generally relate to the use of an in-vehicle latent layer to convert software version-specific signals into a general signal representation. GENERAL STATE OF THE ART

[0002] Connected vehicles can send data to a cloud system. User-controlled insurance (UBI) is a type of vehicle insurance where premium costs depend on a driver's driving behavior. A UBI device can be connected to a vehicle network via a connector, such as an On-Board Diagnostics II (OBD-II) port, to collect vehicle operating data and send it to a remote server for analysis. In other examples, the vehicle's telematics control unit (TCU) can collect the vehicle operating data and send it to the remote server for analysis.

[0003] An autoencoder is a type of artificial neural network used in unsupervised learning for data compression and feature extraction. An autoencoder comprises two main parts: an encoder, which compresses the input data into a representation of latent space, and a decoder, which reconstructs the original data from this compressed form. SUMMARY

[0004] In one or more illustrative examples, a method for using universal signals to determine vehicle metrics involves: training a latent space model, which includes an encoder and a decoder, based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals; the decoder generates reconstructed vehicle signals from the universal signals; sending the encoder to the one or more vehicles; receiving universal signals from the one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals; and applying the universal signals to an analysis model to determine metrics relating to the operation of the one or more vehicles from the latent representation.

[0005] In one or more illustrative examples, the vehicle signals received from one or more vehicles are filtered according to user filter guidelines that determine which of the vehicle signals are to be included in the universal signals.

[0006] In one or more illustrative examples, the procedure further involves varying parameters and / or conditions that affect the vehicle signals, such as changes in driving style, weather conditions, road types, or vehicle load; performing a re-simulation using a generative model to generate new instances of the vehicle signals, simulating the vehicle behavior under the varying parameters and / or conditions; and adding the re-simulated vehicle signals to the vehicle signals as an extended dataset for training the latent space model.

[0007] In one or more illustrative examples, the procedure further involves performing joint training of the analysis model and the latent space model using a loss function that takes into account loss in the latent space model as well as loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model.

[0008] In one or more illustrative examples, the procedure further involves defining the universal signals as a vector of a first version of the vehicle signals in combination with additional elements of a future version that are set to a default value, wherein the vehicle signals received by one or more vehicles are of a second version of the vehicle signals, and training ensures the accuracy of the first version of the vehicle signals and the second version of the vehicle signals through the latent space model.

[0009] In one or more illustrative examples, training involves training a multitude of latent space models on subsets of the vehicle signals, each of the multitude of latent space models containing an encoder-and-decoder pair; and combining the outputs of a multitude of encoders of the encoder-and-decoder pairs to generate the universal signals.

[0010] In one or more illustrative examples, the analysis model determines metrics related to vehicle maintenance.

[0011] In one or more illustrative examples, the analysis model determines metrics related to usage-based insurance.

[0012] In one or more illustrative examples, the procedure further involves detecting a change in the distribution of the vehicle signals and / or the presence of outlier events in the vehicle signals; retraining the latent space model to generate an updated encoder; and sending the updated encoder to the one or more vehicles.

[0013] In one or more illustrative examples, a system for using universal signals to determine vehicle metrics includes one or more computing devices, comprising non-transient data storage and a processor, configured to: train a latent space model, comprising an encoder and a decoder, based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals; the decoder generates reconstructed vehicle signals from the universal signals; send the encoder to the one or more vehicles; and receive universal signals from the one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals.and to apply the universal signals to an analysis model in order to determine metrics relating to the operation of one or more vehicles from the latent representation.

[0014] In one or more illustrative examples, the vehicle signals received from one or more vehicles are filtered according to user filter guidelines that determine which of the vehicle signals are to be included in the universal signals.

[0015] In one or more illustrative examples, the one or more computing devices are further configured to vary parameters and / or conditions affecting the vehicle signals, such as changes in driving style, weather conditions, road types, or vehicle load; to perform a re-simulation using a generative model to generate new instances of the vehicle signals, simulating the vehicle behavior under the varying parameters and / or conditions; and to add the re-simulated vehicle signals to the vehicle signals as an extended dataset for training the latent space model.

[0016] In one or more illustrative examples, the one or more computing devices are further configured to perform joint training of the analysis model and the latent space model using a loss function that takes into account loss in the latent space model and also loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model.

[0017] In one or more illustrative examples, the one or more computing devices are further configured to define the universal signals as a vector of a first version of the vehicle signals in combination with additional elements of a future version that are set to a default value, wherein the vehicle signals received by one or more vehicles are of a second version of the vehicle signals, and the training ensures the accuracy of the first version of the vehicle signals and the second version of the vehicle signals through the latent space model.

[0018] In one or more illustrative examples, the one or more computing devices are further configured to train a plurality of latent space models on subsets of the vehicle signals, each of the plurality of latent space models including an encoder-and-decoder pair; and to combine the outputs of a plurality of encoders of the encoder-and-decoder pairs to generate the universal signals.

[0019] In one or more illustrative examples, the analysis model determines metrics related to vehicle maintenance.

[0020] In one or more illustrative examples, the analysis model determines metrics related to usage-based insurance.

[0021] In one or more illustrative examples, a non-transient computer-readable medium includes instructions which, when executed by one or more processors of one or more computing devices, cause the one or more computing devices to perform operations, including training a model of latent space comprising an encoder and a decoder, based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals, and wherein the decoder generates reconstructed vehicle signals from the universal signals; sending the encoder to the one or more vehicles; and receiving universal signals from the one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals.and applying the universal signals to an analysis model to determine metrics relating to the operation of one or more vehicles from the latent representation.

[0022] In one or more illustrative examples, the non-transient computer-readable medium further includes instructions which, when executed by one or more processors, cause one or more computing devices to perform operations including jointly training the analysis model and the latent space model using a loss function that accounts for loss in the latent space model and also loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model.

[0023] In one or more illustrative examples, the non-transient computer-readable medium further includes instructions which, when executed by the one or more processors, cause the one or more computing devices to perform operations including: training a plurality of latent space models on subsets of the vehicle signals, each of the plurality of latent space models comprising an encoder-and-decoder pair; and combining the outputs of a plurality of encoders of the encoder-and-decoder pairs to generate the universal signals. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 illustrates an exemplary system for using an in-vehicle latent layer to convert software version-specific signals into a general signal representation; Fig. Figure 2 illustrates an exemplary autoencoder architecture for use in the system from Fig. 1; Fig. 3A illustrates a version A of the signals as the output value for the universal signals; Fig. 3B illustrates a version B of the signals, which are encoded as universal signals; Fig. 3C illustrates version A of the signals, which are considered the universal signals for the encoder. Fig. 3B will be applied; Fig. Figure 4 illustrates an alternative autoencoder architecture in which signals from separate subsystems of the vehicle are processed separately; Fig. Figure 5 illustrates an example of jointly training the autoencoder with the analysis model; Fig. Figure 6 illustrates an exemplary loop-like process for using the universal signals to determine metrics using the analysis models; and Fig. Figure 7 illustrates an exemplary computing device for using universal signals to determine vehicle metrics. DETAILED DESCRIPTION

[0024] Depending on the requirements, detailed embodiments of the present invention are disclosed in this document; however, it is understood that the disclosed embodiments are merely exemplary of the invention, which can be implemented in various and alternative forms. The figures are not necessarily to scale; some features may be greatly enlarged or reduced to show details of specific components. Therefore, specific structural and functional details disclosed in this document are not to be interpreted as limiting, but merely as a representative basis to teach those skilled in the art the diverse applications of the present invention.

[0025] Vehicle operating models integrate more low-level signals, such as those from driver assistance features and / or autonomous driving features, to improve model accuracy. However, these low-level signals can differ between software versions because the underlying algorithms that generate them can change. For example, new signals may be added, existing signals may be modified, or signals may be deleted and no longer available. Furthermore, there may be changes to signal naming, interface structures, and the underlying hardware.

[0026] Collecting signal data for each version can be expensive, and retraining the model for each software update can be problematic. Therefore, aspects of the disclosure relate to a method for converting a set of user-defined, low-level vehicle signals into a general set of signals for use with the vehicle operating model.

[0027] Fig. Figure 1 illustrates an exemplary system 100 for using an in-vehicle latent layer to convert software version-specific signals into a general signal representation. The system 100 includes one or more vehicles 102, each vehicle 102 containing a plurality of controllers 104 and sensors 106. Each vehicle 102 also includes one or more vehicle buses 108 for communication between the controller 104, the sensors 106, and a telematics control unit (TCU) 110. The TCU 110 includes a modem 112 configured to facilitate communication via, or otherwise access, a communication network 114. The TCU 110 may include a processor 116 and a data storage device 118. The TCU 110 can acquire signals 122 and manage them in the data storage device 118. The data storage 118 can also manage an event processing application 138 and an encoder 124.The event processing application 138 can use the encoder 124 to encode the signals 122 into universal signals 128 and can send the universal signals 128 to a cloud server 120. The cloud server 120 can also be configured to run a vehicle data service 136, which uses one or more analysis models 132 to operate with universal signals 128 in order to determine various metrics 134. In an example, the metrics 134 can also be provided to client devices 140 in response to client requests 142 to facilitate the provision of insurance rates for the vehicles 102 and / or the scheduling of maintenance for the vehicles 102. It should be noted that the system 100 is only an example and that systems 100 can be used with more, fewer, or different components.

[0028] Vehicle 102 can be any type of automobile, soft-roader (crossover utility vehicle - CUV), off-road vehicle (sport utility vehicle - SUV), truck, motorhome, boat, aircraft, or other mobile machinery for transporting people or goods. Such Vehicle 102 can be human-driven or autonomous. In many cases, Vehicle 102 can be powered by an internal combustion engine. Alternatively, Vehicle 102 can be a battery electric vehicle (BEV), powered by one or more electric motors.Alternatively, Vehicle 102 could be a hybrid electric vehicle (HEV), powered by both an internal combustion engine and one or more electric motors, such as a series hybrid electric vehicle (SHEV), a parallel hybrid electric vehicle (PHEV), or a parallel / series hybrid electric vehicle (PSHEV). Alternatively, Vehicle 102 could be an autonomous vehicle (AV). The level of automation could range from various levels of driver assistance technology to a fully automated, driverless vehicle. Since the type and configuration of Vehicle 102 can vary, its capabilities could also vary accordingly.Among other possibilities, vehicles 102 can have different capabilities with regard to passenger capacity, towing capability and capacity, and storage space. For registration, inventory, and other purposes, vehicles 102 can be assigned unique identifiers, such as vehicle identification numbers (VINs). It should be noted that while motor vehicles 102 are used as examples of road users, other types of road users, such as bicycles, scooters, and pedestrians, can be used additionally or alternatively.

[0029] The vehicle 102 can include a variety of controllers 104 configured to perform and manage various functions of the vehicle 102 using the power of the vehicle's battery and / or powertrain. As shown, the exemplary vehicle controllers 104 are represented as discrete controllers 104 (i.e., controllers 104A to 104G). However, the vehicle controllers 104 can share physical hardware, firmware, and / or software in such a way that the functionality of several controllers 104 can be integrated into a single controller 104, and the functionality of several such controllers 104 can be distributed across a variety of controllers 104.

[0030] As some non-restrictive examples of vehicle controls 104, a powertrain control 104A may be configured to provide control of engine operating components (e.g., idle control components, fuel delivery components, emission control components, etc.) and to monitor the status of such engine operating components (e.g., engine code status); a body control 104B may be configured to manage various performance control functions, such as exterior lighting, interior lighting, keyless entry, remote start, and verification of the status of access points (e.g.,The vehicle 102 may control the closing status of the hood, doors, and / or trunk; a radio transceiver control 104C may be configured to communicate with radio keys, mobile devices, or other local devices of the vehicle 102; an autonomous control 104D may be configured to provide commands to control the powertrain, steering, or other aspects of the vehicle 102; a climate control management control 104E may be configured to provide control for heating and cooling system components (e.g., compressor clutch, blower fan, temperature sensors, etc.).) to provide; a controller 104F for a global navigation satellite system (GNSS) can be configured to provide vehicle location information; and a controller 104G for a human machine interface (HMI) can be configured to receive user input via various buttons or other controls and to provide a driver with vehicle status information, such as fuel level information, engine operating temperature information and the current location of the vehicle 102.

[0031] The vehicle 102's controllers 104 can use various sensors 106 to receive information about the vehicle 102's environment. For example, these sensors 106 can include one or more cameras (e.g., cameras of an advanced driver-assistance system (ADAS)), ultrasonic sensors, radar systems, and / or lidar systems.

[0032] One or more vehicle buses 108 can include various communication methods available between the vehicle controllers 104 and between the TCU 110 and the vehicle controllers 104. As some non-limiting examples, the vehicle bus 108 can include one or more of a Controller Area Network (CAN) for the vehicle, an Ethernet network, and a media-oriented system transfer (MOST) network.

[0033] The TCU 110 can include network hardware configured to facilitate communication between the vehicle controllers 104 and with other devices of the system 100. For example, the TCU 110 can include or otherwise access a modem 112 configured to facilitate communication over a communication network 114. Accordingly, the TCU 110 can be configured to communicate with the communication network 114 using various protocols, such as a network protocol (like Uu). The TCU 110 can also be configured to communicate using a peer-to-peer transmission protocol (like PC5) to facilitate cellular vehicle-to-everything (C-V2X) communication with devices such as other vehicles 102.It should be noted that these protocols are merely examples and different peer-to-peer and / or mobile communication technologies may be used.

[0034] The TCU 110 can include various types of computing devices to support the performance of the functions of the TCU 110 described in this document. For example, the TCU 110 can include one or more processors 116 configured to execute computer instructions and a data storage medium 118 on which the computer-executable instructions and / or data can be managed. A computer-readable data storage medium (also called a processor-readable medium or processor-readable data storage 118) includes any non-transient (e.g., physical) medium involved in providing data (e.g., instructions) that can be read by a computer (e.g., by the processor(s) 116). Generally, the processor 116 receives instructions and / or data, e.g., from the data storage 118, etc., in memory and executes the instructions using the data, thereby executing one or more processes, including one or more of the processes described in this document. Computer-executable instructions can be assembled or interpreted by computer programs created using a variety of programming languages ​​and / or technologies, including, without limitation, and either alone or in combination, Java, C, C++, C#, Fortran, Pascal, Visual Basic, Python, JavaScript, Perl, etc.

[0035] The TCU 110 can be configured to include one or more interfaces from which information from the vehicle 102 can be sent and received. This information can be sampled, recorded, and sent to one or more cloud servers 120. In one example, the cloud server 120, similar to the TCU 110, can also include one or more processors (not shown) configured to execute computer instructions and a data storage medium (not shown) on which the computer-executable instructions and / or data can be managed.

[0036] The TCU 110 can be configured to facilitate the collection of vehicle signals 122 from the vehicle controllers 104 connected to one or more vehicle buses 108. These signals may include, for example, ADAS signals generated by the ADAS functions of the vehicle 102. Although only a single vehicle bus 108 is illustrated, it should be noted that many examples involve multiple vehicle buses 108, with a subset of the controllers 104 typically connected to each vehicle bus 108. Accordingly, to access a given controller 104, the TCU 110 can be configured to manage a mapping of which vehicle buses 108 are connected to which controllers 104, and to access the appropriate vehicle bus 108 for a controller 104 when communication with that specific controller 104 is desired.

[0037] As used in this document, the vehicle signals 122 (e.g., ADAS signals and the like) can refer to various binary, multi-state, integer, floating-point, and / or continuous parameters that can be generated or otherwise augmented by the vehicle control unit 104 and / or sensors 106. The signals 122 can include varying types of units, such as time-series data from different frequency and event streams, and / or different object types, such as floating-point, array, matrices, nested data types, etc.As some non-restrictive examples, vehicle signals 122 may include one or more of the following: latitude, longitude, time, heading angle, speed, throttle position, brake status, steering angle, headlight status, windscreen wiper status, outside temperature, turn signal status, ambient temperature or other weather conditions, attention status, hands off steering wheel status, all-wheel drive (AWD) status, front object detection, side object detection status, rear object detection status, etc.

[0038] The signals 122 present at vehicle 102 can vary based on the software and hardware versions of the controllers 104 and / or sensors 106 of vehicle 102. Thus, instead of sending the signals 122 as they are present at vehicle 102, the TCU 110 and the cloud server 120 can jointly use an autoencoder architecture.

[0039] An autoencoder is a type of neural network designed to efficiently learn compressed representations of input data in an unsupervised manner. For example, the autoencoder can be trained to capture the key features of signals 122 in a way that allows the signals 122 to be accurately reconstructed from a compressed representation. To do this, the autoencoder includes two main components: the encoder 124 and a decoder 126. It should be noted that this is only one exemplary embodiment; other embodiments are possible. Some other possibilities include a variant of an autoencoder, such as a varying, contradictory, denoising, stacked, conditional, and / or multimodal neural network.In other examples, other architectures can be used, such as a converter network, statistical machine translation, or even a recurrent neural network (RNN) with an attention mechanism.

[0040] Fig. Figure 2 illustrates an exemplary autoencoder architecture 200 for use in System 100. As in Fig. 2 shown, and with continued reference to Fig. 1. Signals 122 are received as input to the autoencoder architecture 200. These signals can be processed by a filter 202 using a user filter policy 204 to generate filtered signals 206, which can then be applied to the encoder 124. The encoder 124 can be installed on the vehicle 102 and can convert a representation of the filtered signals 206 into universal signals 128. The universal signals 128 can be transmitted by the vehicle 102 to the cloud server 120. If desired, the universal signals 128 can be converted into reconstructed signals 130 by a decoder 126, which corresponds to the encoder 124. The universal signals 128 can also be processed by one or more analysis models 132 on the cloud server 120 to generate metrics 134.

[0041] Filter 202 can be configured to remove information from signals 122, as desired by the user of vehicle 102. For example, one user might allow cloud server 120 to process all of vehicle 102's signals 122. Alternatively, another user might allow only a subset of signals 122. For instance, one user might prefer to allow signals 122, such as steering input, speed, lane position, etc., while another user might prefer to prevent the use of some of these signals 122, such as providing speed but not steering input. It should be noted that filter 202 can also perform filtering operations other than a signal level filter (e.g., a Kalman filter, a particle filter) and / or generate sensor fusion output.One or more of these processes can be optional and also configurable by a user. These user preferences can be stored in the vehicle 102 as a user filter policy 204, which can be applied by the filter 202 to the signals 122 to generate filtered signals 206.

[0042] The Encoder 124 is a neural network that receives the filtered signals 206 and compresses them into a smaller, lower-dimensional representation called the latent space. This latent space captures the essential features of the filtered signals 206, while discarding less important information, thus enabling a more compact representation. The output of the Encoder 124 is referred to in this document as the universal signals 128.

[0043] Decoder 126 is another neural network that takes the latent representation from Encoder 124 (e.g., the universal signals 128) and attempts to reconstruct the original signals 122 from the universal signals 128. Decoder 126 reverses the encoding process, thereby expanding the compressed latent space back into the dimensions of the original data.

[0044] A training process can be performed for the autoencoder to minimize the difference between the original signals 122 input to the encoder 124 and the reconstructed signals 130 output by the decoder 126. This can be achieved using a loss function, such as the mean squared error, or another suitable function (such as a domain-specific loss function). Thus, the autoencoder can learn to compress data into a smaller form that can later be reconstructed with minimal information loss. Furthermore, the autoencoder can also perform denoising of the signals 122 to repair potentially corrupted data by learning to reconstruct a clean version of the data from its noisy counterpart.Monitoring a denoising autoencoder loss metric can also serve as a plausibility check of the autoencoder translation, either in the training process or potentially in the vehicle.

[0045] With renewed reference to Fig. 1. The analysis model 132 can be any of several different machine learning models trained to determine metrics 134 based on the signals 122 (here, the universal signals 128). In one example, an analysis model 132 can be configured to derive metrics 134 relating to vehicle 102 based on training of the analysis model 132 using universal signals 128 from vehicles 102 with known results. In another example, an analysis model 132 can be trained on maintenance data for vehicles 102 based on data from the universal signal 128 to enable the analysis model 132 to determine metrics 134 relating to likely maintenance required by vehicle 102.In another example, an analysis model 132 can be trained on insurance data for vehicles 102 based on data from the universal signal 128 to enable the analysis model 132 to determine metrics 134 in relation to likely incidents that occur as a result of how the vehicle 102 is driven.

[0046] The system 100 can further include one or more client devices 140 configured to access the cloud server 120 via the communication network 114. Using the services of the vehicle data service 136 of the cloud server 124, the one or more client devices 140 can be configured to perform queries 142 for the metrics 134 for various information, e.g., to prepare insurance quotes for the vehicles 102 and / or to schedule maintenance for the vehicles 102.

[0047] Fig. 3A and Fig. Section 3B illustrates together examples of using a version of signals 122A from a vehicle 102 as a basis for the universal signals 128. As in example 300A from Fig. As shown in Figure 3A, a version of the signals 122A (here version A) is used as an output value for the universal signals 128. No encoder 124 or decoder 126 is used here. Instead, the signals 122A are used directly as a section of the universal signals 128. Additionally, elements 302 of the future version are defined as the remainder of the universal signals 128. These elements 302 of the future version can initially be assigned a default value, such as one or zero, or even a random distribution of values. With such an approach, the version A section of the universal signals 128 is easily understood, as it is consistent with the signals 122A of version A itself.

[0048] As in example 300B from Fig. As shown in Figure 3B, a second version of signal 122B (here version B) is used with the same defined universal signals 128. Here, an encoder 124B of version B is used to encode the signals 122B into the same representation of the universal signals 128. It should be noted that the encoder 124B and the decoder 126B can be trained with a loss function that ensures the accuracy of the signals 122A of version A as well as the signals 122B of version B.

[0049] As in the example 300C from Fig. As shown in 3C, the first version of the signals 122A is applied to the decoder 126B as the universal signals 128. In this example, the signals 122A can be incorporated into the version B signals 122B using the trained decoder 126B. Such an approach can be useful for downstream tasks that depend on the version B signals. Notably, in each of the examples 300A, 300B, and 300C, the same analysis models 132 can use the universal signals 128 to process and generate the metrics 134.

[0050] Fig. Figure 4 illustrates an alternative autoencoder architecture 400 in which the signals 122 from separate subsystems of the vehicle 102 are processed separately. In one such example, the subsystems may relate to different sensors 106. In another example, the different subsystems may relate to different controllers 104 of the vehicle 102. In further analysis models 132, the different subsystems may relate to different functional groupings of functionality of the vehicle 102, such as ADAS functionality, steering, internal combustion engine, etc., independent of the sensors 106 and / or the controller 104 used for the functionality.

[0051] As shown, the subsystem signals 122-1 to 122-N from each of the different subsystems are processed separately by the filters 202-1 to 202-N into filtered signals 206-1 to 206-N and, in turn, encoded by the encoders 124-1 to 124-N into respective sections 128-1 to 128-N of universal signals 128. These sections 128-1 to 128-N of universal signals can be chained together or otherwise combined to form the complete universal signals 128. In such an approach, the complexity of the encoders 124-1 to 124-N can be reduced compared to a single, complete encoder 124, since each of the encoders 124 can be operated separately. This can reduce the processing required by vehicle 102 to generate the universal signal 128, and can also speed up the training of the autoencoder.

[0052] In such an approach, the individual encoders 124-1 to 124-N can be trained together with the decoder 126, just as would be done to train a single monolithic encoder 124.

[0053] Fig. Figure 5 illustrates an example 500 of joint training of the autoencoder with the analysis model 132. This joint training can be performed to improve the ability to preferably capture a useful representation of the signals 122. This joint learning approach can accordingly ensure the accuracy of the universal signals 128 when the input vector of the signal 122 is transformed into the representation of the latent space.

[0054] Analysis Model 132 can be trained together with the autoencoder, so that the latent space is optimized for both the reconstruction and the subsequent analysis task of Analysis Model 132. The joint training process includes adding an additional loss function for the task of Analysis Model 132 and training the entire network end-to-end.

[0055] In one example, the encoder 124 and the decoder 126 can form a variational autoencoder (VAE), and the combined loss function can account for both the VAE reconstruction loss and the analysis task loss. The autoencoder loss 502 can include a reconstruction loss, which is a measure of how well the decoder 126 reconstructs the input (e.g., implemented as mean squared error (MSE) or binary cross-entropy loss). The autoencoder loss 502 can also include Kullback-Leibler (KL) divergence loss, which can ensure that the latent space of VAEs follows a Gaussian distribution.

[0056] The analysis loss 504 can depend on the task performed by the analysis model 132. For example, if the analysis model 132 is a classifier, then the analysis loss 504 may include a classification loss, such as a cross-entropy loss. If the analysis model 132 is a regression model, the analysis loss 504 may include a regression loss, such as MSE.

[0057] An example of a total loss function might be as follows: Total loss = α⋅Autoencoder loss + β⋅Analysis loss where α and β are hyperparameters that balance the contribution of each loss term. During training, the total loss can be backpropagated through the entire network, and the weights of encoder 124, decoder 126, and analysis model 132 can be updated simultaneously. It should be noted that in some examples, the approach to stabilizing the training may begin by training the autoencoder alone and then fine-tuning the entire network after introducing analysis model 132.

[0058] Fig. Figure 6 illustrates an exemplary loop-like process 600 for using the universal signals 128 to determine metrics 134 using the analysis models 132. In an example, the loop-like process 600 can be carried out using the system 100 with one or more of the architectures discussed in detail in this document. As shown, process 600 is a possible application of the universal signals approach 128 for determining metrics 134 about the behavior of the vehicles 102.

[0059] In process 602, signals 122 are collected from the various sensors 106 and controllers 104 of the vehicle 102, including cameras, LiDAR, radar, and other relevant inputs. This data is rich in information about the environment, performance, and driver behavior of the vehicle 102 and forms the basis for the subsequent processes. Once the signals 122 are collected, they can be normalized to ensure consistency and reliability across different sensor types and software versions, making them ready for further processing.

[0060] In process 604, the signals 122 undergo re-simulations, replaying and / or regenerating sensor inputs to improve the coverage of the acquired data. To use the re-simulation to generate training data for creating the autoencoder for the vehicle signals 122, the initial dataset of signals 122 from the vehicles 102 collected in process 602 can be used as input. These signals 122 can include various sensor readings, such as speed, acceleration, combustion engine temperature, fuel levels, and more, collected over time under varying driving conditions. The re-simulation can use a model to mimic or replicate the behavior of these signals 122 under varying conditions that may not be present in the original dataset.

[0061] In one example, the existing vehicle signals 122 can be analyzed to understand their patterns, correlations, and any anomalies. A generative model can, for instance, be trained on this initial dataset to learn the underlying distribution and behavior of the signals 122. Once trained, this model can re-simulate the vehicle signals 122 by varying parameters or conditions that affect them, such as changes in driving style, weather conditions, road types, or vehicle load. The generative model can then generate new instances of the signals 122 that simulate how the vehicle 102 might behave under conditions not originally captured. The re-simulated signals form an expanded dataset representing a broader range of scenarios and variations in the vehicle 102's performance.This extended dataset can then be used as the training data for the autoencoder.

[0062] In process 606, the signals 122 (e.g., as simulated again) are fed into an autoencoder model that includes the encoder 124 and the decoder 126. The encoder 124 is trained to compress the signals 122 into a lower-dimensional latent space that represents essential features in the form of the universal signals 128, enabling efficient signal translation and representation across different software versions. The decoder 126 section of the autoencoder is also trained to reconstruct the original signals 122 to ensure that the latent space retains the information needed for accurate analysis and subsequent modeling tasks. In some examples, subsystems of the vehicle 102 are trained using separate autoencoder models, while in other examples, the collection of signals 122 is used for a single autoencoder model.Training the autoencoder can be performed independently of training the analysis models 132 or in a joint learning approach with training the analysis models 132.

[0063] In process 608, the universal signals 128 generated by the autoencoder are used to generate metrics 134 by the analysis models 132. The analysis model 132 can interpret the features of the universal signals 128 to generate insights into vehicle wear, driver behavior, and / or other applications. By using the universal signals 128 instead of a specific version of the signals 122, the system 100 ensures that the metrics 134 are consistent and comparable regardless of the specific software version or sensor setup of the vehicle 102. Furthermore, the analysis models 132 do not require retraining for each different version of the signals 122.

[0064] In combination with the generation of metrics 134, the system 100 continuously monitors for events of interest, such as sudden changes in speed, sharp turns, or other outlier behavior. In another example, the cloud server 124 can detect a change in the distribution of the data received in the signals 122 and trigger retraining. These events can trigger a model estimation process in which the autoencoder and / or the analysis model 132 is retrained to adjust its predictions and risk assessments based on the new data. This dynamic adjustment allows the analysis model 132 to respond to changing conditions, ensuring that the generated metrics 134 accurately reflect the current state of the vehicle 102 and its operating environment.

[0065] In process 612, the insights gained from the monitored events and model estimates are fed back into the updating of the data collection parameters of vehicle 102. Based on the identified trends, anomalies, or areas for improvement, the system adjusts its data collection strategies. This might include, for example, sending an updated encoder 124 to the vehicles 102 based on retraining performed in process 610. This updated approach improves the next data collection cycle by ensuring that the process continuously improves itself, leading to an even more accurate and reliable analysis of the signals 122.

[0066] Variations in the process are possible. For example, dimensionality reduction and feature engineering techniques, such as manifold learning, can generate varying latent dimensionality using features that are not of great interest to an engineer. In some examples, the disclosed approach can be used to provide feature engineering and dimensionality reduction in a repeatable manner.

[0067] Fig. Figure 7 illustrates an exemplary computing device 702 for using universal signals 128 to determine vehicle metrics 134. With reference to Fig. 7 and with reference to Fig.References 1-6, the vehicle 102, the controllers 104, the sensors 106, the TCU 110, and the cloud server 124 can be examples of such computing devices 702. Computing devices 702 generally contain computer-executable instructions, such as those of the vehicle data server 136 and the event processing application 138, the instructions being executable by one or more computing devices 702. Computer-executable instructions can be compiled or interpreted by computer programs created using a variety of programming languages ​​and / or technologies, including, but not limited to, Java™, C, C++, C#, Visual Basic, JavaScript, Python, Perl, etc., either individually or in combination. Generally, a processor (e.g., a microprocessor) receives instructions, for example, from memory, a computer-readable medium, etc., and executes these instructions, thereby carrying out one or more processes that include one or more of the processes described herein. Such instructions and other data, such as Signals 122, Encoders 124, Decoders 136, Universal Signals 128, Reconstructed Signals 130, Analysis Model 132, Metrics 134, etc., can be stored and transmitted using a variety of computer-readable media.

[0068] As shown, the computing device 702 can include a processor 704, which is operatively connected to a data storage device 706, a network device 708, an output device 710, and an input device 712. It should be noted that this is only an example and computing devices 702 can be used with more, fewer, or different components.

[0069] The 704 processor can include one or more integrated circuits that implement the functionality of a central processing unit (CPU) and / or graphics processing unit (GPU). In some examples, the 704 processors are a system-on-a-chip (SoC) that integrates the functionality of both the CPU and the GPU. The SoC can optionally include other components, such as the 706 data storage device and the 708 networking device, in a single integrated device. In other examples, the CPU and GPU are interconnected via a peripheral interconnect device, such as Peripheral Component Interconnect Express (PCI Express) or another suitable peripheral data connection.In one example, the CPU is a commercially available central processing device that executes a set of instructions, such as one from the x86, ARM or Power instruction set family, or a microprocessor without interlocked pipeline stages (MIPS) instruction set family.

[0070] Regardless of the specifics, the 704 processor executes stored program instructions during operation, which are retrieved from the 706 data memory. The stored program instructions accordingly include software that controls the operation of the 704 processors to perform the operations described herein. The 706 data memory can include both non-volatile and volatile memory devices. The non-volatile memory includes solid-state memory, such as non-AND flash memory (not-AND NAND), magnetic and optical storage media, or any other suitable storage device that retains data when the system is powered off or its power supply is interrupted. The volatile memory includes static and dynamic random-access memory (RAM) on which 100 program instructions and data are stored during system operation.

[0071] The GPU can include hardware and software for displaying at least two-dimensional (2D) and optionally three-dimensional (3D) graphics on the output device 710. The output device 710 can include a graphic or visual display device, such as an electronic display screen, a projector, a printer, or any other suitable device that reproduces a graphic display. As another example, the output device 710 can include an audio device, such as a loudspeaker or headphones. As yet another example, the output device 710 can include a tactile device, such as a mechanically raised device, which in one example can be configured to display Braille or other physical output that can be touched to provide information to a user.

[0072] The input device 712 can include any of the various devices that enable the computing device 702 to receive control inputs from users. Examples of suitable input devices 712 that receive inputs via a human interface can include keyboards, mice, trackballs, touchscreens, microphones, graphics tablets, and the like.

[0073] The Network Devices 708 can each include any of the various devices that enable the described components to send and / or receive data from external devices over networks. Examples of suitable Network Devices 708 include an Ethernet interface, a Wi-Fi transceiver, a cellular transceiver, a Bluetooth or Bluetooth Low Energy (BLE) transceiver, or any other network adapter or peripheral connection device that receives data from another computer or external data storage device, which can be useful for efficiently receiving large datasets.

[0074] With regard to the processes, systems, procedures, heuristics, etc., described in this document, it is understood that although the steps of such processes, etc., have been described as occurring according to a specific, ordered sequence, such processes could be implemented in practice, with the described steps being carried out in a sequence that differs from the sequence described in this document. Furthermore, it is understood that certain steps could be carried out simultaneously, other steps added, or certain steps described herein omitted. In other words, the descriptions of processes herein serve the purpose of illustrating certain embodiments and should in no way be interpreted as limiting the patent claims.

[0075] Accordingly, it is understood that the foregoing description is intended to be illustrative and not limiting. Many other embodiments and applications beyond the examples provided will become apparent from reading the preceding description. The scope should not be determined by reference to the foregoing description, but instead by reference to the attached claims, together with the full scope of equivalents to which these claims entitle. It is expected and intended that there may be future developments in the technologies discussed in this document and that the disclosed systems and methods may be incorporated into such future embodiments. Overall, it is understood that the application may be modified and varied.

[0076] All terms used in the claims shall be assigned their most comprehensive and comprehensible constructions and their general meanings as they would be known to persons skilled in the art in the art of the techniques described herein, unless expressly stated otherwise. In particular, the use of singular articles such as "a", "an", "the", "a", etc., shall be understood as referring to one or more of the elements indicated, unless a claim expressly limits this to the contrary.

[0077] The summary of disclosure is provided to enable the reader to quickly grasp the nature of the technical disclosure. It is submitted on the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, it is evident from the foregoing detailed description that, for the purpose of simplifying the presentation of the disclosure, various features in different embodiments have been grouped together. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly mentioned in each claim. Rather, as reflected in the following claims, the subject matter of the invention consists of fewer than all the features of any single disclosed embodiment.The following patent claims are hereby included in the detailed description, each patent claim being a separately claimed subject matter.

[0078] While exemplary embodiments have been described above, it is not intended that these embodiments describe all possible forms of the disclosure. Rather, the terms used in the description are descriptive rather than limiting, and it is understood that various modifications can be made without departing from the spirit and scope of the disclosure. Furthermore, the features of different implementing embodiments can be combined to form further embodiments of the disclosure.

[0079] According to the present invention, a method for using universal signals to determine vehicle metrics comprises the following: training a latent space model comprising an encoder and a decoder based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals, and wherein the decoder generates reconstructed vehicle signals from the universal signals; transmitting the encoder to the one or more vehicles; receiving universal signals from the one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals; and applying the universal signals to an analysis model to determine metrics relating to the operation of the one or more vehicles from the latent representation.

[0080] In one aspect of the invention, the vehicle signals received from one or more vehicles are filtered according to user filter guidelines that determine which of the vehicle signals are to be included in the universal signals.

[0081] In one aspect of the invention, the method includes the following: varying parameters and / or conditions that affect the vehicle signals, such as changes in driving style, weather conditions, road types, or vehicle load; performing a re-simulation using a generative model to generate new instances of the vehicle signals, simulating the vehicle behavior under the varying parameters and / or conditions; and adding the re-simulated vehicle signals to the vehicle signals as an extended dataset for training the latent space model.

[0082] In one aspect of the invention, the method involves performing joint training of the analysis model and the latent space model using a loss function that takes into account loss in the latent space model and also loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model.

[0083] In one aspect of the invention, the method comprises: defining the universal signals as a vector of a first version of the vehicle signals in combination with additional elements of a future version that are set to a standard value, wherein the vehicle signals received by one or more vehicles are of a second version of the vehicle signals, and the training ensures the accuracy of the first version of the vehicle signals and the second version of the vehicle signals through the latent space model.

[0084] In one aspect of the invention, the training involves: training a plurality of latent space models on subsets of the vehicle signals, wherein each of the plurality of latent space models includes an encoder-and-decoder pair; and combining the outputs of a plurality of encoders of the encoder-and-decoder pairs to generate the universal signals.

[0085] In one aspect of the invention, the analysis model determines metrics relating to vehicle maintenance.

[0086] In one aspect of the invention, the analysis model determines metrics related to usage-based insurance.

[0087] In one aspect of the invention, the method further includes: detecting a change in the distribution of the vehicle signals and / or the presence of outlier events in the vehicle signals; retraining the latent space model to generate an updated encoder; and sending the updated encoder to the one or more vehicles.

[0088] According to the present invention, a system for using universal signals to determine vehicle metrics is provided, comprising: one or more computing devices including non-transient data storage and a processor, configured to: train a latent space model comprising an encoder and a decoder based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals, and wherein the decoder generates reconstructed vehicle signals from the universal signals; transmit the encoder to the one or more vehicles; and receive universal signals from the one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals.and applying the universal signals to an analysis model to determine metrics relating to the operation of one or more vehicles from the latent representation.

[0089] According to one embodiment, the vehicle signals received from one or more vehicles are filtered according to user filter guidelines that determine which of the vehicle signals are to be included in the universal signals.

[0090] According to one embodiment, the one or more computing devices are further configured to: vary parameters and / or conditions that affect the vehicle signals, such as changes in driving style, weather conditions, road types, or vehicle load; perform a re-simulation using a generative model to generate new instances of the vehicle signals, simulating the vehicle behavior under the varying parameters and / or conditions; and add the re-simulated vehicle signals to the vehicle signals as an extended dataset for training the latent space model.

[0091] According to one embodiment, the one or more computing devices are further configured to perform joint training of the analysis model and the latent space model using a loss function that takes into account loss in the latent space model and also loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model.

[0092] According to one embodiment, the one or more computing devices are further configured to: define the universal signals as a vector of a first version of the vehicle signals in combination with additional elements of a future version that are set to a default value, wherein the vehicle signals received by one or more vehicles are of a second version of the vehicle signals, and the training ensures the accuracy of the first version of the vehicle signals and the second version of the vehicle signals through the latent space model.

[0093] According to one embodiment, the one or more computing devices are further configured to: train a plurality of latent space models on subsets of the vehicle signals, wherein each of the plurality of latent space models includes an encoder-and-decoder pair; and combine the outputs of a plurality of encoders of the encoder-and-decoder pairs to generate the universal signals.

[0094] According to one embodiment, the analysis model determines metrics related to vehicle maintenance.

[0095] According to one embodiment, the analysis model determines metrics related to usage-based insurance.

[0096] According to the present invention, a non-transient, computer-readable medium is provided, comprising instructions which, when executed by one or more processors of one or more computing devices, cause the one or more computing devices to perform operations, including: training a latent space model comprising an encoder and a decoder, based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals, and wherein the decoder generates reconstructed vehicle signals from the universal signals; transmitting the encoder to the one or more vehicles; and receiving universal signals from the one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals.and applying the universal signals to an analysis model to determine metrics relating to the operation of one or more vehicles from the latent representation.

[0097] According to one embodiment, the invention is further characterized by instructions which, when executed by one or more processors, cause one or more computing devices to perform operations including jointly training the analysis model and the latent space model using a loss function that takes into account loss in the latent space model and also loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model.

[0098] According to one embodiment, the invention is further characterized by instructions which, when executed by one or more processors, cause one or more computing devices to perform operations comprising: training a plurality of latent space models on subsets of the vehicle signals, wherein each of the plurality of latent space models includes an encoder-decoder pair; and combining the outputs of a plurality of encoders of the encoder-decoder pairs to generate the universal signals.

Claims

[1] Method for using universal signals to determine vehicle metrics, comprising: Training a latent space model comprising an encoder and a decoder based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals, and wherein the decoder generates reconstructed vehicle signals from the universal signals; Sending the encoder to one or more vehicles; Receiving universal signals from one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals; and Applying the universal signals to an analysis model to determine metrics relating to the operation of one or more vehicles from the latent representation. [2] Method according to claim 1, wherein the vehicle signals received from one or more vehicles are filtered according to user filter guidelines which determine which of the vehicle signals are to be included in the universal signals. [3] Method according to claim 1, further comprising: Variations in parameters and / or conditions that affect vehicle signals, such as changes in driving style, weather conditions, road types, or vehicle load; Performing a re-simulation using a generative model to generate new instances of the vehicle signals, simulating the vehicle behavior under the varying parameters and / or conditions; and Adding the re-simulated vehicle signals to the vehicle signals as an extended dataset for training the latent space model. [4] Method according to claim 1, further comprising performing joint training of the analysis model and the latent space model using a loss function that takes into account loss in the latent space model and also loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model. [5] The method of claim 1, further comprising: Defining the universal signals as a vector of a first version of the vehicle signals in combination with additional elements of a future version that are set to a default value, wherein the vehicle signals received by one or more vehicles are of a second version of the vehicle signals, and the training ensures the accuracy of the first version of the vehicle signals and the second version of the vehicle signals through the latent space model. [6] The method of claim 1, wherein the training comprises: Training a multitude of latent space models on subsets of the vehicle signals, each of the multitude of latent space models including an encoder-decoder pair; and Combining the outputs of a variety of encoders, the encoder and decoder pairs, to generate the universal signals. [7] Method according to claim 1, wherein the analysis model determines metrics relating to one or more of vehicle maintenance or usage-based insurance. [8] Method according to claim 1, further comprising: Detecting a change in the distribution of vehicle signals and / or the presence of outlier events in the vehicle signals; Retraining the latent space model to generate an updated encoder; and Sending the updated encoder to the one or more vehicles. [9] System for using universal signals to determine vehicle metrics, comprising: one or more computing devices that include non-transient data storage and a processor and are configured to: Training a latent space model comprising an encoder and a decoder based on vehicle signals received from one or more vehicles, wherein the encoder generates universal signals as a latent representation of the vehicle signals, and wherein the decoder generates reconstructed vehicle signals from the universal signals; Sending the encoder to one or more vehicles; Receiving universal signals from one or more vehicles, wherein the universal signals are generated by using the encoder on the vehicle signals; and Applying the universal signals to an analysis model to determine metrics relating to the operation of one or more vehicles from the latent representation. [10] System according to claim 9, wherein the vehicle signals received from one or more vehicles are filtered according to user filter guidelines which determine which of the vehicle signals are to be included in the universal signals. [11] System according to claim 9, wherein the one or more computing devices are further configured to: Variations in parameters and / or conditions that affect vehicle signals, such as changes in driving style, weather conditions, road types, or vehicle load; Performing a re-simulation using a generative model to generate new instances of the vehicle signals, simulating the vehicle behavior under the varying parameters and / or conditions; and Adding the re-simulated vehicle signals to the vehicle signals as an extended dataset for training the latent space model. [12] System according to claim 9, wherein the one or more computing devices are further configured to perform joint training of the analysis model and the latent space model using a loss function that takes into account loss in the latent space model and also loss in the analysis model, thereby ensuring the accuracy of the universal signals with respect to the analysis model. [13] System according to claim 9, wherein the one or more computing devices are further configured to: Defining the universal signals as a vector of a first version of the vehicle signals in combination with additional elements of a future version that are set to a default value, wherein the vehicle signals received by one or more vehicles are of a second version of the vehicle signals, and the training ensures the accuracy of the first version of the vehicle signals and the second version of the vehicle signals through the latent space model. [14] System according to claim 9, wherein the one or more computing devices are further configured to: Training a multitude of latent space models on subsets of the vehicle signals, each of the multitude of latent space models including an encoder-decoder pair; and Combining the outputs of a variety of encoders, the encoder and decoder pairs, to generate the universal signals. [15] System according to claim 9, wherein the analysis model determines metrics relating to one or more of vehicle maintenance or usage-based insurance.

Images