Method for detecting anomalies in vehicle calibration sets

Abstract

A method for detecting anomalies in vehicle calibration sets, the method comprising: Receiving a plurality of vehicle calibration sets, each vehicle calibration set containing corresponding vehicle parameters; Calculating an anomaly score for each vehicle parameter of each vehicle calibration set; Detecting an anomaly associated with at least one vehicle parameter of a vehicle calibration set of the plurality of vehicle calibration sets based on the anomaly score for the vehicle parameter and a defined threshold; Modifying the vehicle parameter of the vehicle calibration set in response to detecting the anomaly; and Transferring the vehicle calibration set including the modified vehicle parameter to a vehicle control module (108) associated with a vehicle to control at least one component of the vehicle; characterized by transmitting a visualization containing the anomaly score associated with the detected anomaly to a display module (106).

Description

INTRODUCTION

[0001] The information provided in this section is intended to provide a general context for the disclosure. The work of the presently named inventors, to the extent described in this section, as well as those aspects of the description that do not otherwise qualify as prior art at the time of filing, are neither expressly nor implicitly acknowledged as prior art to the present disclosure.

[0002] The present disclosure relates to visualizations and anomaly detection for vehicle calibration sets. Reference is made to DE 10 2015 014 478 A1 and DE 10 2021 121 715 A1 for prior art.

[0003] Vehicles rely heavily on complex calibrations that are essential for defining and optimizing vehicle performance metrics such as vibration, range, battery efficiency, miles per gallon, emissions, etc. A set of calibrations (e.g., parameter values) for a specific vehicle is typically deployed to that vehicle during manufacturing (e.g., flashing at the factory) or in a post-manufacturing scenario (e.g., via over-the-air (OTA) updates or flashing at the dealership). In some cases, a data management system may be employed during the calibration process to verify the calibration values. In other cases, a manual process may be required, evaluating a single comparison of each calibration parameter. SUMMARY

[0004] The present invention is defined by the features of the appended independent claim 1. Advantageous further developments are specified in the following description and in the dependent claims.

[0005] A method for detecting anomalies in vehicle calibration sets includes receiving a plurality of vehicle calibration sets, each vehicle calibration set including corresponding vehicle parameters, calculating an anomaly score for each vehicle parameter of each vehicle calibration set, detecting an anomaly associated with at least one vehicle parameter of a vehicle calibration set of the plurality of vehicle calibration sets based on the anomaly score for the vehicle parameter and a defined threshold in response to detecting the anomaly, modifying the vehicle parameter of the vehicle calibration set, and transmitting the vehicle calibration set including the modified vehicle parameter to a vehicle control module associated with a vehicle for controlling at least one component of the vehicle.

[0006] According to other features, the plurality of vehicle calibration sets include a master vehicle calibration set and one or more successor vehicle calibration sets.

[0007] According to other features, calculating the anomaly score includes calculating a gradient for each corresponding vehicle parameter of the plurality of vehicle calibration sets, calculating one or more numerical relationships for each corresponding vehicle parameter of each successor vehicle calibration set against a corresponding vehicle parameter of the parent vehicle calibration set, and calculating the anomaly score based on a cost function, a weighted value of the gradient, and the weighted values ​​of the one or more numerical relationships.

[0008] According to other features, the one or more numerical relationships include a Euclidean distance and a linear regression.

[0009] According to other features, calculating the anomaly score includes calculating a gradient for each corresponding vehicle parameter of the plurality of vehicle calibration sets and calculating one or more numerical relationships for each corresponding vehicle parameter of each vehicle calibration set against a corresponding vehicle parameter of all other vehicle calibration sets.

[0010] According to other features, calculating the anomaly score includes calculating a preliminary anomaly score for each corresponding vehicle parameter of the plurality of vehicle calibration sets based on a cost function, a weighted value of the gradient, and the weighted values ​​of the one or more numerical relationships, setting the vehicle calibration set having the corresponding vehicle parameter with a lowest preliminary anomaly score as a reference vehicle calibration set, and calculating the anomaly score by setting the preliminary anomaly score for each corresponding vehicle parameter based on the reference vehicle calibration set.

[0011] According to other features, the one or more numerical relationships include a Euclidean distance and a linear regression.

[0012] According to other features, calculating the anomaly score includes calculating a distance for each corresponding vehicle parameter of each successor vehicle calibration set against a corresponding vehicle parameter of the main vehicle calibration set and calculating the anomaly score based on the distance.

[0013] According to other features, calculating the anomaly score includes calculating a normalized value of the distance and setting the anomaly score equal to the normalized value.

[0014] According to other features, calculating the anomaly score includes calculating an average for each corresponding vehicle parameter of the plurality of vehicle calibration sets, calculating a distance for each corresponding vehicle parameter of each vehicle calibration set against the calculated average for that vehicle parameter, and calculating the anomaly score based on the distance.

[0015] According to other features, calculating the anomaly score includes calculating a normalized value of the distance and setting the anomaly score equal to the normalized value.

[0016] In other features, the method further includes detecting whether the plurality of vehicle calibration sets are in a scalar format.

[0017] According to other features, calculating the anomaly score in response to the plurality of vehicle calibration sets being in scalar format includes calculating a distance for each corresponding vehicle parameter of each set of vehicle calibrations against a reference value and calculating the anomaly score based on the distance.

[0018] In other features, the method further includes detecting whether the plurality of vehicle calibration sets are in a vector format or a map format.

[0019] According to other features, calculating the anomaly score specific to each set of vehicle calibrations includes calculating the anomaly score based on a cost function in response to the plurality of vehicle calibration sets being in vector format or map format.

[0020] According to the invention, the method further includes transmitting a visualization containing the anomaly score associated with the detected anomaly to a display module.

[0021] According to other features, the visualization includes at least one chart graphically representing the anomaly score.

[0022] In other features, the method further includes receiving a user input in response to the transmitted visualization, wherein modifying the vehicle parameter of the vehicle calibration set includes modifying the vehicle parameter of the vehicle calibration set in response to the received user input.

[0023] In other features, the method further includes calculating one or more statistical relationships of corresponding vehicle parameters of the plurality of vehicle calibration sets and transmitting a visualization including at least one table graphically representing the one or more statistical relationships to a display module.

[0024] According to other features, the method further includes receiving a defined auto-correction threshold.

[0025] According to other features, modifying the vehicle parameter of the vehicle calibration includes auto-correcting the vehicle parameter of the vehicle calibration to a reference calibration in response to the detected anomaly being greater than the defined auto-correcting threshold.

[0026] In other features, transmitting the vehicle calibration set including the modified vehicle parameter to the vehicle control module includes transmitting the vehicle calibration set to the vehicle control module associated with the vehicle via an over-the-air (OTA) update.

[0027] A method for detecting anomalies in vehicle calibration sets includes receiving a plurality of vehicle calibration sets, each vehicle calibration set including corresponding vehicle parameters divided into two or more sections, calculating an anomaly score for each section of each vehicle calibration set, detecting an anomaly associated with at least a section of a vehicle calibration set of the plurality of vehicle calibration sets based on the anomaly score for the section and a defined threshold, modifying a vehicle parameter in the section of the vehicle calibration set in response to detecting the anomaly, and transmitting the vehicle calibration set including the modified vehicle parameter to a vehicle control module associated with a vehicle for controlling at least one component of the vehicle.

[0028] A system for detecting anomalies in vehicle calibration sets includes a vehicle control module associated with a vehicle for controlling at least one component of the vehicle, and a control module in communication with the vehicle control module.The control module is configured to receive a plurality of vehicle calibration sets, each vehicle calibration set including corresponding vehicle parameters, calculate an anomaly score for each vehicle parameter of each vehicle calibration set, detect an anomaly associated with at least one vehicle parameter of a vehicle calibration set of the plurality of vehicle calibration sets based on the anomaly score for the vehicle parameter and a defined threshold, modify the vehicle parameter of the vehicle calibration set in response to detecting the anomaly, and transmit the vehicle calibration set including the modified vehicle parameter to a vehicle control module associated with the vehicle to control at least one component of the vehicle.

[0029] Further areas of applicability of the present disclosure will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are for purposes of illustration only and are not intended to limit the scope of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present disclosure will be more fully understood from the detailed description and the accompanying drawings, in which: Fig. 1 is a functional block diagram of an exemplary system for detecting anomalies in vehicle calibration sets and generating statistical visualizations of the vehicle calibration sets in accordance with the present disclosure; Fig.2-5 are graphs illustrating anomaly scores for corresponding vehicle parameters of various calibration sets when using a main calibration set, in accordance with the present disclosure; Fig. 6-9 are graphs illustrating anomaly scores for corresponding vehicle parameters of various calibration sets when no main calibration set is used, according to the present disclosure; Fig. 10-13 are graphs illustrating the anomaly scores for the corresponding vehicle parameters of four separate sections of two calibration sets according to the present disclosure; Fig. 14 is a diagram showing the anomaly scores for the vehicle parameters of four common sections from one of the two calibration sets according to the Fig. 10-13 according to the present disclosure; Fig.15 is a diagram showing the anomaly scores for the vehicle parameters of four common sections of another of the two calibration sets according to the Fig. 10-13 according to the present disclosure; Fig. 16 is a diagram showing the anomaly scores for the vehicle parameters of four common sections of the two calibration sets according to the Fig. 10-13 according to the present disclosure; Fig. 17-22 illustrate flowcharts of exemplary control processes for detecting anomalies in vehicle calibration sets, generating statistical visualizations of the vehicle calibration sets, and / or modifying vehicle calibrations in accordance with the present disclosure; Fig.23 is a functional block diagram of an exemplary system for updating one or more calibrations and transmitting the new calibrations to a manufacturing facility and / or vehicle after manufacture in accordance with the present disclosure; Fig. 24 a vehicle that uses sections of the system according to Fig. 1, according to the present disclosure; and Fig. 25 is a graphical representation showing the interactions between nine vehicle calibration sets according to the present disclosure.

[0031] In the drawings, reference symbols may be used multiple times to identify similar and / or identical elements. DETAILED DESCRIPTION

[0032] Calibrations are used in vehicles to define and optimize vehicle performance metrics, such as vibration, range, battery efficiency, miles per gallon, emissions, etc. The number of calibrations per vehicle is significant. For example, in a vehicle, the electric drive system alone may utilize over 18,000 calibratable parameters critical to optimal vehicle performance. Other controllable vehicle components may utilize a similar number of parameters. The total number of parameters for the vehicle itself can expand into the hundreds of thousands. Such calibrations (e.g., parameter values) are provided to vehicles during manufacturing or in post-manufacturing scenarios (e.g., flashed into a vehicle control module's memory circuit or its associated memory circuit).In some cases, faulty calibrations can be inadvertently released to vehicles, which may require recalls. For example, faulty calibrations pose significant risks to users (e.g., drivers, passengers, bystanders, etc.) and vehicles, compromising user safety and vehicle performance. Additionally, faulty calibrations can impose significant and burdensome economic consequences should recalls become necessary after production. While recalls are undesirable due to their huge financial and safety implications, recalls occur more frequently than desired, with calibration outliers / anomalies often being the primary cause.

[0033] Some data management systems are available for verifying vehicle calibrations. However, such systems cannot handle large data sets and / or are unreliable when processing large data sets of calibrations across multiple vehicle programs (e.g., particularly comparing curves and maps across multiple data sets, which is very laborious). As a result, companies often resort to a manual approach, a process that involves comparing each calibration parameter individually. This manual approach is both time-consuming and prone to human error. Like existing data management systems themselves, the manual approach is impractical and unreliable.

[0034] The systems and methods according to the present disclosure provide technical solutions for implementing anomaly detection algorithms that can examine scalar and complex calibration maps (e.g., 1D, 2D, 3D, etc. calibration maps) to accurately identify anomalies in parameter values ​​associated with various vehicle calibration sets containing thousands (and in some cases, hundreds of thousands) of parameters, and then for providing clear visualizations of the vehicle calibration sets. With such systems and methods, an array of complex calibration anomalies can be summarized into a single, easily interpretable numerical representation, providing calibration engineers with a powerful tool to locate and correct discrepancies prior to integration into the vehicle production line, as further explained herein.Additionally, the systems and processes remain functional after manufacturing, providing a safety net through the ability to push necessary repairs or solutions to the customer's vehicle via over-the-air (OTA) updates, thereby avoiding potential recalls while improving vehicle safety and quality in real time.

[0035] The detection of anomalies in parameter values ​​offers numerous additional benefits. For example, by enabling rapid responses to calibration errors, the technical solutions significantly support vehicle safety and proactively avert potential hazards. In addition, the technical solutions facilitate the early detection and correction of calibration errors, avoiding the significant costs associated with vehicle recalls. Furthermore, when manufacturing a vast range of vehicles, each of which includes thousands (and in some cases hundreds of thousands) of unique calibrations and numerous variants, the technical solutions enable scalability that seamlessly integrates into expanding production ecosystems.Furthermore, the technical solutions here help to detect safety-critical calibration errors before the errors reach customers, save millions of dollars otherwise spent on repairing recalls, and improve performance by updating feature calibrations.

[0036] In addition, the technical solutions here significantly reduce the time and cost requirements associated with vehicle calibration processes. For example, the technical solutions can provide a reduction in engineering time, on average across several different vehicle programs, of at least sixteen weeks per engineer per year compared to conventional systems that do not utilize the technical solutions here. As a result of such significant engineering time savings, engineers can focus their attention on other requirements, costs can be significantly reduced, etc.

[0037] In Fig. 1, a block diagram of an exemplary system 100 for detecting anomalies in vehicle calibration sets and generating statistical visualizations of the vehicle calibration sets is shown. The system 100 according to Fig. 1 may be applicable to any suitable vehicle, such as an electric vehicle (e.g., a pure electric vehicle, a plug-in hybrid vehicle, etc.), an internal combustion engine vehicle, etc. Additionally, system 100 may be applicable to autonomous vehicles, such as semi-autonomous vehicles and fully autonomous vehicles. By way of example, and as further explained, the vehicle calibration sets analyzed and, in some cases, modified by system 100 may be provided for one or more vehicle control modules in any suitable vehicle.

[0038] As in Fig.1, the system 100 generally includes a control module 102, a vehicle calibration database 104, a display module 106, and at least one vehicle control module 108. According to such examples, the control module 102 may be in communication with any of the other components, as shown in Fig. 1. Although Fig. 1 illustrates system 100 as including specific modules, it should be appreciated that one or more other modules may be used if desired. Additionally, while system 100 is shown as including multiple separate modules, any combination of the modules and / or their functionality may be incorporated into one or more modules.

[0039] According to the example Fig.1, the vehicle control module 108 may be any suitable controller associated with a vehicle for controlling at least one component of the vehicle. According to such examples, the vehicle control module 108 may be any control module within the vehicle that receives calibratable parameters. For example, the vehicle control module 108 may be an engine control module (ECM) that controls a motor / engine (e.g., an electric motor, an internal combustion engine, etc.), a battery control module for controlling battery operations (e.g., output current, charging current, etc.), an inverter control module for controlling outputs to the battery, etc.

[0040] While the system 100 in Fig.1 is shown as including only one vehicle control module 108, it should be appreciated that system 100 may include multiple vehicle control modules. According to such examples, each vehicle control module may receive dedicated calibratable parameters specific to that vehicle control module.

[0041] The display module 106 may be any suitable device having a display. For example, the display module 106 may be a user device such as a laptop, a telephone, a monitor, a desktop computing device, etc. According to such examples, the display module 106 displays one or more visualizations related to the analysis of vehicle calibration sets, as further explained herein.

[0042] The vehicle calibration database 104 according to Fig.1 stores various vehicle calibration sets, each of which can be received by the control module 102. Each set of vehicle calibrations can, for example, contain multiple parameters (e.g., thousands, etc.) to control various aspects of the vehicle. Each parameter can have an associated value. While the system 100 is searching for Fig. 1 is shown as containing only one vehicle calibration database 104, it should be appreciated that the system 100 may contain multiple databases if desired.

[0043] According to various embodiments, the vehicle calibration sets and / or the parameters therein may be in different formats. For example, each calibration set may contain a combination of scalar, curve, and map formats. The vehicle calibrations in a calibration set may, for example, include a scalar format where the parameters are mapped to a single value, a vector (or curve) format where the parameters are mapped to multiple values ​​in a line (e.g., the values ​​on one axis alone), and / or a map format where the parameters are mapped to multiple values ​​across multiple lines (e.g., a grid of values ​​mapped to two axes). According to some examples, the scalar format, the vector format, and the map format may be referred to as a 1D (one-dimensional) format, a 2D (two-dimensional) format, and a 3D (three-dimensional) format, respectively.

[0044] According to some examples, the stored vehicle calibration sets may include master and slave vehicle calibration sets. For example, a master vehicle calibration set may include vehicle calibrations associated with a vehicle program (e.g., a specific type of manufactured vehicle). Similarly, the slave vehicle calibration sets may include the vehicle calibrations associated with various vehicle configurations for the vehicle program or associated programs that share components with the vehicle program (e.g., various example configurations for the specific type of manufactured vehicle). According to various embodiments, each vehicle program may have over fifty vehicle calibration sets, with each vehicle calibration set having thousands of parameters.

[0045] According to other examples, the vehicle calibration sets may not include a master vehicle calibration set. According to such examples, the successor vehicle calibration sets may be analyzed from the same vehicle program (e.g., a specific type of manufactured vehicle) or from different vehicle programs (e.g., from two or more types of manufactured vehicles), as further explained below.

[0046] Continue in Fig.1, the control module 102 receives the vehicle calibration sets from the vehicle calibration database 104 and / or another suitable source. The control module 102 may, for example, import any number of vehicle calibration sets, such as all vehicle calibration sets for one vehicle program, all or selected vehicle calibration sets for multiple vehicle programs, etc. Additionally, the control module 102 may import and support multiple data formats, including DCM, CSV, etc. According to such examples, users (e.g., technicians, engineers, etc.) may be able to request the desired calibration sets in various formats via user input (e.g., an input signal 110 to Fig.1). According to various embodiments, multiple vehicle calibration sets can be imported simultaneously, providing flexibility. According to other examples, the vehicle calibration sets can be imported sequentially if desired.

[0047] According to some examples, control module 102 may filter and / or sort the received vehicle calibration sets and / or the parameters therein. For example, a user may select an input option to filter one or more vehicle calibration sets. In this way, the analysis of the vehicle calibration sets may be focused on specific subsets of calibrations for targeted comparisons, allowing the user to focus on specific calibration parameters of primary interest.

[0048] The control module 102 may be operable to perform a calibration comparison process. The control module 102 may, for example, calculate (e.g., substantially in real time) one or more statistical relationships of corresponding vehicle parameters of the vehicle calibration sets received from the vehicle calibration database 104 and / or another suitable source. The statistical relationships (e.g., the statistical measures) may, for example, include a mean, a standard deviation, a variance, and / or any other suitable computable relationship between the parameters. According to various embodiments, the calculated statistical relationships may be defined by a user. According to some examples, the control module 102 may automatically begin such calculations in response to receiving the vehicle calibration sets, in response to user input (e.g., via signal 110, etc.).

[0049] According to various embodiments, the control module 102 calculates the anomaly scores associated with some or all of the received vehicle calibration sets. More specifically, the control module 102 may calculate the anomaly scores for the vehicle parameters in the vehicle calibration sets and / or sections (e.g., a collection of vehicle parameters) within the vehicle calibration sets. In this way, the control module 102 may identify one or more vehicle parameter outliers in one set of vehicle calibrations compared to corresponding vehicle parameters in other sets of vehicle calibrations.

[0050] According to the example Fig.1, the control module 102 may calculate the anomaly scores differently based on one or more conditions. For example, and as further explained below, the control module 102 may calculate the anomaly scores for the vehicle parameters in a scalar format in a different manner than for the vehicle parameters in a vector or map format. According to such examples, the control module 102 may identify the format type based on the data (e.g., a size of the data, etc.).

[0051] For example, the control module 102 may detect whether the received vehicle calibration sets are in a scalar format or a vector or map format. Then, the control module 102 may calculate the anomaly scores using different algorithms based on the format type. For example, in response to the vehicle calibration sets being in scalar format, the control module 102 may calculate a distance for each corresponding vehicle parameter of each set of vehicle calibrations against a reference value and then calculate an anomaly score based on the distance, as further explained below. Alternatively, in response to the vehicle calibration sets being in vector or map format, the control module 102 may calculate an anomaly score based on a cost function, as further explained below.

[0052] Additionally, the control module 102 may calculate the anomaly scores in a different manner based on whether a master vehicle calibration set is known. According to such examples, the control module 102 may detect whether a vehicle calibration set is a master set based on user input (e.g., the set identified by a user), a label provided with the set of vehicle calibrations from the vehicle calibration database 104, etc.

[0053] For example, the control module 102 may determine that the received vehicle calibration sets include a master vehicle calibration set and one or more slave vehicle calibration sets. According to such examples, the control module 102 may treat the master vehicle calibration set as a reference set of vehicle calibrations for comparisons to the slave vehicle calibration sets. According to other examples, the received vehicle calibration sets may not include a master vehicle calibration set and / or the control module 102 may not be able to determine a master vehicle calibration set (e.g., if received). According to such examples, the control module 102 may calculate the reference values ​​and / or identify one of the vehicle calibration sets (e.g., under the same vehicle program or different vehicle programs) as a reference set of vehicle calibrations, as further explained below.

[0054] The following are example operations performed by control module 102 depending on whether a master vehicle calibration set is available and the type of data format (e.g., scalar, vector, or map). While control module 102 is described as performing such operations, it should be recognized that other example operations (e.g., calculations, algorithms, etc.) may be performed if desired.

[0055] As mentioned above, if the vehicle calibration sets are in a scalar format, the control module 102 may calculate an anomaly score based on a certain distance for each corresponding vehicle parameter of each set of vehicle calibrations against a reference value. For example, if a master vehicle calibration set is available, the control module 102 calculates a distance for each corresponding vehicle parameter of each successor vehicle calibration set against a corresponding vehicle parameter of the master vehicle calibration set, and then calculates the anomaly score for that vehicle parameter based on the calculated distance. According to various embodiments, the control module 102 may calculate a normalized value of the distance and then set the normalized value as the anomaly score for that vehicle parameter.

[0056] If no master vehicle calibration set is available, the control module 102 may alternatively calculate reference values. For example, the control module 102 calculates an average value for each corresponding vehicle parameter of the received vehicle calibration sets. Next, the control module 102 calculates a distance for each corresponding vehicle parameter of each set of vehicle calibrations against the calculated average value for that vehicle parameter, and then calculates the anomaly score for that vehicle parameter based on the calculated distance. Similar to the above, the control module 102 may calculate a normalized value of the distance (with respect to the average value) and then set the normalized value as the anomaly score for that vehicle parameter.

[0057] However, if the vehicle calibration sets are in a vector or map format (e.g., a 2D or 3D format), the control module 102 may calculate an anomaly score based on a cost function, as mentioned above. For example, if a main vehicle calibration set is available, the control module 102 calculates a gradient for each corresponding vehicle parameter of the vehicle calibration sets. According to such examples, the gradient may be calculated in conventional ways.

[0058] The control module 102 then calculates one or more numerical relationships for each corresponding vehicle parameter of each follower vehicle calibration set against a corresponding vehicle parameter of the master vehicle calibration set. For example, the control module 102 may calculate a Euclidean distance and a linear regression relationship for each corresponding follower vehicle parameter against a corresponding master vehicle parameter (e.g., a reference parameter).

[0059] Next, the control module 102 implements a cost function relationship to calculate the anomaly score for each vehicle parameter. For example, the control module 102 may use a cost function that sums the gradient, Euclidean distance, and linear regression associated with a vehicle parameter to obtain the anomaly score. According to various embodiments, each of the gradient, Euclidean distance, and linear regression may be weighted before summing the values. For example, the gradient may be multiplied by a defined weighted value (e.g., a value between 0.1 and 0.9), the Euclidean distance may be multiplied by a defined weighted value (e.g., a value between 0.1 and 0.9), and the linear regression may be multiplied by a defined weighted value (e.g., a value between 0.1 and 0.9).

[0060] Alternatively, if no master vehicle calibration set is available for a vector or mapped format analysis, the control module 102 may identify one of the vehicle calibration sets as a reference set of vehicle calibrations and then determine the anomaly score for each vehicle parameter based on that reference set. For example, the control module 102 calculates a gradient for each corresponding vehicle parameter of the vehicle calibration sets, as explained above.

[0061] The control module 102 then calculates one or more numerical relationships (e.g., a Euclidean distance and a linear regression relationship) for each corresponding vehicle parameter of each set of vehicle calibrations against a corresponding vehicle parameter of all other vehicle calibration sets. A Euclidean distance between a vehicle parameter in one calibration set with respect to a corresponding vehicle parameter of another calibration set may be, for example, 20, while a Euclidean distance between the same vehicle parameter in one calibration set with respect to another corresponding vehicle parameter of yet another calibration set may be, for example, 30. The control module 102 then separately averages the Euclidean distances and the linear regression values ​​for each parameter (e.g.,A map of a calibration set with respect to the others to obtain a single Euclidean distance and a single linear regression value for that calibration set parameter. The single Euclidean distance may, for example, continue to be 25 (e.g., (20 + 30) / 2) with respect to the above example.

[0062] The control module 102 then calculates a preliminary anomaly score for each corresponding vehicle parameter based on a cost function, as explained above. The cost function may, for example, add the gradient, the individual (e.g., averaged) Euclidean distance, and the individual (e.g., averaged) linear regression value associated with a vehicle parameter to obtain the preliminary anomaly score. According to various embodiments, each of the gradient, the Euclidean distance, and the linear regression may be weighted before summing the values, as explained above. According to such examples, the weighted values ​​may be different for when the main vehicle calibration set is unavailable and for when the main vehicle calibration set is available.

[0063] Next, the control module 102 determines which preliminary anomaly score is lowest for a given parameter across the calibration sets, and then sets the corresponding vehicle calibration set with the lowest preliminary anomaly score as the reference set. For example, the control module 102 may set a value of the parameter corresponding to the lowest preliminary anomaly score to zero.

[0064] The control module 102 then sets the other preliminary anomaly scores for the corresponding vehicle parameters of the other calibration sets based on the reference set of vehicle calibrations. For example, if the preliminary anomaly scores are 18.5, 30.5, or 48, the control module 102 sets a value of the parameter corresponding to the lowest preliminary anomaly score (18.5) to zero. The control module 102 then subtracts 18.5 from each of the other preliminary anomaly scores for the corresponding vehicle parameters of the other calibration sets. In other words, the control module 102 sets the other preliminary anomaly scores to 12 (e.g., 30.5-18.5) and 29.5 (e.g., 48-18.5). The set preliminary anomaly scores (e.g., 0, 12, 29.5) can then be provided as the anomaly scores.

[0065] Although the above example operations performed by the control module 102 have been described with respect to determining an anomaly score specific to a parameter in a vehicle calibration set, it should be appreciated that similar calculations may be implemented for a collection of vehicle parameters in a vehicle calibration set (e.g., there could be tens of thousands per calibration set). For example, each vehicle calibration set may include vehicle parameters divided into two or more sections. According to such examples, an anomaly score for each section may be determined in a manner similar to that explained above with respect to an anomaly score for a vehicle parameter.

[0066] According to various embodiments, the control module 102 may detect an anomaly associated with at least one vehicle parameter of a set of vehicle calibrations based on a calculated anomaly score for the vehicle parameter. For example, the control module 102 may compare the calculated anomaly score to a defined threshold (e.g., a single value, a range of values, etc.) and detect an anomaly if the anomaly score is greater than the defined threshold.

[0067] By way of example only, the defined threshold may be a percentage (e.g. 10%, 12%, 15%, 17%, 20%, etc.) of the reference value used as explained above.

[0068] Additionally, according to some examples, the control module 102 may transmit at least one visualization to the display module 106 and / or another suitable module for displaying the visualization. The visualization may include a calculated anomaly score, whether or not the anomaly score is associated with a detected anomaly. For example, the control module 102 may transmit the visualization with the anomaly score in response to the anomaly score being associated with a detected anomaly. According to other examples, the control module 102 may transmit the visualization with the anomaly score even if no anomaly is detected.

[0069] According to various embodiments, the control module 102 may compile the calculated data in various visual formats for viewing by the user. For example, the control module 102 may leverage 2D and 3D visualization techniques to make complex calibration comparisons easily digestible and visually recognizable for users. For example, the control module 102 may provide the visualization with one or more charts graphically representing the anomaly scores, or with one or more tables graphically representing anomaly scores, etc. By way of example only, the Fig. 2-14 different charts 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200, 1300, 1400, 1500, 1600, showing different visualizations of the anomaly scores. According to the examples after the Fig. 2-16 the vehicle parameter refers to battery current values.

[0070] The diagrams 200, 300, 400, 500 according to the Fig. 2-5, for example, represent the anomaly scores for a corresponding parameter across four different calibration sets when a main vehicle calibration set was used as a reference. According to such examples, the diagram 200 represents Fig. 2 represents the parameter of the main vehicle calibration set with an anomaly score of zero (0). The diagram 300 according to Fig. 3 represents the corresponding parameter of a follower vehicle calibration set (successor No. 2) with an anomaly score of zero (0). As shown, the diagram 300 is Fig. 3 to the diagram 200 according to Fig. 2 completely equal. The diagram 400 after Fig. 4 represents the corresponding parameter of another successor vehicle calibration set (successor No. 3) with an anomaly score of 45.7535, while the diagram 500 after Fig.5 shows the corresponding parameter of yet another follower vehicle calibration set (follower No. 4) with an anomaly score of 46.2335. As shown, diagrams 400 and 500 differ according to the Fig. 4-5 significantly from diagram 200 to Fig. 2, which indicates parameter outliers (and possible anomalies). More specifically, the middle sections and the lower right sections of the graphs 400, 500 differ according to the Fig. 4-5 significantly from the corresponding sections of diagram 200 to Fig. 2.

[0071] The diagrams 600, 700, 800, 900 according to the Fig.6-9 represent the anomaly scores for a corresponding parameter across four different calibration sets when no main vehicle calibration set was available as a reference. According to such examples, one of the calibration sets is set as a reference, as explained above. According to this example, the diagram 600 represents Fig. 6 represents the parameter of the calibration set used as the reference (e.g., the calibration set with the lowest preliminary anomaly, as explained above). As such, the anomaly score associated with the chart 600 is zero (0). The chart 700 according to Fig. Figure 7 shows the corresponding parameter of another set of calibrations (set no. 2) with an anomaly score of 0.17544 (e.g., after the settings as explained above). As shown, the diagram 700 is after Fig. 7 to diagram 600 according to Fig.6 almost identical. The diagram 800 after Fig. Figure 8 shows the corresponding parameter of another set of calibrations (set no. 3) with an anomaly score of 14.7258, while the diagram 900 after Fig. 9 shows the corresponding parameter of yet another set of calibrations (set no. 4) with an anomaly score of 15.1944. As shown, the diagrams 800, 900 differ according to the Fig. 8-9 significantly from diagram 600 to Fig. 6, which indicates parameter outliers (and possible anomalies).

[0072] According to the examples in the Fig. 2-9, the parameter values ​​of the different vehicle calibration sets are the same, with the only difference that a main calibration set is created according to the examples after the Fig. 2-5 and not according to the examples in the Fig.6-9. As shown, regardless of whether a main calibration set (e.g., an initial reference set) is employed, the control module 102 can accurately calculate anomaly scores and identify anomalies (e.g., charts 400, 500, 800, 900). This ensures effectiveness in all practical scenarios and demonstrates unprecedented versatility and innovation in addressing industry requirements. For example, in some scenarios, a new vehicle program may be introduced that has no main calibration set and therefore no starting point. According to such examples, the control module 102 can accurately calculate the anomaly scores and identify the anomalies without this starting point.

[0073] The diagrams 1000, 1100, 1200, 1300 according to the Fig.10-13 depict the anomaly scores for four different corresponding sections of two different calibration sets. In such configurations, a user can identify one or more specific sections that are causing problems (e.g., due to outlier parameter values ​​in the specific sections), thereby providing a granular view of the analysis. Specifically, chart 1000 depicts a first section 1002-1 of the parameters for a master calibration set 1002 and a corresponding first section 1004-1 of the parameters for a successor calibration set 1004.Diagrams 1100, 1200, and 1300 illustrate similar characteristics for the main calibration set 1002 and the successor calibration set 1004, but diagram 1100 focuses on the second sections 1002-2, 1004-2, diagram 1200 focuses on the third sections 1002-3, 1004-3, and diagram 1300 focuses on the fourth sections 1002-4, 1004-4. As shown, the main calibration set 1002 is generally located below the successor calibration set 1004.

[0074] In addition, the example shows the Fig.10-13, each section of the main calibration set 1002 has an anomaly score of zero (0), the first section 1004-1 of the successor calibration set 1004 has an anomaly score of 44.8055, the second section 1004-2 of the successor calibration set 1004 has an anomaly score of 65.5232, the third section 1004-3 of the calibration set 1004 has a successor anomaly score of 34.9958, and the fourth section 1004-4 of the calibration set 1004 has a successor anomaly score of 19.9895. According to such examples, a particular section (e.g., the second section) may be identified as causing the most problems because it has the highest anomaly score.

[0075] The diagrams 1400, 1500, 1600 according to the Fig. 14-16 show other examples of the representations of diagrams 1000, 1100, 1200, 1300 according to the Fig. 10-13. The diagram 1400 after Fig.For example, Figure 14 shows the four sections of diagrams 1000, 1100, 1200, 1300 for the main calibration set 1002, while diagram 1500 is Fig. 15 shows the four sections of diagrams 1000, 1100, 1200, 1300 for the successor calibration set 1004. In addition, diagram 1600 represents Fig. 16 shows the four sections of the diagrams 1000 (anomaly score 44.8055), 1100 (anomaly score 65.5232), 1200 (anomaly score 34.9958) and 1300 (anomaly score 19.9895) for both the main calibration set 1002 and the successor calibration set 1004.

[0076] Continue in Fig.1, the control module 102 may transmit at least one visualization comprising at least one table graphically depicting various variables and / or the previously calculated statistical relationships. According to such examples, the control module 102 may transmit such data to the display module 106 and / or another suitable module for displaying the table. For example, the table may contain different parameter values ​​(in different formats, such as scalar, vector, and map) for different calibration sets of the same vehicle program. According to other examples, the table may contain different parameter values ​​(in different formats, such as scalar, vector, and map) for different vehicle programs.

[0077] Table 1 below is an exemplary illustration of the data in a visualization that may be displayed on the display module 106 to a user. As shown below, Table 1 contains corresponding parameters for different calibration sets (Cal Set 1, Cal Set 2, ..., Cal Set 59, Cal Set 60) of the same vehicle program (VP1). According to this example, the parameters are shown in different formats, such as scalar, vector, and map. According to various embodiments, the calibration set (Cal Set 1) may be a main calibration set, while the other calibration sets (Cal Set 2, ..., Cal Set 59, Cal Set 60) may be successor calibration sets associated with the main calibration set in a vehicle program. While Table 1 shows sixty (60) calibration sets for the vehicle program (VP1), it should be recognized that more or fewer may be implemented.

[0078] As shown below, the scalar parameters are represented by "value," the scalar parameters are represented by arrows, and the map parameters are represented by grids. According to such examples, each scalar parameter (value) is a single value (e.g., 50, 80, 52, 45, etc.), each vector parameter can include different values ​​along one axis, and each map parameter can have different values ​​along two axes. For example, a vector parameter can include x-axis values ​​(e.g., battery current values, etc.) of 1000, 2000, 3000, 4000 and corresponding parameter values ​​(e.g., torque values, etc.) of 10, 20, 30, and 40 for the x-axis values, respectively. A map parameter can contain x-axis values ​​(e.g. battery current values, etc.) of 1000, 2000, 3000, 4000, y-axis values ​​(e.g. vehicle speed values, etc.) of 100, 200, 300, 400, and sixteen parameter values ​​for the grid of x- and y-axis values.

[0079] Additionally, each scalar parameter (value) in Table 1 can be any suitable statistical measure requested by the user. Any of the scalar parameters can be, for example, a mean, a mean difference, a standard deviation, a covariance, a percentage difference, a relative error, a maximum difference, a minimum difference, an anomaly score, an absolute raw value, etc. Such statistical relationships can be calculated by the control module 102, as explained above. Table 1 Cal Set 1 (VP1) Cal Set 2 (VP1) ... Cal Set 4 (VP1) Cal Set 5 (VP1) Scalar 1 (example parameter) Value Value ... Value Value Scalar 2 (example parameter) Value Value ... Value Value Vector 3 (example parameter) ↗ ↗ ... ↗ ↗ Map 4 (example parameter) ... ... ... ... ... ... ... Scalar 2000 (example parameter) Value Value ... Value Value

[0080] Table 2 below is another example illustration of the data in a visualization that can be displayed on the display module 106 for a user. As shown below, Table 2 contains different variants (Variant 1, Variant 2, ..., Variant 5) for different vehicle programs (VP1, VP2, VP30). According to such examples, each variant can represent different parameters in different formats, such as scalar, vector, and map, for different vehicle calibration sets. Similar to Table 1 above, the scalar parameters in Table 2 are represented by "value," the scalar parameters in Table 2 are represented by arrows, and the map parameters in Table 2 are represented by grids.

[0081] According to this example, the total number of variants for a single vehicle program is five, differentiated by vehicle features (e.g., options, etc.), such as rear-wheel drive (RWD), all-wheel drive (AWD), long-range battery, etc. Additionally, according to this example, the total number of vehicle programs. While the total number of variants for a vehicle program and the total number of vehicle programs are specified as thirty-five, it should be recognized that such values ​​are only examples and that more or fewer variants per vehicle program and / or vehicle programs may be used if desired, depending on, for example, the available vehicle features (e.g., options, etc.), the number of manufacturer variants, the number of manufacturer vehicle programs, etc.

[0082] According to various embodiments, Table 2 may represent an analysis for a motor calibration control module. According to other examples, the same or a different table (e.g., format, variants, etc.) may be used for a battery calibration control module, an inverter calibration control module, a diagnostic calibration control module, etc. Furthermore, and as shown below in Table 2, any combination of variants and vehicle programs may be compared to find one or more possible anomalies, since many sub-features of one vehicle (e.g., one vehicle program) may have similar calibrations to the sub-features of another vehicle (e.g., another vehicle program). Table 2 Variant 1 (VP1) Variant 2 (VP2) ... Variant 4 (VP29) Variant 5 (VP30) Scalar 1 (example parameter) Value Value ... Value Value Scalar 2 (example parameter) Value Value ... Value Value Vector 3 (example parameter) ↗ ↗ ... ↗ ↗ Map 4 (example parameter) ... .................. ... ... ... ... ... Scalar 2000 (example parameter) Value Value ... Value Value

[0083] Continue in Fig.1, the control module 102 may modify one or more vehicle parameters of a vehicle calibration set. For example, in response to detecting an anomaly associated with a vehicle parameter based on an anomaly score (as explained herein), the control module 102 may automatically modify a value of that vehicle parameter. According to such examples, the control module 102 may change the vehicle parameter to a defined value (e.g., as defined by a user, etc.).

[0084] For example, once the anomaly is detected and the anomaly is above a defined threshold set by a user, the control module 102 may implement an automated auto-correction feature to correct the errors in the vehicle parameters (calibrations) compared to the reference calibrations. According to such examples, the control module 102 may, if the sections of a calibration map (e.g., as shown in the Fig. 10-16) have different anomaly scores, implement the automated autocorrection feature in the sections that are above the defined threshold set by the user, so that only those sections can be autocorrected and automated across all of the thousands of calibration parameters.

[0085] According to other examples, the control module 102 may modify one or more vehicle parameters of a vehicle calibration set based on a user input. For example, the control module 102 may display the calculated anomaly scores associated with the vehicle parameters and / or sections in the form of charts (e.g., any one or more of the charts according to the Fig. 2-16), tables, and / or another visualization (e.g., to the display module 106). According to other examples, the control module 102 may transmit the parameter values ​​and statistical relationships (e.g., Table 1, Table 2, etc.) along with the anomaly scores.

[0086] A user can then analyze the displayed data and determine whether a modification is desired. For example, in some scenarios, it may be desirable to have an outlier parameter value, or a modification of one parameter may affect another parameter. As such, the user can decide whether a modification of the vehicle parameter is desired. According to such examples, the display module 106 and / or another suitable device can transmit a signal (via a user input) to the control module 102 instructing the control module 102 to modify the vehicle parameter (possibly to a defined value). The control module 102 can then modify the vehicle parameter in response to the received signal.

[0087] According to other examples, a user may prefer to modify a vehicle parameter of the vehicle calibration set even if no anomaly associated with the vehicle parameter is detected. For example, in cases where the vehicle parameter may be a bit of an outlier but is not a problem, the user may provide user input to control module 102 to modify the vehicle parameter. According to such examples, while the modification is not necessary to correct a problem, it may assist in achieving better vehicle performance (e.g., optimizing it).

[0088] According to various embodiments, the control module 102 may transmit one or more vehicle calibration sets to the vehicle control module 108 after Fig.1. For example, the control module 102 may transmit the vehicle calibration sets with one or more previously modified vehicle parameters, as explained above. According to other examples, the control module 102 may transmit the vehicle calibration sets with one or more unmodified vehicle parameters if no modifications are required. According to such examples, the vehicle calibrations may be flashed into the memory of the vehicle control module 108 and / or provided to the vehicle control module 108 in another suitable manner.

[0089] According to some examples, the control module 102 may transmit the vehicle calibration sets at different times. For example, the control module 102 may transmit (e.g., push) the vehicle calibration sets containing one or more modified vehicle parameters during the vehicle manufacturing process. According to other examples, the anomalies associated with one or more vehicle parameters may not be visible until after the vehicle is manufactured. According to such examples, the control module 102 may transmit (e.g., push) the vehicle calibration sets containing one or more modified vehicle parameters to the vehicle control module 108 in a vehicle after manufacturing (e.g., during a recall process, OTA updates, etc.).

[0090] The Fig. 17-22 illustrate exemplary control processes 1700, 1800, 1900, 2000, 2100, 2200 executed by the control module 102 according to Fig.1 for detecting anomalies in vehicle calibration sets, generating statistical visualizations of the vehicle calibration sets, and / or modifying the vehicle calibrations. Although the exemplary control processes 1700, 1800, 1900, 2000, 2100, 2200 with respect to the system 100 of Fig. 1, which includes the control module 102, each of the control processes 1700, 1800, 1900, 2000, 2100, 2200 may be implemented by any suitable system.

[0091] As in Fig.17, control begins at 1702, where the control module 102 receives or imports the vehicle calibration sets, as explained herein. The vehicle calibration sets may contain, for example, the vehicle parameters with the same or different data formats (e.g., DCM, CSV, etc.). Control then continues to 1704, where the control module 102 may optionally filter and / or sort the data in the received vehicle calibration sets, as explained above. Control then continues to 1706 and 1708.

[0092] At 1706, the control module 102 may optionally calculate or otherwise determine one or more statistical relationships (e.g., statistical measures such as a mean, a standard deviation, a variance, and / or any other suitable computable relationship between parameters) for the vehicle calibration sets. According to such examples, the control module 102 may automatically initiate such calculations in response to receiving the vehicle calibration sets, in response to filtering, in response to user input, etc.

[0093] At 1708, the control module 102 detects a data type of one or more of the received vehicle calibration sets. By way of example, and as explained above, the control module 102 may detect whether the received vehicle calibration sets are in a scalar format, a vector format, or a map format. According to such examples, the control module 102 may identify the format type based on the data (e.g., a size of the data, etc.), based on the tagged data, etc. Control then proceeds to 1710.

[0094] At 1710, the control module 102 determines whether the detected vehicle calibration sets are in a scalar format. If so, control proceeds to 1712. However, if the vehicle calibration sets are not in a scalar format (e.g., are instead in a vector format or a map format), control proceeds to 1718.

[0095] At 1712, the control module 102 determines from the vehicle calibration sets whether a master calibration set is present. By way of example, and as explained above, the control module 102 may detect whether one of the vehicle calibration sets is a master calibration set based on user input (e.g., the set identified by a user), a label provided with the set of vehicle calibrations, etc.

[0096] If the main calibration set is present, control transfers to 1714, where the control module 102 calculates the anomaly scores based on the existing main calibration set, as explained herein. Control then transfers to 1728. However, if the main calibration set is not present or cannot be identified, control transfers to 1716, where the control module 102 calculates the anomaly scores based on all vehicle calibration sets, as explained herein. Control then transfers to 1728.

[0097] At 1718, the control module 102 determines from the vehicle calibration sets whether a main calibration set is present, as explained herein. If the main calibration set is present and the data is not in scalar format (e.g., is instead in vector or map format), control continues to 1720. At 1720, the control module 102 calculates an anomaly score based on the main calibration set and the weighting factors, as explained herein. Control then continues to 1728.

[0098] However, if the main calibration set is not present and the data is not in a scalar format (e.g., is instead in a vector format or map format), control continues to 1722. At 1722, the control module 102 calculates the anomaly scores (e.g., preliminary anomaly scores) based on all vehicle calibration sets and weighting factors, as explained herein. The control module 102 may, for example, calculate a preliminary anomaly score for each corresponding vehicle parameter of all vehicle calibration sets based on a cost function. As such, the preliminary anomaly score for a vehicle parameter of each vehicle calibration set may be related to the corresponding vehicle parameter of the other vehicle calibration sets. Fig.Figure 25 illustrates an exemplary graphical representation 2500 showing such an interaction between nine different vehicle calibration sets 2502, 2504, 2506, 2508, 2510, 2512, 2514, 2516, 2518. While the graphical representation 2500 according to Fig. 25 contains nine vehicle calibration sets, it should be recognized that more or fewer vehicle calibration sets (e.g., six vehicle calibration sets, twelve vehicle calibration sets, 24 vehicle calibration sets, etc.) may be used.

[0099] Continue in Fig. 17, control then proceeds to 1724 and 1726. At 1724, the control module 102 establishes the vehicle calibration set with the lowest preliminary anomaly score as a reference set. Then, at 1726, the control module 102 sets the other calculated preliminary anomaly scores based on the reference set, as explained herein. Control then proceeds to 1728.

[0100] At 1728, the control module 102 generates and outputs one or more visualizations with the calculated anomaly scores (from 1714 or 1716 or 1720 or 1726) and / or the calculated statistical relationships (from 1706). By way of example, and as explained herein, the control module 102 may compile the calculated data in various visual formats for viewing by the user, such as charts, tables, etc. Then, the control module 102 may send the charts, tables, etc. to the display module 106 after Fig. 1 and / or another suitable module for displaying the one or more visualizations, as explained herein. Control then proceeds to 1730.

[0101] At 1730, the control module 102 determines whether any input is received. For example, a user may analyze the data provided to the display module 106 and determine whether a parameter modification is desired, as explained herein. If so, the user may transmit user input (e.g., via the display module 106 and / or another suitable device) to the control module 102. The user input may instruct the control module 102 to modify one or more specific vehicle parameters of specific calibration sets. If no user input is received, control continues to 1734.

[0102] However, if control module 102 receives a user input, control transfers to 1732, where control module 102 corrects or modifies the specific vehicle parameters. Control then transfers to 1734.

[0103] At 1734, the control module 102 transmits one or more vehicle calibration sets to a vehicle control module (e.g., the vehicle control module 108 of Fig. 1). By way of example, and as explained above, the control module 102 may transmit the vehicle calibration sets with one or more previously modified vehicle parameters (e.g., at 1732), as explained above. According to other examples, the control module 102 may transmit the vehicle calibration sets with one or more unmodified vehicle parameters if no modifications are required. According to such examples, the vehicle calibrations may be flashed into the memory of the vehicle control module 108 and / or provided to the vehicle control module 108 in another suitable manner. Control may then end.

[0104] According to various embodiments, the control module 102 may optionally compare the calculated anomaly scores (from 1714 or 1716 or 1720 or 1726) to one or more defined thresholds to detect anomalies. This step may occur before or after the control module 102 generates and outputs the one or more visualizations. According to some examples, the control module 102 may only generate and output one or more visualizations if one or more anomalies are detected.

[0105] The tax process 1800 after Fig. 18 shows an exemplary implementation of step 1714 according to Fig.17, wherein the control module 102 calculates the anomaly scores for scalar data based on an existing master calibration set. As shown, control begins at 1802, where the control module 102 selects one of the received successor calibration sets. Control then continues to 1804 and 1806.

[0106] At 1804, the control module 102 calculates a distance for each vehicle parameter of the selected successor calibration set against the corresponding vehicle parameters of the master calibration set, as explained herein. Then, at 1806, the control module 102 calculates the one or more anomaly scores for the one or more successor vehicle parameters based on the one or more calculated distances. As explained herein, the control module 102 may calculate the one or more anomaly scores based on the normalized distances. Control then continues to 1808.

[0107] At 1808, the control module 102 determines whether any additional successor calibration sets remain. If so, control transfers to 1810, where the control module 102 selects the next successor calibration set. Control then returns to 1804 and 1806, where the one or more anomaly scores for the next calibration set are calculated. However, if no additional successor calibration sets remain at 1808, control transfers to 1812, where the control module 102 outputs or otherwise makes available the one or more calculated anomaly scores. Control may then end.

[0108] The tax process 1900 after Fig. 19 shows an exemplary implementation of step 1716 Fig.17, wherein the control module 102 calculates the anomaly scores for scalar data based on other vehicle calibration sets (e.g., where no master calibration set is present or identified), as explained herein. As shown, control begins at 1902, where the control module 102 calculates an average of all calibration sets. By way of example, and as explained above, the control module 102 may calculate an average for each corresponding vehicle parameter of the vehicle calibration sets. Control then continues to 1904, where the control module 102 selects one of the received successor calibration sets. Control then continues to 1906 and 1908.

[0109] At 1906, the control module 102 calculates a distance for each vehicle parameter of the selected successor calibration set against a corresponding calculated average, as explained herein. Then, at 1908, the control module 102 calculates the one or more anomaly scores for the one or more vehicle parameters based on the one or more calculated distances. As explained herein, the control module 102 may calculate the one or more anomaly scores based on the normalized distances. Control then continues to 1910.

[0110] At 1910, the control module 102 determines whether any more calibration sets remain. If so, control transfers to 1912, where the control module 102 selects the next calibration set. Control then returns to 1906 and 1908, where the one or more anomaly scores for the next set are calculated. However, if no more calibration sets remain at 1910, control transfers to 1914, where the control module 102 outputs or otherwise makes available the one or more calculated anomaly scores. Control may then end.

[0111] The tax process 2000 after Fig. 20 shows an exemplary implementation of step 1720 Fig.17, wherein the control module 102 calculates the anomaly scores for vector or map data based on an existing master calibration set. As shown, control begins at 2002, where the control module 102 selects defined weighting factors (e.g., including various defined weights). Control then continues to 2004, where the control module 102 calculates a gradient for each corresponding vehicle parameter of the vehicle calibration sets, as explained herein. Control then continues to 2006, where the control module 102 selects one of the received successor calibration sets. Control then continues to 2008 and 2010.

[0112] At 2008, the control module 102 calculates the numerical relationships for each corresponding vehicle parameter of the follower calibration set with respect to the main calibration set. By way of example, and as explained above, the control module 102 may calculate a Euclidean distance and a linear regression relationship for each corresponding vehicle parameter of each follower calibration set against a corresponding vehicle parameter of the main calibration set.

[0113] At 2010, the control module 102 calculates the one or more anomaly scores for the one or more vehicle parameters in the selected calibration set. By way of example, and as explained above, the control module 102 may calculate an anomaly score by implementing a defined cost function that sums the calculated gradient, the calculated Euclidean distance, and the calculated linear regression, each weighted according to the defined weight factors. Control then proceeds to 2012.

[0114] At 2012, the control module 102 determines whether any additional successor calibration sets remain. If so, control transfers to 2014, where the control module 102 selects the next successor calibration set. Control then returns to 2008 and 2010, where the one or more anomaly scores for the next set are calculated. However, if no additional calibration sets remain at 2012, control transfers to 2016, where the control module 102 outputs or otherwise makes available the one or more calculated anomaly scores. Control may then end.

[0115] The tax process 2100 after Fig. 21 shows an exemplary implementation of step 1722 Fig.17, wherein the control module 102 calculates the anomaly scores for vector or map data when no main calibration set is present or identified. As shown, control begins at 2102 and 2104, where the control module 102 selects the defined weighting factors (e.g., including various defined weights). According to various embodiments, the defined weighting factors selected at 2102 may be different values ​​than those selected at 2102 after Fig. 20 must be selected. Control then proceeds to 2104.

[0116] At 2104, the control module 102 calculates a gradient for each corresponding vehicle parameter of the vehicle calibration sets, as explained herein. Control then proceeds to 2106, where the control module 102 selects one of the received calibration sets. Control then proceeds to 2108, 2110, 2112.

[0117] At 2108, the control module 102 calculates the numerical relationships (e.g., a Euclidean distance, a linear regression relationship, etc.) for each corresponding vehicle parameter of the calibration set with respect to each other calibration set. At 2110, the control module 102 calculates an average of each calculated numerical relationship (e.g., an average of all Euclidean distance values ​​for the selected vehicle calibration set, an average of all linear regression values ​​for the selected vehicle calibration set, etc.).

[0118] At 2112, the control module 102 calculates the one or more anomaly scores for the one or more vehicle parameters in the selected calibration set. By way of example, and as explained above, the control module 102 may calculate an anomaly score (e.g., a preliminary anomaly score) by implementing a defined cost function that sums the calculated gradient, the calculated average Euclidean distance, and the calculated average linear regression, each weighted according to the defined weight factors (selected at 2102). Control then proceeds to 2114.

[0119] At 2114, the control module 102 determines whether any more calibration sets remain. If so, control transfers to 2116, where the control module 102 selects the next calibration set. Control then returns to 2108, 2110, 2112, where the one or more anomaly scores for the next set are calculated. However, if no more calibration sets remain at 2114, control transfers to 2118, 2120, 2122.

[0120] At 2118, the control module 102 establishes the vehicle calibration set with the lowest (e.g., preliminary) anomaly score as a reference set. Then, at 2120, the control module 102 sets the other calculated (e.g., preliminary) anomaly scores based on the reference set, as explained herein. Control then proceeds to 2122, where the control module 102 outputs or otherwise makes available the one or more calculated anomaly scores. Control may then end.

[0121] The tax process 2200 after Fig. 22 illustrates an exemplary implementation of the automatic auto-correction of the vehicle parameters. According to various embodiments, the control process 2200 may be implemented in conjunction with the control process 1700, such as before or after step 1728 of Fig. 17. According to other examples, the control process 2200 may repeat steps 1728, 1730, 1732, 1734 after Fig. Replace 17.

[0122] As shown, control begins at 2202, where control module 102 receives a defined anomaly threshold. According to various embodiments, the defined anomaly threshold may be set by the user. As such, control module 102 may receive the threshold via user input. Control then proceeds to 2204.

[0123] At 2204, the control module 102 receives a reference calibration set, for example, including various calibrations (e.g., parameters). According to various embodiments, the reference calibrations may be user-defined. The reference calibrations may be, for example, a master calibration set or another suitable reference set, as explained herein. Control then proceeds to 2206.

[0124] At 2206, the control module 102 determines each detected anomaly that has a score greater than the defined anomaly threshold. If yes at 2206, control proceeds to 2208, where the control module 102 performs the auto-correction of the calibrations (e.g., the calibrations in a specific section) according to the detected anomaly. According to such examples, the control module 102 may set the calibrations to be equal to the reference calibrations. Control proceeds to 2210. If no at 2206, control proceeds to 2210, where the control module 102 transmits one or more vehicle calibration sets to a vehicle control module (e.g., the vehicle control module 108 to Fig. 1) as explained here. Control can then end.

[0125] According to various embodiments, after the detection and correction of an anomaly, the one or more updated calibrations may be transferred to a manufacturing facility for producing one or more vehicles. This enables the integration of the one or more new calibrations with the software, thereby facilitating the flashing process. Furthermore, in cases where a vehicle is in a post-manufacturing stage and already in the customer's possession, the one or more updated calibrations may be delivered via over-the-air (OTA) technology. This methodology serves to avoid the need for a physical recall of the vehicle, thereby saving both time and financial resources.

[0126] Fig. For example, Figure 23 illustrates a system 2300 that includes the control module 102 according to Fig.1. As shown, the control module 102 receives the vehicle calibration sets (e.g., (if applicable) the master calibration sets, the slave calibration sets, etc.) relating to a vehicle program (1) 2310, a vehicle program (2) 2320, and a vehicle program (N) 2330. According to such examples, N may be any suitable positive value, such as thirty, thirty-five, forty, etc. According to various embodiments, each vehicle calibration set may be generated according to multiple process stages, such as requirements elicitation, system design and development, software development, software integration, functional integration, and functional calibration.

[0127] Then, the control module 102 may detect and subsequently correct one or more anomalies associated with the one or more calibrations of a particular calibration set for the vehicle program (1) 2310, the vehicle program (2) 2320, and / or the vehicle program (N) 2330, as explained herein. Next, the control module 102 may transmit the one or more new / corrected calibrations to various locations. By way of example, and as described in Fig.23, the control module 102 may transmit the one or more new / corrected calibrations to a manufacturing facility 2340 for manufacturing one or more vehicles. Additionally and / or alternatively, the control module 102 may transmit the one or more new / corrected calibrations to a customer-owned vehicle 2350 after manufacture. According to such examples, the control module 102 may transmit the one or more new / corrected calibrations via the over-the-air (OTA) technique.

[0128] Fig. 24 illustrates a vehicle 2400 which, for example, has the vehicle control module 108 according to Fig. 1. According to the example in Fig. 24, the vehicle control module 108 is connected to the control module 102 Fig.1 to receive calibrations including one or more possible new / corrected calibrations. According to various embodiments, the vehicle 2400 may be a post-manufactured vehicle owned by a customer. According to such examples, the vehicle control module 108 may receive the one or more new / corrected calibrations via OTA technology. According to other examples, the vehicle 2400 may be a vehicle in a manufacturing facility (e.g., in the process of being manufactured). According to such examples, the one or more new / corrected calibrations may be flashed into the memory associated with the vehicle control module 108, as explained herein.

[0129] The foregoing description is merely illustrative and is in no way intended to limit the disclosure, its application, or uses. The broad teachings of the disclosure may be implemented in various forms. Therefore, while this disclosure contains specific examples, the true scope of the disclosure should not be so limited because other modifications will become apparent upon a study of the drawings, the specification, and the following claims. It should be recognized that one or more steps within a method may be performed in different orders (or simultaneously) without altering the principles of the present disclosure.Furthermore, although each of the embodiments has been described above with certain features, one or more of those features described with respect to any embodiment of the disclosure may be implemented in any of the other embodiments and / or combined with the features of any of the other embodiments, even if that combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and permutations of one or more embodiments with each other remain within the scope of this disclosure.

[0130] Spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using various terms, including "connected," "engaged," "coupled," "adjacent," "beside," "on top of," "over," "below," and "disposed." When a relationship between a first and a second element is described in the above disclosure, that relationship may be a direct relationship, with no other intervening elements present between the first and second elements, but may also be an indirect relationship, with one or more intervening elements (either spatial or functional) present between the first and second elements if not explicitly described as "direct."As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A OR B OR C) using a non-exclusive logical OR, and should not be construed to mean "at least one of A, at least one of B, and at least one of C."

[0131] In the figures, the direction of an arrow, as indicated by the arrowhead, generally demonstrates the flow of information (such as data or instructions) of interest for the illustration. For example, if element A and element B exchange different information, but the information transmitted from element A to element B is relevant for the illustration, the arrow may point from element A to element B. This unidirectional arrow does not imply that no other information is transmitted from element B to element A. Further, for the information sent from element A to element B, element B may send requests for or acknowledgments of receipt of the information to element A.

[0132] In this application, including the definitions below, the term "module" or the term "controller" may be replaced by the term "circuit". The term "module" may refer to, be part of, or contain: an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field-programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores the code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system on a chip.

[0133] The module may include one or more interface circuits. According to some examples, the interface circuits may include wired or wireless interfaces connected to a local area network (LAN), the Internet, a wide area network (WAN), or combinations thereof. The functionality of any given module of the present disclosure may be distributed among multiple modules connected via interface circuits. For example, multiple modules may enable load balancing. According to another example, a server module (also known as a remote or cloud module) may perform some functionality on behalf of a client module.

[0134] The term code, as used above, may include software, firmware, and / or microcode, and may refer to programs, routines, functions, classes, data structures, and / or objects. The term shared processor circuitry includes a single processor circuit that executes some or all of the code from multiple modules. The term group processor circuitry includes a processor circuit that, in combination with additional processor circuitry, executes some or all of the code from one or more modules. References to multiple processor circuits include multiple processor circuits on discrete dies, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above.The term shared memory circuit refers to a single memory circuit that stores some or all of the code from multiple modules. The term group memory circuit refers to a memory circuit that, in combination with additional memories, stores some or all of the code from one or more modules.

[0135] The term memory circuit is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not include transient electrical or electromagnetic signals that propagate through a medium (such as on a carrier wave); the term computer-readable medium can therefore be considered tangible and non-transitory. Non-limiting examples of a non-transitory, tangible computer-readable medium include non-volatile memory circuits (such as a flash memory circuit, an erasable programmable read-only memory circuit, or a mask read-only memory circuit), volatile memory circuits (such as a static random access memory circuit or a dynamic random access memory circuit), magnetic storage media (such as an analog or digital magnetic tape or a hard disk drive), and optical storage media (such asa CD, a DVD or a Blu-ray Disc).

[0136] The devices and methods described in this application may be implemented partially or entirely by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in computer programs. The function blocks, flowchart components, and other elements described above serve as software specifications that can be translated into computer programs through the routine work of a trained technician or programmer.

[0137] The computer programs contain processor-executable instructions stored on at least one non-transitory, tangible, computer-readable medium. The computer programs may also contain or rely on stored data. The computer programs may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with special-purpose devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0138] The computer programs may contain: (i) descriptive text to be parsed, such as: B. HTML (Hypertext Markup Language), XML (Extensible Markup Language) or JSON (JavaScript Object Notation), (ii) assembly code, (iii) object code generated from source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. By way of example only, the source code may be written using the syntax of languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5 (Hypertext Markup Language, 5th Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK and Python®.

Claims

[1] Method for detecting anomalies in vehicle calibration sets, the method comprising: Receiving multiple vehicle calibration sets, each containing corresponding vehicle parameters; Calculating an anomaly score for each vehicle parameter of each vehicle calibration set; Detecting an anomaly that is associated with at least one vehicle parameter of one of several vehicle calibration sets, based on the anomaly score for the vehicle parameter and a defined threshold; Modifying the vehicle parameter of the vehicle calibration set in response to the detection of the anomaly; and Transferring the vehicle calibration set including the modified vehicle parameter to a vehicle control module (108) assigned to a vehicle to control at least one component of the vehicle; characterized by the transfer of a visualization containing the anomaly score associated with the detected anomaly to a display module (106). [2] Method according to claim 1, wherein: the multiple vehicle calibration sets include a main vehicle calibration set and one or more successor vehicle calibration sets; and The calculation of the anomaly score includes: Calculating a gradient for each corresponding vehicle parameter of the multiple vehicle calibration sets; Calculating one or more numerical relationships for each corresponding vehicle parameter of each successor vehicle calibration set against a corresponding vehicle parameter of the main vehicle calibration set; and Calculating the anomaly score based on a cost function, a weighted value of the gradient, and the weighted values ​​of one or more numerical relationships. [3] Method according to claim 1, wherein the calculation of the anomaly point score includes: Calculating a gradient for each corresponding vehicle parameter of the multiple vehicle calibration sets; and Calculating one or more numerical relationships for each corresponding vehicle parameter of each vehicle calibration set against a corresponding vehicle parameter of all other vehicle calibration sets. [4] Method according to claim 3, wherein the calculation of the anomaly point score includes: Calculating a preliminary anomaly score for each corresponding vehicle parameter of the multiple vehicle calibration sets based on a cost function, a weighted value of the gradient, and weighted values ​​of one or more numerical relationships; Defining the vehicle calibration set with the corresponding vehicle parameter with the lowest preliminary anomaly score as a reference vehicle calibration set; and Calculating the anomaly score by setting the preliminary anomaly score for each relevant vehicle parameter based on the reference vehicle calibration set. [5] Method according to claim 1, wherein: the multiple vehicle calibration sets include a main vehicle calibration set and one or more successor vehicle calibration sets; and The calculation of the anomaly score includes: Calculating a distance for each corresponding vehicle parameter of each successor vehicle calibration set against a corresponding vehicle parameter of the main vehicle calibration set; and Calculating the anomaly score based on the distance. [6] Method according to claim 1, wherein the calculation of the anomaly point score includes: Calculating an average value for each corresponding vehicle parameter of the multiple vehicle calibration sets; Calculating a distance for each corresponding vehicle parameter of each vehicle calibration set against the calculated mean value for that vehicle parameter; and Calculating the anomaly score based on the distance. [7] Method according to claim 1, further comprising receiving a user input in response to the transmitted visualization, wherein modifying the vehicle parameter of the vehicle calibration set includes modifying the vehicle parameter of the vehicle calibration set in response to the received user input. [8] The method of claim 1, further comprising: Calculating one or more statistical relationships between the corresponding vehicle parameters of the multiple vehicle calibration sets; and Transferring a visualization containing at least one table that graphically represents one or more statistical relationships to a display module (106) [9] Method according to claim 1, wherein: The procedure further includes receiving a defined autocorrect threshold; and Modifying the vehicle parameter of the vehicle calibration includes the auto-correction of the vehicle parameter of the vehicle calibration to a reference calibration in response to the detected anomaly being larger than the defined auto-correction threshold.

Images