Statistical visualization and anomaly detection for vehicle calibration sets

Abstract

A method for detecting anomalies in vehicle calibration sets includes: receiving a plurality of vehicle calibration sets, each vehicle calibration set including a corresponding vehicle parameter; calculating an anomaly score for each vehicle parameter of each vehicle calibration set; detecting an anomaly associated with at least one vehicle parameter of a vehicle calibration set of the plurality of vehicle calibration sets based on an anomaly score for the vehicle parameter and a defined threshold; in response to detecting the anomaly, modifying the vehicle parameters of the vehicle calibration set; the vehicle calibration set including the modified vehicle parameters is transmitted to a vehicle control module associated with the vehicle for controlling at least one component of the vehicle. Other example methods and systems for detecting anomalies in a vehicle calibration set are also disclosed.

Description

Technical Field

[0002] The present disclosure relates to visualization and anomaly detection for vehicle calibration sets. Background Art

[0003] Vehicles depend heavily on complex calibrations that are essential to defining and optimizing vehicle performance metrics such as vibration, mileage, battery efficiency, miles per gallon, emissions, etc. A calibration set (e.g., parameter values) for a specific vehicle is typically provided to the vehicle during production (e.g., flashing at the manufacturing plant) or in a post-production scenario (e.g., via an over-the-air (OTA) update or dealer flash). In some cases, a data management system may be employed to check calibration values ​​during the calibration process. In other examples, a manual process may be necessary in which a one-by-one comparison of each calibration parameter is evaluated. Summary of the invention

[0004] A method for detecting anomalies in a vehicle calibration set, comprising: receiving a plurality of vehicle calibration sets, each vehicle calibration set including corresponding vehicle parameters; calculating an anomaly score for each vehicle parameter of each vehicle calibration set; detecting an anomaly associated with at least one vehicle parameter of a vehicle calibration set in the plurality of vehicle calibration sets based on the anomaly score of the vehicle parameter and a defined threshold; modifying the vehicle parameter of the vehicle calibration set in response to detecting the anomaly; and transmitting the vehicle calibration set including the modified vehicle parameter to a vehicle control module associated with a vehicle, the vehicle control module being used to control at least one component of the vehicle.

[0005] In other features, the plurality of vehicle calibration sets includes a master vehicle calibration set and one or more child vehicle calibration sets.

[0006] In other features, calculating the anomaly score includes: calculating the gradient of each corresponding vehicle parameter of the multiple vehicle calibration sets; calculating one or more numerical relationships of each corresponding vehicle parameter of each sub-vehicle calibration set relative to (against) the corresponding vehicle parameter of the main vehicle calibration set; and calculating the anomaly score based on a cost function, a weighted value of the gradient, and a weighted value of the one or more numerical relationships.

[0007] In other features, the one or more numerical relationships include Euclidean distance and linear regression.

[0008] In other features, calculating the anomaly score includes calculating a gradient for each corresponding vehicle parameter of the plurality of vehicle calibration sets and calculating one or more numerical relationships of each corresponding vehicle parameter of each vehicle calibration set relative to corresponding vehicle parameters of all other vehicle calibration sets.

[0009] In other features, calculating the anomaly score includes: calculating a temporary anomaly score for each corresponding vehicle parameter of the plurality of vehicle calibration sets based on a cost function, a weighted value of the gradient, and a weighted value of the one or more numerical relationships; setting a vehicle calibration set having corresponding vehicle parameters with a lowest temporary anomaly score as a reference vehicle calibration set; and calculating the anomaly score by adjusting the temporary anomaly score for each corresponding vehicle parameter based on the reference vehicle calibration set.

[0010] In other features, the one or more numerical relationships include Euclidean distance and linear regression.

[0011] In other features, calculating the anomaly score includes: calculating a distance of each corresponding vehicle parameter of each child vehicle calibration set relative to a corresponding vehicle parameter of the host vehicle calibration set; and calculating the anomaly score based on the distance.

[0012] In other features, calculating the anomaly score includes: calculating a normalized value of the distance; and setting the anomaly score equal to the normalized value.

[0013] In other features, calculating the anomaly score includes: calculating a mean of each corresponding vehicle parameter of the plurality of vehicle calibration sets; calculating a distance of each corresponding vehicle parameter of each vehicle calibration set relative to the calculated mean of the vehicle parameter; and calculating the anomaly score based on the distance.

[0014] In other features, calculating the anomaly score includes: calculating a normalized value of the distance; and setting the anomaly score equal to the normalized value.

[0015] In other features, the method further comprises detecting whether the plurality of vehicle calibration sets are in a scalar format.

[0016] In other features, calculating the anomaly score includes, in response to the plurality of vehicle calibration sets being in a scalar format, calculating a distance of each corresponding vehicle parameter of each vehicle calibration set relative to a reference value, and calculating the anomaly score based on the distance.

[0017] In other features, the method further comprises detecting whether the plurality of vehicle calibration sets are in a vector format or a map format.

[0018] In other features, calculating the anomaly score specific to each vehicle calibration set includes calculating the anomaly score based on a cost function in response to the plurality of vehicle calibration sets being in a vector format or a map format.

[0019] In other features, the method further comprises transmitting a visualization including an anomaly score associated with the detected anomaly to a display module.

[0020] In other features, the visualizing comprises charting at least one graph of the anomaly scores.

[0021] In other features, the method further comprises receiving user input in response to the transmitted visualization, wherein modifying the vehicle parameters of the vehicle calibration set comprises modifying the vehicle parameters of the vehicle calibration set in response to the received user input.

[0022] In other features, the method further comprises: calculating one or more statistical relationships of corresponding vehicle parameters of the plurality of vehicle calibration sets; and transmitting to a display module a visualization including at least one table plotting the one or more statistical relationships.

[0023] In other features, the method further comprises receiving a defined autocorrect threshold.

[0024] In other features, modifying the vehicle parameter of the vehicle calibration includes automatically correcting the vehicle parameter of the vehicle calibration to a reference calibration in response to the detected anomaly being greater than a defined automatic correction threshold.

[0025] In other features, transmitting the vehicle calibration set including the modified vehicle parameters to the vehicle control module includes transmitting the vehicle calibration set to the vehicle control module associated with the vehicle via an over-the-air (OTA) update.

[0026] A method for detecting anomalies in a vehicle calibration set, comprising: receiving a plurality of vehicle calibration sets, each vehicle calibration set comprising corresponding vehicle parameters divided into two or more parts; calculating an anomaly score for each part of each vehicle calibration set; detecting an anomaly associated with at least one part of a vehicle calibration set in the plurality of vehicle calibration sets based on the anomaly score of the part and a defined threshold; in response to detecting the anomaly, modifying the vehicle parameters in the part of the vehicle calibration set; and transmitting the vehicle calibration set including the modified vehicle parameters to a vehicle control module associated with a vehicle, the vehicle control module being used to control at least one component of the vehicle.

[0027] A system for detecting anomalies in a vehicle calibration set includes: a vehicle control module associated with a vehicle for controlling at least one component of the vehicle; and a control module in communication with the vehicle control module. The control module is configured to: receive a plurality of vehicle calibration sets, each vehicle calibration set including corresponding vehicle parameters; calculate an anomaly score for each vehicle parameter of each vehicle calibration set; detect an anomaly associated with at least one vehicle parameter of a vehicle calibration set in the plurality of vehicle calibration sets based on the anomaly score of the at least one vehicle parameter of the vehicle calibration set and a defined threshold; modify the vehicle parameter of the vehicle calibration set in response to detecting the anomaly; and transmit the vehicle calibration set including the modified vehicle parameter to a vehicle control module associated with the vehicle, the vehicle control module for controlling at least one component of the vehicle.

[0028] This application provides the following examples.

[0029] 1. A method for detecting anomalies in a vehicle calibration set, the method comprising: receiving a plurality of vehicle calibration sets, each vehicle calibration set comprising corresponding vehicle parameters; An anomaly score is calculated for each vehicle parameter of each vehicle calibration set; detecting an anomaly associated with at least one vehicle parameter of a vehicle calibration set of the plurality of vehicle calibration sets based on an anomaly score of the vehicle parameter and a defined threshold; In response to detecting the anomaly, modifying the vehicle parameters of the vehicle calibration set; and The vehicle calibration set including the modified vehicle parameters is transmitted to a vehicle control module associated with a vehicle, the vehicle control module being used to control at least one component of the vehicle.

[0030] 2. The method of embodiment 1, wherein: The plurality of vehicle calibration sets include a master vehicle calibration set and one or more child vehicle calibration sets; and Calculating the anomaly score includes: calculating a gradient of each corresponding vehicle parameter of the plurality of vehicle calibration sets, One or more numerical relationships of each corresponding vehicle parameter of each child vehicle calibration set relative to a corresponding vehicle parameter of the host vehicle calibration set are calculated, and an anomaly score is calculated based on a cost function, a weighted value of the gradient, and a weighted value of the one or more numerical relationships.

[0031] 3. The method of embodiment 2, wherein the one or more numerical relationships include Euclidean distance and linear regression.

[0032] 4. The method of embodiment 1, wherein calculating the anomaly score comprises: calculating a gradient of each corresponding vehicle parameter of the plurality of vehicle calibration sets; and One or more numerical relationships are calculated for each corresponding vehicle parameter of each vehicle calibration set relative to corresponding vehicle parameters of all other vehicle calibration sets.

[0033] 5. The method of embodiment 4, wherein calculating the anomaly score comprises: calculating a temporary anomaly score for each corresponding vehicle parameter of the plurality of vehicle calibration sets based on a cost function, a weighted value of the gradient, and a weighted value of the one or more numerical relationships; setting a vehicle calibration set having corresponding vehicle parameters with the lowest temporary anomaly score as a reference vehicle calibration set; and calculating an anomaly score by adjusting the temporary anomaly score of each corresponding vehicle parameter based on the reference vehicle calibration set.

[0034] 6. The method of embodiment 4, wherein the one or more numerical relationships include Euclidean distance and linear regression.

[0035] 7. The method of embodiment 1, wherein: The plurality of vehicle calibration sets include a master vehicle calibration set and one or more child vehicle calibration sets; and Calculating the anomaly score includes: Calculating the distance of each corresponding vehicle parameter of each child vehicle calibration set relative to the corresponding vehicle parameter of the main vehicle calibration set, and An anomaly score is calculated based on the distance.

[0036] 8. The method of embodiment 7, wherein calculating the anomaly score comprises: calculating a normalized value of the distance; and The anomaly score is set equal to the normalized value.

[0037] 9. The method of embodiment 1, wherein calculating the anomaly score comprises: calculating a mean value of each corresponding vehicle parameter of the plurality of vehicle calibration sets; Calculating a distance of each corresponding vehicle parameter of each vehicle calibration set relative to the calculated mean of the vehicle parameter; and calculating an anomaly score based on the distance.

[0038] 10. The method of embodiment 9, wherein calculating the anomaly score comprises: calculating a normalized value of the distance; and The anomaly score is set equal to the normalized value.

[0039] 11. The method according to embodiment 1 further includes detecting whether the multiple vehicle calibration sets are in a scalar format, wherein calculating the anomaly score includes: in response to the multiple vehicle calibration sets being in a scalar format, calculating the distance of each corresponding vehicle parameter of each vehicle calibration set relative to a reference value, and calculating the anomaly score based on the distance.

[0040] 12. The method according to embodiment 1 further includes detecting whether the multiple vehicle calibration sets are in vector format or map format, wherein calculating an anomaly score specific to each vehicle calibration set includes: calculating the anomaly score based on a cost function in response to the multiple vehicle calibration sets being in vector format or map format.

[0041] 13. The method of embodiment 1 further comprises transmitting a visualization including anomaly scores associated with the detected anomalies to a display module.

[0042] 14. The method of embodiment 13, wherein the visualization comprises plotting at least one graph of the anomaly score.

[0043] 15. The method of embodiment 13, further comprising: receiving user input in response to the transmitted visualization, wherein modifying the vehicle parameters of the vehicle calibration set comprises modifying the vehicle parameters of the vehicle calibration set in response to the received user input.

[0044] 16. The method of embodiment 1, further comprising: calculating one or more statistical relationships of corresponding vehicle parameters of the plurality of vehicle calibration sets; and A visualization including at least one table plotting the one or more statistical relationships is transmitted to a display module.

[0045] 17. The method of embodiment 1, wherein: The method further includes receiving a defined auto-correction threshold; and Modifying the vehicle parameters of the vehicle calibration includes automatically correcting the vehicle parameters of the vehicle calibration to a reference calibration in response to the detected anomaly being greater than a defined automatic correction threshold.

[0046] 18. The method of embodiment 1 wherein transmitting the vehicle calibration set including modified vehicle parameters to the vehicle control module comprises transmitting the vehicle calibration set to the vehicle control module associated with the vehicle via an over-the-air (OTA) update.

[0047] 19. A method for detecting anomalies in a vehicle calibration set, the method comprising: receiving a plurality of vehicle calibration sets, each vehicle calibration set comprising corresponding vehicle parameters divided into two or more parts; calculating an anomaly score for each part of each vehicle calibration set; detecting an anomaly associated with at least one portion of vehicle calibration sets in the plurality of vehicle calibration sets based on an anomaly score of the portion and a defined threshold; In response to detecting the anomaly, modifying vehicle parameters in the portion of the vehicle calibration set; and The vehicle calibration set including the modified vehicle parameters is transmitted to a vehicle control module associated with a vehicle, the vehicle control module being used to control at least one component of the vehicle.

[0048] 20. A system for detecting anomalies in a vehicle calibration set, the system comprising: a vehicle control module associated with the vehicle for controlling at least one component of the vehicle; and a control module in communication with the vehicle control module, the control module being configured to: receiving a plurality of vehicle calibration sets, each vehicle calibration set comprising corresponding vehicle parameters; An anomaly score is calculated for each vehicle parameter of each vehicle calibration set; detecting an anomaly associated with at least one vehicle parameter of a vehicle calibration set of the plurality of vehicle calibration sets based on an anomaly score of the vehicle parameter and a defined threshold; In response to detecting the anomaly, modifying the vehicle parameters of the vehicle calibration set; and The vehicle calibration set including the modified vehicle parameters is transmitted to a vehicle control module associated with a vehicle, the vehicle control module being used to control at least one component of the vehicle.

[0049] Further areas of applicability of the present disclosure will become apparent from the detailed description, claims, and drawings.The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present disclosure will become more fully understood from this detailed description and the accompanying drawings, in which: Figure 1 is a functional block diagram of an example system for detecting anomalies in a vehicle calibration set and generating a statistical visualization of the vehicle calibration set according to the present disclosure; Figure 2-5 is a graph depicting anomaly scores of corresponding vehicle parameters of different calibration sets when a master calibration set is employed according to the present disclosure; Figure 6-9is a graph depicting anomaly scores of corresponding vehicle parameters of different calibration sets when a master calibration set is not employed according to the present disclosure; Figure 10-13 is a graph depicting anomaly scores for corresponding vehicle parameters for four separate portions of two calibration sets in accordance with the present disclosure; Fig.14 According to the present disclosure, Figure 10-13 Graphs of anomaly scores of vehicle parameters for four collective parts of one of the two calibration sets; Fig.15 According to the present disclosure, Figure 10-13 Graphs of anomaly scores of vehicle parameters for four collective parts in the other of the two calibration sets; Fig.16 According to the present disclosure, Figure 10-13 Plots of anomaly scores of vehicle parameters for four collective parts of two calibration sets; Figure 17-22 is a flow chart of an example control process for detecting anomalies in a vehicle calibration set, generating a statistical visualization of the vehicle calibration set, and / or modifying a vehicle calibration according to the present disclosure; Fig.23 is a functional block diagram of an example system for updating one or more calibrations and transmitting the new calibrations to a manufacturing facility and / or a post-production vehicle according to the present disclosure; Fig.24 According to the present disclosure, Figure 1 the systems of the various parts of the vehicle; and Fig.25 is a diagram illustrating the interaction between nine vehicle calibration sets according to the present disclosure.

[0051] In the drawings, reference numerals may be repeated to identify similar and / or identical elements. DETAILED DESCRIPTION

[0052] Calibration is used in vehicles to define and optimize vehicle performance indicators, such as vibration, mileage, battery efficiency, miles per gallon, emissions, etc. The number of calibrations for each vehicle is significant. For example, in a vehicle, the electric propulsion system alone can utilize more than 18,000 calibrable parameters, which are critical for optimal vehicle performance. Other controllable vehicle components can utilize a similar number of parameters. As such, the total number of parameters for a vehicle can extend to hundreds of thousands. Such calibrations (e.g., parameter values) are provided to the vehicle during production or in a post-production scenario (e.g., flash storage in a memory circuit of a vehicle control module or associated with a vehicle control module). In some cases, a defective calibration may be inadvertently released for a vehicle that may need to be recalled. For example, a defective calibration may bring significant risks to users (e.g., drivers, passengers, bystanders, etc.) and vehicles by endangering user safety and vehicle performance. Additionally, a defective calibration may cause significant and heavy economic impacts if a recall becomes necessary after production. While recalls are undesirable as they have huge financial and safety impacts, recalls occur more frequently than desired and many times calibration outliers / anomalies are the root cause.

[0053] Some data management systems are available for checking vehicle calibration. However, such systems are not capable of handling large data sets and / or are unreliable when handling large data sets of calibrations across multiple vehicle programs (e.g., especially comparing curves and maps with multiple data sets that are very tedious). As a result, companies frequently resort to manual methods - a process that involves comparing each calibration parameter one by one. This manual method is both time consuming and prone to human error. As such, the manual method, like existing data management systems, is unfeasible and unreliable.

[0054] The system and method according to the present disclosure provide a technical solution for implementing anomaly detection algorithms that are capable of examining scalar and complex calibration maps (e.g., 1D, 2D, 3D, etc. calibration maps) to accurately identify anomalies in parameter values ​​associated with various vehicle calibration sets having thousands (and in some cases hundreds of thousands) of parameters, and the technical solution then provides a unique visualization of the vehicle calibration set. Utilizing such a system and method, a series of intricate calibration anomalies can be distiled into a single, easily interpretable digital representation, enabling calibration engineers to use effective tools to pinpoint and correct discrepancies prior to integration into a vehicle production line, as further explained herein. In addition, the system and method maintain functionality after production to provide a safety net by the ability to transmit necessary fixes or solutions to customer vehicles via over-the-air (OTA) updates, thereby avoiding potential recalls while enhancing vehicle safety and quality in real time.

[0055] Many other benefits arise from detecting anomalies in parameter values. For example, by enabling a quick response to calibration errors, the technical solution significantly enhances vehicle safety, thereby proactively avoiding potential dangers. Additionally, the technical solution facilitates early detection and correction of calibration errors, thereby avoiding the huge costs associated with vehicle recalls. Moreover, with the production of a wide variety of vehicles, each covering thousands (and in some cases hundreds of thousands) of unique calibrations and numerous variants, the technical solution achieves scalability, thereby harmoniously integrating into the vast production ecosystem. In addition, the technical solution of the present invention helps to catch safety-critical calibration errors, even before the error reaches the customer, saving millions of dollars that would otherwise be spent on repairing recalls, and improving performance by updating feature calibrations.

[0056] In addition, the technical solution herein significantly reduces the time and cost requirements associated with the vehicle calibration process. For example, the technical solution can provide an average engineering time reduction of at least sixteen weeks per engineer per year across multiple different vehicle programs compared to a conventional system that does not employ the technical solution herein. As a result of such significant engineering time savings, engineers can focus their attention on other requirements, costs can be significantly reduced, and so on.

[0057] Reference now Figure 1 , a block diagram of an example system 100 for detecting anomalies in a vehicle calibration set and generating a statistical visualization of the vehicle calibration set is presented. Figure 1 The system 100 may be applicable to any suitable vehicle, such as an electric vehicle (e.g., a pure electric vehicle, a plug-in hybrid electric vehicle, etc.), an internal combustion engine vehicle, etc. Additionally, the system 100 may be applicable to autonomous vehicles, such as semi-autonomous vehicles and fully autonomous vehicles. For example, and as further explained, the vehicle calibration set analyzed and, in some cases, modified by the system 100 may be provided to one or more vehicle control modules in any suitable vehicle.

[0058] like Figure 1 As shown in FIG. 1 , the system 100 generally includes a control module 102, a vehicle calibration database 104, a display module 106, and at least one vehicle control module 108. In this example, the control module 102 may be connected to a vehicle calibration database 104, a display module 106, and at least one vehicle control module 108. Figure 1 Each of the other components shown communicates with each other. Figure 1 System 100 is illustrated as including specific modules, but it should be appreciated that one or more other modules may be employed if desired. Additionally, while system 100 is shown as including multiple separate modules, any combination of these modules and / or their functionality may be integrated into one or more modules.

[0059] exist Figure 1 In the example of , the vehicle control module 108 can be any suitable control device associated with the vehicle, which is used to control at least one component of the vehicle. In this example, the vehicle control module 108 can be any control module in the vehicle that receives calibrable parameters. For example, the vehicle control module 108 can be an engine control module ECM for controlling a motor / engine (e.g., an electric motor, an internal combustion engine, etc.), a battery control module for controlling battery operation (e.g., output current, charging current, etc.), an inverter control module for controlling output to the battery, etc.

[0060] Although Figure 1 The system 100 is shown as including only one vehicle control module 108, but it should be appreciated that the system 100 may include multiple vehicle control modules. In such an example, each vehicle control module may receive dedicated calibratable parameters specific to that vehicle control module.

[0061] The display module 106 can be any suitable device having a display. For example, the display module 106 can be a user device such as a laptop, a phone, a monitor, a desktop computing device, etc. In such an example, the display module 106 displays one or more visualizations related to the analysis of the vehicle calibration set, as further explained herein.

[0062] Figure 1 The vehicle calibration database 104 stores various vehicle calibration sets, each of which can be received by the control module 102. For example, each vehicle calibration set can include multiple parameters (e.g., thousands, etc.) for controlling different aspects of the vehicle. Each parameter can have an associated value. Figure 1 The system 100 is shown as including only one vehicle calibration database 104 , but it should be appreciated that the system 100 may include multiple databases, if desired.

[0063] In various embodiments, the vehicle calibration set and / or the parameters therein may be in different formats. For example, in each calibration set, there may be a combination of scalar format, curve format, and map format. For example, the vehicle calibration in the calibration set may have: a scalar type format, in which the parameter is associated with a single value; a vector (or curve) type format, in which the parameter is associated with multiple values ​​in a line (e.g., values ​​along an axis); and / or a map type format, in which the parameter is associated with multiple values ​​across multiple lines (e.g., a grid of values ​​associated with two axes). In some examples, the scalar format, the vector format, and the map format may be referred to as a 1D (one-dimensional) format, a 2D (two-dimensional) format, and a 3D (three-dimensional) format, respectively.

[0064] In some examples, the stored vehicle calibration sets may include a master vehicle calibration set and a sub-vehicle calibration set. For example, the master vehicle calibration set may include a vehicle calibration associated with a vehicle program (e.g., a specific type of vehicle manufactured). Relatedly, a sub-vehicle calibration set may include vehicle calibrations associated with different vehicle settings for the vehicle program or associated programs that share components with the vehicle program (e.g., different example settings for a specific type of vehicle manufactured). In various embodiments, each vehicle program may have more than fifty vehicle calibration sets, each of which has thousands of parameters.

[0065] In other examples, the vehicle calibration set may not include a master vehicle calibration set. In such examples, child vehicle calibration sets for the same vehicle program (e.g., a particular type of vehicle manufactured) or different vehicle programs (e.g., two or more types of vehicles manufactured) may be analyzed, as further explained below.

[0066] Continue to refer Figure 1 , the control module 102 receives the vehicle calibration set from the vehicle calibration database 104 and / or other suitable sources. For example, the control module 102 can import any number of vehicle calibration sets, such as all vehicle calibration sets for a vehicle program, all vehicle calibration sets for multiple vehicle programs, or selected vehicle calibration sets, etc. Additionally, the control module 102 can be able to import and support multiple data formats, including DCM, CSV, etc. In such an example, a user (e.g., a technician, an engineer, etc.) can be able to select a vehicle calibration set via user input (e.g., Figure 1 The input signal 110 of the vehicle calibration set can be used to control the desired calibration set in various formats. In various embodiments, multiple vehicle calibration sets can be imported at the same time, thereby providing flexibility. In other examples, if desired, the vehicle calibration sets can be imported continuously.

[0067] In some examples, the control module 102 can filter and / or sort the received vehicle calibration sets and / or parameters therein. For example, a user can select an input option to filter one or more vehicle calibration sets. In doing so, the analysis of the vehicle calibration sets can be focused on a specific calibration subset for target comparison, thereby allowing the user to focus on specific calibration parameters of primary interest.

[0068] The control module 102 may be operable to perform a calibration comparison process. For example, the control module 102 may (e.g., substantially in real time) calculate one or more statistical relationships of corresponding vehicle parameters of a vehicle calibration set received from the vehicle calibration database 104 and / or another suitable source. The statistical relationship (e.g., statistical metric) may include, for example, a mean, a standard deviation, a variance, and / or any other suitable computable relationship between parameters. In various embodiments, the calculated statistical relationship may be defined by a user. In some examples, the control module 102 may automatically initiate such calculations in response to receiving a vehicle calibration set, in response to user input (e.g., via signal 110, etc.).

[0069] In various embodiments, the control module 102 calculates anomaly scores associated with some or all received vehicle calibration sets. More specifically, the control module 102 can calculate anomaly scores for vehicle parameters in a vehicle calibration set and / or portions (e.g., sets of vehicle parameters) in a vehicle calibration set. In doing so, the control module 102 can identify one or more vehicle parameter outliers in the vehicle calibration set compared to corresponding vehicle parameters in other vehicle calibration sets.

[0070] exist Figure 1 In an example of the present invention, the control module 102 may calculate the anomaly score differently based on one or more conditions. For example, and as further explained below, the control module 102 may calculate the anomaly score for a vehicle parameter in a scalar format differently than for a vehicle parameter in a vector format or a map format. In such an example, the control module 102 may identify the format type based on the data (e.g., the size of the data, etc.).

[0071] For example, the control module 102 can detect whether the received vehicle calibration set is in a scalar format or a vector or map format. Then, based on the type of format, the control module 102 can calculate the anomaly score using different algorithms. For example, in response to the vehicle calibration set being in a scalar format, the control module 102 can calculate the distance of each corresponding vehicle parameter of each vehicle calibration set relative to a reference value, and then calculate the anomaly score based on the distance, as further explained below. Alternatively, in response to the vehicle calibration set being in a vector or map format, the control module 102 can calculate the anomaly score based on a cost function, as further explained below.

[0072] Additionally, the control module 102 can calculate the anomaly score in different ways based on whether the master vehicle calibration set is known. In such an example, the control module 102 can detect whether the vehicle calibration set is the master set based on user input (e.g., a set identified by a user), a tag provided with the vehicle calibration set from the vehicle calibration database 104, etc.

[0073] For example, the control module 102 may determine that the received vehicle calibration set includes a master vehicle calibration set and one or more child vehicle calibration sets. In such an example, the control module 102 may treat the master vehicle calibration set as a reference vehicle calibration set for comparison with the child vehicle calibration sets. In other examples, the received vehicle calibration set may not include the master vehicle calibration set, and / or the control module 102 may not be able to determine the master vehicle calibration set (e.g., if received). In such an example, the control module 102 may calculate a reference value and / or identify one of the vehicle calibration sets (e.g., in the same vehicle program or a different vehicle program) as a reference vehicle calibration set, as further explained below.

[0074] The following are example operations taken by the control module 102 depending on whether a host vehicle calibration set is available and the data format type (e.g., scalar, vector, or map). Although the control module 102 is described as performing such operations, it should be appreciated that other example operations (e.g., calculations, algorithms, etc.) may be taken if desired.

[0075] As mentioned above, if the vehicle calibration set is in scalar format, the control module 102 can calculate the anomaly score based on the determined distance of each corresponding vehicle parameter of each vehicle calibration set relative to the reference value. For example, if the master vehicle calibration set is available, the control module 102 calculates the distance of each corresponding vehicle parameter of each child vehicle calibration set relative to the corresponding vehicle parameter of the master vehicle calibration set, and then calculates the anomaly score of the vehicle parameter based on the calculated distance. In various embodiments, the control module 102 can calculate a normalized value of the distance, and then set the normalized value as the anomaly score of the vehicle parameter.

[0076] Alternatively, if the host vehicle calibration set is not available, the control module 102 may calculate a reference value. For example, the control module 102 calculates the mean of each corresponding vehicle parameter of the received vehicle calibration set. Next, the control module 102 calculates the distance of each corresponding vehicle parameter of each vehicle calibration set relative to the calculated mean of the vehicle parameter, and then calculates the abnormality score of the vehicle parameter based on the calculated distance. Similar to the above, the control module 102 may calculate a normalized value of the distance (relative to the mean), and then set the normalized value as the abnormality score of the vehicle parameter.

[0077] However, if the vehicle calibration set is in a vector or map format (e.g., 2D or 3D format), the control module 102 can calculate the anomaly score based on the cost function, as mentioned above. For example, if the host vehicle calibration set is available, the control module 102 calculates the gradient of each corresponding vehicle parameter of the vehicle calibration set. In such an example, the gradient can be calculated in a conventional manner.

[0078] Then, the control module 102 calculates one or more numerical relationships of each corresponding vehicle parameter of each sub-vehicle calibration set relative to the corresponding vehicle parameter of the host vehicle calibration set. For example, the control module 102 may calculate the Euclidean distance and linear regression relationship of each corresponding sub-vehicle parameter relative to the corresponding host vehicle parameter (e.g., reference parameter).

[0079] Next, the control module 102 implements a cost function relationship to calculate an anomaly score for each vehicle parameter. For example, the control module 102 may utilize a cost function that adds the gradient, Euclidean distance, and linear regression associated with the vehicle parameter to obtain an anomaly score. In various embodiments, each of the gradient, Euclidean distance, and linear regression may be weighted before the values ​​are summed. For example, the gradient may be multiplied by a defined weight value (e.g., a value between 0.1 and 0.9), the Euclidean distance may be multiplied by a defined weight value (e.g., a value between 0.1 and 0.9), and the linear regression may be multiplied by a defined weight value (e.g., a value between 0.1 and 0.9).

[0080] Alternatively, if the host vehicle calibration set is not available for vector or map format analysis, the control module 102 may identify one of the vehicle calibration sets as a reference vehicle calibration set and then determine an anomaly score for each vehicle parameter based on the reference set. For example, the control module 102 calculates the gradient of each corresponding vehicle parameter of the vehicle calibration set, as explained above.

[0081] The control module 102 then calculates one or more numerical relationships (e.g., Euclidean distances and linear regression relationships) of each corresponding vehicle parameter of each vehicle calibration set relative to corresponding vehicle parameters of all other vehicle calibration sets. For example, the Euclidean distance between one vehicle parameter in one calibration set relative to a corresponding vehicle parameter in another calibration set may be 20, and the Euclidean distance between the same vehicle parameter in one calibration set relative to another corresponding vehicle parameter in yet another calibration set may be 30. The control module 102 then individually averages the Euclidean distances and linear regression values ​​of each parameter (e.g., map) of the calibration set relative to the other parameters to obtain a single Euclidean distance and a single linear regression value for that parameter of the calibration set. For example, continuing with the example above, the single Euclidean distance may be 25 (e.g., (20+30) / 2).

[0082] The control module 102 then calculates a temporary anomaly score for each corresponding vehicle parameter based on a cost function, as explained above. For example, the cost function may add the gradient, a single (e.g., average) Euclidean distance, and a single (e.g., average) linear regression value associated with the vehicle parameter to obtain the temporary anomaly score. In various embodiments, each of the gradient, Euclidean distance, and linear regression may be weighted before the values ​​are summed, as explained above. In such an example, the weighted values ​​may be different when the host vehicle calibration set is not available and when the host vehicle calibration set is available.

[0083] Next, the control module 102 determines which temporary anomaly score is the lowest for a given parameter across the calibration sets, and then sets the corresponding vehicle calibration set with the lowest temporary anomaly score as the reference set. For example, the control module 102 may set the value of the parameter corresponding to the lowest temporary anomaly score to zero.

[0084] Then, the control module 102 adjusts the other temporary anomaly scores of the corresponding vehicle parameters of the other calibration sets based on the reference vehicle calibration set. For example, if the temporary anomaly scores are 18.5, 30.5, 48, the control module 102 sets the value of the parameter corresponding to the lowest temporary anomaly score (18.5) to zero. The control module 102 then subtracts 18.5 from each of the other temporary anomaly scores of the corresponding vehicle parameters of the other calibration sets. In other words, the control module 102 adjusts the other temporary anomaly scores to 12 (e.g., 30.5-18.5) and 29.5 (e.g., 48-18.5). Then, the adjusted temporary anomaly scores (e.g., 0, 12, 29.5) can be provided as anomaly scores.

[0085] Although the above example operations taken by the control module 102 are described with respect to determining an anomaly score specific to one parameter in a vehicle calibration set, it should be appreciated that similar calculations may be implemented for a collection of vehicle parameters in a vehicle calibration set (e.g., each calibration set may have tens of thousands of vehicle parameters). For example, each vehicle calibration set may include vehicle parameters that are divided into two or more parts. In such an example, the anomaly score for each part may be determined in a manner similar to that explained above with respect to the anomaly score for the vehicle parameters.

[0086] In various embodiments, the control module 102 may detect an anomaly associated with at least one vehicle parameter in the vehicle calibration set based on the calculated anomaly score of the vehicle parameter. For example, the control module 102 may compare the calculated anomaly score to a defined threshold (e.g., a single value, a range of values, etc.), and if the anomaly score is greater than the defined threshold, an anomaly is detected. For example only, the defined threshold may be a percentage (e.g., 10%, 12%, 15%, 17%, 20%, etc.) from a reference value employed as explained above.

[0087] Additionally, in some examples, control module 102 may transmit at least one visualization to display module 106 and / or another suitable module for displaying the visualization. The visualization may include the calculated anomaly score, whether or not the anomaly score is associated with a detected anomaly. For example, control module 102 may transmit a visualization with an anomaly score in response to the anomaly score being associated with a detected anomaly. In other examples, control module 102 may transmit a visualization with an anomaly score even if no anomaly is detected.

[0088] In various embodiments, the control module 102 can compile the calculated data in different visual formats for viewing by the user. For example, the control module 102 can utilize 2D and 3D visualization techniques to make complex calibration comparisons easily digestible and visually discernible to the user. For example, the control module 102 can provide a visualization having one or more charts plotting the anomaly scores, one or more tables plotting the anomaly scores, etc. For example only, Figure 2-14 Various graphs 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200, 1300, 1400, 1500, 1600 are depicted showing different visualizations of anomaly scores. Figure 2-16 In the example of FIG. , the vehicle parameter is related to the battery current value.

[0089] For example, Figure 2-5 The graphs 200, 300, 400, 500 depict the anomaly scores of corresponding parameters across four different calibration sets when the host vehicle calibration set is taken as a reference. In this example, Figure 2 Graph 200 depicts parameters for a host vehicle calibration set having an anomaly score of zero (0). Figure 3 The graph 300 of depicts the corresponding parameters of a sub-vehicle calibration set (sub #2) having an anomaly score of zero (0). As shown, Figure 3 The chart of 300 and Figure 2 The same is true for the chart 200. Figure 4The graph 400 of FIG. 4 depicts the corresponding parameters of another sub-vehicle calibration set (sub #3) having an anomaly score of 45.7535, and Figure 5 The graph 500 of depicts the corresponding parameters of yet another sub-vehicle calibration set (sub #4) having an anomaly score of 46.2335. As shown, Figure 4-5 The chart of 400, 500 and Figure 2 The graph 200 of is significantly different, thereby indicating a parameter outlier (and possible anomaly). More specifically, Figure 4-5 The middle and lower right parts of the chart 400 and 500 are Figure 2 The corresponding portion of graph 200 is significantly different.

[0090] Figure 6-9 The graphs 600, 700, 800, 900 depict anomaly scores for corresponding parameters across four different calibration sets when the host vehicle calibration set is not available as a reference. In this example, one of the calibration sets is set as the reference, as explained above. In this example, Figure 6 The graph 600 depicts the parameters of a calibration set used as a reference (eg, the calibration set with the lowest temporary anomaly as explained above). As such, the anomaly score associated with the graph 600 is zero (0). Figure 7 Graph 700 of depicts corresponding parameters for another calibration set (set #2) having an anomaly score of 0.17544 (e.g., after adjustment as explained above). As shown, Figure 7 The chart of 700 and Figure 6 The chart for 600 is almost identical. Figure 8 The graph 800 of FIG. 8 depicts the corresponding parameters for another calibration set (set #3) with an anomaly score of 14.7258, and Fig. 9 The graph 900 of depicts the corresponding parameters of yet another calibration set (set #4) with an anomaly score of 15.1944. As shown, Figure 8-9 The charts of 800, 900 and Figure 6 600 is significantly different, indicating a parameter outlier (and possible anomaly).

[0091] exist Figure 2-9 In the example of , the parameter values ​​of different vehicle calibration sets are the same, with the only difference being: Figure 2-5 The master calibration set is identified in the example and Figure 6-9In the example of , the master calibration set is not identified. As shown, the control module 102 is able to calculate accurate anomaly scores and identify anomalies (e.g., charts 400, 500, 800, 900) regardless of whether a master calibration set (e.g., an initial reference set) is employed. This ensures effectiveness in all practical scenarios and demonstrates unparalleled versatility and innovation in addressing industry needs. For example, in some scenarios, a new vehicle program may be introduced that does not have a master calibration set and therefore does not have a starting point. In such an example, the control module 102 can accurately calculate anomaly scores and identify anomalies without the starting point.

[0092] Figure 10-13 The charts 1000, 1100, 1200, 1300 depict anomaly scores for four different corresponding parts of two different calibration sets. With this configuration, a user can identify one or more specific parts that cause problems (e.g., due to outlier parameter values ​​in these specific parts), thereby providing a granular view of the analysis. Specifically, the chart 1000 depicts a first part 1002-1 of the parameters of the main calibration set 1002 and a corresponding first part 1004-1 of the parameters of the sub-calibration set 1004. The charts 1100, 1200, 1300 depict similar characteristics of the main calibration set 1002 and the sub-calibration set 1004, but wherein the chart 1100 focuses on the second part 1002-2, 1004-2, the chart 1200 focuses on the third part 1002-3, 1004-3, and the chart 1300 focuses on the fourth part 1002-4, 1004-4. As shown, the main calibration set 1002 is generally below the sub-calibration set 1004 .

[0093] Additionally, in Figure 10-13 In the example of FIG. 1 , each portion of the main calibration set 1002 has an anomaly score of zero (0), the first portion 1004-1 of the sub-calibration set 1004 has an anomaly score of 44.8055, the second portion 1004-2 of the sub-calibration set 1004 has an anomaly score of 65.5232, the third portion 1004-3 of the sub-calibration set 1004 has an anomaly score of 34.9958, and the fourth portion 1004-4 of the sub-calibration set 1004 has an anomaly score of 19.9895. In this example, a particular portion (e.g., the second portion) may be identified as causing the most problems because it has the highest anomaly score.

[0094] Figure 14-16 The graphs 1400, 1500, 1600 depict Figure 10-13 Other examples of diagrams 1000, 1100, 1200, 1300 are shown. For example, Fig.14The diagram 1400 depicts four portions of the diagrams 1000, 1100, 1200, 1300 for the master calibration set 1002, and Fig.15 The diagram 1500 depicts four portions of the diagrams 1000, 1100, 1200, 1300 for the sub-calibration set 1004. Additionally, Fig.16 Graph 1600 depicts four portions of graphs 1000 (anomaly score of 44.8055), 1100 (anomaly score of 65.5232), 1200 (anomaly score of 34.9958), 1300 (anomaly score of 19.9895) for both the main calibration set 1002 and the sub-calibration set 1004 .

[0095] Continue to refer Figure 1 , the control module 102 may transmit at least one visualization having at least one table plotting various variables and / or previously calculated statistical relationships. In such examples, the control module 102 may transmit such data to the display module 106 and / or another suitable module for displaying the table. For example, the table may include various parameter values ​​(in different formats, such as scalars, vectors, and maps) for different calibration sets of the same vehicle program. In other examples, the table may include various parameter values ​​(in different formats, such as scalars, vectors, and maps) for different vehicle programs.

[0096] Table 1 below is an example illustration of data in a visualization that may be displayed to a user on the display module 106. As shown below, Table 1 includes corresponding parameters for different calibration sets (calibration set 1, calibration set 2 ... calibration set 59, calibration set 60) for the same vehicle program (VP1). In this example, the parameters are shown in different formats (such as scalars, vectors, and maps). In various embodiments, the calibration set (calibration set 1) may be a master calibration set, and the other calibration sets (calibration set 2 ... calibration set 59, calibration set 60) may be sub-calibration sets associated with the master calibration set in one vehicle program. Although Table 1 shows sixty (60) calibration sets for a vehicle program (VP1), it should be appreciated that more or fewer calibration sets may be implemented.

[0097] As shown below, scalar parameters are represented by "values", vector parameters are represented by arrows, and map parameters are represented by grids. In this example, each scalar parameter (value) is a single value (e.g., 50, 80, 52, 45, etc.), each vector parameter can be a different value along an axis, and each map parameter can be a different value along two axes. For example, a vector parameter may include x-axis values ​​of 1000, 2000, 3000, 4000 (e.g., battery current values, etc.) and corresponding parameter values ​​of 10, 20, 30, 40 for the x-axis values, respectively (e.g., torque values, etc.). A map parameter may include x-axis values ​​of 1000, 2000, 3000, 4000 (e.g., battery current values, etc.), y-axis values ​​of 100, 200, 300, 400 (e.g., vehicle speed values, etc.), and sixteen parameter values ​​for a grid of x and y axis values.

[0098] Additionally, each scalar parameter (value) in Table 1 may be any suitable user-requested statistical measure. For example, any of the scalar parameters may be a mean, mean difference, standard deviation, covariance, percentage difference, relative error, maximum difference, minimum difference, anomaly score, absolute raw value, etc. Such statistical relationships may be calculated by the control module 102 as explained above.

[0099]

[0100] Table 2 below is another example illustration of data in a visualization that may be displayed to a user on the display module 106. As shown below, Table 2 includes various variants (Variant 1, Variant 2, ... Variant 5) for different vehicle programs (VP1, VP2, VP30). In this example, each variant may represent different parameters in different formats (such as scalar, vector, and map) for different vehicle calibration sets. Similar to Table 1 above, scalar parameters in Table 2 are represented by "values", vector parameters in Table 2 are represented by arrows, and map parameters in Table 2 are represented by grids.

[0101] In this example, the total number of variants for a single vehicle program is five and is differentiated by vehicle features (e.g., options, etc.), such as rear wheel drive (RWD), all wheel drive (AWD), long range battery, etc. Additionally, in this example, the total number of vehicle programs is thirty. Although the total number of variants for the vehicle programs and the total number of vehicle programs are specified as five and thirty, it should be appreciated that such values ​​are merely examples and that more or fewer variants per vehicle program and / or multiple vehicle programs may be employed if desired, depending on, for example, available vehicle features (e.g., options, etc.), the number of manufacturer variants, the number of manufacturer vehicle programs, etc.

[0102] In various embodiments, Table 2 may represent an analysis for a motor calibration control module. In other examples, the same or different tables (e.g., formats, variations, etc.) may be used for a battery calibration control module, an inverter calibration control module, a diagnostic calibration control module, etc. In addition, and as shown below in Table 2, any combination of variations and vehicle programs may be compared to find one or more possible anomalies, as many sub-features of one vehicle (e.g., one vehicle program) may have similar calibrations as sub-features of another vehicle (e.g., another vehicle program).

[0103]

[0104] Continue to refer Figure 1 , the control module 102 may modify one or more vehicle parameters of the vehicle calibration set. For example, in response to detecting an anomaly associated with a vehicle parameter based on an anomaly score (as explained herein), the control module 102 may automatically modify the value of the vehicle parameter. In such an example, the control module 102 may modify the vehicle parameter to a defined value (e.g., as defined by a user, etc.).

[0105] For example, once an anomaly is detected and the anomaly is above a defined threshold set by the user, the control module 102 may implement an automated auto-correction feature to fix the error in the vehicle parameters (calibration) compared to the reference calibration. Figure 10-16 ), the control module 102 can automate the auto-correction feature for the parts above the defined threshold set by the user, so that only these parts can be automatically corrected and automated across all thousands of calibration parameters.

[0106] In other examples, the control module 102 may modify one or more vehicle parameters of the vehicle calibration set based on user input. For example, the control module 102 may display the vehicle calibration set in a graph (e.g., Figure 2-16 The calculated anomaly scores associated with vehicle parameters and / or parts may be transmitted (e.g., to display module 106) in the form of any one or more of graphs, tables, and / or another visualization. In other examples, control module 102 may transmit parameter values ​​and statistical relationships (e.g., Table 1, Table 2, etc.) along with the anomaly scores.

[0107] The user can then analyze the presented data and determine whether to expect a modification. For example, in some scenarios, it may be desirable to have a parameter value outlier, or a modification to one parameter may affect another parameter. In this manner, the user can decide whether a modification of a vehicle parameter is desirable. In such an example, the display module 106 and / or another suitable device can transmit a signal (via user input) to the control module 102 that instructs the control module 102 to modify the vehicle parameter (and possibly to a defined value). The control module 102 can then modify the vehicle parameter in response to the received signal.

[0108] In other examples, even if an anomaly associated with a vehicle parameter is not detected, a user may prefer to modify a vehicle parameter of the vehicle calibration set. For example, in a situation where a vehicle parameter may have a slight outlier value but is not a problem, a user may provide user input to the control module 102 to modify the vehicle parameter. In such an example, the modification, while not necessary to correct the problem, may help achieve better (e.g., optimized) vehicle performance.

[0109] In various embodiments, the control module 102 may Figure 1 The control module 102 may transmit one or more vehicle calibration sets to the vehicle control module 108. For example, the control module 102 may transmit a vehicle calibration set with previously modified vehicle parameter(s), as explained above. In other examples, if modification is not necessary, the control module 102 may transmit a vehicle calibration set with unmodified vehicle parameter(s). In such examples, the vehicle calibration may be flash stored in a memory of the vehicle control module 108 and / or provided to the vehicle control module 108 in another suitable manner.

[0110] In some examples, the control module 102 may transmit the vehicle calibration set at different times. For example, the control module 102 may transmit (e.g., push) a vehicle calibration set including modified (one or more) vehicle parameters during the manufacturing process of the vehicle. In other examples, an anomaly associated with (one or more) vehicle parameters may not become apparent until after the vehicle is manufactured. In such an example, the control module 102 may transmit (e.g., push) the vehicle calibration set including the modified (one or more) vehicle parameters to the vehicle control module 108 in the manufactured vehicle (e.g., during a recall process, OTA update, etc.).

[0111] Figure 17-22 The diagram shows the Figure 1The example control processes 1700, 1800, 1900, 2000, 2100, 2200 employed by the control module 102 of the present invention for detecting anomalies in a vehicle calibration set, generating statistical visualizations of a vehicle calibration set, and / or modifying vehicle calibration. Although the example control processes 1700, 1800, 1900, 2000, 2100, 2200 are related to a control module 102 including Figure 1 Although described with reference to system 100, any of the control processes 1700, 1800, 1900, 2000, 2100, 2200 may be employed by any suitable system.

[0112] like Fig.17 As shown in , control begins at 1702, where the control module 102 receives or imports a vehicle calibration set, as explained herein. For example, the vehicle calibration set includes vehicle parameters having the same or different data formats (e.g., DCM, CSV, etc.). Control then proceeds to 1704, where the control module 102 can optionally filter and / or sort the data in the received vehicle calibration set, as explained above. Control then proceeds to 1706 and 1708.

[0113] At 1706, the control module 102 may optionally calculate or otherwise determine one or more statistical relationships (e.g., statistical metrics such as mean, standard deviation, variance, and / or any other suitable computable relationship between parameters) for the vehicle calibration set. In such examples, the control module 102 may automatically initiate such calculations in response to receiving the vehicle calibration set, in response to filtering, in response to user input, etc.

[0114] At 1708, the control module 102 detects a data type of one or more of the received vehicle calibration sets. For example, and as explained above, the control module 102 can detect whether the received vehicle calibration set is in a scalar format or a vector format or a map format. In such an example, the control module 102 can identify the format type based on the data (e.g., the size of the data, etc.), based on the data being tagged, etc. Control then proceeds to 1710.

[0115] At 1710 , the control module 102 determines whether the detected vehicle calibration set is in scalar format. If so, control proceeds to 1712 . However, if the vehicle calibration set is not in scalar format (eg, instead in vector format or map format), control proceeds to 1718 .

[0116] At 1712, the control module 102 determines whether there is a master calibration set in the vehicle calibration sets. For example, and as explained above, the control module 102 can detect whether one of the vehicle calibration sets is a master calibration set based on user input (e.g., a set identified by a user), a tag provided with the vehicle calibration set, etc.

[0117] If the master calibration set exists, control proceeds to 1714 where the control module 102 calculates an anomaly score based on the existing master calibration set, as explained herein. Control then proceeds to 1728. However, if the master calibration set does not exist or cannot be identified, control proceeds to 1716 where the control module 102 calculates an anomaly score based on all vehicle calibration sets, as explained herein. Control then proceeds to 1728.

[0118] At 1718, the control module 102 determines whether a master calibration set exists in the vehicle calibration set, as explained herein. If a master calibration set exists, and the data is not in scalar format (e.g., instead in vector format or map format), control proceeds to 1720. At 1720, the control module 102 calculates an anomaly score based on the master calibration set and the weighting factor, as explained herein. Control then proceeds to 1728.

[0119] However, if the master calibration set does not exist and the data is not in scalar format (e.g., instead, in vector format or map format), control proceeds to 1722. At 1722, the control module 102 calculates anomaly scores (e.g., temporary anomaly scores) based on all vehicle calibration sets and weighting factors, as explained herein. For example, the control module 102 can calculate temporary anomaly scores for each corresponding vehicle parameter of all vehicle calibration sets based on a cost function. As such, the temporary anomaly scores for the vehicle parameters of each vehicle calibration set can be referenced relative to the corresponding vehicle parameters of other vehicle calibration sets. Fig.25 An example diagram 2500 is depicted showing such interactions between nine different vehicle calibration sets 2502, 2504, 2506, 2508, 2510, 2512, 2514, 2516, 2518. Fig.25 Diagram 2500 includes nine vehicle calibration sets, but it should be appreciated that more or fewer vehicle calibration sets may be employed (e.g., six vehicle calibration sets, twelve vehicle calibration sets, 24 vehicle calibration sets, etc.).

[0120] Continue to refer Fig.17 , control then proceeds to 1724 and 1726. At 1724, the control module 102 sets the vehicle calibration set with the lowest temporary anomaly score as the reference set. Then at 1726, the control module 102 adjusts the other calculated temporary anomaly scores based on the reference set, as explained herein. Control then proceeds to 1728.

[0121] At 1728, the control module 102 generates and outputs one or more visualizations having the calculated anomaly scores (from 1714 or 1716 or 1720 or 1726) and / or the calculated statistical relationships (from 1706). For example, and as explained herein, the control module 102 may compile the calculated data in different visual formats (such as charts, tables, etc.) for viewing by the user. The control module 102 may then transmit the charts, tables, etc. to Figure 1 Control then proceeds to 1730 .

[0122] At 1730, the control module 102 determines whether any input was received. For example, a user may analyze the data provided to the display module 106 and determine whether a parameter modification is desired, as explained herein. If so, the user may transmit the user input to the control module 102 (e.g., via the display module 106 and / or another suitable device). The user input may instruct the control module 102 to modify one or more specific vehicle parameters of a specific calibration set. If no user input is received, control proceeds to 1734.

[0123] However, if the control module 102 receives user input, control proceeds to 1732 where the control module 102 corrects or modifies certain vehicle parameters. Control then proceeds to 1734.

[0124] At 1734, the control module 102 transmits one or more vehicle calibration sets to a vehicle control module (eg, Figure 1 1732). For example, and as explained above, the control module 102 may transmit a vehicle calibration set having previously modified vehicle parameter(s) (e.g., modified at 1732), as explained above. In other examples, if modification is not necessary, the control module 102 may transmit a vehicle calibration set having unmodified vehicle parameter(s). In such examples, the vehicle calibration may be flash stored to a memory of the vehicle control module 108 and / or provided to the vehicle control module 108 in another suitable manner. Control may then end.

[0125] In various embodiments, the control module 102 may optionally compare the calculated anomaly score (from 1714 or 1716 or 1720 or 1726) to one or more defined thresholds to detect anomalies. This step may occur before or after the control module 102 generates and outputs one or more visualizations. In some examples, the control module 102 may only generate and output one or more visualizations if one or more anomalies are detected.

[0126] Fig.18 The control process 1800 depicts Fig.17 In one example implementation of step 1714, the control module 102 calculates an anomaly score for the scalar data based on the existing master calibration set. As shown, control begins at 1802, where the control module 102 selects one of the received sub-calibration sets. Control then proceeds to 1804 and 1806.

[0127] At 1804, the control module 102 calculates the distance of each vehicle parameter of the selected sub-calibration set relative to the corresponding vehicle parameter of the main calibration set, as explained herein. Then at 1806, the control module 102 calculates (one or more) anomaly scores for the (one or more) sub-vehicle parameters based on the (one or more) calculated distances. As explained herein, the control module 102 can calculate (one or more) anomaly scores based on normalized distances. Control then proceeds to 1808.

[0128] At 1808, the control module 102 determines whether there are any more sub-calibration sets remaining. If so, control proceeds to 1810, where the control module 102 selects the next sub-calibration set. Control then returns to 1804 and 1806, where the anomaly score(s) for the next calibration set are calculated. However, if at 1808 no more sub-calibration sets remain, control proceeds to 1812, where the control module 102 outputs or otherwise makes available the calculated anomaly score(s). Control may then end.

[0129] Fig.19 The control process 1900 depicts Fig.17 1716, in which the control module 102 calculates an abnormality score for the scalar data based on other vehicle calibration sets (e.g., no master calibration set exists or no master calibration set is identified), as explained herein. As shown, control begins at 1902, where the control module 102 calculates the mean of all calibration sets. For example, and as explained above, the control module 102 can calculate the mean of each corresponding vehicle parameter of the vehicle calibration set. Control then proceeds to 1904, where the control module 102 selects one of the received sub-calibration sets. Control then proceeds to 1906 and 1908.

[0130] At 1906, the control module 102 calculates the distance of each vehicle parameter of the selected subcalibration set relative to the corresponding calculated mean, as explained herein. Then at 1908, the control module 102 calculates anomaly scores for the vehicle parameter(s) based on the calculated distance(s). As explained herein, the control module 102 may calculate anomaly scores based on normalized distances. Control then proceeds to 1910.

[0131] At 1910, the control module 102 determines whether there are any more calibration sets remaining. If so, control proceeds to 1912, where the control module 102 selects the next calibration set. Control then returns to 1906 and 1908, where the anomaly score(s) for the next set are calculated. However, if at 1910 no more calibration sets remain, control proceeds to 1914, where the control module 102 outputs or otherwise makes available the calculated anomaly score(s). Control may then end.

[0132] Fig. 20 The Control Process 2000 depicts Fig.17 1720 , in which the control module 102 calculates an anomaly score for the vector or map data based on the existing master calibration set. As shown, control begins at 2002, where the control module 102 selects a defined weight factor (e.g., including various defined weights). Control then proceeds to 2004, where the control module 102 calculates a gradient for each corresponding vehicle parameter of the vehicle calibration set, as explained herein. Control then proceeds to 2006, where the control module 102 selects one of the received sub-calibration sets. Control then proceeds to 2008 and 2010.

[0133] At 2008, the control module 102 calculates a numerical relationship of each corresponding vehicle parameter of the sub-calibration set relative to the main calibration set. For example, and as explained above, the control module 102 may calculate a Euclidean distance and a linear regression relationship of each corresponding vehicle parameter of each sub-calibration set relative to the corresponding vehicle parameter of the main calibration set.

[0134] At 2010, the control module 102 calculates anomaly scores for the vehicle parameter(s) in the selected calibration set. For example, and as explained above, the control module 102 may calculate anomaly scores by implementing a defined cost function that sums the calculated gradient, the calculated Euclidean distance, and the calculated linear regression, each of which is weighted according to a defined weight factor. Control then proceeds to 2012.

[0135] At 2012, the control module 102 determines whether there are any more sub-calibration sets remaining. If so, control proceeds to 2014, where the control module 102 selects the next sub-calibration set. Control then returns to 2008 and 2010, where the anomaly score(s) for the next set are calculated. However, if at 2012 no more calibration sets remain, control proceeds to 2016, where the control module 102 outputs or otherwise makes available the calculated anomaly score(s). Control may then end.

[0136] Fig.21 The control process 2100 depicts Fig.17 In one example implementation of step 1722, when no master calibration set is present or not identified, the control module 102 calculates an anomaly score for the vector or map data. As shown, control begins at 2102 and 2104, where the control module 102 selects a defined weight factor (e.g., including various defined weights). In various embodiments, the defined weight factor selected at 2102 may be the same as that selected at Fig. 20 The value selected at 2002 is a different value. Control then proceeds to 2104.

[0137] At 2104, the control module 102 calculates the gradient of each corresponding vehicle parameter of the vehicle calibration set, as explained herein. Control then proceeds to 2106, where the control module 102 selects one of the received calibration sets. Control then proceeds to 2108, 2110, 2112.

[0138] At 2108, the control module 102 calculates a numerical relationship (e.g., Euclidean distance, linear regression relationship, etc.) of each corresponding vehicle parameter of the calibration set relative to each other calibration set. At 2110, the control module 102 calculates an average of each calculated numerical relationship (e.g., an average of all Euclidean distance values ​​for the selected vehicle calibration set, an average of all linear regression values ​​for the selected vehicle calibration set, etc.).

[0139] At 2112, the control module 102 calculates anomaly scores for the vehicle parameter(s) in the selected calibration set. For example, and as explained above, the control module 102 may calculate anomaly scores (e.g., interim anomaly scores) by implementing a defined cost function that sums the calculated gradient, the calculated average Euclidean distance, and the calculated average linear regression, each of which is weighted according to a defined weight factor (selected at 2102). Control then proceeds to 2114.

[0140] At 2114, the control module 102 determines whether there are any more calibration sets remaining. If so, control proceeds to 2116, where the control module 102 selects the next calibration set. Control then returns to 2108, 2110, 2112, where the anomaly score(s) for the next set are calculated. However, if at 2114 no more calibration sets remain, control proceeds to 2118, 2120, 2122.

[0141] At 2118, the control module 102 sets the vehicle calibration set with the lowest (e.g., temporary) anomaly score as the reference set. Then, at 2120, the control module 102 adjusts other calculated (e.g., temporary) anomaly scores based on the reference set, as explained herein. Control then proceeds to 2122, where the control module 102 outputs or otherwise makes available the calculated anomaly score(s). Control may then end.

[0142] Fig. 22 The control process 2200 of FIG. 1 depicts an example implementation of automatically correcting vehicle parameters. In various embodiments, the control process 2200 can be implemented in conjunction with the control process 1700, such as, for example, in Fig.17 Before or after step 1728. In other examples, the control process 2200 can replace Fig.17 Steps 1728, 1730, 1732, 1734.

[0143] As shown, control begins at 2202, where the control module 102 receives a defined abnormality threshold. In various embodiments, the defined abnormality threshold may be user-set. As such, the control module 102 may receive the threshold via user input. Control then proceeds to 2204.

[0144] At 2204, the control module 102 receives a reference calibration set having, for example, various calibrations (e.g., parameters). In various embodiments, the reference calibration may be user defined. For example, the reference calibration may be a master calibration set or another suitable reference set, as explained herein. Control then proceeds to 2206.

[0145] At 2206, the control module 102 determines if any detected anomalies have a score greater than a defined anomaly threshold. If yes at 2206, control proceeds to 2208, where the control module 102 automatically corrects the calibration corresponding to the detected anomaly (e.g., the calibration in a particular section). In such an example, the control module 102 may set the calibration equal to the reference calibration. Control proceeds to 2210. If no at 2206, control proceeds to 2210, where the control module 102 transmits one or more vehicle calibration sets to a vehicle control module (e.g., Figure 1 The vehicle control module 108 is described herein as explained in detail. Control may then end.

[0146] In various embodiments, after detecting and correcting the anomaly, the (one or more) updated calibrations may be transmitted to the manufacturing facility used to make one or more vehicles. This enables the (one or more) new calibrations to be integrated with the software, thereby facilitating the flash storage process. In addition, in instances where the vehicle is in the post-production stage and is already owned by the customer, the (one or more) updated calibrations may be delivered via over-the-air (OTA) technology. This approach is used to avoid the necessity of actually recalling the vehicle, thereby effectively saving both time and financial resources.

[0147] For example, Fig.23 Depicted include Figure 1 23. As shown, the control module 102 receives vehicle calibration sets (e.g., master calibration sets (if applicable), sub-calibration sets, etc.) associated with vehicle program (1) 2310, vehicle program (2) 2320, and vehicle program (N) 2330. In this example, N may be any suitable positive value, such as thirty, thirty-five, forty, etc. In various embodiments, each vehicle calibration set may be generated according to a plurality of process stages, such as requirements engineering, system design and development, software development, software integration, functional integration, and functional calibration.

[0148] The control module 102 may then detect and subsequently correct one or more anomalies associated with the calibration(s) of a particular calibration set of vehicle program (1) 2310, vehicle program (2) 2320, and / or vehicle program (N) 2330, as explained herein. Next, the control module 102 may transmit the new / corrected calibration(s) to various locations. For example, and as Fig.23 As shown in , the control module 102 may transmit the new / corrected calibration(s) to a manufacturing facility 2340 for making one or more vehicles. Additionally and / or alternatively, the control module 102 may transmit the new / corrected calibration(s) to a customer-owned post-production vehicle 2350. In such an example, the control module 102 may transmit the new / corrected calibration(s) via OTA technology.

[0149] For example, Fig.24 Depicted include Figure 1 The vehicle control module 108 of the vehicle 2400. Fig.24 In the example of Figure 1The vehicle 2400 may communicate with the control module 102 to receive a calibration, including possible (one or more) new / corrected calibrations. In various embodiments, the vehicle 2400 may be a post-production vehicle owned by a customer. In such an example, the vehicle control module 108 may receive (one or more) new / corrected calibrations via OTA technology. In other examples, the vehicle 2400 may be a vehicle in a manufacturing facility (e.g., in the process of being manufactured). In such an example, (one or more) new / corrected calibrations may be flash stored in a memory associated with the vehicle control module 108, as explained herein.

[0150] The foregoing description is merely illustrative in nature and is by no means intended to limit the present disclosure, its application or use. The broad teachings of the present disclosure can be implemented in a variety of forms. Therefore, although the present disclosure includes specific examples, the true scope of the present disclosure should not be so limited, because other modifications will become apparent when studying the drawings, the specification and the following claims. It should be understood that, without changing the principles of the present disclosure, one or more steps in the method can be performed in different orders (or simultaneously). In addition, although each embodiment is described above as having certain features, any one or more of those features described with respect to any embodiment of the present disclosure can be implemented in the features of any other embodiment and / or combined with the features of any other embodiment, even if the combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and one or more embodiments and each other's permutation (permutation) are still within the scope of the present disclosure.

[0151] Spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using various terms, including "connected," "engaged," "coupled," "adjacent," "immediately adjacent," "on," "above," "below," and "disposed." Unless explicitly described as "direct," when describing a relationship between a first and a second element in the above disclosure, the relationship may be a direct relationship in which no other intervening elements exist between the first and second elements, but may also be an indirect relationship in which one or more intervening elements (either spatially or functionally) exist between the first and second elements. As used herein, the phrase "at least one of A, B, and C" should be interpreted to mean a logical (A or B or C) using a non-exclusive logical "or," and should not be interpreted to mean "at least one of A, at least one of B, and at least one of C."

[0152] In the various figures, the direction of the arrow, as indicated by the arrow head, generally indicates the flow of information (such as data or instructions) of interest to the illustration. For example, when element A and element B exchange various information, but the information transmitted from element A to element B is relevant to the illustration, the arrow may point from element A to element B. The unidirectional arrow does not imply that no other information is transmitted from element B to element A. In addition, for information sent from element A to element B, element B may send a request or a receipt confirmation of the information to element A.

[0153] In this application, including the definitions below, the term "module" or the term "controller" may be replaced with the term "circuit". The term "module" may refer to, be part of, or include: an application specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or grouped) that executes code; a memory circuit (shared, dedicated, or grouped) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system on a chip.

[0154] The module may include one or more interface circuits. In some examples, the interface circuit may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functions of any given module of the present disclosure may be distributed among multiple modules connected via the interface circuit. For example, multiple modules may allow load balancing. In a further example, a server (also referred to as a remote or cloud) module may implement some functions on behalf of a client module.

[0155] The term "code" as used above may include software, firmware, and / or microcode, and may refer to a program, a routine, a function, a class, a data structure, and / or an object. The term "shared processor circuit" covers a single processor circuit that executes some or all code from multiple modules. The term "group processor circuit" covers a processor circuit that executes some or all code from one or more modules in conjunction with an additional processor circuit. References to multiple processor circuits cover multiple processor circuits on discrete dies, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" covers a single memory circuit that stores some or all code from multiple modules. The term "group memory circuit" includes a memory circuit that stores some or all code from one or more modules in conjunction with additional memory.

[0156] The term "memory circuit" is a subset of the term "computer-readable medium". The term "computer-readable medium" as used herein does not encompass non-transitory electrical or electromagnetic signals propagated through a medium (such as on a carrier wave); therefore, the term "computer-readable medium" may be considered to be tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0157] The apparatus and method described in the present application can be implemented in part or in whole by a special-purpose computer, which is created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The functional blocks, flow chart components and other elements described above act as software specifications, which can be translated into computer programs by the routine work of skilled technicians or programmers.

[0158] The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include or rely on stored data. The computer program may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0159] A computer program may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation), (ii) assembly code, (iii) object code generated by a compiler from source code, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. By way of example only, source code may be written using syntax from languages ​​including: C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Fortran, Perl, Pascal, Curl, OCaml, HTML5 (Hypertext Markup Language Version 5), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Lua, MATLAB, SIMULINK, and

Claims

1. A method for detecting anomalies in a vehicle calibration set, the method comprising: receiving a plurality of vehicle calibration sets, each vehicle calibration set comprising corresponding vehicle parameters; An anomaly score is calculated for each vehicle parameter of each vehicle calibration set; detecting an anomaly associated with at least one vehicle parameter of a vehicle calibration set of the plurality of vehicle calibration sets based on an anomaly score of the vehicle parameter and a defined threshold; In response to detecting the anomaly, modifying the vehicle parameters of the vehicle calibration set; as well as The vehicle calibration set including the modified vehicle parameters is transmitted to a vehicle control module associated with a vehicle, the vehicle control module being used to control at least one component of the vehicle.

2. The method according to claim 1, wherein: The plurality of vehicle calibration sets include a main vehicle calibration set and one or more sub-vehicle calibration sets; as well as Calculating the anomaly score includes: calculating a gradient of each corresponding vehicle parameter of the plurality of vehicle calibration sets, calculating one or more numerical relationships of each corresponding vehicle parameter of each child vehicle calibration set relative to the corresponding vehicle parameter of the host vehicle calibration set, and An anomaly score is calculated based on a cost function, a weighted value of the gradient, and a weighted value of the one or more numerical relationships.

3. The method of claim 1, wherein calculating an anomaly score comprises: calculating a gradient of each corresponding vehicle parameter of the plurality of vehicle calibration sets; as well as One or more numerical relationships are calculated for each corresponding vehicle parameter of each vehicle calibration set relative to corresponding vehicle parameters of all other vehicle calibration sets.

4. The method of claim 3, wherein calculating anomaly score comprises: calculating a temporary anomaly score for each corresponding vehicle parameter of the plurality of vehicle calibration sets based on a cost function, a weighted value of the gradient, and a weighted value of the one or more numerical relationships; setting the vehicle calibration set having corresponding vehicle parameters with the lowest temporary anomaly score as a reference vehicle calibration set; as well as An anomaly score is calculated by adjusting a temporary anomaly score for each corresponding vehicle parameter based on the reference vehicle calibration set.

5. The method according to claim 1, wherein: The plurality of vehicle calibration sets include a main vehicle calibration set and one or more sub-vehicle calibration sets; as well as Calculating the anomaly score includes: Calculating the distance of each corresponding vehicle parameter of each child vehicle calibration set relative to the corresponding vehicle parameter of the main vehicle calibration set, and An anomaly score is calculated based on the distance.

6. The method of claim 1, wherein calculating an anomaly score comprises: calculating a mean value of each corresponding vehicle parameter of the plurality of vehicle calibration sets; calculating a distance of each corresponding vehicle parameter of each vehicle calibration set relative to a calculated mean of the vehicle parameter; as well as An anomaly score is calculated based on the distance.

7. The method according to claim 1, further comprising: A visualization including anomaly scores associated with the detected anomalies is transmitted to a display module.

8. The method according to claim 7, further comprising: A user input is received in response to the transmitted visualization, wherein modifying the vehicle parameter of the vehicle calibration set includes modifying the vehicle parameter of the vehicle calibration set in response to the received user input.

9. The method according to claim 1, further comprising: calculating one or more statistical relationships of corresponding vehicle parameters of the plurality of vehicle calibration sets; as well as A visualization including at least one table plotting the one or more statistical relationships is transmitted to a display module.

10. The method of claim 1, wherein: The method further includes receiving a defined auto-correction threshold; as well as Modifying the vehicle parameters of the vehicle calibration includes automatically correcting the vehicle parameters of the vehicle calibration to a reference calibration in response to the detected anomaly being greater than a defined automatic correction threshold.

Images