Vehicle actuation based on software updates

Abstract

The invention provides vehicle actuation based on software updates. A computer includes a memory and a memory including instructions executable by a processor to receive data regarding a specified aspect of vehicle performance corresponding to a plurality of vehicles. A validity of a software update for a specified aspect of vehicle performance is determined based on the data, and when the determined validity is below a specified threshold, the system actuates a change in at least one of the plurality of vehicles.

Description

Technical Field

[0001] The present disclosure relates to a vehicle actuation based on software update. Background Art

[0002] Software updates can be used to address various aspects of vehicle performance. For example, electric vehicles can benefit from software updates (e.g., over-the-air updates) to maintain battery performance. Due to the complex interactions of different vehicles and varying operating conditions, software updates may affect vehicles in different ways. Additionally, during the release of a specific software update designed to address a specific aspect of vehicle performance, other software updates may also be pushed to the vehicle at or near the same time. Summary of the Invention

[0003] The present disclosure provides techniques for actuating changes in the state of a vehicle when using causal machine learning to determine the effectiveness of a software update for a specified aspect of vehicle performance. Based on the effectiveness of the software update, the system can actuate changes to the vehicle, which may include restoring the software to a previous version and / or disabling components of the vehicle. The software update can address, for example, electric vehicles that may benefit from software updates (e.g., over-the-air updates) to maintain battery performance. Software updates can be provided to address specific issues in a fleet of vehicles, such as battery drain caused by area lighting remaining on after the vehicle is turned off. Due to the complex interactions and varying operating conditions of different vehicles, software updates may affect various vehicles in different ways. Additionally, during a particular software update that is provided to address a specific aspect of vehicle performance, other software updates may also be pushed to the vehicle at or near the same time.

[0004] In an example, a software update targeting a specified aspect of vehicle performance is rolled out to a group of vehicles. Data, including vehicle characteristics, warranty information, and vehicle telematics (e.g., diagnostic trouble codes), is collected for vehicles with the software update (i.e., the treatment group) and vehicles without the software update (i.e., the control group). Conventional causal machine learning techniques (e.g., x-learners, s-learners, causal trees, etc.) are applied to the collected data to construct a causal model for performing a conditional average treatment effect (CATE) calculation. The causal model is then used to perform a two-level causal analysis. The first level calculates the CATE of the software update for outcomes related to vehicle performance. In other words, the first level indicates how the remedial action (i.e., the software update) changes the aspect of vehicle performance of interest. For example, the first level may determine whether the software update increases or decreases the incidence of a specific diagnostic trouble code (DTC). The second level calculates the average treatment effect (ATE) of the feature group on the CATE calculated from the previous first-level step. The second level indicates how different groups of features affect changes in the CATE. This can be used to indicate which vehicle characteristics are associated with changes in vehicle performance caused by the software update. The causal effect of multiple software updates over time is evaluated by summing the effectiveness of the software update and one or more subsequent software updates over time.

[0005] A system is disclosed herein that includes a computer having a processor and a memory. The memory includes instructions executable by the processor to receive data corresponding to a plurality of vehicles regarding a specified aspect of vehicle performance. The system determines the effectiveness of a software update for the specified aspect of vehicle performance based on the data, and when the determined effectiveness is below a specified threshold, initiates a change in at least one of the plurality of vehicles.

[0006] The instructions for determining the effectiveness of the software update may include instructions for applying a causal model to the received data.

[0007] The instructions for applying the causal model may include instructions for determining a conditional average treatment effect of the software update based on the data.

[0008] The instructions for determining a conditional average treatment effect of the software update based on the data may include instructions for determining using an S-learner, an X-learner, or a causal tree algorithm.

[0009] The instructions for applying the causal model may include instructions for determining an average treatment effect of the vehicle feature based on the determined conditional average treatment effect of the software update.

[0010] The instructions for applying the causal model may include instructions for summing the effectiveness of the software update and one or more subsequent software updates.

[0011] The instructions for applying the causal model may include instructions for grouping the plurality of vehicles by a characteristic.

[0012] The instructions for actuating the change to at least one of the plurality of vehicles may include instructions for reverting the software update to a previous version and / or disabling a component of at least one of the plurality of vehicles.

[0013] The received data may include warranty claim information and / or vehicle diagnostic trouble codes.

[0014] The instructions for actuating a change in at least one of the plurality of vehicles may include instructions for reverting a software update to a previous version and / or disabling components of a group of vehicles of the plurality of vehicles having a specified characteristic.

[0015] A method for actuating a change in a vehicle is disclosed herein. The method includes receiving data corresponding to a plurality of vehicles regarding a specified aspect of vehicle performance. The method may include determining, based on the data, the effectiveness of a software update for the specified aspect of vehicle performance; and actuating the change in at least one of the plurality of vehicles when the determined effectiveness is below a specified threshold.

[0016] Determining the effectiveness of the software update may include applying a causal model to the received data.

[0017] Applying the causal model may include determining a conditional average treatment effect of the software update based on the data.

[0018] Determining the conditional average treatment effect of the software update based on the data may include using an S-learner, an X-learner, or a causal tree algorithm.

[0019] Applying the causal model may include determining an average treatment effect of the vehicle feature based on the determined conditional average treatment effect of the software update.

[0020] Applying the causal model may include summing the effectiveness of the software update and one or more subsequent software updates.

[0021] Applying causal patterns may include grouping multiple vehicles by characteristics.

[0022] Actuating the change to at least one of the plurality of vehicles may include reverting a software update to a previous version and / or disabling a component of at least one of the plurality of vehicles.

[0023] The received data may include warranty claim information and / or vehicle diagnostic trouble codes.

[0024] Actuating the change to at least one of the plurality of vehicles may include reverting a software update to a previous version and / or disabling components of a group of vehicles of the plurality of vehicles having a specified characteristic. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a block diagram of an example vehicle.

[0026] Figure 2 is a process flow diagram illustrating an example process for vehicle change actuation based on a software update.

[0027] Figure 3 This is a block diagram of software effectiveness analysis based on causal machine learning.

[0028] Figure 4 It is a block diagram of the first-level causal analysis.

[0029] Figure 5 is a histogram showing the conditional average treatment effect.

[0030] Figure 6 It is a block diagram of the second-level causal analysis.

[0031] Figure 7 is a histogram showing the average treatment effect.

[0032] Figure 8 is a graph showing the effect of software updates over time. DETAILED DESCRIPTION

[0033] Figure 1 is a diagram of an example system 100. System 100 includes a vehicle 110 that can be operated by a user and / or under the control of one or more computing devices 115, which may include one or more vehicle electronic control units (ECUs) or computers, such as are known in the art, which may include additional hardware, software, and / or programming, as described herein. Computing device 115 can receive data regarding the operation of vehicle 110 from sensors 116. Computing device 115 can operate vehicle 110 or components thereof in lieu of, or in combination with, control by a human user. System 100 can also include a remote (i.e., external to the vehicle) server computer 120 that can communicate with vehicle 110 via network 130.

[0034] The computing device 115 may include one or more processors and one or more memory devices, such as are known in the art. Furthermore, the memory may include one or more forms of computer-readable media and store instructions that are executable by the processor to perform various operations (including the operations disclosed herein). For example, the computing device 115 may include programming to operate one or more of vehicle braking, propulsion (e.g., controlling acceleration of the vehicle 110 by controlling one or more of an internal combustion engine, an electric motor, a hybrid engine, etc.), steering, climate control, interior lights, and / or exterior lights, etc.

[0035] The computing device 115 may include more than one computing device, such as a controller, ECU, or the like included in the vehicle 110 for monitoring and / or controlling various vehicle subsystems (e.g., the propulsion subsystem 112, the braking subsystem 113, the steering subsystem 114, etc.), or may be communicatively coupled to the more than one computing device, such as via a vehicle communication bus as further described below. The computing device 115 is typically arranged to communicate on a vehicle communication network (e.g., including a bus in the vehicle 110, such as a controller area network (CAN), etc.). The vehicle network may additionally or alternatively include known wired or wireless communication mechanisms, such as Ethernet or other communication protocols.

[0036] The computing device 115 can transmit messages to and / or receive messages from various devices in the vehicle (e.g., controllers, actuators, sensors (including sensor 116), etc.) via the vehicle network. Alternatively or additionally, where the computing device 115 includes multiple devices, the vehicle communication network can be used for communication between the devices represented in this disclosure as computing device 115. Furthermore, as mentioned below, various controllers or sensing elements (such as sensor 116) can provide data to the computing device 115 via the vehicle communication network.

[0037] Additionally, the computing device 115 may be configured to communicate with a remote server computer 120 (such as a cloud server) via a network 130 through a vehicle-to-infrastructure (V-to-I) interface, as described below, which includes hardware, firmware, and software that permits the computing device 115 to communicate with a remote server computer 120 (such as a cloud server) via a vehicle-to-infrastructure (V-to-I) interface, such as a wireless Internet connection. The V2X interface 111 may thus include a computer configured to communicate with the remote server computer 120 using various wired and / or wireless networking technologies (e.g., cellular, The computing device 115 may include a processor, memory, transceiver, and the like, such as Bluetooth Low Energy (BLE), Ultra-Wideband (UWB), peer-to-peer communication, UWB-based radar, IEEE 802.11, and / or other wired and / or wireless packet networks or technologies. The computing device 115 may be configured to communicate with other vehicles via a V2X (vehicle-to-everything) interface 111 using, for example, a vehicle-to-vehicle (V-to-V) network formed between nearby vehicles 110 on a mobile ad hoc basis or via an infrastructure-based network (e.g., based on cellular communication (C-V2X) wireless communication, dedicated short-range communication (DSRC), and / or similar communications). The computing device 115 also includes a non-volatile memory, such as is known in the art. The computing device 115 may record data by storing it in the non-volatile memory for later retrieval and transmission to a server computer 120 or a user's mobile device via the vehicle communication network and the vehicle-to-infrastructure (V2X) interface 111.

[0038] As already mentioned, the instructions stored in the memory and executable by the processor of the computing device 115 generally include programming for operating one or more vehicle components (e.g., braking, steering, propulsion, etc.). Using data received at the computing device 115 (e.g., sensor data from sensors 116, server computer 120, etc.), the computing device 115 can make various determinations and / or control various vehicle components and / or operations.

[0039] Each of the subsystems 112, 113, 114 may include a corresponding processor and memory and / or one or more actuators. The subsystems 112, 113, 114 may be programmed and connected to a vehicle communication bus, such as a controller area network (CAN) bus or a local interconnect network (LIN) bus, to receive instructions from the computing device 115 and control the actuators based on the instructions.

[0040] Vehicle 110 is typically a land-based vehicle 110 having three or more wheels, such as a passenger car, light truck, etc. Vehicle 110 includes one or more sensors 116, a V2X interface 111, a computing device 115, and one or more subsystems 112, 113, and 114. Sensors 116 can collect data related to vehicle 110 and the operating environment of vehicle 110. By way of example and not limitation, sensors 116 can include, for example, altimeters, cameras, lidars, radars, ultrasonic sensors, infrared sensors, pressure sensors, accelerometers, gyroscopes, temperature sensors, pressure sensors, Hall sensors, optical sensors, voltage sensors, current sensors, mechanical sensors (such as switches), and the like. Sensors 116 can be used to sense the operating environment of vehicle 110. For example, sensors 116 can detect phenomena such as weather conditions (rainfall, external ambient temperature, etc.), road grade, road position (e.g., using road edges, lane markings, etc.), or the location of target objects (such as neighboring vehicles). Sensors 116 can also be used to collect data, including dynamic vehicle data related to the operation of vehicle 110, such as speed, yaw rate, steering angle, engine speed, brake pressure, oil pressure, power levels applied to subsystems 112, 113, 114 in vehicle 110, connectivity between components, and accurate and timely performance of components of vehicle 110.

[0041] The server computer 120 generally has features in common with the V2X interface 111 and computing device 115 of the vehicle 110, such as a computer processor and memory and a configuration for communicating via the network 130, and therefore these features will not be described further. The server computer 120 can be used to develop and train software that can be transferred to the computing device 115 in the vehicle 110.

[0042] Figure 2 1 is a flow chart illustrating an example process 200 for actuating a vehicle change based on a software update. The process 200 may be implemented in a computing device 115 included in the vehicle 110 and / or a server computer 120 in communication with the computing device 115 via a network 130. The process 200 includes a plurality of blocks that may be executed in the order shown. Alternatively or in addition, the process 200 may include more or fewer blocks and / or may include blocks executed in a different order.

[0043] Process 200 begins at block 202 when, for example, a server computer 120 pushes new software or a software update to a plurality of vehicles or a fleet of vehicles. In the context of this document, "pushing" software or a software update means providing the software to the computing device 115 via the network 130, typically at a scheduled time and / or a scheduled event (e.g., the vehicle 110 reaches a non-operating state, such as a parked, propulsion-off, and / or ignition-off state, at or after a specified time). In an example, the software update may be pushed via an over-the-air (OTA) update. Once the software update is created and before it is pushed to a fleet of vehicles, the software update may be tested using A / B testing (i.e., testing in which random variations are provided to different vehicles in a controlled manner to evaluate the effects of the variations) on test vehicles. Observational data from testing the test vehicles may be used to update the software as needed prior to deployment on consumer vehicles. In some examples, a software update may be sent via OTA to update a specified target ECU module (target ECU T * ).

[0044] As an example, certain vehicle models may experience unexpected or abnormal battery drain (e.g., above a threshold of typically expected battery drain) after the vehicle is turned off when the vehicle is not in use, resulting in battery drain and, in some cases, premature replacement of the battery under warranty. Battery drain issues may also be associated with specific DTCs. In such cases, a software update may be provided to address this issue by changing the target ECU T * The logic used for battery usage in the .

[0045] At block 204, the server computer 120 collects data regarding the results of the software update. The server computer 120 may receive data corresponding to a plurality of vehicles regarding specified aspects of vehicle performance, such as specific DTCs associated with abnormal or unexpected battery drain. The received data may include, for example, warranty claim information, vehicle diagnostic trouble codes, and specific performance parameters, such as the percentage of vehicle key cycles with abnormal battery drain or a decrease in state of charge (SOC). In an example, abnormal battery drain is a drain that is greater than a specified drain threshold (e.g., eight percent) within a threshold amount of time (e.g., one hour) following a key-off event. The drain threshold and time threshold may be based on empirical testing of a specific battery platform and / or design parameters.

[0046] In addition to observational data regarding the results, server computer 120 also receives vehicle characteristics, including vehicle features. Highly correlated vehicle features are grouped together to reduce the number of features to be analyzed, and representative features can be selected for each group. A feature grouping algorithm identifies these highly correlated features. The algorithm can determine the correlation / association of each pair of features based on statistical metrics, including correlation algorithms, interaction information, and the like. Feature groups are matched to the treatment and control groups so that both groups have similar feature groups. Vehicle features are components, groups of components, and / or systems controlled or influenced by software. In one example, these features can be grouped by functional packages, such as an entertainment package that provides audio and / or video playback and typically includes components such as a screen, video player, and speakers. The control and treatment groups may have the same feature groups, but the distribution of features in the treatment and control groups may differ. For example, in the control group, a feature value may range from 0 to 10, while in the treatment group, the value of that feature may only range from 9 to 10. This matching step helps ensure that the distribution of feature values in the control and treatment groups is similar.

[0047] At box 206, the server computer 120 performs a causal machine learning analysis to determine the effectiveness of the software update. Once the data is collected in box 204, including vehicle characteristics, warranty information, and vehicle telematics (e.g., diagnostic trouble codes) for vehicles with the software update (i.e., the treatment group) and vehicles without the software update (i.e., the control group), a causal machine learning algorithm (e.g., an x-learner, an s-learner, a causal tree, etc.) is applied to the collected data to build a causal model for performing the CATE calculation. Figures 3 to 7Interpretation is then used to perform two-level causal analysis using causal models. Causal machine learning is a branch of machine learning focused on understanding causal relationships in data. Causal machine learning estimates the causal effect of treatment T on outcome Y for a vehicle with observed covariate features X. For example, CausalML is a Python package that provides examples of suitable machine learning algorithms for estimating CATE from experimental or observational data.

[0048] At decision block 208, the server computer 120 evaluates the effectiveness of the software update for the specified aspect of vehicle performance based on the data. The effectiveness can be positive, negative, or no effect. For example, continuing with the example of abnormal (i.e., unexpected or unusual) battery drain, positive effectiveness can be indicated by a decrease in the frequency of a particular DTC associated with battery drain. Negative effectiveness can be indicated by an increase in the frequency of a particular DTC. No effect can be indicated by no change in the frequency of a particular DTC.

[0049] If the software update has no effect, process 200 proceeds to block 210 for further investigation. If the software update has a negative effect, process 200 proceeds to block 212 to initiate changes to the vehicles. If the software update has a positive effect, the update continues for some or all eligible vehicles. Typically, a software update is applied to vehicles for which it has been determined that the software update will have a positive effect. In some examples, there may be a group or subset (typically a minority) of vehicles (e.g., vehicles with or without certain characteristics) that have no or a negative effect from the software update. In these cases, the software update determined to have no effect and / or a negative effect may be stopped or not provided to the subset of vehicles.

[0050] At block 210 , if the software update had no effect, server computer 120 may send data regarding that result to a remote computer such as server 120 , for example, to notify a software update team. Process 200 typically ends after block 210 .

[0051] At block 212 , if the software update had a negative effect, server computer 120 may activate changes to the vehicles on which the update had a negative effect. When the determined effectiveness is below a specified threshold, for example, when the effectiveness is less than zero, server computer 120 may activate the changes to the vehicles. In some examples, the effectiveness threshold need not be zero and may be a positive or negative value close to zero, for example, 0.001 or -0.001. Activating changes to the corresponding vehicles may include reverting the software update to a previous version and / or disabling components and / or features of the vehicle. For example, in the battery drain example, area lighting may be disabled. In this or other examples, cameras, lights, etc. may be turned off, and features may be disabled or reactivated.

[0052] At block 214 , if the software update has a positive effect, the server computer 120 may continue updating the vehicles. In some examples, there may be a small group or subset of vehicles (e.g., vehicles with or without certain features) that experience no or negative effects from the software update. In these cases, software updates may be discontinued for the subset of vehicles.

[0053] refer to Figure 3 The causal analysis block 206 may include a first stage 302 and a second stage 304. The first stage 302 calculates the CATE of the designed software update for an outcome related to vehicle performance. The second stage 304 calculates the average treatment effect (ATE) of the feature group on the CATE values calculated from the previous first stage 302. The CATE is a quantity representing the individualized causal effect and is defined as the difference between the expected outcome Y 308 for the treatment group (T=1) 306 and the control group (T=0) conditional on the covariate X 310: CATE(x)=E[Y(T=1)-Y(T=0)|Y(T=1)-Y(T=0)X=x]. The ATE is defined as the average difference in outcome Y 314 between the control group (T=1) 312 and the control group (T=0): ATE=E[Y(T=1)-Y(T=0)].

[0054] refer to Figure 4 and Figure 5 In the example, the first stage 302 calculates a CATE, where the process T 306 (i.e., the cause) is a software update and the result Y 308 (i.e., the effect) is a specified aspect of vehicle performance (e.g., warranty claim, DTC, or performance). The first stage calculates the CATE for the designed software update for outcomes related to vehicle performance. In other words, the first stage indicates how the remedial action (i.e., the software update) changes the vehicle performance aspect of interest. For example, the first stage may determine whether the software update increases or decreases the occurrence of a particular diagnostic trouble code (DTC). In this example, covariates X 310 may include vehicle part content, trip summary / weather information, and / or other software updates.

[0055] Figure 5 500 is a histogram of the distribution of CATE values in the fleet relative to the frequency of a particular DTC (i.e., battery drain). Mean 502 is negative, indicating that the process (i.e., software update) caused the value of the result (i.e., DTC) to decrease from 1 (problem) to 0 (no problem). In other words, the software update reduced the incidence of DTCs associated with the aspect of vehicle performance of interest (e.g., battery drain). Therefore, when the mean of the CATE value is negative, effectiveness is positive.

[0056] based on Figure 5The t-test results 504 shown in FIGURE 5 indicate that the treatment was effective for this fleet of vehicles. The t-test analysis can help prevent misattribution of a reduction in DTC frequency due to a covariate as due to the treatment. The t-test is an inferential statistic that can be used to determine whether there is a significant or meaningful difference between the means of two groups and how they are related.

[0057] refer to Figure 6 and Figure 7 , the second level 304 calculates the ATE, where the treatment T 312 (i.e., the cause) is the feature being investigated, and the outcome Y 314 (i.e., the effect) is the CATE from the first level 302. The second level calculates the average treatment effect (ATE) of the feature group on the CATE calculated from the previous first level step. The second level 304 indicates how different groups of features affect the change in CATE. This can be used to indicate which vehicle features are associated with changes in vehicle performance caused by the software update. Thus, the second level 304 calculates the causal effect of each feature on the outcome.

[0058] In this case, treatment T 312 is the feature of interest (e.g., area lighting), and covariate X 316 is the other feature, while outcome Y 314 now becomes the treatment effect (CATE) calculated in the first stage 302. The calculated causal effect of each feature on the outcome can be used to rank groups of features to identify which features have the greatest effect on the outcome for further investigation.

[0059] In addition to understanding whether the treatment is effective at the fleet level, it is also possible to investigate the impact of different features on the treatment's CATE value. In this example, the top representative feature group includes area lighting. Figure 7 Histograms 702 and 704 in FIG. 1 illustrate the impact of lighting zone features on the processing effect. Histogram 702 shows the distribution of ATE values relative to the frequency of a specific DTC (i.e., battery drain) in vehicles with the zone lighting feature. Histogram 704 shows the distribution of ATE values relative to the frequency of a specific DTC (i.e., battery drain) in vehicles without the zone lighting feature. Comparing the mean values 706 and 708 of histogram 702 with zone lighting and histogram 704 without zone lighting, respectively, it can be seen that the zone lighting feature causes the mean ATE to be negative (i.e., lowers the incidence of DTCs), while the lighting feature of vehicles without zone lighting is less affected by the software update.

[0060] like Figure 8As shown, the causal effect of multiple software updates over time is evaluated by summing the effectiveness of a target software update and one or more subsequent software updates. In this exemplary graph 800, the performance aspect is the percentage of key cycles that experience high battery drain (e.g., SOC drop > 8%). Battery drain issues in these vehicles can be addressed by, for example, software updates to change the logic of battery usage. For example, a software update can be designed to update a specified target ECU module (Target ECU T * During the time period of the process effectiveness evaluation, other ECU modules may also be updated, and those updates may also affect the battery consumption issue. Each vehicle will update the software in different time frames (the time period may be weeks or months or longer), which naturally forms data with vehicles having different software versions in a given time period. At a given time T, the process effect can be expressed as

[0061] Formula 1.

[0062]

[0063] Where N is the number of vehicles [1,2,3…n,…N]; M is the number of software updates [A,B,C…,m,…M]; indicates that a treatment was applied, and Indicates that no processing was applied; E * Is a software update (applied to the target ECU T * The expected result below δ m Indicates whether the vehicle has undergone the mth software update (1 or 0).

[0064] The treatment effect is the difference between the expected value with the treatment applied (solid line 804) and the expected value without the treatment applied (dashed line 802), and is expressed as the causal difference in Equation 2.

[0065]

[0066] If the causal difference is less than zero, then the update causes the battery consumption percentage to decrease, indicating that the treatment is effective; if the causal difference is greater than zero, then it indicates an adverse effect; if the causal difference is equal to zero, then there is no effect. * The causal difference 806 between the expected value with the treatment applied (solid line 804) and the expected value without the treatment applied (dashed line 802) is positive, indicating that the effectiveness is negative, thus having an adverse effect. However, at time TA2 * Thereafter, the causal difference 808 between the expected value with the treatment applied (solid line 804) and the expected value without the treatment applied (dashed line 802) is negative, indicating positive effectiveness.

[0067] Computing devices such as those described herein typically each include commands that can be executed by one or more computing devices such as those identified above and used to implement the blocks or steps of the processes described above. For example, the process blocks described above can be embodied as computer-executable commands.

[0068] The computer executable instructions may be compiled or interpreted by a computer program created using a variety of programming languages and / or technologies, including but not limited to the following, either singly or in combination: Java TM , C, C++, Python, Julia, SCALA, Visual Basic, Java Script, Perl, HTML, etc. Typically, a processor (e.g., a microprocessor) receives commands, for example, from a memory, a computer-readable medium, etc., and executes these commands, thereby performing one or more processes including one or more of the processes described herein. A variety of computer-readable media can be used to store such commands and other data in files and to transmit such commands and other data. A file in a computing device is typically a collection of data stored on a computer-readable medium such as a storage medium, random access memory, or the like.

[0069] Computer-readable media (also known as processor-readable media) include any non-transitory (i.e., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media can take many forms, including but not limited to non-volatile media and volatile media. Instructions can be transmitted via one or more transmission media, including optical fiber, wires, wireless communications, including internal components that make up a system bus coupled to a processor of a computer. Common forms of computer-readable media include, for example, RAM, PROM, EPROM, FLASH-EEPROM, any other memory chip or cartridge, or any other medium from which a computer can read.

[0070] Unless otherwise expressly indicated herein, all terms used in the claims are intended to be given their ordinary and customary meanings as understood by those skilled in the art. Specifically, unless a claim recites an explicit limitation to the contrary, use of singular articles such as "a," "an," "the," and "said" should be construed to recite one or more of the indicated elements.

[0071] The adverb “approximately” modifying a value or result means that the shape, structure, measurement, value, determination, calculation, etc. may deviate from the exactly described geometry, distance, measurement, value, determination, calculation, etc. due to imperfections in materials, machining, manufacturing, sensor measurement, calculation, processing time, communication time, etc.

[0072] In the accompanying drawings, the same candidate marks indicate the same elements. In addition, some or all of these elements can be changed. With respect to the media, processes, systems, methods, etc. described herein, it should be understood that although the steps or frames of such processes, etc. have been described as occurring according to a sequence in a specific order, such processes can be put into practice by performing the described steps in an order other than the order described herein. It should be understood that certain steps can be performed simultaneously, other steps can be added, or certain steps described herein can be omitted. In other words, the description of the process herein is provided for the purpose of illustrating certain embodiments and should in no way be interpreted as limiting the claimed invention. Any use of "based on" and "in response to" herein (including with reference to the media, processes, systems, methods, etc. described herein) indicates a causal relationship, not just a temporal relationship.

[0073] According to the present invention, a system is provided, comprising: a computer including a processor and a memory, the memory including instructions executable by the processor to: receive data corresponding to a plurality of vehicles regarding specified aspects of vehicle performance; determine, based on the data, the effectiveness of a software update for the specified aspects of vehicle performance; and actuate a change in at least one of the plurality of vehicles when the determined effectiveness is below a specified threshold.

[0074] According to an embodiment, the instructions for determining the effectiveness of the software update include instructions for applying a causal model to the received data.

[0075] According to an embodiment, the instructions for applying the causal model include instructions for determining a conditional average treatment effect of the software update based on the data.

[0076] According to an embodiment, the instructions for determining a conditional average treatment effect of a software update based on the data include instructions for determining using an S-learner, an X-learner, or a causal tree algorithm.

[0077] According to an embodiment, the instructions for applying the causal model include instructions for determining an average treatment effect of a vehicle feature based on the determined conditional average treatment effect of the software update.

[0078] According to an embodiment, the instructions for applying the causal model include instructions for summing the effectiveness of the software update and one or more subsequent software updates.

[0079] According to an embodiment, the instructions for applying the causal model include instructions for grouping the plurality of vehicles by a characteristic.

[0080] According to an embodiment, the instructions for actuating a change in at least one of the plurality of vehicles include instructions for reverting a software update to a previous version and / or disabling a component of at least one of the plurality of vehicles.

[0081] According to an embodiment, the received data includes warranty claim information and / or vehicle diagnostic trouble codes.

[0082] According to an embodiment, the instructions for actuating a change in at least one of the plurality of vehicles include instructions for reverting a software update to a previous version and / or disabling components of a group of vehicles of the plurality of vehicles having a specified characteristic.

[0083] According to the present invention, a method for actuating changes in vehicles includes: receiving data corresponding to multiple vehicles regarding specified aspects of vehicle performance; determining the effectiveness of a software update for the specified aspects of vehicle performance based on the data; and actuating changes in at least one of the multiple vehicles when the determined effectiveness is below a specified threshold.

[0084] In one aspect of the invention, determining the effectiveness of a software update includes applying a causal model to the received data.

[0085] In one aspect of the invention, applying the causal model includes determining a conditional average treatment effect of the software update based on the data.

[0086] In one aspect of the invention, determining the conditional average treatment effect of the software update based on the data includes using an S-learner, an X-learner, or a causal tree algorithm.

[0087] In one aspect of the invention, applying the causal model includes determining an average treatment effect of the vehicle feature based on the determined conditional average treatment effect of the software update.

[0088] In one aspect of the invention, applying the causal model includes summing the effectiveness of the software update and one or more subsequent software updates.

[0089] In one aspect of the invention, applying the causal model includes grouping the plurality of vehicles by a characteristic.

[0090] In one aspect of the invention, actuating the change to at least one of the plurality of vehicles includes reverting a software update to a previous version and / or disabling a component of at least one of the plurality of vehicles.

[0091] In one aspect of the invention, the received data includes warranty claim information and / or vehicle diagnostic trouble codes.

[0092] In one aspect of the invention, actuating the change to at least one of the plurality of vehicles includes reverting a software update to a previous version and / or disabling components of a group of vehicles of the plurality of vehicles having designated characteristics.

Claims

1. A system comprising: A computer comprising a processor and a memory, the memory comprising instructions executable by the processor to: receiving data corresponding to a plurality of vehicles regarding specified aspects of vehicle performance; determining, based on the data, the effectiveness of a software update for the specified aspect of vehicle performance; as well as When the determined effectiveness is below a specified threshold, a change in at least one of the plurality of vehicles is actuated. 2 . The system of claim 1 , wherein the instructions for determining the effectiveness of the software update include instructions for applying a causal model to received data. 3 . The system of claim 2 , wherein the instructions for applying the causal model include instructions for determining a conditional average treatment effect of the software update based on the data.

4. The system of claim 3, wherein the instructions for determining a conditional average treatment effect of a software update based on the data include instructions for determining using an S-learner, an X-learner, or a causal tree algorithm. 5 . The system of claim 3 , wherein the instructions for applying the causal model include instructions for determining an average treatment effect of a vehicle feature based on the determined conditional average treatment effect of the software update.

6. The system of claim 2, wherein the instructions for applying the causal model include instructions for summing the effectiveness of the software update and one or more subsequent software updates.

7. The system of claim 1, wherein the received data includes warranty claim information and / or vehicle diagnostic trouble codes.

8. The system of any one of claims 1 to 7, wherein the instructions for actuating a change to the at least one of the plurality of vehicles include instructions for reverting a software update to a previous version and / or disabling a component of the at least one of the plurality of vehicles.

9. A method for actuating a change in a vehicle, comprising: receiving data corresponding to a plurality of vehicles regarding specified aspects of vehicle performance; determining, based on the data, the effectiveness of a software update for the specified aspect of vehicle performance; as well as When the determined effectiveness is below a specified threshold, a change in at least one of the plurality of vehicles is actuated.

10. The method of claim 9, wherein determining the effectiveness of the software update comprises applying a causal model to the received data.

11. The method of claim 10, wherein applying the causal model comprises determining a conditional average treatment effect of the software update based on the data.

12. The method of claim 11, wherein determining a conditional average treatment effect of the software update based on the data comprises using an S-learner, an X-learner, or a causal tree algorithm. 13 . The method of claim 11 , wherein applying the causal model comprises determining an average treatment effect of a vehicle feature based on the determined conditional average treatment effect of the software update. The method of claim 10 , wherein applying the causal model comprises grouping the plurality of vehicles by characteristics.

15. The method of any one of claims 10 to 14, wherein actuating the change to the at least one of the plurality of vehicles comprises reverting the software update to a previous version and / or disabling components of a group of vehicles of the plurality of vehicles having specified characteristics.

Images