Method and system for sending an update model

Abstract

A method of sending an update model, a server comprising a processor operable to carry out the method, and a device operable to receive an update model, wherein the update model is an Artificial Intelligence Model, and / or a Machine Learning model. The method comprising sending, by a server, a command to update software on a device. The method further comprising receiving, by the device, an indication of current software on the device. The method further comprising generating, by the server, an update model that comprises a set of data wherein the set of data is part of the update software and is not part of the current software, and wherein the update model is smaller in size than the update software or the current software. The method further comprising sending, by the server, the update model to the device.

Description

[0001] Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0002] 1

[0003] METHOD AND SYSTEM FOR SENDING AN UPDATE MODEL

[0004] The present invention relates to a method of sending an update model and a server operable to send the update model.

[0005] BACKGROUND OF THE INVENTION

[0006] Artificial Intelligence (Al) models, such as language and large language models, are continuously evolving. With this evolution, the size and complexity of Al models is also increasing. This places a large burden on the various devices that store and employ Al models because larger storage means and higher processing capabilities are required by such devices. Due to the evolving nature and rapid increase in research and development of Al, it is also a necessity to update devices from legacy Al models to current Al models, thereby ensuring that those devices benefit from the most recent advancements in Al model technology.

[0007] Present means of updating a device to include the most recent Al model require the full Al model to be transferred from a network or a server to the device. The capabilities of the device and the network are of utmost importance given the large size of Al models and that an update may occur as frequently as once per day and that it is expected that update frequency will increase in the future, the capabilities of the device and network. With present means, the device has to store the full current Al model and download the full updated Al model. Furthermore, current servers have to transfer the entire updated Al model to the device and, possibly, multiple devices that require updating, leading to inefficient duplicate data transmission because it is usually not the entire Al model that requires updating, but only a portion of the Al model. Further still, uploading and downloading a large Al model takes a significant amount of time, thereby reducing the ability of providing real-time adaptation of updates for devices employing Al models (such as for autonomous vehicles which may require an immediate safety update). Without efficient load balancing, existing systems may experience increased overhead and latency during the model update process.

[0008] It is an aim of this invention to address the above-mentioned issues, for example, by providing more efficient data transmission and by leveraging unused utilities in the network (for example, edge computing capabilities). Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0009] 2

[0010] SUMMARY

[0011] To achieve the above objectives, the invention sets out a method, a server, a device, and a system of sending an update model as in the claims below.

[0012] In a preferred embodiment, a method of sending an update model is provided. The update model is an Artificial Intelligence (Al) and / or Machine Learning (ML) model. The method includes sending, by a server, a command to update software on one or more devices. The method further includes receiving, by the one or more devices, an indication of current software on the one or more devices. The method further includes generating, by the server, an update model that comprises a set of data wherein the set of data is part of the update software and is not part of the current software, and wherein the update model is smaller in size than the update software or the current software. The method further includes sending, by the server, the update model to the one or more devices. By generating an update model that comprises less data than an entire update software / program, less data has to be transmitted to the one or more devices and processed by the one or more devices. Advantageously, bandwidth usage is reduced during sending of the update model. Furthermore, transmission times are reduced, due to the reduced amount of data transmitted. Further still, the computational requirements of the one or more devices is reduced due to the reduced amount of data that has to be processed. Advantageously, updates of artificial intelligence models and machine learning algorithms can be split into sets of legacy data and sets of newer data related to the update material. Thus, frequent updates of Al models and / or ML algorithms can be performed with decreased computational requirements on the one or more devices and decreased bandwidth usage on the network.

[0013] In an embodiment, generating, by the server, the update model includes pruning, quantization, knowledge distillation, weight clustering, tensor decomposition, sparsity induction, low-rank factorization, and / or hybrid quantization. Advantageously, Al model sizes can be reduced, facilitating faster transmission and requiring less storage on the one or more devices.

[0014] In an embodiment, generating, by the server, the update model includes generating the update model using generative model compression. Advantageously, further reductions in model sizes can be achieved, surpassing the capabilities of traditional compression techniques.

[0015] In an embodiment, the update model includes is generated using a federated learning system. Advantageously, this enables collaborative model training across the one or more devices without centralising data, thus preserving privacy and leveraging distributed data Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0016] 3 sources for more robust models. In automotive applications, this is particularly beneficial as it allows for continuous improvement of Al models used in vehicles (for example, for autonomous driving or predictive maintenance) while ensuring that sensitive data, such as driver behavior or vehicle sensor data, remains on the local device and is not exposed to potential security risks.

[0017] In an embodiment, sending, by the server, the update model to the one or more devices includes identifying a plurality of channels between the server and the one or more devices, and distributing packets of the update model across the plurality of channels. Advantageously, the update load across the network is distributed which prevents bottlenecks and ensures a smooth and efficient update process for all connected devices. This has the further benefit of leading to significant energy savings due to the reduced computational load.

[0018] In an embodiment sending, by the server, the update model to the one or more devices includes identifying a plurality of channels between the server and the one or more devices, sending an acknowledgement, ACK, signal from the server to the one or more devices on each of the plurality of channels, determining, based on each of the ACK signals a current bandwidth availability on each of the plurality of channels, selecting a set of the plurality of channels, wherein the set of the plurality of channels comprises at least one of the plurality of channels, and sending the update model on the set of the plurality of channels. Advantageously, the transmission strategy can be dynamically adjusted based on the current network conditions, optimising the use of available bandwidth and minimising transmission times.

[0019] In an embodiment, the method further includes determining a threshold bandwidth requirement based on the size of the update model and identifying the set of the plurality of channels based on the threshold bandwidth requirement. Advantageously, the transmission strategy can be further dynamically adjusted based on the size of the update model.

[0020] In an embodiment, the method further includes sending the update model to at least two of the one or more devices, and training the server with training data to predict bandwidth availability on each of the plurality of channels and to identify the set of the plurality of channels based on the predicted bandwidth availability on each of the plurality of channels. The training data comprises the current bandwidth of each of the plurality of channels, the bandwidth of each of the plurality of channels when the update model is being sent, the bandwidth of each of the plurality of channels after the update model has been sent, and the size of the update model for each of the at least two devices. The method further includes identifying the set of the plurality of channels for a future update model. Advantageously, the network conditions can Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0021] 4 be anticipated based on navigation data and available infrastructure, thus providing a more dynamic transmission strategy.

[0022] In an embodiment, the method further includes sending the update model on one channel of the set of the plurality of channels, or distributing the sending of the update model across the set of the plurality of channels. Sending the update model on one channel is advantageous in scenarios with sufficient bandwidth and reliability, the transmission process is simplified. Sending the update model across the set of the plurality of channels enhances robustness and reliability by allowing other channels to continue transmission if one fails, ensuring quick and seamless updating, optimizing bandwidth use, and reducing bottlenecks. This is especially valuable when one or more edge devices (for example vehicle(s)) operate in changing environments (for example, when they are moving) and the access to the network is changing dynamically.

[0023] In an embodiment, sending, by the server, the update model to the one or more devices includes sending the update model via a blockchain network. Advantageously, an additional layer of security and transparency for the distribution process is provided, ensuring the integrity of model updates through decentralized verification.

[0024] In an embodiment, the one or more devices include at least one vehicle, at least one edge device, or a combination of the above. Advantageously, updated models can be provided in a versatile manner to a plurality of different devices such as automotive vehicles, roadside devices, or a combination thereof.

[0025] In an embodiment, the one or more devices include at least one vehicle and at least one edge device, and the method further includes sending the update model to the at least one edge device, processing the update model on the at least one edge device, and commanding the at least one edge device to send the processed update model from the at least one edge device to the at least one vehicle. Advantageously, computational load can be distributed throughout the network according to where storage and / or processing capabilities are available. For example, if a server’s processing capabilities are overloaded the processing requirements can be outsourced to a vehicle and / or any other edge device if the vehicle and / or edge device’s processor have extra processing capabilities.

[0026] In an embodiment, the method further includes receiving one or more error messages from the one or more devices, the one or more error messages comprising faults, training an Al and / or ML algorithm of the server to identify the one or more error messages, generating a second update model, based on the received error messages, the second update model not Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0027] 5 including the faults, and sending, by the server, the second update model to the one or more devices. Advantageously, performance data from the one or more devices can be collected to facilitate continuous improvement of the updated Al models. This allows for the personalization of models to meet the specific needs of different devices and operational contexts, improving the overall effectiveness and user experience.

[0028] In an embodiment, the faults may include one or more corrupted data packets in the update model, incomplete transmission of the one or more data packets from the server to the one or more devices, an incompatibility of the update model with the current software, latency (and / or time delays) exceeding a predefined threshold, insufficient bandwidth between the server and the one or more devices, checksum mismatches, an unexpected reboot or crash of the one or more devices, a failure to install the update model on the one or more devices, one or more performance metrics being below a predefined threshold, or any combination thereof.

[0029] In an embodiment, the method further includes determining, by the server, whether free memory in the one or more devices is above a predetermined threshold. The method further includes storing the update model on the one or more devices if the free memory is above the predetermined threshold. The method further includes sending, by the server, a command to the one or more devices to send the update model to the other one or more devices.

[0030] In an embodiment, the method further includes determining, by the server, whether free processing capacity of the one or more devices is above a predetermined threshold. The method further includes sending, by the server, a command to the one or more devices to install the update model on the one or more devices if the free processing capacity is above the predetermined threshold. The method further includes sending, by the server, a command to the one or more devices to send the installed update model to the other one or more devices. Advantageously, immediate, on-device decision-making through real-time data analysis can be provided to enhance the responsiveness and functionality of the one or more devices.

[0031] In a preferred embodiment, a server for sending an update model is provided. The server includes a processor operable to carry out the features as described above. By generating an update model that comprises less data than an entire update software / program, less data has to be transmitted to the one or more devices and processed by the one or more devices. Advantageously, bandwidth usage is reduced during sending of the update model. Furthermore, transmission times are reduced, due to the reduced amount of data transmitted. Further still, the computational requirements of the one or more devices is reduced due to the reduced amount of data that has to be processed. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0032] 6

[0033] In a preferred embodiment a device for receiving and processing an update model is provided. The device includes memory, and a processor, the processor is operable to send an indication of current software stored on the memory and receive an update model as set out above.

[0034] In a preferred embodiment, a system for sending an update model is provided. The system includes the server and the at least one device as described above.

[0035] In an embodiment, the at least one device includes at least one vehicle and at least one edge device.

[0036] In an embodiment, the edge device includes any one of a vehicle, a smart infrastructure device, such as Smart City Infrastructure (e.g., a smart road sign, smart traffic lights, smart cat’s eyes, a smart street lamp, a smart buoy, a smart electricity pole; a smart road); Drones or Unmanned Aerial Vehicles (UAVs); robots and autonomous systems; mobile robots; network equipment; a smart appliance, a smart building, a user equipment (UE) device, a server, or any combination of the above.

[0037] BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The features, objects, and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings in which like reference numerals refer to similar elements.

[0039] Figure 1 shows an example of a cloud infrastructure including a server for sending an update, a network, and edge devices;

[0040] Figure 2 a further example of a cloud infrastructure including the network of figure 1 and examples of edge devices;

[0041] Figure 3 depicts a flow chart of a method of sending an update model according to the invention;

[0042] Figure 4 depicts a flow chart of further methods of sending an update model according to alternative embodiments of the invention;

[0043] Figure 5 depicts a flow chart of further methods of sending an update model according to alternative embodiments of the invention; and

[0044] Figure 6 depicts a flow chart of yet further methods of sending an update model according to alternative embodiments of the invention. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0045] 7

[0046] DETAILED DESCRIPTION

[0047] Figure 1 shows a system (for example, a cloud infrastructure) including a server 102 (for example, an over the air (OTA) server) for storing and sending data (such as update data), a network 110, and a group of edge devices 112 for storing and processing data. The group of edge devices 112 may include one edge device 114 or a plurality of edge devices 114a - 114c. Three edge devices 114a - 114c are shown in figure 1 to demonstrate example communication arrangements between multiple edge devices. This is merely an example number of edge devices and the invention may comprise one, two, three or any other number of edge devices 114n. The server 102 communicates with the group of edge devices 112 (e.g., one or more edge devices 114n) and the group of edge devices 112 communicate with the server 102 by means of a communication path. The communication path may include a wired connection between the one or more edge devices 114n and the server 102 (for example, an ethernet cable or any other suitable communication cable), a wireless connection by means of the network 110 (for example, using WiFi, Bluetooth, LTE, 4G, 5G, 6G or any other suitable wireless connection), or a combination of the two. In an embodiment, the group of edge devices 112 and the server 102 may communicate with one direct communication path. That direct communication path may include multiple wired or wireless connections (such as multiple physical cables and / or multiple bandwidths). In an embodiment, each edge device 114n of the group of edge devices 112 may separately communicate with the server 102. That direct communication path may include multiple wired or wireless connections (such as multiple physical cables and / or multiple bandwidths). In an embodiment, one or more edge devices 114n of the group of edge devices 112 may communicate separately with the server 102 (as described above) and the other edge devices 112 may communicate with the server 102 by means of one direct communication path.

[0048] The server 102 may include computing means, for example, a memory operable to store data and instructions to carry out operations and a processor operable to carry out the instructions. The data may include software for the one or more edge devices 114n. The software may include a full program or a partial program for one or more operations of the one or more edge devices 114n. The software / program may be stored on the server 102 in an installed state such that it can be run on the one or more edge devices 114n without requiring installation on the one or more edge devices 114n. The software / program may be stored on the server 102 in an uninstalled state and which is smaller in size than an installed state, thus reducing the storage requirement on the server 102. The software / program may be installed by Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0049] 8 the processor of the server 102 and subsequently transmitted to the one or more edge devices 114n, thus reducing the computational requirements of the one or more edge devices 114n. Alternatively, the software / program may be transmitted to the one or more edge devices 114n in the uninstalled state to reduce the bandwidth taken during transmission and to reduce the computational load (for example, processing requirements) on the server 102.

[0050] In some embodiments, the software and / or program may be a model such as an artificial intelligence (Al) model, a neural network (NN) model, a large language model (LLM), or a combination of the above. The software, program, and / or model may be an update to a previous software, program, and / or model.

[0051] The server may include a distribution & management unit 104, a model optimization unit 106, and a data processing & feedback unit 108. The distribution & management unit 104, the model optimization unit 106, and the data processing & feedback unit 108 may each be software units or hardware units. As a software unit, each of distribution & management unit 104, the model optimization unit 106, and the data processing & feedback unit 108 may use a dedicated section of the computing means of the server 102 (for example, each unit 104, 106 and 108 may have a dedicated section of the memory of the server 102 and a dedicated section of the processor of the server 102). As a hardware unit, each of the distribution & management unit 104, the model optimization unit 106, and the data processing & feedback unit 108 may have a memory operable to store instructions and data and a processor operable to carry out instructions.

[0052] The distribution & management unit 104 is a central hub within the server 102 that manages how and when data (for example, program(s), software, model(s), and / or update(s)) are sent from the server 102 to the one or more edge devices 114n. The distribution & management unit 104 can interact directly with each of the one or more edge devices 114n and between different types of edge devices as described below with regard to figure 2. The distribution & management unit 104 sends data (for example, program(s), software, model(s), and / or update(s)) stored on the server 102 to the one or more edge devices 114n. If the data is update data (for example, an update to a program, update to software, update to an AI / NN / LL or other model), the data, the distribution & management unit 104 sends a command to the one or more devices 114n to update software on the one or more devices 114n. The command may be a message containing information related to the update data such as a version of the update data, what changes are within the update data and / or what additional information is present within the update data. The command may additionally include a request to the one or more Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0053] 9 edge devices to send an indication of the current data / software present on the one or more edge devices 114n. This indication may be a message containing information related to the current data / software such as a version of the current data / software, and / or what information is or is not present in the current data / software compared to the update data. The distribution & management unit 104 (and thus, the server 102) receives this indication of the current software / data present on the one or more edge devices. The model optimization unit 106 may determine, from the indication, which parts of the data / software present on the one or more edge devices 114n requires updating and generates an update model, as described below.

[0054] In an embodiment, the distribution & management unit 104 performs one or more load balancing operations between the server 102 and the one or more edge devices 114n. This involves real-time monitoring of channel characteristics such as bandwidth, latency, and current load, dynamically selecting optimal channels, splitting the update model into smaller packets, and transmitting these packets in parallel across the selected channels. The server continuously adjusts the load balancing strategy based on feedback from edge devices to ensure efficient and reliable transmission. This includes determining whether the communication path between the server 102 and the one or more edge devices 114n has a plurality of channels. This can be done, for example, by sending a test signal from the server to the one or more edge devices 114n. Alternatively or additionally this can be achieved by querying the network interface controllers by the server 102 on each node in the network graph and to receive information on available connections, by performing networking operations (such as Multipath TCP (MPTCP), Network Path Discovery / channel bonding (IEEE 802.3ad), Software-Defined Networking), or any combination thereof. In an embodiment, the number of channels may be pre-determined by the server 102 and / or by the one or more edge devices 114n. Where a single communication path between the server 102 and the group of edge devices 112 is present, a plurality of channels may be present within the single communication path. Where a plurality of communication paths is present between the server 102 and the group of edge devices 112 (for example, because one or more of the edge devices 114n have a direct and dedicated communication path to the server 102) each communication path may include one or more channels. The distribution & management unit 104 may distribute the data to be sent (for example, program(s), software, model(s), and / or update(s)) from the server 102 to the one or more edge devices 114n across the plurality of channels. This may include splitting the data into a plurality of packets and distributing the sending of the plurality of packets across the plurality of channels (for example, by sending one set of packets across one of the channels and another set of packets across a Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0055] 10 different channel). The packets can be distributed evenly or non-evenly. As a non-limiting example, when four channels are present a set of data can be split into 100 packets and the 100 packets may be distributed evenly across each of the four channels such that each channel carries 25 packets from the server 102 to the one or more edge devices 114n. In an alternative non-limiting example, the data can be split unevenly such that one of the channels carries 1 packet and the other channels carry a different number of packets. The data does not have to be split among all available channels. For example, out of four available channels, the distribution & management unit may determine to only send data across one, two or three available channels. Advantageously, the data load across the network 110 is distributed which prevents bottlenecks (reduces latency and prevents increased overhead on one channel) and ensures a smooth and efficient sending / update process for all connected devices 114n. This has the further benefit of leading to significant energy savings due to the reduced computational load required by one channel. Furthermore, the data can be transmitted dynamically depending on the bandwidth availability. Sending the update model on one channel is advantageous in scenarios with sufficient bandwidth and reliability, the transmission process is simplified. Sending the update model across the set of the plurality of channels enhances robustness and reliability by allowing other channels to continue transmission if one fails, ensuring quick and seamless updating, optimizing bandwidth use, and reducing bottlenecks. This is especially valuable when one or more edge devices (for example vehicle(s)) operate in changing environments (for example, when they are moving) and the access to the network is changing dynamically.

[0056] The one or more load balancing operations as described herein may include a weighted round robin (for example, where data packets are distributed across channels based on their bandwidth, latency, reliability, or a combination thereof). The one or more load balancing operations may include dynamically adjusting the load based on „feedback“ from the edge devices / network. In an example in the automotive context, initial channel identification may take place. Therein, a central server (for example server 102) identifies multiple communication channels (as described above) available for transmitting updates to a fleet of connected vehicles (for example edge devices 104n as described above). These channels might include 4G, 5G, Wi-Fi, and dedicated short-range communications (DSRC). Real-Time monitoring and selection may be applied after this. In an example, this may include the server 102 monitoring the real-time status of these channels, noting that the 5G channel currently has high bandwidth but moderate latency, while the Wi-Fi channel has moderate bandwidth and Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0057] 11 low latency (the terms 5G and Wi-Fi are only used as examples and are exchangeable with any other type of channel). The server 102 may then decide to use both channels, prioritizing the 5G channel for the bulk of the data due to its higher bandwidth and the Wi-Fi channel for critical, smaller packets due to its low latency. Packet distribution may take place subsequently, where the update model is divided into smaller packets. Larger packets are sent over the 5G channel in parallel, while smaller, latency-sensitive packets are sent over the Wi-Fi channel. This may be dynamically adjusted. For example, if the server 102 detects increased congestion on the 5G channel, it dynamically shifts more packets to the Wi-Fi channel or other available channels to maintain optimal performance. Vehicles (for example edge devices 104n) provide feedback on packet reception, allowing the server 102 to further refine its load balancing strategy.

[0058] The distribution & management unit 104 may distribute the data to be sent (sent (for example, program(s), software, model(s), and / or update(s)) from the server 102 to the one or more edge devices 114n across the plurality of channels based on a current location and one or more predicted future location(s) of the one or more edge devices 114n. The distribution & management unit 104 receives current location from the one or more edge devices 114n and may predict one or more future location(s) based on receive navigation information or from a route that is predicted by the one or more edge devices 114n or the server 102. For example, this may include determining a future location based on a navigation route entered into a navigation system of the one or more edge devices 114n (for example, a vehicle). This may include predicting that the one or more edge devices 114n (for example, a vehicle) will be at a certain location at a certain time based on previous patterns. For example, a vehicle may be predicted to be parked at a user’s home in the evening and / or the vehicle may be predicted to be parked at the user’s work place on certain days and times of the week.

[0059] The distribution & management unit 104 may trigger distribution of the data (for example, program(s), software, model(s), and / or update(s)) to be sent via a network channel (for example, a 4G, 5G, LTE, or similar type of network) when the one or more edge devices 114n (for example, a vehicle) is outside of reach of a high bandwidth (for example, Wi-Fi) channel. The distribution & management unit 104 may only trigger distribution of the data via the network channel when it is determined that the data (for example, an update) is urgent.

[0060] If it is predicted by the distribution & management unit 104 that the one or more edge devices 114n will be at a destination (for example, a user’ s home, a user’ s workplace, or similar) with a high bandwidth channel (for example, Wi-Fi), and the data (for example, the update) is Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0061] 12 determined not to be urgent, then the distribution & management unit 104 may prevent the data from being distributed over the network channel (for example, 4G, 5G, LTE, etc.) and instead trigger the distribution of the data when the one or more edge devices 114n are at the destination via the high bandwidth channel (Wi-Fi). Advantageously, the network channel data usage (which is generally more expensive to the user, the network provider, and / or the original equipment manufacturer (OEM)) can be preserved, while maintaining that the one or more edge devices 114n are up to date.

[0062] In an embodiment, the distribution & management unit 104 performs adaptive bandwidth utilization between the server 102 and the one or more edge devices 114n. The distribution & management unit 104 can identifying a plurality of channels between the server 102 and the one or more edge devices 114n as described above. Subsequently, the distribution & management unit 104 performs a bandwidth utilization test on each of the plurality of channels. This can be done, for example, by sending an acknowledgement (ACK) signal from the server 102 to the one or more edge devices 114n on each of the plurality of channels and determining, based on each of the ACK signals, the current bandwidth availability on each of the plurality of channels. The distribution & management unit 104 may select a set of the plurality of channels, wherein the set of the plurality of channels includes one or more channels of the plurality of channels. The ACK signal provides a real-time status update from the edge devices back to the server, indicating whether the data packets were successfully received. This feedback allows the distribution & management unit 104 to assess the current load and availability of each channel. If an ACK signal indicates a high bandwidth availability, the corresponding channel is included in the set of channels for data transmission. Conversely, if the ACK signal suggests low bandwidth availability, that channel is excluded from the set. The set of channels may be chosen depending on the current bandwidth availability on each of the plurality of channels. For example, the distribution & management unit 104 may determine that one or more of the plurality of channels have a current bandwidth availability that is above a threshold bandwidth requirement to send data to the one or more edge devices 114n. The distribution & management unit 104 may then select the one or more of the plurality of channels having the current bandwidth availability above the threshold bandwidth requirement (i.e. the set of the plurality of channels) to send the data on. Similarly, the distribution & management unit 104 may determine that one or more of the plurality of channels have a current bandwidth availability below a predetermined threshold and the unit 104 may decide not to send the data on that one or more plurality of channels. The threshold bandwidth requirement may be Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0063] 13 determined by the distribution & management unit 104 depending on the size of the data that is to be sent to the one or more edge devices 114n. For example, data with a relatively large size will require a larger amount of bandwidth. Similarly, data with a relatively small size will require a smaller amount of bandwidth. Accordingly, bandwidth allocation can be adapted depending on the data that is sent. Other methods, such as sending periodic pings or utilizing network management protocols (e.g., SNMP), can also be employed for monitoring and adjusting bandwidth allocation. Advantageously, the transmission strategy can be dynamically adjusted based on the current network conditions, optimising the use of available bandwidth and minimising transmission times.

[0064] The distribution & management unit 104 may dynamically adapt bandwidth utilization of the network 110 based on current network conditions. Furthermore, the unit 104 may collect information (such as bandwidth utilization and bandwidth availability of the plurality of channels) from the current and past network conditions to anticipate future network conditions and to dynamically adapt future bandwidth utilization of the network 110. Advantageously, the unit 104 optimises the use of available bandwidth and minimises transmission times. The features described above relating to the distribution & management unit 104 facilitate a collaborative ecosystem where devices (such as edge devices 114n) can share computational tasks and data insights, enhancing overall system efficiency and effectiveness. Advantageously, quick and efficient deployment of the data to the one or more edge devices 114n can be achieved. Furthermore, the computational load on edge devices can be reduced and data transmission can be optimised for efficiency, leading to significant energy savings.

[0065] The distribution & management unit 104 may perform load balancing, as described above, in combination with adaptive bandwidth utilization as also described above.

[0066] The model optimization unit 106 performs compression of data sent from the server 102 to the one or more edge devices 114n. In particular, the model optimization unit 106 receives (from the distribution & management unit 104) the indication of the current data / software on the one or more edge devices 114n and generates an update model to be sent from the server 102 to the one or more edge devices 114n. The purpose of generating an update model is to generate data (for example, program(s), software, AI / NN / LL model(s), update(s)) that is smaller in size than the data that is stored on the server 102 and that is intended to be stored / installed / operated on the one or more edge devices 114n. For example, if outdated software is installed on the one or more edge devices and it is intended to update the outdated software on the one or more edge devices, the model optimization unit 106 can determine which Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0067] 14 the differences between the outdated software and the updated software. The model optimization unit 106 can then isolate those differences and generate data (i.e. an update model) that only comprises those differences in addition to any other functional code that is required to execute a successful update. This update model is then sent and distributed to the one or more edge devices 114n with the distribution & management unit 104 as described above. Accordingly, the update model comprises a set of data (the updated component s)) that is part of the updated software and that is not part of the outdated software (i.e. the current software) on the one or more edge devices 114n. Because the update model only comprises a set of data of the updated software and not the entire updated software, the update model is smaller in size than the update software and is also smaller in size than the current / outdated software on the one or more edge devices. By generating an update model that comprises less data than an entire update software / program, less data has to be transmitted to the one or more devices and processed by the one or more devices. Advantageously, bandwidth usage is reduced during sending of the update model. Furthermore, transmission times are reduced, due to the reduced amount of data transmitted. Further still, the computational requirements of the one or more devices is reduced due to the reduced amount of data that has to be processed. This is much more efficient than traditional update mechanisms which often involve sending the entire model, program, software even if only a small portion has changed, which is inefficient in terms of network usage and update speed.

[0068] In an embodiment, the update model is an Artificial Intelligence (Al) Model, a Machine Learning (ML) Model, and / or a large language model (LLM). Advantageously, updates of artificial intelligence models and machine learning algorithms can be split into sets of legacy data and sets of newer data related to the update material. Thus, frequent updates of Al models and / or ML algorithms can be performed with decreased computational requirements on the one or more devices and decreased bandwidth usage on the network.

[0069] In an embodiment, the model optimization unit 106 generates the update model by applying techniques such as pruning, quantization, knowledge distillation, weight clustering, tensor decomposition, sparsity induction, low-rank factorization, hybrid quantization, and others. Pruning involves removing unnecessary weights and nodes from the model, effectively reducing its size without significantly impacting performance. Quantization reduces the precision of the model's weights, converting them from high-precision (e.g., 32-bit floating point) to lower precision (e.g., 8-bit integer), which decreases model size and computational requirements. Knowledge distillation transfers knowledge from a larger, more complex model Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0070] 15

[0071] (teacher) to a smaller, simpler model (student), retaining much of the performance with significantly fewer parameters. Weight clustering groups the model's weights into clusters and then assigns each weight to the nearest cluster centroid, which reduces the number of unique weights and can significantly compress the model. Tensor decomposition decomposes large tensors (multi-dimensional arrays) in the model into smaller, more manageable components, thus reducing the overall computational complexity and memory footprint of the model. Sparsity induction introduces sparsity into the model by zeroing out certain weights, reducing the number of non-zero parameters and leading to significant compression. Low-rank factorization decomposes weight matrices into products of lower-rank matrices, reducing the number of parameters while maintaining performance. Hybrid quantization combines different levels of quantization precision within the same model to balance performance and compression. Advantageously, Al model sizes can be reduced, facilitating faster transmission and requiring less storage on the one or more devices. In the context of automotive applications, this is particularly important as vehicles are becoming increasingly reliant on Al for functions such as autonomous driving, predictive maintenance, and advanced driver-assistance systems. Efficient model updates are crucial for maintaining the safety, performance, and reliability of these systems. In addition to this, frequent updates and customization of Al models can be accommodated for by allowing for quick and efficient deployment of updated models. By creating update models in reduced size as described above, updates to large model sizes (such as Al models and especially deep learning models) can be distributed over networks efficiently without the need of distributing the entire model. This is exceptionally useful where bandwidth is limited or costly. Furthermore, some edge devices (such as edge devices 114n) have constrained computational power and storage capacity. Accordingly, it may not be possible for those devices 114n to receive large and full updates (which may include an entire new updated program, software, model, etc.) because the storage within those devices 114n may be limited. Additionally or alternatively, such devices 114n may not have the processing power to install large and full updates. By providing an update model as described above that is smaller in size, computational load can be reduced, thus being able to provide updates quickly and efficiently. Moreover, because the amount of data that needs to be sent over the network, multiple updates can be provided to the one or more edge devices 114n in a shorter period of time, which is becoming increasingly more popular with the increased complexity of edge devices (for example, autonomous vehicles which may receive updates multiple times per day). Reducing Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0072] 16 the computational load on the one or more edge devices 114n and optimizing data transmission for efficiency also leads to significant energy savings.

[0073] In an embodiment, the model optimization unit 106 generates the update model using generative model compression. This technique involves training a generative model, such as a variational autoencoder (VAE) or a generative adversarial network (GAN), on the original Al model to learn a more compact representation. The generative model can then be used to reconstruct the Al model with significantly fewer parameters, thus reducing its size. The process includes:

[0074] (1) Training the generative model on the original Al model to capture its essential features and patterns.

[0075] (2) Encoding the original model into a latent space representation using the trained generative model.

[0076] (3) Decoding the latent representation back into a smaller, yet functionally equivalent Al model.

[0077] Advantageously, further reductions in model sizes can be achieved, surpassing the capabilities of traditional compression techniques. This method not only reduces storage requirements but also speeds up the transmission and deployment of updates to edge devices, which is particularly beneficial in automotive applications where frequent and efficient updates are necessary for maintaining vehicle safety and performance.

[0078] In an embodiment, the model optimization unit 106 generates the update model using a federated learning system. Federated learning involves training machine learning models across multiple edge devices 114n (for example, vehicles, smart infrastructure) without the need to centralize data. The federated learning system can operate with the OTA server 102 communicating directly with the plurality of the edge devices 114n. The federated learning system can operate in a distributed environment with the OTA server 102 communicating with one or more of the plurality of edge devices 114n via one or more nodes and / or gateways. The federated learning system can operate as a hierarchical federated learning system. The process may include:

[0079] (1) Distributing the initial model to a plurality of edge devices 114n (all participating edge devices). The initial model may be an initial standardised model (for example an initial standardised update model), such as the update model as described above.

[0080] (2) Each of the plurality of participating edge device 114n locally trains the model (such as the initial standardised (update) model) using data from the respective participating edge device. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0081] 17

[0082] (3) The locally trained models' updates (for example, gradients or weights) are sent back to a central server (for example OTA server 102).

[0083] (4) The central server 102 aggregates these updates (i.e. the locally trained models updates) to form a consensus model (i.e. an aggregated model or aggregated update model), which is then redistributed to the edge devices 114n for further training.

[0084] This iterative process continues until the model converges to a desired level of accuracy. This may include one redistribution or any number of redistributions to the edge devices 114n for further training. At the desired level of accuracy, the consensus model corresponds to a global model that may be distributed to the edge devices 114n. Advantageously, this enables collaborative model training across the one or more devices without centralising data, thus preserving privacy and leveraging distributed data sources for more robust models. By utilizing federated learning for collaborative model training, the system enables the one or more edge devices 114n to benefit from shared improvements without centralizing data, thereby preserving privacy and complying with data protection regulations. In automotive applications, this is particularly beneficial as it allows for continuous improvement of Al models used in vehicles (for example, for autonomous driving or predictive maintenance) while ensuring that sensitive data, such as driver behavior or vehicle sensor data, remains on the local device and is not exposed to potential security risks.

[0085] By providing a federated learning system with a hierarchical federated learning system, the plurality of edge devices 114n (for example, a fleet of vehicles, internet of thing devices (such as smart road infrastructure), or a combination thereof) provide a communication overhead for the amount of data transmitted (for example, model parameters, gradients, update models, global models, etc.). This is particularly useful when large quantities of clients (for example, edge devices 114n) participate in federated learning, where there is a risk of far greater computational overhead (i.e. computational efficiency is compromised). This is further enhanced with the use of pruning, quantization, knowledge distillation (as described above), and federated averaging.

[0086] Pruning (for example, train-time pruning or post-train pruning) may remove the least impactful parameters from a model. For example, an autonomous driving model may be trained on data from various sensors (such as multiple cameras, LIDAR, HD Map, etc.) across different vehicles. Not all parameters in the model are equally important for every vehicle. Pruning removes the least impactful parameters from a model, so each vehicle only transmits critical data, reducing communication load while maintaining most of the model’s performance. Train- Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0087] 18 time pruning allows for continuous optimization of Al models while they are being trained. This can reduce the model size and complexity, which is crucial for deploying efficient and lightweight Al models on vehicles with limited computational resources. For instance, in autonomous driving systems, train-time pruning can help in maintaining high performance while ensuring that the models remain compact and efficient enough to be run in real-time on in-vehicle processors. Post-train pruning can be applied after the training has been completed. Post-train pruning can be particularly useful for updating Al models in (Software Defined Vehicles) SDVs without the need for retraining. This allows further reduction in model size, making it ideal for OTA updates where bandwidth and storage space are limited. The flexibility of both train-time pruning and / or post-train pruning allows developers to choose the most appropriate pruning method based on the specific requirements of the vehicle and the operational context. For example, train-time pruning can be used during the initial development and training phases (for a small fleet of vehicles that support better HW), while post-train pruning can be employed for subsequent updates and optimizations.

[0088] Quantization (for example, post training quantization or quantization aware training) provides an arrangement to lower the precision of collected data. For example, a model used for lane detection in autonomous vehicles can be trained using high-precision data (for example, 32 bit floating-point), however for updates transmitted during training, these parms can be quantized into lower precision, such as 8-bit. This can provide a significant (in this case 75%) reduction in the size of transmitted data, and the impact on performance might be negligible.

[0089] In knowledge distillation, each of the plurality of participating edge devices 114n (for example, each vehicle) may train a smaller, "distilled" model locally, which is later aggregated to form the global model. The global model is distributed back as the operating model and the new distilled model is chosen for training on each of the participating edge devices 114n (vehicles). The process may include:

[0090] (1) Local training: Each participating edge device 114n (vehicle) collects real-time data from its environment and trains a lightweight student model locally on this data. This model is typically smaller in size than a fully-fledged teacher model, making local computation feasible even on resource-constrained vehicles.

[0091] (2) Distilled knowledge transmission: After training, instead of sending the full raw data (which can be large) or the complete teacher model, the edge device 114n (vehicle) sends distilled knowledge (the trained student model's parameters or its smaller representation) back to the central server (for example, the OTA server 102). Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0092] 19

[0093] (3) Central aggregation: The central server aggregates student models (i.e., the learned parameters or gradients) from multiple edge devices 114n (vehicles), learning from the diverse environments and experiences of all participating vehicles. It can use these aggregated models to update the global model and then send a new global version to the plurality of participating edge devices (for example, a fleet of vehicles).

[0094] An edge device 114n (a vehicle) might collect a large amount of data during a drive (this may belO gigabyte (GB) or more), but after training and distillation, the student model to be sent back to the server is reduced to an order of a few megabytes. This greatly reduces the bandwidth requirements. It also supports privacy, as no real data is send back to the server 102. Edge devices 114n (vehicles) encounter different environments and situations (for example, urban vs. rural driving), so they may train on different types of data. By sending results back to the central server 102, the federated learning framework allows the central model to capture these diverse data distributions, preventing the global model drift that can occur when each vehicle’s data varies significantly.

[0095] In an embodiment, instead of sending all gradients during model updates, sparse updates can be used, where only the gradients that have changed significantly during training are sent. This reduces the communication volume. In a non-limiting example, a vehicle that trains a model to detect traffic signs may benefit from this. After local training, only a small portion of the model’s parameters (such as those related to recognizing specific shapes or colors) might change significantly. Using sparse updates, only these significantly changed parameters are transmitted back to the server 102, reducing the amount of data by certain percentage compared to sending the entire gradient set.

[0096] In an embodiment, federated learning may include federated averaging (FedAvg) where clients (such as the plurality of participating edge devices 114n) may train locally for several iterations before sending an averaged model update to the server 102. In a non-limiting example, each vehicle of a plurality of vehicles can run local training on data gathered from its sensors (for example to detect, driver behavior, road conditions, etc.) for a set period (such as 10 training rounds). After completing these rounds the vehicle may communicate with the server 102 (or a node in between), sending a compressed version of the averaged model. This reduces communication frequency, effectively lowering overhead while maintaining acceptable global model performance. Furthermore, by combining federated averaging and knowledge distillation (as described above), global model drift can be reduced and avoided. Global model drift is a symptom found in legacy federated learning systems where locally available data point Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0097] 20 distribution (for example, data collected by each of the plurality of participating edge devices 114n) may not represent the overall data distribution because the data distribution varies widely between the clients (for example the participating edge devices 114n) leading to global model drift.

[0098] In an embodiment, the federated learning system may include network optimization. For example, an edge device 114n (for example, a vehicle) that is in a region with poor network connectivity may reduce its communication frequency, waiting until it has access to a faster or more reliable connection (such as in urban areas). Additionally, the edge device 114n (vehicle) can use dynamic compression, sending more compressed model updates when network bandwidth is low and higher-quality updates when bandwidth is available. The federated learning system may additionally adaptively select which updates to send (such as important updates for safety-critical models) while deferring less critical updates for later.

[0099] In a Hierarchical Federated Leaning system (for example, in a smart city), edge devices 114n such as traffic lights or roadside units (RSUs) can act as intermediaries between vehicles and the central server 102. Vehicles send updates to the nearest RSU, which aggregates these updates and forwards a single combined update to the central server using all above methods. This reduces the number of vehicles directly communicating with the central server, decreasing overall communication overhead.

[0100] In an embodiment, federated averaging may include averaging the data and also a reweighting stage. The re-weighting stage may take into account the amount of data each client holds and how representative that data is of the overall distribution. If an edge device 114n (vehicle) gathers more data or data that is more representative of the overall distribution (for example, a vehicle driving through both urban and rural environments), its update are given a larger weight during aggregation. In contrary, if a vehicle operates in a very specific or niche environment (for example, an off-road vehicle collecting limited, specialized data), its updates are given less weight to prevent it from skewing the global model. In a non-limiting example, the context of a fleet of vehicles where some of the vehicles operate primarily in urban areas with frequent stops, traffic lights, and pedestrians, while other vehicles drive long distances on highways with different driving patterns is considered below. Without reweighting, the global model might drift towards urban driving patterns if the number of urban vehicles is higher, despite the diversity in the data. By using weighted FedAvg, the central server 102 can assign higher weights to vehicles that collect a broader range of driving conditions (and not where we Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0101] 21 just have more data), ensuring that the global model reflects both highway and urban driving. Accordingly, global model drift may be reduced or prevented.

[0102] In an embodiment, the federated learning system may include clustered federated learning, to create multiple global models, each representing a cluster of clients with similar data distributions. Clustered federated learning prevents all vehicles to contribute to a single global model and, instead, groups clients into clusters based on similarities in their local data (for example, vehicles in urban areas form one cluster, and vehicles in rural areas form another, country (different rules), climate). Each cluster then trains its own model, with periodic synchronization between clusters to share generalized knowledge. Accordingly, global model drift may be reduced or prevented.

[0103] To further reduce or prevent global model drift, regularization techniques may be utilized. These are applied in local training to prevent overfitting to local data, thus helping to reduce the global model drift when the models are aggregated. For example, a vehicle driving in extreme conditions (such as in a mountainous area in snow or rain) may overfit to those specific conditions during training. Regularization techniques prevent the model from becoming too specialized in these extreme conditions, helping the global model remain useful across all driving environments. In some embodiments, the regularization techniques may include L2 regularization (ridge regression) that gives a penalty to large model weights, which helps prevent vehicles from overfitting to their local data. In some embodiments, the regularization techniques include regularizing each vehicle's local model updates based on the global model. Each vehicle can be encouraged to keep its model parameters close to those of the current global model, minimizing the extent to which it deviates from the global norm.

[0104] In an embodiment, the federated learning system includes a periodic synchronization of the global model with local models. Advantageously, global model drift is reduced or prevented. Each of the participating edge devices 114n (vehicles) can also perform local finetuning on top of the global model. Accordingly, after several rounds of training, the central server 102 sends the global model to each vehicle for local fine-tuning. This helps each vehicle’s model remain aligned with the global model while still being specialized for its specific driving conditions. In a non-limiting example, every few weeks, the global model might be pushed out to vehicles for fine-tuning. Each vehicle model might fine-tune the global model with its specific data, ensuring the model remains accurate for its unique conditions while preventing global drift. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0105] 22

[0106] The data from the respective participating edge devices 114n is locally collected data, reflecting the specific environments of those edge devices 114n (for example, those vehicles, their driving conditions, and their sensor configurations). The locally collected data may be partitioned. In an embodiment, partitioning the locally collected data may include task-driven partitioning where each vehicle focuses on training a model for tasks that are relevant to its environment, sensor setup, and hardware capabilities. This way, each vehicle contributes specialized knowledge to the global model without being overwhelmed by the need to handle all types of data or tasks. Accordingly, task-driven partitioning ensures that each vehicle trains a specialized model based on the tasks relevant to its context, contributing meaningful knowledge to the global model. Federated Averaging with task prioritization ensures that contributions are weighted based on the relevance of the tasks being handled, improving the global model’s performance in critical areas. Data augmentation can be used instead of synthetic data to balance the dataset without introducing the risks of overfitting to simulated conditions. So to give a bit more details - in task-driven partitioning, each vehicle.

[0107] In an embodiment, task-driven partitioning may include specialization based on environment. Therein, vehicles operating in different environments (urban, rural, highway) can specialize in tasks that are most relevant to their conditions. The global model benefits from this diversity because it combines specialized sub-models from various regions, making it more versatile.

[0108] In an embodiment, task-driven partitioning may include adaptation to sensor setup. Therein, each vehicle has different sensor configurations (for example, different LIDAR placements, camera angles). Task-driven partitioning allows each vehicle to train models that are specific to its sensor setup, ensuring that the trained model is optimized for the vehicle's capabilities. A vehicle with front-facing cameras focuses on training models for front-view object detection, while a vehicle with side-mounted cameras specializes in blind-spot detection. This ensures that each vehicle uses its hardware efficiently without being burdened with tasks irrelevant to its configuration.

[0109] In an embodiment, task-driven partitioning may include prioritizing critical tasks. Some vehicles may have resource constraints (e.g., limited computing power or memory). In such cases, task-driven partitioning allows the vehicle to prioritize critical tasks that align with its environment and hardware, reducing the computational load. A resource-constrained vehicle may prioritize simpler tasks like lane-keeping or speed control, while high-powered vehicles handle more complex tasks like 3D mapping or multi-sensor fusion. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0110] 23

[0111] In an embodiment, task-driven partitioning may include federated averaging with task prioritization. When aggregating models from different vehicles, Federated Averaging (FedAvg) can be adapted to prioritize the contributions of models that perform well on tasks relevant to the global model's needs. For example, if the global model needs to improve in pedestrian detection, the contributions from vehicles that specialize in this task (such as those in urban environments) are weighted more heavily during the aggregation process. In a further example, if there is a need for better snow condition handling, the global model will prioritize contributions from vehicles in snowy regions (such as in Finland).

[0112] In an embodiment, the system may comprise a blockchain network (not shown). The blockchain network may be a public blockchain network, a private blockchain network, a hybrid blockchain network, a consortium blockchain network or any other suitable blockchain network at an output side of the server 102. The blockchain network may be integrated in the generation of the update model using the federated learning system. The blockchain network may include using a blockchain at an input stage of the OTA server 102 (for example, for data received from each of the plurality of participating edge devices 114n) which may be located at the input side of the OTA server 102, at an output side of each of the plurality of participating edge devices 114n, or at the input side of the OTA server 102 and at an output side of each of the plurality of participating edge devices 114n. The blockchain network may additionally or alternatively include using a blockchain at an output stage of the OTA server 102 (for example, when distributing the initial standardised model or the consensus model).

[0113] At the input stage of the OTA server 102, the blockchain can be used for data collection (for example, data received from each of the plurality of participating edge devices 114n), for data pre-processing, or for both. Advantageously, this ensures data integrity, transparency, and accountability in the data collection process. The blockchain can be used to verify and record the provenance (from where it comes) of data before it is used in the training process as set out above. This is particularly beneficial in a federated learning context, where data does not leave the local device (for example, the one or more participating edge devices 114n), and ensuring the trustworthiness of local data is critical for the overall system.

[0114] Blockchain records may be created at an output of each of the plurality of participating edge devices 114n and may be used to log and verify the data that each of the plurality of participating edge devices 114n collects, ensuring that the data used for training is authentic and hasn't been tampered with. Each time a participating edge devices 114n collects a dataset, it is hashed and the hash is stored on the blockchain, making it easy to verify later if that data Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0115] 24 has been altered. A participating edge device 114n (for example, a vehicle) may collect sensor data from LIDAR, cameras, and GPS while driving in a city. Before training begins, the raw data is hashed, and a record of this hash is stored on the blockchain. This ensures that if the data is altered (for example, because the LIDAR data is modified or manipulated), it can be easily detected by comparing the current hash with the one recorded on the blockchain.

[0116] Blockchain can be used to track the source and ownership of data. This is particularly useful in systems where multiple stakeholders are involved (for example, different vehicle manufacturers or fleet operators). Each data contribution can be linked back to a specific edge device 114n (for example, vehicle or entity), ensuring transparency. This is particularly useful in a large autonomous vehicle fleet with multiple OEMs. The blockchain can keep a record of which vehicle collected specific data and when. This information is also valuable for regulatory compliance and / or auditing purposes. The blockchain can additionally or alternatively enable smart contracts to incentivize vehicles (for example, edge devices 114n) to collect and contribute high-quality data. The vehicles can earn tokens or credits for contributing diverse, valuable, or underrepresented data (for example, data from rare driving conditions like snow).

[0117] The blockchain network may be used during the model aggregation stage to ensure that the updates from the plurality of participating edge devices 114n (for example model parameters or gradients provided by the vehicles) are verified and trustworthy. This is important for securing the federated learning process. Blockchain can be used to track and verify the locally trained models’ updates (such as gradients and / or model weights) shared between vehicles and the OTA server 102 (or edge servers in a hierarchical federated learning system). Each update is hashed and recorded on the blockchain, preventing malicious clients from submitting corrupted updates. For example, each vehicle submits its locally trained model updates (for example, after processing data from its specific driving environment) to the OTA server 102. The blockchain records a hash of the update, ensuring that the model has not been tampered with before aggregation.

[0118] In an embodiment, the blockchain network may include consensus algorithms (such as Proof of Stake or Proof of Work), to ensure that only valid, verified model updates are aggregated into the global model. This adds an extra layer of security to the aggregation process, ensuring that no single edge device 114n or group of edge devices 114n can maliciously influence the global model. For example, a consensus mechanism can be applied to decide whether model updates from the plurality of participating edge devices 114n are legitimate before they are aggregated. Only updates that meet certain criteria (for example, verification Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0119] 25 from multiple clients or meeting the blockchain's smart contract conditions) are accepted into the global model.

[0120] In federated learning, there is a risk of model poisoning attacks, where malicious clients intentionally submit bad updates to corrupt the global model. Blockchain helps prevent this by making the model updates verifiable and transparent. Any suspicious or malicious update can be easily identified and discarded. For example, a participating edge device 114n (for example, a vehicle) may submit an update that attempts to skew the model toward certain false patterns (such as misclassifying stop signs). Since the update is recorded on the blockchain, it can be traced, reviewed, and rejected if it’s flagged as abnormal by consensus or validation algorithms.

[0121] At the output stage of the OTA server 102, the blockchain can be used to ensure the integrity and transparency of the global model distributed back to the plurality of participating edge devices 114n (for example, vehicles). For example, the distribution & management unit 104 may send the update data via the blockchain network at the output side of the server 102. After the models’ updates have been aggregated, blockchain can help ensure that the distributed global model is a correct, verified version and has not been tampered with during deployment. Subsequent to aggregation, the global model’s hash can be recorded on the blockchain. When one or more of the participating edge devices 114n (e.g., vehicles) receive the global model, they can verify that the model they received matches the hashed version stored on the blockchain. This ensures that the global model distributed is authentic and has not been altered. For example, a fleet of vehicles may receive the global model after an aggregation round. Before deploying the global model, each vehicle verifies the model hash against the one stored on the blockchain to ensure they have the correct version.

[0122] The blockchain can be used to track version control for the global model updates. Each new version of the global model is hashed and stored on the blockchain, allowing the plurality of participating edge devices 114n (vehicles) to know which version they are running and ensuring that outdated or compromised global models are not deployed. The blockchain can provide an auditable trail of global model performance across different participating edge devices 114n (vehicles). This is useful for verifying whether a global model performs well in certain environments or conditions, and for regulatory compliance in autonomous vehicle operations. For example, after a global model is deployed, vehicles record their performance metrics (such as accuracy in pedestrian detection, lane-keeping success rates) on the blockchain. This provides a transparent, auditable trail for monitoring the global model's effectiveness across various regions and conditions. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0123] 26

[0124] Accordingly, the use of a blockchain network provides an additional layer of security and transparency for the distribution process, ensuring the integrity of model updates through decentralized verification. By leveraging blockchain technology, the system addresses security concerns associated with tampering or interception of transmission, which are common in systems limited to basic security protocols. The blockchain network operates as a decentralized ledger, recording each transaction related to the update process. This includes the creation and distribution of Al model updates. Each update package distributed to edge devices includes a cryptographic hash, which serves as a unique digital fingerprint of the update. This hash is also recorded on the blockchain. When an edge device receives an update, it recalculates the hash of the received package and compares it with the hash stored on the blockchain. This verification step ensures that the update has not been altered during transmission, thereby protecting against data corruption and unauthorized modifications. This can be used together or instead of standard validation / authorization process. The additional benefit is the (built-in) transparency of the blockchain technology that enables tracking and auditing of updates. All transactions and updates recorded on the blockchain are visible to authorized participants. This transparency is crucial for maintaining the integrity of the system, especially in scenarios where compliance with security protocols is mandatory.

[0125] The blockchain network also enhances security by decentralizing the management of permissions. This ensures that only authorized entities can push updates or access certain functionalities, thereby reducing the risk associated with a single point of control. In the event of an attempt to intercept or tamper with the update process, the cryptographic security mechanisms of the blockchain provide a robust defence, allowing devices to detect and reject compromised updates.

[0126] The data processing & feedback unit 108 may collect performance data and feed this into a feedback loop operable to analyse the efficiency of data being sent between the server 102 and the one or more edge devices 114n. The performance data may include current bandwidth and past bandwidth availability of each of the plurality of channels, an amount of time taken to send data from the server 102 to each of the one or more edge devices 114n, error messages, data packet loss rates, device-specific processing times, latency variations, and energy consumption metrics, or a combination of the above. An error message may include any issue or fault relating to the sending of data from the server 102 to the one or more edge devices 114n. For example, this may include the data not having been received by one or multiple of the one or more edge devices 114n. Alternatively, or additionally, the error message Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0127] 27 may include one or more faults within the update model. The one or more faults may be corrupted data or data packets within the update model. Alternatively or additionally, the error message may include performance data not being within a predetermined threshold, such as bandwidth availability on one or more channels being below the measurement of the bandwidth availability on the one or more channels. The error message may include an amount of time taken to send data from the server 102 to each of the one or more edge devices 114n being above a predetermined threshold. The one or more faults may include one or more corrupted data packets in the update model, incomplete transmission of the one or more data packets from the server to the one or more devices, an incompatibility of the update model with the current software, latency (and / or time delays) exceeding a predefined threshold, insufficient bandwidth between the server and the one or more devices, checksum mismatches, an unexpected reboot or crash of the one or more devices, a failure to install the update model on the one or more devices, one or more performance metrics being below a predefined threshold, or any combination thereof. Additionally, the feedback loop might consider the effectiveness of compression techniques used, the success rate of update installations, the frequency of updates, and user or device feedback on the performance of the updated models. By incorporating these additional metrics, the feedback loop can provide a more comprehensive analysis to optimize future data transmissions and model updates.

[0128] The data processing & feedback unit 108 may receive one or more error messages from the one or more edge devices 114n. The one or more error message may include one or more faults, as described above. The data processing & feedback unit may receive performance data as described above. The one or more error messages may be fed into an Al and / or ML algorithm as training data. The performance data may also be fed into the Al and / or ML algorithm as training data. The Al and / or ML algorithm may be trained to identify the error messages as faults within the sending and / or generating process as described above and as carried out by the distribution & management unit 104 and / or the model optimization unit 106. The trained Al and / or ML algorithm can allocate faults and / or performance data and can allocate resources from the server 102 to improve the sending and / or generating. If one or more faults (such as corrupted data) are detected within the update model, the Al and / or ML algorithm, can detect this type of fault and instruct the model optimization unit 106 to generate a second update model based on the received error messages, the second update model being without (i.e. not including) the fault (for example, by replacing the corrupted data with non-corrupted data). The server 102 can then send the second update model to the one or more edge devices 114n. If Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0129] 28 performance data is measured incorrectly by the distribution & management unit 104, then the trained Al and / or ML algorithm of the data processing & feedback unit 108 can instruct the distribution & management unit 104 to alter its measurement process. For example, if the distribution & management unit 104 determines that a channel has a high bandwidth availability, but it is determined that the actual bandwidth availability on that channel is slightly lower than determined, future measurements taken by the distribution & management unit 104 can be calibrated to more accurately reflect the actual bandwidth availability on that channel. The data processing & feedback unit 108 may implement a feedback loop for continuous improvement of the update model (for example, by scanning for further error messages, receiving those further error messages relating to additional faults and training the Al and / or ML algorithm to generate additional update models based on the further received error messages to not include those faults). This ensures improved performance and reliability of the one or more devices 114n. Advantageously, performance data from the one or more devices 114n can be collected to facilitate continuous improvement of the updated Al models. This allows for the personalization of models to meet the specific needs of different devices and operational contexts, improving the overall effectiveness and user experience.

[0130] In an embodiment, the one or more devices 114n continuously monitor their performance and log relevant metrics (for example, error messages and faults as described above). The one or more devices 114n report the logged metrics and report them as error messages and faults to the server 102. The server 102 may also monitor transmission metrics (for example, bandwidth as described above) between the server 102 and the one or more devices 114n. The performance and relevant metrics may include transmission metrics (for example, bandwidth utilization, latency or time delays, data packet loss rates, acknowledgment (ACK) / not acknowledge (NACK) signals, etc.), device performance metrics (for example, processing times, memory usage, storage capacity, energy consumption, reboot logs, crash logs, etc.), and error data (for example types of errors (such as corrupted data packets, incomplete transmissions), frequency of errors, severity of faults, incompatibility issues, etc.). The server 102 may perform a pre-update analysis, where the server 102 uses historical performance data (including the data logged by the one or more devices 114n as set out above and sent to the server 102) to predict optimal transmission strategies. The optimal transmission strategies may include selecting appropriate channels and determining bandwidth requirements before sending the update model. The server 102 may perform real-time adjustments. This includes continuous monitoring of performance data and allows dynamic adjustment of the transmission Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0131] 29 process, such as rerouting data through different channels if a failure is detected during the update model transmission. The server 102 may perform post-update evaluation after the update model has been transmitted and installed. This includes analyzing the collected data (logs etc.) to identify any discrepancies or failures that occurred during the update to improve future update models. The server may perform Al and / or ML algorithm training (as also described above) where error data and performance metrics are used to train an AI / ML algorithm to recognize and predict faults, leading to the development of more robust update models and efficient transmission strategies.

[0132] The data processing & feedback unit 108 include a reinforcement learning engine (i.e. a closed loop) that adapts how the program, software and / or model stored on the server 102 are distributed to the one or more edge devices 114n. The data processing & feedback unit 108 can be trained with training data to predict bandwidth availability on each of the plurality of channels and to identify the (optimal) set of the plurality of channels based on the predicted bandwidth availability on each of the plurality of channels. The training data is collected by sending the update model to at least two of the one or more edge devices, by sending the update model and any future update models to one or more of the edge devices, or a combination of the above. The unit 108 collects training data related to each occurrence of sending update model(s) and / or future update model(s). The training data includes the current bandwidth of each of the plurality of channels, the bandwidth of each of the plurality of channels when the update model is being sent, the bandwidth of each of the plurality of channels after the update model has been sent, the size of the update model for each of the at least two devices, or any combination of the above. Following training, the data processing & feedback unit 108 identifies a set of the plurality of channels (for example, the set of the plurality of channels as described above) to use for the sending of a future update model. The data processing & feedback unit 108 can instruct the distribution & management unit 104 to use that set of the plurality of channels for a future update model. The process as described above is repeated after each occurrence of sending an update model to improve the ability of the data processing & feedback unit to predict bandwidth availability. The above process is also be performed to optimise load balancing as described above. Advantageously, the network conditions are anticipated based on usage patterns, navigation data and available infrastructure, thus providing a more dynamic transmission strategy. Furthermore, the reinforcement learning engine optimizes distribution strategies and bandwidth utilization in real-time, adapting to changing network conditions and device requirements. This leads ultimately to reduced bandwidth Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0133] 30 consumption because each of the plurality of channels can be utilised to their maximum potential. Further still, this provides a more accurate adaptive bandwidth utilization mechanism which allows the system to adjust dynamically to changing network conditions and device requirements.

[0134] The data processing & feedback unit 108 may additionally or alternatively include an edge analytics module operable to perform real-time data analysis for each of the one or more edge devices 114n for immediate decision making. The data processing & feedback unit 108 may be operable to determine the available computing capabilities of the one or more edge devices 114n in communication with the server 102. Advantageously, the data processing & feedback unit 108 may determine that one or more tasks may be performed on one or more of the edge devices 114n. The one or more tasks may include preprocessing data (for example, unpacking a compressed folder, separating one or more specific parts of a full program from the full program, installing the program, filtering and aggregating sensor data, executing preliminary data validation and error correction, or compressing and encrypting data for secure transmission). Advantageously, processing and / or storage means of the one or more edge devices 114n may be used which frees up processing and / or storage means on the server 102. Furthermore, latency in the network 110 may be reduced by sending less data (for example, compressed data) from the server 102 to the one or more edge devices 114n instead of, for example, uncompressed data.

[0135] The edge analytics module can also perform predictive maintenance tasks by analysing real-time data from the edge devices to predict potential failures and schedule maintenance proactively. This predictive approach reduces downtime and maintenance costs. Moreover, the module can facilitate localized decision-making for autonomous driving functions in vehicles, ensuring quick response times without relying on central server processing. In addition, the edge analytics module can enhance data privacy by processing sensitive information locally on the edge devices, minimizing the need to transmit such data over the network. This approach ensures compliance with data protection regulations and reduces the risk of data breaches. Furthermore, the edge analytics module can support dynamic resource allocation by continuously monitoring the performance and workload of edge devices and redistributing tasks to maintain optimal system performance. This dynamic allocation ensures that no single device becomes a bottleneck, thereby improving the overall efficiency and responsiveness of the network. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0136] 31

[0137] In an embodiment, the data processing & feedback unit 108 may send the update model to at least one edge device 114n (such as a device with computing means that is connected to the network 110 and has available computing capacity) and instruct the at least one edge device 114n to process the update model (this may include installing the update model, replacing part(s) of the current data / software on the at least one edge device 114n with the update model, or similar operations). The at least one edge device 114n may process the update model accordingly. The data processing & feedback unit 108 may command the at least one edge device 114n to send the processed update model (for example, the installed updated data / software) from the at least one edge device 114n to a different one or more edge devices 114n*. The different one or more edge devices 114n* may be one or more vehicles or other edge devices that require the updated data / software. Thus, the first at least one edge device 114n provides computational load for the different one or more edge devices 114n*. Advantageously, computational load can be distributed throughout the network according to where storage and / or processing capabilities are available. For example, if a server’s processing capabilities are overloaded the processing requirements can be outsourced to a vehicle and / or any other edge device if the vehicle and / or edge device’s processor have extra processing capabilities.

[0138] In an embodiment, the data processing & feedback unit 108 may perform real-time data analysis for each of the one or more edge devices 114n. This may include determining computational capacities (such as available memory, available storage, available processing capabilities, or a combination of the above) of each of the one or more edge devices 114n. For example, this may include determining, by the server 102, whether free memory in the one or more edge devices 114n is above a predetermined threshold. If the free memory is above the predetermined threshold, the data processing & feedback unit 108 may send a command to the one or more edge devices 114n to store the update model on the one or more devices 114n. The unit 108 may then send a command to the one or more devices 114n to send the update model to the other one or more devices. This may alternatively or additionally include determining, by the data processing & feedback unit 108, whether free processing capacity of the one or more devices 114n is above a predetermined threshold. If the free processing capacity is above the predetermined threshold, the unit 108 may send a command to the one or more devices 114n to install the update model on the one or more devices 114n. The unit 108 may subsequently send a command to the one or more devices 114n to send the installed update model to the other one or more devices. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0139] 32

[0140] The data processing & feedback unit 108 can instruct the distribution & management unit 104 to outsource processing and / or storage of update models to one or more edge devices 114n based on this information. Advantageously, the computational capabilities of the one or more edge devices 114n can be leveraged for data storage, preprocessing, generating of update models and analysis, reducing the load on the server 102 and decreasing latency (because data stored on the one or more edge devices 114n can be sent directly to a different edge device without having to be stored on the server 102). This reduces scalability issues because a central server faces issues when storing, generating and distributing update models to a large number of edge devices. Efficient use of edge device resources, model compression and selective update mechanisms ensure that edge devices operate efficiently, even with limited resources. Furthermore, leveraging edge computing capabilities for data storage, preprocessing and local decision-making reduces the reliance on central servers and cloud resources, minimizing latency in model deployment and application.

[0141] Figure 2 shows examples of the one or more edge devices 114n coupled to the network 110. The one or more edge devices 114n may be one or more vehicles, as indicated in 114a. The vehicle 114a shown in figure 2 is a road-going vehicle. However, the invention is not limited to road going vehicles and may also include off-road vehicles, rail vehicles, airborne vehicles, amphibious vehicles, or a combination of the above. The one or more vehicles 114a may be vehicles that comprise computing means (for example, memory and a processor) and provide data for aggregation and leveraging external computing resources from Other Edge Devices (OED), such as edge devices 114b and 114c described below. The one or more vehicles 114a may be a vehicle as described above, a drone or unmanned aerial vehicle (UAV), a robot and / or an autonomous system, or any combination thereof. The one or more edge devices 114n may include one or more stationary infrastructure as depicted by reference numeral 114b. Such stationary infrastructure may be a smart infrastructure device, such as smart city infrastructure and may include, for example, a smart electricity pole, a smart road, a smart road sign, a smart appliance, a smart building or any other smart infrastructure element that may include computing means (for example, memory and a processor) and be connected to the network 110. The one or more edge devices may be one or more interactive elements as depicted by reference numeral 114c. This may include a user equipment (UE) device, network equipment, a server, smart traffic lights, smart cats eyes, smart street lamps, smart buoys or any other suitable smart interactive infrastructure device that may include computing means (for example, memory and a processor) and be connected to the network. The one or more edge Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0142] 33 devices 114n and the various types of edge device 114a, 114b, 114c may each communicate with one another without communicating to the server 102 via the network.

[0143] Each of the one or more edge devices 114n comprise a memory and a processor. The processor is operable to send an indication of current software and / or data stored on the memory to another edge device 114n and / or to a server (for example, server 102 as described above). The processor is operable to receive an update model as described above.

[0144] Each of the one or more edge devices 114n is operable to collect data to generate an update model using a federated learning system, as described above with regard to figure 1. Each of the one or more vehicles 114a may have hardware to collect the data. The hardware may include one or more LIDAR sensors, one or more radars, and / or one or more cameras on the front, sides or rear of the vehicle 114a.

[0145] Global model drift can occur at each of the one or more vehicles 114a due to variability in the hardware used for data collection from each of the plurality of vehicles 114a. For example, the LIDAR sensor(s) may be positioned at varying heights or angles on each of the plurality of vehicles 114a. This will produce data with different perspectives, which means the learned model on one vehicle may not be directly transferable to another vehicle with a different sensor setup. The cameras on the front, sides, or rear of a vehicle capture very different scenes, and if not properly accounted for, the resulting models can diverge from the global objective. And each model might have them installed at different places. Some of the vehicles 114a may have more advanced hardware (for example, multiple high-resolution cameras, LIDARs, or radars) that generate more detailed data, while others might have simpler setups, leading to further divergence in model performance.

[0146] Global modal drift can also occur due to a variability in the driving conditions, traffic patterns, weather conditions, and / or road rules across regions of the plurality of vehicles 114a. For example, vehicles will experience vastly different driving conditions depending on whether they are operating in urban, suburban, or highway environments. Urban driving involves stop- and-go traffic, frequent lane changes, interactions with pedestrians, and complex traffic signals. This generates a different set of training data than highway driving, which focuses on highspeed lane-keeping, overtaking, and long stretches of uninterrupted driving. For example, a vehicle operating in a busy city like New York will gather data on frequent stop-start traffic and complex intersection navigation, while a vehicle driving on a rural highway in Finland might mostly collect data on long, uninterrupted road segments. Weather conditions (for example, snow, rain, fog) and climate variations (such as hot, dry, cold) impact how the sensors (such as Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0147] 34 the camera(s) and LIDAR) capture data and how the vehicle responds to steering and other driving inputs. This variability can affect how well a model trained in one environment generalizes to others. For example, a vehicle in Florida may frequently deal with rain and humid conditions, affecting camera and sensor performance, while a vehicle in Finland may drive in icy conditions, influencing how it handles object detection and vehicle control. Training models on vastly different weather data can cause inconsistencies when aggregated into a global model. Traffic behavior and road-user interactions differ significantly between countries. For example, road users in some countries may drive more aggressively, whereas drivers in other countries may generally drive in a more reserved and law-abiding manner. Different countries have varying road rules, such as speed limits, yielding practices, roundabout usage, and lane priorities. These regional variations can lead to discrepancies in how local models learn and make decisions. Not to mention left / right side driving. A vehicle in Tokyo may frequently interact with pedestrians, cyclists, and other road users in tight urban spaces, while a vehicle in Arizona may collect data on wide open roads with minimal traffic or human interaction.

[0148] To address the global model drift caused by these issues, the federated learning system as described above can utilize the understanding of vehicle hardware and vehicle software modelling and vehicle operating area and other factors. In an embodiment, this includes personalized federated learning. Personalized federated learning may address the hardware and environmental heterogeneity by allowing each vehicle 114a to train a model that is customized to its unique characteristics while still contributing to the global model. Each vehicle 114a can train a model tailored to its specific sensor setup, whether it is a LIDAR at a particular height or cameras mounted at different angles. Advantageously, the global model remains generalizable, while each vehicle 114a fine-tunes it to suit its unique hardware. This ensures that the global model can handle data from various sensor configurations without degrading in performance. The input data can also be transformed into generalized data by adjustments based on vehicle characteristic - which is based on available vehicle information. The personalized federated learning may be applied to regional adaptation (vehicles driving in different conditions, such as in snowy or sunny conditions) and can fine-tune the global model to adapt to their local environments as based on data from the central server 102 it can be aware of how it operates and specialize learning for certain scenarios. For example, vehicles in cold climates can adjust the model to better handle icy road conditions, while vehicles in urban environments can focus on pedestrian and stop-and-go traffic interactions. Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0149] 35

[0150] In an embodiment, the federated learning system includes partitioning the model for sensor-aware parts. Given the above-mentioned hardware variabilities, each of the one or more vehicles 114a may be operable to train only the model components relevant to their specific sensors, such as LIDAR, cameras, or radar, reducing computational overhead while still contributing to a comprehensive global model. A vehicle with front-facing cameras may, for example, focus on training the model for front- view object detection. A vehicle with side- mounted cameras may, for example, work on blind-spot detection, while vehicles with one or more LIDARs mounted at different heights may, for example, focus on distance and heightbased object recognition. In a hierarchical federated learning system (such as the hierarchical federated learning system as described above), roadside units (RSUs) or gateway nodes can aggregate the models from multiple vehicles based on their sensor inputs before forwarding a combined model to the central server. This hierarchical approach ensures that sensor-specific insights are shared across the fleet while reducing the communication overhead on the central server.

[0151] In an embodiment, the federated learning system includes federated multi-task learning. In this arrangement the one or more vehicles 114a can specialize in different driving tasks based on their environments. This helps avoid task overlap and ensures that the vehicles 114a contribute meaningful, environment-specific data to the global model. A vehicle primarily driving on highways may, for example, focus on tasks like lane-keeping and high-speed navigation, while a vehicle in a dense urban environment may, for example, prioritize pedestrian detection, traffic light recognition, and stop-and-go traffic management. These specialized models can then aggregated (as described above with regard to federated learning) to create a more versatile global model capable of handling a range of tasks.

[0152] Mismatches of data and data accuracy are important to ensure that the model is trained correctly. Based on Vehicle HW&SW Inventory (something that the described system possesses) the client on the vehicle can perform data completeness test before starting local training. The data completes test searches for missing or incomplete data based on predefined criteria. For example, for a model for autonomous driving that requires data from LIDAR, front camera, and GPS, the vehicle can check whether it has complete data from all three sources for each required time period. If LIDAR data is determined to be missing for a portion of the training period, the vehicle can either request a resampling of the data (if possible) or omit incomplete records from training to avoid training on faulty data. Vehicles 114a can maintain a log of missing or incomplete data points. Data may be determined crucial for training (for Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0153] 36 example, pedestrian detection in a busy city). If crucial data is missing the vehicle can flag this data to be either retrained later or reported to the central server 102 for further correction or augmentation.

[0154] In an embodiment, statistical analysis may be performed for dataset imbalance. The dataset imbalance may include - feature distribution analysis which includes checking how often certain features (such as pedestrians, stop signs, rainy conditions) appear in the data. If certain features are underrepresented, the vehicle can be flagged for potential imbalance. The dataset imbalance may additionally or alternatively include label distribution analysis which includes identifying if there are underrepresented classes in the dataset. For example, if a vehicle has very few instances of lane changes but many instances of straight-line driving, this can lead to bias in the model's decision-making. This can be provided to the central server 102 based on which it will adjust the class weights etc.

[0155] A LIDAR sensor from a first OEM may collect point cloud data at a different resolution or frequency compared to a LIDAR sensor from a second (and different) OEM. Similarly, a camera from one model may have a different field of view than another. Without proper standardization, combining data from these different sources may cause inconsistencies in the training process. The same applies to setting the same LIDAR data in different position / place at the vehicle. In an embodiment, the system may include a data preprocessing pipeline on each of the plurality of edge devices 114n (including each of the vehicles 114a) to prevent mismatching of data. The preprocessing pipeline on each edge device 114n ensures that the raw sensor data collected by each edge device 114n is converted into a predetermined required format before training begins. This preprocessing step may include, rescaling sensor data (for example, adjusting LIDAR point clouds to a common resolution), image normalization for camera inputs (for example, ensuring consistent brightness, contrast, or color profiles), and / or modifying sensor data by adding information on where the sensor is located.

[0156] Figure 3 shows a flow chart of a method 300 of sending an update model, as described above with reference to figures 1 and 2, according to the invention. The method includes sending, by a server (such as server 102), a command to update software (for example, the data, program(s), software, update(s), models(s) as described above) on one or more devices (for example, one or more edge devices 114n as described above in figures 1 and 2) as shown at 302. The method further includes at 304 receiving, by the one or more devices (e.g. devices 114n), an indication of current software on the one or more devices 114n as described above with regard to figures 1 and 2. The method further includes at 306 generating, by the server Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0157] 37

[0158] 102, an update model (as described above) that comprises a set of data wherein the set of data is part of the update software and is not part of the current software, and wherein the update model is smaller in size than the update software or the current software as described above with reference to figures 1 and 2. The method further includes at 308 sending, by the server 102, the update model to the one or more devices 114n as described above. By generating an update model that comprises less data than an entire update software / program, less data has to be transmitted to the one or more devices and processed by the one or more devices. Advantageously, bandwidth usage is reduced during sending of the update model. Furthermore, transmission times are reduced, due to the reduced amount of data transmitted. Further still, the computational requirements of the one or more devices is reduced due to the reduced amount of data that has to be processed.

[0159] Figures 4, 5 and 6 show additional method steps of sending an update model which can be combined with the method as described above with regard to figure 3 and with the system(s) as described with regard to figures 1 and 2 above.

[0160] In Figure 4, the method may include generating, by the server (for example, server 102), the update model by pruning, quantization, knowledge distillation, weight clustering, tensor decomposition, sparsity induction, low-rank factorization, and / or hybrid quantization as shown at 402. The method may include generating the update model using generative model compression as shown at 404. The method may include generating the update model using generative model compression as shown at 406. Not all of the features referred to at 402, 404 and 406 have to be performed. Only one, more than one or all of the features referred to at 402, 404 and 406 may be performed. Those features may be performed in any order as described above with reference to Figures 1 to 3.

[0161] In Figure 5 the method may further include identifying a plurality of channels between the server and the one or more devices; and distributing packets of the update model across the plurality of channels as shown at 502. The method may include sending the update model via a blockchain network as shown at 504. The method may include identifying a plurality of channels between the server and the one or more devices, sending an acknowledgement (ACK) signal from the server to the one or more devices on each of the plurality of channels, determining, based on each of the ACK signals a current bandwidth availability on each of the plurality of channels, selecting a set of the plurality of channels, wherein the set of the plurality of channels comprises at least one of the plurality of channels, and sending the update model on the set of the plurality of channels as shown at 506. Not all of the features referred to at 502, Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0162] 38

[0163] 504 and 506 have to be performed. Only one, more than one or all of the features referred to at 502, 504 and 506 may be performed. Those features may be performed in any order as described above with reference to Figures 1 to 3.

[0164] The method may, subsequent to 506, include determining a threshold bandwidth requirement based on the size of the update model, and identifying the set of the plurality of channels based on the threshold bandwidth requirement as shown at step 508. The method may, subsequent to 508, include sending the update model to at least two of the one or more devices, training the server with training data to predict bandwidth availability on each of the plurality of channels and to identify the set of the plurality of channels based on the predicted bandwidth availability on each of the plurality of channels, and identifying the set of the plurality of channels for a future update model as shown at 510. The training data may include the current bandwidth of each of the plurality of channels, the bandwidth of each of the plurality of channels when the update model is being sent, the bandwidth of each of the plurality of channels after the update model has been sent, and the size of the update model for each of the at least two devices. The method may, subsequent to 506, 508, or 510, include sending the update model on one channel of the set of the plurality of channels, or distributing the sending of the update model across the set of the plurality of channels as shown at 512. The above features may be performed as described above with reference to Figures 1 to 3.

[0165] In Figure 6, the method may include receiving one or more error messages from the one or more devices, the one or more error messages comprising faults, training an Al and / or ML algorithm of the server to identify the one or more error messages, generating a second update model based on the received error messages, the second update model not including the faults, and sending, by the server, the second update model to the one or more devices as shown at 602. The method may include determining (by the server) whether free memory in the one or more devices is above a predetermined threshold, storing the update model on the one or more devices if the free memory is above the predetermined threshold, and sending (by the server) a command to the one or more devices to send the update model to the other one or more devices as shown at 604. The method may include determining (by the server) whether free processing capacity of the one or more devices is above a predetermined threshold, sending (by the server) a command to the one or more devices to install the update model on the one or more devices if the free processing capacity is above the predetermined threshold, and sending (by the server) a command to the one or more devices to send the installed update model to the other one or more devices as shown at 606. Not all of the features referred to at 602, 604 and 606 have to Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-05

[0166] 39 be performed. Only one, more than one or all of the features referred to at 602, 604 and 606 may be performed. Those features may be performed in any order as described above with reference to Figures 1 to 3.

Claims

Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-0540CLAIMS1. A method of sending an update model, wherein the update model is an Artificial Intelligence, Al, Model and / or a Machine Learning, ML, Model, the method comprising: sending, by a server, a command to update software on one or more devices; receiving, by the one or more devices, an indication of current software on the one or more devices; generating, by the server, an update model that comprises a set of data wherein the set of data is part of the update software and is not part of the current software, and wherein the update model is smaller in size than the update software or the current software; and sending, by the server, the update model to the one or more devices.

2. The method of claim 1, wherein generating, by the server, the update model comprises generating the update model using generative model compression.

3. The method of claims 1 to 2, wherein, the update model is generated using a federated learning system.

4. The method of claims 1 to 3, wherein sending, by the server, the update model to the one or more devices comprises: identifying a plurality of channels between the server and the one or more devices; and distributing packets of the update model across the plurality of channels.

5. The method of claims 1 to 4, wherein sending, by the server, the update model to the one or more devices comprises: sending the update model via a blockchain network.

6. The method of claims 1 to 5, wherein the one or more devices comprise: at least one vehicle; at least one edge device; orHarman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-0541 a combination of the above.

7. The method of claims 1 to 6, wherein the one or more devices comprise at least one vehicle and at least one edge device, the method further comprising: sending the update model to the at least one edge device; processing the update model on the at least one edge device; and commanding the at least one edge device to send the processed update model from the at least one edge device to the at least one vehicle.

8. The method of claims 1 to 7, further comprising: receiving one or more error messages from the one or more devices, the one or more error messages comprising faults; training an Al and / or ML algorithm of the server to identify the one or more error messages; generating a second update model based on the received error messages, the second update model not including the faults; and sending, by the server, the second update model to the one or more devices.

9. The method of claim 8, wherein the faults comprise: one or more corrupted data packets in the update model; incomplete transmission of the one or more data packets from the server to the one or more devices; an incompatibility of the update model with the current software; latency (and / or time delays) exceeding a predefined threshold; insufficient bandwidth between the server and the one or more devices; checksum mismatches; an unexpected reboot or crash of the one or more devices; a failure to install the update model on the one or more devices; one or more performance metrics being below a predefined threshold; or any combination thereof.

10. The method of claims 1 to 9, further comprising:Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-0542 determining, by the server, whether free memory in the one or more devices is above a predetermined threshold; storing the update model on the one or more devices if the free memory is above the predetermined threshold; and sending, by the server, a command to the one or more devices to send the update model to the other one or more devices.

11. The method of claims 1 to 10, further comprising: determining, by the server, whether free processing capacity of the one or more devices is above a predetermined threshold; sending, by the server, a command to the one or more devices to install the update model on the one or more devices if the free processing capacity is above the predetermined threshold; and sending, by the server, a command to the one or more devices to send the installed update model to the other one or more devices.

12. A server comprising a processor operable to carry out the method of claims 1 to 11.

13. A device comprising: memory; and a processor, the processor is operable to: send an indication of current software stored on the memory; receive an update model as set out in claims 1 to 11.

14. A system for sending an update model, the system comprising: the server of claim 12; and at least one device of claim 13.

15. The system of claim 14, wherein the at least one device comprises at least one vehicle and at least one edge device.

16. The system of claim 15, wherein the edge device comprises any one of:Harman Becker Automotive Systems GmbH P240108WO Karlsbad, DE 2024-12-0543 a vehicle; a smart infrastructure device; a drone; an autonomous system; network equipment; a smart appliance; a smart building; a user equipment device; a server; or any combination of the above.

Images