Method and system for online training of a first machine learning model

Abstract

Method and technical system (100) for online training of a first machine learning model (202) that is executable on a control unit (204) of the technical system (100).

Description

[0001] R. 415120

[0002] - 1 -

[0003] Description

[0004] title

[0005] Method and system for online training of a first machine learning model

[0006] The invention relates to a method for online training of a first machine learning model. Furthermore, the invention relates to a system for online training of a first machine learning model.

[0007] State of the art

[0008] With the advent of the 6G communication standard, the use of digital twins is becoming a key component of modern technology. Particularly in the field of mobility, digital twins can transmit information to road users. This information can be used with the help of AI and machine learning (ML) solutions processed in the cloud, instead of having to be calculated directly on the devices.

[0009] This leads to a complex decision about whether certain functions should be performed on the device or in the cloud. Both approaches have potential drawbacks: Computing in the cloud requires access to sensor data, which can slow down data transmission and thus processing. On the other hand, traffic scenarios can be so complex that the locally deployed model reaches its limits. These challenges necessitate a balanced decision to ensure optimal performance and efficiency.

[0010] It is an object of the invention to provide an improved method and / or an improved device. R. 415120

[0011] - 2 -

[0012] The problem is solved by a method according to the features of claim 1. The problem is solved by a device according to the features of claim 7.

[0013] Disclosure of the invention

[0014] According to a first aspect, a method for online training of a first machine learning model is proposed, which is executable on a control unit of a technical system. The method comprises the steps of: providing a trained second machine learning model, which is executable on a server; providing sensor data from a sensor of the technical system, on the basis of which a first output value is generated by the first machine learning model, on the basis of which a first action can be executed for the technical system; storing the sensor data, the first output value, and the first action in a memory of the control unit, preferably when a predetermined time criterion is met; transmitting the sensor data, preferably the first output value and preferably the first action, to the server when a predetermined storage criterion is met.Generating a second output value and a second action executable based on it, using the second machine learning model and the transmitted sensor data; transmitting the second output value and the second action to the control unit to compare the first action with the second action; and if the comparison of the first action with the second action meets a predetermined comparison criterion, training the first machine learning model online based on the sensor data and the second output value.

[0015] It is understood that the steps according to the invention, as well as further optional steps, do not necessarily have to be carried out in the sequence shown, but can also be carried out in a different sequence. Furthermore, further intermediate steps may be provided. The individual steps may also comprise one or more substeps without thereby departing from the scope of the method according to the invention. According to a second aspect, a system for online training of a first machine learning model is proposed, wherein the first machine learning model is R. 415120

[0016] - 3 -

[0017] The learning model is executable on an electronic control unit (ECU), the system comprising a server and the ECU communicatively connected to the server, the server being configured to execute a trained second machine learning model, the system comprising a sensor configured to provide sensor data, the first machine learning model being configured to generate a first output value based on the sensor data, on the basis of which a first action can be executed for the system; the ECU being configured to store the sensor data, the first output value, and the first action in a memory of the ECU depending on a predetermined time criterion, and to transmit the sensor data, preferably the first output value and preferably the first action, to the server depending on a predetermined storage criterion;wherein the second machine learning model is configured to generate a second output value and a second action executable on the basis of the transmitted sensor data, wherein the server is configured to transmit the second output value and the second action to the control unit for comparison of the first action with the second action, and wherein the first machine learning model can be trained online depending on a predetermined comparison criterion of comparing the first action with the second action based on the sensor data and the second output value.

[0018] Ideally, the server should include an edge server or a cloud server.

[0019] Two different machine learning models are used here. The second machine learning model has a high degree of complexity and adaptability, but can only be executed on the server and not on the ECU, as it requires significant computing power. The first machine learning model has lower technical complexity and adaptability and also requires less computing power. Therefore, the first machine learning model can be executed on the ECU. The more powerful second machine learning model preferentially uses information from the digital twin (the first machine learning model). The first machine learning model has access to digitized data from the R. 415120.

[0020] - 4 - physical system and its environment. The second machine learning model is trained using a large amount of data prior to the present procedure and can only be executed on the external server, e.g., an edge server or a cloud server. The first machine learning model can be executed on the ECU and is designed to optimally utilize the ECU's computing resources.

[0021] The second machine learning model can preferably also be improved by using more contextual information provided by digital twins and other online resources (official maps, official traffic information, long-term sensor information from base stations, historical data, Collective Perception Messages (CPMs), as described in an ETSI technical report, etc.). The second machine learning model is superior to the first machine learning model.

[0022] By preferably requesting the output values ​​and actions of the second machine learning model on a regular basis, the first machine learning model preferably receives ground-truth data that can be used to improve the first machine learning model through online learning. While the first machine learning model learns from the complex second machine learning model, its computational complexity will not increase. In other words, the first machine learning model remains the most efficient, compressed version of a preferably continuously improving second machine learning model.

[0023] The invention thus utilizes the power of a server solution to overcome the limitations of an on-device solution. It therefore improves the on-device first machine learning model and makes the control unit more functional, even with limited computing resources and in situations with limited communication coverage. This is particularly advantageous for autonomous driving, enabling the end user to be provided with a highly functional control unit.

[0024] The first machine learning model is not a standalone system, but is preferably embedded in the overall function of the control unit. R. 415120

[0025] - 5 -

[0026] The statements made regarding the procedure apply accordingly to the device. It is understood that linguistic modifications of procedurally formulated features can be reformulated for the device according to common linguistic practice, without such formulations needing to be explicitly listed here.

[0027] The processing power of a cloud solution can be used to improve the deployed machine learning (ML) model. As a result, the ML model can be more easily adapted to changes in the environment. While these changes may be minor within a few years, they can be significant over a period of ten years or more.

[0028] In another aspect, it is proposed that the predetermined storage criterion includes a number of sensor data and / or output values ​​and / or actions, and / or wherein the storage criterion includes a predetermined, in particular storage space-dependent, time interval.

[0029] A predetermined storage criterion is defined, encompassing a specific number of sensor data points, output values, and / or actions. Alternatively or additionally, the storage criterion can include a predetermined time interval, which depends in particular on the available storage space.

[0030] During the deployment of an electronic control unit (ECU), the device's internal first machine learning model is used multiple times. After several frames, the second machine learning model is preferably used to annotate the sensor data, which then serves as the basis for improving the first machine learning model.

[0031] For example, data is stored in memory until a storage capacity is reached. Once a storage capacity is reached, historical data points can be deleted and replaced with current data points. The predetermined time interval is preferably freely selectable. R. 415120

[0032] - 6 -

[0033] An internal dataset can also be maintained, which grows over time, and only when the dataset reaches a maximum size are some entries replaced by new observations. Furthermore, entries that are too old, e.g., older than 3 months, can be removed from this dataset.

[0034] If at least one data point is stored in the memory, it can be transmitted to the server for further processing.

[0035] In another aspect, it is proposed that the comparison criterion indicates that the first action matches the second action, or that a difference between the first action and the second action satisfies a predetermined threshold criterion.

[0036] Preferably, a comparison criterion is defined that specifies that the first action is identical to the second action. Alternatively, the comparison criterion can specify that a difference between the first and second actions meets a predetermined threshold criterion.

[0037] This ensures that only the information from the second model that does not change the system's behavior is used. This is useful when the action is discrete (classification). If the action is continuous (e.g., steering angle), the condition can be favored by taking a difference and then comparing the results.

[0038] In another aspect, it is proposed that during the generation of the fourth step, provisioning according to the second step and saving according to the third step should take place.

[0039] In a further aspect, it is proposed that the first action and / or the second action involves a discrete classification based on sensor data or a continuous control action of the technical system, in particular braking or acceleration or adjusting a steering angle. Other actions are also conceivable. R. 415120

[0040] - 7 -

[0041] Another aspect is proposed: the sensor data is preprocessed through data preprocessing steps to generate the output value.

[0042] The invention can be used, purely by way of example, in autonomous driving. In this context, data is collected from the vehicle's sensors. This data can be preprocessed, if necessary, by dedicated hardware or software. The result of this preprocessing is the sensor data. This sensor data is fed into the first machine learning model. This yields the first output values. It can be assumed that the output values ​​are in the range of real numbers.

[0043] The initial output values ​​can also be higher-dimensional. Based on these initial output values, a decision is made that leads to at least the first action; for example, collision detection leads to deceleration or active braking, or traffic sign recognition delivers the traffic sign with the highest probability to a driver's display or to the autonomous vehicle.

[0044] In another aspect, the control unit is included in a vehicle with an autonomous driving function and / or a robotics system and / or an industrial machine.

[0045] The present method can be used for the analysis of (image and / or video) data acquired by a sensor. In this context, the term "image and / or video data" can also be replaced by "sensor data." The sensor can detect measurements of the environment in the form of sensor signals, which may be provided, for example, by the following elements: digital images (e.g., video, radar, lidar, ultrasound), motion, thermal images, audio signals, and / or specific data, such as 1D data (e.g., in production). In principle, it is also possible to obtain information about elements encoded by a sensor signal based on that signal. In other words, an indirect measurement can be performed. R. 415120

[0046] - 8 - based on a sensor signal used as a direct measurement. This is also known as virtual sensing.

[0047] Furthermore, the present method can be used to classify and / or categorize and / or segment the sensor data, in particular to detect the presence or absence of objects in the sensor data and / or to perform semantic segmentation of the sensor data, e.g. with regard to traffic signs and / or road surfaces and / or pedestrians and / or vehicles and / or other elements.

[0048] The present method can also be used to determine a continuous value or several continuous values, i.e., to perform a regression analysis, e.g., regarding the distance and / or velocity and / or acceleration and / or tracking of an element, e.g., an object, in the data. This is preferably carried out based on low-level features (e.g., edges or pixel attributes for images).

[0049] The present method can also be used to calculate a control signal for controlling a technical system, such as a computer-controlled machine, a robot system, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system.

[0050] This involves analyzing data (e.g., scalar time series), particularly from at least one sensor, such as a radar sensor and / or a video sensor. The technical system can then be operated accordingly. The present method can be executed on a specific physical system. For example, the technical system can be used in a vehicle with autonomous driving or advanced driving assistance systems (ADAS). R. 415120

[0051] - 9 -

[0052] The present method can be used in the field of active learning and / or active testing, for example, in a test bench. It can be used to select suitable data points for training a machine learning model. The present method preferably involves training a machine learning model and / or generating training data for training a machine learning model. The machine learning models trained in this way can then be used for inference.

[0053] In another aspect, a computer program with program code is claimed to execute at least parts of the present method in one of its aspects when the computer program is executed on a computer. In other words, a computer program (product) is claimed to comprise instructions that, when executed by a computer, cause it to execute the method(s) in one of its aspects. In particular, the computer program comprises instructions that cause the proposed system to execute the proposed method.

[0054] In a further aspect, a computer-readable data carrier containing the program code of a computer program is proposed to execute at least parts of the present method in one of its aspects when the computer program is executed on a computer. In other words, the invention relates to a computer-readable (storage) medium comprising instructions that, when executed by a computer, cause it to execute the method / steps of the method in one of its aspects. In particular, the proposed computer program, which includes instructions that cause the proposed system to execute the proposed method, is stored on the data carrier.

[0055] The described configurations and training programs can be combined in any way desired.

[0056] Further possible embodiments, developments and implementations of the invention also include combinations of previously mentioned or R. 415120 that are not explicitly mentioned.

[0057] - 10 - features of the invention described below with regard to the exemplary embodiments.

[0058] Brief description of the drawings

[0059] The accompanying drawings are intended to provide a further understanding of the embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain the principles and concepts of the invention.

[0060] Other embodiments and many of the aforementioned advantages become apparent with reference to the drawings. The elements depicted in the drawings are not necessarily shown to scale.

[0061] Fig. 1 shows a schematic flowchart of the present

[0062] Method according to one embodiment.

[0063] Fig. 2 shows a schematic block diagram of one of the present

[0064] System according to one embodiment.

[0065] In the figures of the drawings, identical reference symbols denote identical or functionally equivalent elements, parts or components, unless otherwise stated.

[0066] Fig. 1 shows a schematic flowchart of a procedure for online training of a first machine learning model, which can be executed on a control unit of a technical system (see Fig. 2).

[0067] The procedure includes at least the following steps:

[0068] In step S1, a trained second machine learning model is deployed, which can be executed on a server.

[0069] In step S2, sensor data from a sensor of the technical system is provided, on the basis of which the first machine R. 415120

[0070] - 11 -

[0071] A learning model generates an initial output value, on the basis of which an initial action can be performed for the technical system. The sensor data can be preprocessed through data preprocessing steps to generate the output value.

[0072] In step S3, the sensor data, the first output value, and the first action are stored in a memory of the control unit, preferably when a predetermined time criterion is met. The storage criterion has a predetermined time interval, which is particularly dependent on memory space.

[0073] In step S4, the sensor data, preferably the first output value and preferably the first action, is transmitted to the server when a predetermined storage criterion is met. The predetermined storage criterion specifies a number of sensor data points and / or output values ​​and / or actions.

[0074] In step S5, a second output value and a second action that can be executed on the basis of this value are generated using the second machine learning model based on the transmitted sensor data.

[0075] In step S6, the second output value and the second action are transmitted to the control unit to compare the first action with the second action.

[0076] In step S7, if the comparison of the first action with the second action meets a predetermined comparison criterion, the first machine learning model is trained online based on the sensor data and the second output value. The comparison criterion specifies that the first action matches the second action, or that a difference between the first action and the second action meets a predetermined threshold criterion.

[0077] Preferably, provisioning S2 and saving S3 occur during generation S4. R. 415120

[0078] - 12 -

[0079] The first action and / or the second action involves a discrete classification based on sensor data or a continuous control action of the technical system, in particular braking or acceleration or adjusting a steering angle.

[0080] Fig. 2 shows a technical system 100 for online training of a first machine learning model 202, which can be executed on a control unit 204. The system 100 comprises a server 206 and the control unit 204, which is communicatively connected to the server 206. The server 206 is configured to execute a trained second machine learning model 208.

[0081] System 100 has a sensor 210 configured to provide sensor data 212, and the first machine learning model 202 is configured to generate a first output value 214 based on the sensor data 212, on the basis of which a first action 216 can be executed for System 100. Control unit 204 is configured to store the sensor data 212, the first output value 214, and the first action 216 in a memory 219 of control unit 204 according to a predetermined time criterion, and to transmit the sensor data 212, the first output value 214, and the first action 216 to server 206 according to a predetermined storage criterion.

[0082] The second machine learning model 208 is configured to generate a second output value 218 and a second action 220 executable based on the transmitted sensor data 212. The server 206 is configured to transmit the second output value 218 and the second action 220 to the control unit 204 for comparison of the first action with the second action. The first machine learning model 202 can then be trained online based on a predetermined comparison criterion for comparing the first action 216 with the second action 220 based on the sensor data 212 and the second output value 218.

Claims

R. 415120 - 13 - Claims 1. Method for online training of a first machine learning model (202) that is executable on a control unit (204) of a technical system (100), the method comprising the steps: Provision (S1) of a trained second machine learning model (208) that is executable on a server (206); Providing (S2) sensor data (212) of a sensor (210) of the technical system (100), on the basis of which a first output value (214) is generated by means of the first machine learning model (202), on the basis of which a first action (216) can be performed for the technical system (100); Storing (S3) the sensor data (212), the first output value (214) and the first action (216) in a memory (219) of the control unit (204), preferably when a predetermined time criterion is met; Transmitting (S4) the sensor data (212), preferably the first output value (214) and preferably the first action (216) to the server (206) when a predetermined storage criterion is met; Generating (S5) a second output value (218) and a second action (220) executable on the basis of the transmitted sensor data (212) using the second machine learning model (208); Transmitting (S6) the second output value (218) and the second action (220) to the control unit (204) to compare the first action (216) with the second action (220); and if the comparison of the first action (216) with the second action (220) meets a predetermined comparison criterion, online training (S7) of the first machine learning model (202) based on the sensor data (212) and the second output value (218). R. 415120 - 14 - 2. Method according to claim 1, wherein the predetermined storage criterion comprises a number of sensor data (212) and / or output values ​​(214) and / or actions (216), and / or wherein the storage criterion comprises a predetermined, in particular storage space-dependent, time interval.

3. Method according to claim 1 or 2, wherein the comparison criterion indicates that the first action (216) is identical to the second action (220), or that a difference between the first action (216) and the second action (220) satisfies a predetermined threshold criterion.

4. Method according to one of the preceding claims, wherein during generation (S4) provisioning (S2) and storage (S3) takes place.

5. Method according to one of the preceding claims, wherein the first action (216) and / or the second action (220) comprises a discrete classification based on the sensor data (212) or a continuous control action of the technical system (220), in particular a braking or an acceleration or an adjustment of a steering angle.

6. Method according to one of the preceding claims, wherein the sensor data (212) are preprocessed by data preprocessing steps to generate the first and / or second output value (214, 218).

7. System (100) for online training of a first machine learning model (202) executable on a control unit (204), wherein the system (100) comprises a server (206) and the control unit (204) communicatively connected to the server (206), wherein the server (206) is configured to execute a trained second machine learning model (208), wherein the system (100) comprises a sensor (210) configured to provide sensor data (212), and wherein the first machine learning model (202) is configured to generate a first machine learning model based on the sensor data (212). R. 415120 - 15 - to generate an output value (214) on the basis of which a first action (216) can be executed for the system (100); wherein the control unit (204) is configured to store the sensor data (212), the first output value (214) and the first action (216) in a memory (219) of the control unit (204) depending on a predetermined time criterion, and to transmit the sensor data (212), preferably the first output value (214) and preferably the first action (216) to the server (206) depending on a predetermined storage criterion;wherein the second machine learning model (208) is configured to generate a second output value (218) and a second action (220) executable on the basis of the transmitted sensor data (212), wherein the server (206) is configured to transmit the second output value (218) and the second action (220) to the control unit (204) for comparison of the first action (216) with the second action (220), and wherein the first machine learning model (202) can be trained online depending on a predetermined comparison criterion of the comparison of the first action (216) with the second action (216) based on the sensor data (212) and the second output value (218).

8. System (100) according to claim 7, wherein the server (206) comprises an edge server or a cloud server.

9. Computer program with program code to execute at least parts of a method according to any one of claims 1 to 6 when the computer program is executed on a computer, wherein in particular the computer program comprises instructions that cause the system (100) according to claim 7 or 8 to execute the method according to any one of claims 1 to 6.

10. Computer-readable data carrier containing program code of a computer program for executing at least parts of a method according to any one of claims 1 to 6 when the computer program is executed on a computer, wherein in particular the R. 415120 - 16 - The computer program according to claim 9 is stored on the data carrier.

Images