Method for extending a learning system, electronic vehicle and computer program

Abstract

Method for extending a learning system (3) which is trained to operate a vehicle (1) at least partially autonomously, wherein - the learning system (3) is trained on the basis of provided data sets (6) with which various scenarios relating to an at least partially autonomous operation of a vehicle (1) are characterized, characterized in that - input data relating to a given scenario (10) are provided, - a comparison of the input data relating to the present scenario (10) with the provided data sets (6) is carried out, - if it is determined on the basis of the comparison carried out that the present scenario (10) differs at least partially from the most diverse scenarios relating to the provided data sets (6), subsystems (12) into which the learning system (3) is divided are checked to determine whether the functionality of at least one subsystem (13) of these subsystems (12) is impaired with regard to the present scenario (10), wherein - another subsystem (14) of the subsystems (12), which is not impaired in its functionality with regard to the present scenario (10), is used to train the at least one impaired subsystem (13) for the present scenario (10), whereby the learning system (3) is expanded with regard to the present scenario (10).

Description

[0001] The invention relates to a method for extending a learning system which is trained to operate a vehicle at least partially autonomously, wherein the learning system is trained on the basis of provided data sets with which various scenarios relating to an at least partially autonomous operation of a vehicle are characterized.

[0002] Furthermore, the invention relates to an electronic vehicle system for an at least partially autonomously operable vehicle with a learning system.

[0003] The invention also relates to a computer program which can be loaded directly into a memory of a control device of an electronic vehicle system.

[0004] When the environment or sensor technology changes, neural networks in vehicles today must be retrained from scratch. For example, stop signs look similar in many countries, although there are exceptions. This is where neural networks reach their limits, as a completely new training process is required, especially in an autonomous vehicle, to learn these differences. These constantly new lessons take a lot of time, incur high costs, and thus slow down autonomous driving overall.

[0005] Even trivial changes can cause significant effort, for example, in the development of autopilots. For example, many autonomous test vehicles currently use cameras with a resolution of 2 megapixels. Replacing these with better models with 8 megapixels essentially changes little. However, this is sufficient to completely retrain the neural network. This is detrimental to the efficient operation of a vehicle that is at least partially or fully autonomous.

[0006] For example, DE 10 2020 118 504 A1 discloses a method for extending a neural network for environment recognition, comprising providing a trained neural network implemented to perform object classification on environment data. The neural network is extended based on analyzing the output of the neural network.

[0007] For example, US 5,532,929 A discloses a device for controlling a vehicle speed or a vehicle performance.

[0008] Furthermore, IN 2022 410 593 70 A discloses a method for detecting and segmenting cars by removing their shadow counterparts, taking into account a "Widrow-Hoff learning level." The system is trained to perform appropriate detection and segmentation based on different vehicle types, their exterior appearance, color, and model.

[0009] One object of the present invention is to be able to train a learning system, such as a neural network of a vehicle, more efficiently in new and previously unknown situations or scenarios.

[0010] This problem is solved by a method, an electronic vehicle system, and a computer program according to the independent patent claims. Useful further developments arise from the appended patent claims.

[0011] One aspect of the invention relates to a method for extending a learning system which is trained to operate a vehicle at least partially autonomously, wherein - in particular, the learning system is trained on the basis of provided data sets with which various scenarios relating to at least partially autonomous operation of a vehicle are characterized, - in particular, input data relating to an existing scenario are provided, - in particular, a comparison of the input data relating to the present scenario with the data sets provided is carried out, - in particular, if it is determined on the basis of the comparison carried out that the present scenario differs at least partially from the various scenarios concerning the data sets provided, subsystems into which the learning system is divided are checked to determine whether the functionality of at least one subsystem is impaired with regard to the present scenario, whereby - in particular, another subsystem of the subsystems, which is not impaired in its functionality with regard to the present scenario, is used to train the at least one impaired subsystem for the present scenario, whereby the learning system is extended with regard to the present scenario.

[0012] The proposed method enables efficient training of the learning system, in particular for expanding the learning system. The proposed method, and in particular the training of a subsystem or subsystem of the learning system, allows computing time, computing power, and resource requirements to be reduced or minimized. Consequently, the learning system can be deployed or used more efficiently and advantageously in vehicles, such as fully autonomous or semi-autonomous vehicles. In road traffic, vehicles may encounter new, complex, and unknown traffic situations, such as scenarios. To ensure safe and efficient, at least partially autonomous operation of a vehicle in these situations, a learning system, as expanded with the present method, is advantageous.The present invention enables a learning system, such as a vehicle's neural network, to be trained in a variety of as yet unknown situations in a time- and effort-minimized manner. This is because the entire learning system is not initially retrained, but rather at least one subsystem of the learning system is selectively trained. In order to achieve further time savings, a corresponding subsystem is taught or trained on the basis of another subsystem of the learning system. In this way, individual subsystems of the learning system learn or train each other from each other in order to be able to react to a new situation, such as a traffic situation, in the simplest way possible.

[0013] As a basis, the adaptive system can be trained using provided data sets. These data sets contain extensive information regarding a wide variety of scenarios, such as traffic situations or traffic scenarios. If a scenario arises that is unknown to at least one subsystem of the adaptive system, this subsystem is trained or taught with the support of another subsystem of the adaptive system that is already essentially trained for this particular scenario and is therefore not impaired in its functionality. Thus, at least one subsystem or several subsystems of the adaptive system model an impaired subsystem of the adaptive system in order to be trained for the particular scenario or situation.This means that the learning system is not completely retrained, but rather individual subsystems are learned individually, allowing the entire learning system to be expanded accordingly. This allows the learning system to be continuously expanded in the simplest way possible.

[0014] Specifically, the subsystems into which the adaptive system is divided are checked to determine whether the functionality of at least one of the subsystems is impaired in the current scenario compared to a reference functionality. This allows a target-actual comparison of functionality to be performed.

[0015] The functionality or a usage state or a deployment state of the at least one impaired subsystem can be checked based on a reference condition, a reference scenario, or a parameter structure. In this case, the at least one subsystem can be assessed as to how it is expected to operate in the given scenario.

[0016] In particular, the verified functionality of at least one impaired subsystem can be used to assess the suitability and / or reliability of the subsystem in the present scenario.

[0017] The at least one impaired subsystem, whose functionality with respect to the current scenario is impaired, may be insufficiently trained for the current scenario. To remedy this, the at least one subsystem can be trained using another subsystem.

[0018] For example, the check may reveal that at least one subsystem is only partially or only partially functional in the given scenario. A subsystem with impaired functionality may, for example, be slow, inefficient, inaccurate, faulty, and / or error-prone. Such an impaired subsystem may lead the vehicle into a critical situation in the given scenario. The subsystem may react incorrectly to the given scenario with regard to at least partially autonomous operation of the vehicle.

[0019] In particular, the functionality test can assess the reliability and / or failure rate of the subsystem. This can include whether the subsystem influences safety-relevant functions of the vehicle. If the subsystem affects a braking system, steering system, or safety system, the subsystem should have a reliability of at least 95%, especially 97%.

[0020] If this is not the case, functionality would be impaired. When assessing the failure rate, it is possible to predict how many failures per unit of time (Failure in Time) could occur. If the predicted, i.e. potential, failures per unit of time exceed a specified threshold, the functionality of the subsystem is impaired. For example, if the subsystem is used to automatically implement comfort or infotainment settings, it may be sufficient for the subsystem to have a reliability of at least 85%, especially 90%. These examples are not intended to be exhaustive, but rather to provide a brief overview of potential influencing factors when testing functionality.

[0021] Specifically, the subsystems are checked to determine whether they contain data or training data that corresponds to the input data of the current scenario. This allows us to check whether the subsystems have been trained based on training data that corresponds to the current scenario. If at least one subsystem is found to be insufficiently trained for the current scenario, i.e., unable to process all input data relating to the current scenario, then this subsystem can be considered a subsystem that still needs to be trained for the current scenario.

[0022] For example, an accuracy measure can be defined on the system side, with which the training status of a subsystem can be checked. Using this accuracy measure, it can be determined at what point a subsystem is sufficiently trained with regard to a given scenario. For this purpose, it can be checked whether, for example, the training status of at least one subsystem falls below a threshold value for the accuracy measure with regard to the given scenario. If at least one subsystem falls below this threshold value, this subsystem is insufficiently trained.

[0023] If necessary, it can be checked whether a particular subsystem can fulfill or perform a proper function and / or task in the given scenario. If one of these subsystems cannot fulfill its intended function, this subsystem may be insufficiently trained.

[0024] For example, input data relating to the current or prevailing scenario can be provided by the vehicle's or communication units' environment detection systems. This particularly concerns a traffic situation or a current situation of the at least partially autonomously operated vehicle, which in turn has the learning system. In other words, the input data can be environmental data and / or sensor data. This provided input data can be compared using the data sets to determine whether or not the current scenario is already known to the learning system. Should at least a partial discrepancy or a delta be detected, meaning that the provided data sets, i.e. training data sets, differ at least slightly from the current situation or scenario, appropriate training then takes place.

[0025] For this purpose, as already mentioned, the individual subsystems of the adaptive system are considered. For example, before the vehicle begins its journey, the adaptive system can be subdivided accordingly. The individual subsystems, which may be neural networks, can be assigned to different vehicle components, vehicle functions, and / or vehicle systems. In other words, a subsystem can be responsible for at least one function and / or unit of the vehicle in order to be able to perform processing operations, control operations, or other operations related to at least partially autonomous operation of the vehicle.

[0026] For example, the comparison might reveal that the current scenario involves new or modified traffic signs, new vehicle types, new environmental conditions, unfamiliar road users, or other unfamiliar traffic situations. In this case, environmental information, such as images, may not be able to be processed or classified. Thus, the situation is at least partially unknown to the adaptive system. To remedy this, the individual subsystems can train each other.

[0027] For example, the subsystems can operate individual vehicle functions and / or vehicle components at least partially autonomously.

[0028] If it is determined, which can be done, for example, with an electronic evaluation unit, that at least one or more subsystems are insufficiently trained for the current scenario, this subsystem is selected. Furthermore, it is checked which of the other subsystems of the adaptive system has information, training data sets, or knowledge regarding the current scenario. This subsystem, which can already be referred to as the trained subsystem, can now be used to teach or train the subsystem that is insufficiently trained for the current scenario.

[0029] Thus, the proposed method extends the learning system by individually training individual parts or subsystems of the learning system.

[0030] For example, the training of at least one subsystem is based on "AI delta learning." This allows the at least one subsystem to be trained through knowledge transfer. In other words, two artificially intelligent or neural networks mutually develop each other.

[0031] In particular, the present method may be a computer-implemented method.

[0032] In one embodiment, training data of the subsystems is provided or transmitted via interfaces with which the subsystems of the learning system are communicatively networked with one another. The subsystem which has already been trained for the current scenario transmits training data suitable for the current scenario to at least one subsystem via the interfaces. The individual subsystems can thus be networked with one another in order to be able to exchange corresponding information, data and / or training data. As a result, the individual subsystems can benefit from the respective other subsystems, since above all training data can be exchanged in order to be able to train another subsystem using one subsystem. In particular, each subsystem can have at least one interface.Communication with other subsystems or other systems can be carried out via at least one interface.

[0033] To better analyze and train the learning system, the learning system can be "cut up" or divided into subsystems. This is done using subsystems, with interfaces representing points of contact or starting points between these subsystems. Using these interfaces, the subsystems can then be combined to form the higher-level learning system. These interfaces can then act as interfaces.

[0034] In one embodiment, it is provided that after a training process of the at least one subsystem has been carried out, a check is carried out to determine whether the at least one subsystem has been trained for the current scenario and, if not, a new training process of the at least one subsystem is carried out. In this way, a continuous learning process or training process of the at least one subsystem can be carried out. In particular, training of the at least one subsystem takes place until the at least one subsystem has been trained with regard to the current scenario. In this case, training can be divided into training processes. After each training process, a review process can be carried out to determine whether the learning or training success has been achieved. If the training was not carried out successfully, training data from another subsystem of the learning-capable system could be used.

[0035] Specifically, after the training session, especially the first one, it can be checked whether the functionality of at least one subsystem is no longer impaired. If an impairment persists, another training session can be performed.

[0036] In one embodiment, it is further provided that after at least one subsystem has been trained, a functional test of the at least one subsystem is carried out, and if the functional test was carried out successfully, the at least one subsystem is released for use in the adaptive system. If the functional test was not carried out successfully, the at least one subsystem is trained again. This makes it possible for a subsystem that was trained based on a current situation or circumstance to be fully used again in the adaptive system only if the trained subsystem is error-free, i.e., essentially fully functional. For example, as a result of the training, the subsystem may be faulty or defective or exhibit other disruptions.If this faulty subsystem then interacts with the other subsystems in the overall adaptive system, these errors can spread to other subsystems or other systems. To prevent this, after training or learning of at least one subsystem, a review or check of this at least one subsystem is performed. This can include, for example, formal checks and / or plausibility checks. If the subsystem has been successfully trained and, in particular, is fully functional, it can be used again in the adaptive system and, in particular, can communicate and work with the other subsystems again.

[0037] In particular, in the event of a faulty or unsuccessful functional test, a corresponding error message can be generated and, for example, output or provided to a higher-level system and / or the learning system and / or a user of the vehicle.

[0038] In one embodiment, it is provided that after at least one subsystem has been trained, a holistic functional test of the learning system is carried out. If the functional test was carried out successfully, the learning system is released for at least partial operation of the vehicle. If the functional test was not carried out successfully, error identification is carried out. For example, after the subsystem has been trained and a functional test of this subsystem has been carried out, a complete system test of the learning system can be carried out again. Otherwise, for example, the holistic functional test can be carried out in parallel to the individual functional tests of the subsystem. It is also conceivable that the holistic functional test is carried out straight away instead of checking the individual trained subsystems.Since the learning system is used in particular to operate or control a vehicle partially or fully autonomously, a fault-free state of the learning system is essential. To avoid operating the vehicle with a learning system that is at least partially faulty, a comprehensive functional test must be carried out during a training process or any other intervention in the learning system. This can ensure that only the learning system operates the car at least partially autonomously, even if it is fully capable of doing so. However, should an error, uncertainty, or potential error be discovered, error identification is first carried out before the learning system is used to operate the vehicle again. This eliminates critical traffic situations and / or other dangerous situations involving the vehicle can be prevented or avoided.

[0039] If a functional test is detected as unsuccessful, a corresponding error message or warning can be issued in the vehicle. Likewise, a corresponding warning can be transmitted to a service facility outside the vehicle.

[0040] In one embodiment, it is provided that during error identification it is checked whether the trained at least one subsystem influences another subsystem, and if so, the another subsystem is trained on the basis of the trained at least one subsystem.

[0041] Due to the networking of the individual subsystems of the learning system and a, in particular, constant, exchange of information and data between them, it can happen that training or adapting one subsystem results in direct or indirect effects or influences on at least one other. This can in turn be determined by means of error identification or a control process. For this purpose, this, in particular identified, faulty additional subsystem can also be trained or taught accordingly. Subsequently, a new holistic functional test of the learning system and / or individual functional tests of the affected subsystems can be carried out. If the errors have been corrected, the learning system can again be fully released accordingly, in particular for use in the vehicle.

[0042] It would also be conceivable for the corresponding vehicle functions, vehicle components and / or vehicle systems, each of which is controlled or influenced by a subsystem, to be at least temporarily deactivated or put into an inactive state until the learning system could be fully released again.

[0043] In one embodiment, it is provided that the at least one subsystem is trained based on machine learning, in particular unsupervised learning, supervised learning, active learning, or a knowledge distillation method. Using machine learning, the at least one subsystem can be trained or taught based on the provided data sets and / or data or training data of the already trained subsystem.

[0044] In supervised or semi-supervised learning, only a small portion of the data has labels that can be used to classify it into a category. The algorithm is therefore trained with unlabeled data. For example, a model trained with labeled data can make predictions for a portion of the unlabeled data. These predictions can then be incorporated into the training data to train another model, such as at least one subsystem, with this expanded dataset.

[0045] In unsupervised learning, an artificial intelligence, such as at least one subsystem, learns using data that has not previously been manually categorized. This allows data to be clustered, features extracted from it, or a new, compressed representation of the input data to be learned without human assistance. Unsupervised learning is primarily used when initializing a neural network. Furthermore, it can reduce the amount of annotated training data. On the other hand, a previously trained neural network can also be adapted to a new domain, such as a new scenario, by attempting to learn a consistent representation of the data. For example, if there is a domain change from day to night with regard to environmental detection, features that the model learned for the day are also applied at night.

[0046] In active learning, algorithms evaluate the training data for a neural network during the training period, such as situations that have not yet occurred. The selection is based, among other things, on uncertainty measures that estimate how the neural network's prediction will turn out. For example, active learning can reduce the effort required for manually annotating video images because only the training data that is essential for later learning needs to be processed.

[0047] Continuous learning involves developing algorithms that can be expanded with new knowledge without loss of knowledge and without having to retrain the entire dataset. Unlike conventional methods, not all data needs to be available at training time. Instead, additional data can be gradually incorporated into the training process later. For example, a neural network can learn to recognize a newly detected object in the vicinity of a vehicle without forgetting previously learned object recognition processes.

[0048] The knowledge distillation process (knowledge transfer) involves transferring knowledge between neural networks, i.e., between subsystems. Knowledge is usually transferred from a more complex model, also called the teacher, to a smaller model, here called the student. In this case, the teacher would be the already trained subsystem, and the student would be at least one subsystem yet to be trained. More complex models usually have a greater knowledge capacity and thus achieve higher prediction accuracies. Through such knowledge transfer, the knowledge contained in the complex neural network can be compressed into a smaller network, with only a slight loss of accuracy to be expected. The knowledge distillation process can also be used in continuous learning to reduce knowledge loss.

[0049] In particular, with these possible examples of machine learning, a wide variety of learning methods can be applied to train the learning system and / or individual subsystems.

[0050] In particular, “AI delta learning” is used to train the subsystem or other subsystems of the learning system.

[0051] In one embodiment, the adaptive system, and in particular the subsystems, are based on a neural network. In particular, the adaptive system can be based on an artificial neural network. By using a neural network, a vehicle system, and thus an at least partially autonomously operable vehicle, can be implemented particularly advantageously and efficiently.

[0052] For example, the learning system could be a learning vehicle system. In particular, the learning system could be an AI (artificial intelligence) model.

[0053] For example, the learning system can be integrated into a higher-level system or a higher-level algorithm of a vehicle system

[0054] A further aspect of the invention relates to an electronic vehicle system for an at least partially autonomously operable vehicle having a learning system, wherein the electronic vehicle system is designed to carry out a method according to the previous aspect or in a further development thereof.

[0055] In particular, a method of the previously described method is carried out with the electronic vehicle system just described.

[0056] With the help of the electronic vehicle system, which can be a driver assistance system and / or a vehicle guidance system, the vehicle can be operated at least partially autonomously or fully autonomously.

[0057] In particular, the electronic vehicle system serves to control and / or operate the vehicle. For this purpose, the electronic vehicle system can, in turn, comprise the learning system, which can be enhanced using the method mentioned above. With the help of the learning system, the electronic vehicle system can be continuously trained and made more intelligent in an intelligent manner, so that, above all, fully automated driving of the vehicle can be carried out more efficiently and, above all, safely.

[0058] A further aspect of the invention relates to a computer program which can be loaded directly into a memory of a control device of an electronic vehicle system according to the previous aspect, with program means for executing a method according to one of the previous aspects or, in an advantageous further development, when the program is executed in the control device of the electronic vehicle system.

[0059] A further independent aspect of the invention relates to an electronically readable data carrier having electronically readable control information stored thereon, which is controlled in such a way that, when the data carrier is used in a control device of a vehicle system according to one of the preceding aspects, it carries out a method according to one of the preceding aspects.

[0060] Advantageous embodiments of one aspect of the invention may be considered advantageous embodiments of another or all other aspects. The same applies in reverse.

[0061] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0062] The invention also includes further developments of the vehicle system according to the invention and the computer program according to the invention that have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the vehicle system according to the invention and the computer program according to the invention are not described again here.

[0063] The invention also includes combinations of the features of the described embodiments.

[0064] Embodiments of one aspect are to be regarded as advantageous embodiments of the other aspects and vice versa.

[0065] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 ; a schematic representation of a vehicle having a vehicle system with a learning system (neural network); and Fig. 2 ; a schematic sequence of a training process of a subsystem of the learning system from Fig. 1.

[0066] The exemplary embodiments explained below are preferred exemplary embodiments of the invention. In the exemplary embodiments, the described components each represent individual, independently considered features of the invention, which also further develop the invention independently of one another and are thus also to be considered as components of the invention, either individually or in a combination other than that shown. Furthermore, the described exemplary embodiments can also be supplemented by further features of the invention already described.

[0067] In the figures, functionally identical elements are provided with the same reference numerals.

[0068] In the Fig. 1 schematically shows a vehicle 1. The vehicle 1 can be at least partially or fully autonomously operated. For this purpose, the vehicle 1 can have an electronic vehicle system 2, such as a vehicle guidance system.

[0069] In particular, vehicle system 2 can implement a fully automated or autonomous driving mode of vehicle 1, which may be a motor vehicle, according to level 5 of the SAE J3016 classification. An electronic vehicle guidance system, such as vehicle system 2, can also be understood as a driver assistance system that supports the driver in partially automated or semi-autonomous driving. In particular, vehicle system 2 can implement a partially automated or semi-autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification.

[0070] In order to enable at least partially autonomous driving or at least partially autonomous operation of the vehicle 1, one or more neural networks can be used to continuously improve, teach, or train the vehicle system 2. For this purpose, the vehicle system 2 can, for example, have a learning system 3. The learning system 3 can be a neural network, i.e., an AI.

[0071] The adaptive system 3, which is trained to enable at least partially autonomous driving, requires a wide variety of training data. This training data can be used to continuously learn or train the adaptive system 3, and thus the vehicle system 2, for a wide variety of traffic situations.

[0072] This requires, above all, a detection of the surroundings or environment of the vehicle 1, in particular a complete detection of the environment. For this purpose, for example, a detection device or a detection system 5 can be provided, with which in particular all areas around the vehicle 1 can be detected. This data can be made available to the learning system 3, for example, as input data. This data is also made available to the vehicle system 2 in order to be able to carry out corresponding driving maneuvers. Furthermore, the learning system 3, which can be implemented completely or at least partially in the vehicle system 2, is provided with data sets 6 relating to a wide variety of traffic situations or traffic events or scenarios, so that the learning system 3 and thus the vehicle system 2 can be extensively trained or taught in order to be able to react to a wide variety of situations in road traffic.Thus, for example, the data sets, in addition to the respective recorded environmental information of the surrounding area 4, are used as input data for training the adaptive system 3. The training of the adaptive system 3 can, in particular, take place continuously or, for example, before each start of the vehicle 1, i.e., before the beginning of a journey.

[0073] For the operation of the vehicle system 2, in particular for the learning system 3, a computer program 7 can be provided, which can be loaded directly into a memory 8 of a control device 9 of the electronic vehicle system 2. When the computer program 7 is executed in the control device 9, the learning system 3 can be trained or expanded, so that the vehicle system 2, in particular, can be continuously made more intelligent in order to make autonomous driving modes of vehicles more efficient and, in particular, safer.

[0074] When vehicle 1 is moving, information regarding a given scenario 10 can be obtained from a current situation in vehicle 1, particularly using the input data which can be provided, for example, by means of the detection system 5 or from other information sources. Here, it can happen that the vehicle system 2, in particular the learning system 3, is insufficiently trained with regard to this scenario 10, so that there is a risk that incorrect reactions or incorrect driving maneuvers will be carried out, which in turn can endanger road users. Such a situation can arise, for example, if the situation is completely different from the trained training scenarios. This can also happen, for example, if the detection system 5 is faulty, recalibrated, exchanged or replaced.Thus, these input data can at least partially differ from the trained data sets 6, so that the learning system 3 can reach its limits here.

[0075] For this purpose, a comparison or a check of the input data relating to the current scenario 10 with the provided data sets 6 can be carried out, for example by means of an electronic evaluation unit 10. If it is determined during this system-side comparison that the current scenario 10 or the current traffic situation differs from the predetermined scenarios covered by the data sets 6, and thus there is a discrepancy or a delta, appropriate machine learning and thus an expansion of the learning system 3 can be carried out. In order to minimize computing power, reduce computing effort and / or minimize processing time while the vehicle 1 is moving, the learning system 3 is not completely trained or retrained with regard to scenario 10. For this purpose, a corresponding subdivision or division takes place.a division of the adaptive system 3 into several subsystems 12 or subsystems. This results in a selection / division of the vehicle of the adaptive system 3, so that, for example, a dedicated subsystem 12 can be selected for the most diverse functions and / or units of the vehicle system 2. This division into several subsystems, which may be partial neural networks, allows for individual and / or selective training exactly where necessary, without the entire adaptive system 3 and thus the vehicle system 2 having to be fully trained.

[0076] In this case, it can now be determined or recognized whether at least one of the subsystems 12 is insufficiently trained or prepared for the current scenario 10. For example, the subsystem 13 may be insufficiently trained or at least only partially trained with regard to the current scenario 10. Consequently, the functionality of the subsystem 13 with regard to the current scenario 10 may be impaired. This can be the case if, in particular, a training state of the subsystem 13 falls below a threshold value of an accuracy measure. Using the accuracy measure, a training state of a subsystem 12 can be assessed. In order for it to be appropriately prepared for this situation and to be able to react accordingly, this subsystem 13 is now specifically trained.In order to be able to carry out an efficient training process without the learning system 3 being severely restricted and a correspondingly high computational effort being necessary, the subsystem 13 is trained on the basis of at least one further subsystem 14 of the subsystems 12, for example by means of a knowledge distillation process (knowledge transfer), which is machine learning. Several subsystems 12 are also conceivable in order to transfer knowledge or information to the insufficiently trained subsystem 13. In other words, the individual subsystems 12 train each other so that the knowledge of a more extensively trained subsystem can be passed on to less trained subsystems. This allows targeted training to take place at specific points within the learning system 3, so that information can be expanded or supplemented without the entire learning system 3 having to be completely retrained.an extension of the learning system can be carried out.

[0077] To enable such a knowledge transfer or information flow between the subsystems 12, the subsystems 12 can be communicatively networked with each other via interfaces 15. This allows data, especially training data, to be exchanged between the subsystems 12. This results in an intelligent training process that reduces time and minimizes computing capacity.

[0078] The vehicle system 2 can, in particular, be a higher-level system. The vehicle system 2 can, for example, include a driving system, an energy system, a body system, an ADAS / DAS system, an occupant interaction system, and / or a data management system as underlying systems. These systems, in turn, can have various subsystems, which can be controlled or trained, for example, using the subsystems 12. For example, a subsystem can be a steering system, wheel system, attachment system, sound system, outside world communication system, or a perception system.

[0079] In the following Fig. Figure 2 illustrates an exemplary process for training subsystem 12. First, in an optional step S1, training of subsystem 2 is started or commenced with the support of subsystem 14.

[0080] In an optional step S1, the difference between scenario 10 and the scenarios in data sets 6 is determined. In a subsequent optional step S2, the subsystems 12 can be checked so that, for example, subsystem 13 can be identified as the subsystem to be trained. Subsequently, in an optional step S3, subsystem 13 can be trained based on subsystem 14 and / or other information from subsystems 12, the adaptive system, or other information sources. After training or after an initial training process, a check or interim query can be performed. If it is determined here that subsystem 13 is not yet sufficiently trained, training can be continued with step S3 and repeated.This training and verification process can lead to termination if a termination criterion is met—that is, if sufficient training success is still not achieved after numerous runs. This can, for example, inform Vehicle 1 or a user of the vehicle that an autonomous driving mode may be impaired.

[0081] If it is determined that the training of subsystem 13 was successful, the process can proceed to an optional step S5. In this step S5, for example, it can be checked whether the basis or basic structure of subsystem 13 was retained. Above all, a functional test of subsystem 13 can be performed here to determine whether subsystem 13 can be safely used in the adaptive system 3.

[0082] If it is determined that the structure of subsystem 13 has been modified too significantly, a different machine learning algorithm can be used in an optional step S6. The process can then continue with step S3.

[0083] For example, after a training process, a clean measurement or measurement of the individual subsystems 12 can be performed via the interfaces 15. For this purpose, test and / or measurement programs or systems can be used. Data can be acquired from data acquisition hardware, image processing hardware, instruments, or CAN buses. Serial protocols such as SPI (Serial Peripheral Interface) and communications can be used. These can then be used for analysis, for example, with the evaluation unit 11. Another conceivable possibility is that, in addition to the purely system-side control or verification, a final confirmation from the user can be performed.For example, after individual subsystems 12 have been tested accordingly by the learning system 3, the final release regarding the interaction of the subsystems 12 can be carried out by a user, such as a user of the vehicle 1.

[0084] If the functional test has been successfully completed, the process can continue to step S7, for example. A comprehensive functional test of the adaptive system 3 can now be performed. Above all, it can be checked here whether the entire system, i.e., the adaptive system 3, functions after the implementation of subsystem 13. If it is determined here that the adaptive system 3 is functional, the learning process can be terminated in an optional step S8, and the adaptive system 3, and thus the vehicle system 2, can safely perform the corresponding autonomous tasks.

[0085] Should it be determined, when checking the function of the learning system 3, that at least one function of the learning system 3 and thus the vehicle system 2 is impaired, an error identification or error check can be carried out or performed in an optional step S9. For example, it can happen that the newly trained subsystem 13 has effects or influences on other subsystems 12. Thus, the trained subsystem 13 can influence the behavior of at least one other subsystem 16. Accordingly, the process can return to step S2 or step S3, depending on the subsystem in question, so that this further identified subsystem 16 can be trained accordingly.Thus, a continuous process can be carried out here, which is carried out in particular until the learning system 3 and thus the vehicle system 2 are functional.

[0086] In particular, the above-mentioned embodiments show how “AI delta learning” can be carried out for a vehicle system, such as a vehicle guidance system. List of reference symbols 1 vehicle 2 electronic vehicle system 3 learning system 4 Environment 5 Recording system 6 records 7 Computer program 8 storage 9 Control device 10 present scenario 11 electronic evaluation unit 12 subsystems 13 impaired subsystem 14 other subsystem 15 interfaces 16 additional subsystems S1 - S9 steps

Claims

[1] Method for extending a learning system (3) which is trained to operate a vehicle (1) at least partially autonomously, wherein - the learning system (3) is trained on the basis of provided data sets (6) with which various scenarios relating to an at least partially autonomous operation of a vehicle (1) are characterized, characterized by , that - input data relating to a given scenario (10) are provided, - a comparison of the input data relating to the present scenario (10) with the provided data sets (6) is carried out, - if it is determined on the basis of the comparison carried out that the present scenario (10) differs at least partially from the most diverse scenarios relating to the provided data sets (6), subsystems (12) into which the learning system (3) is divided are checked to determine whether the functionality of at least one subsystem (13) of these subsystems (12) is impaired with regard to the present scenario (10), wherein - another subsystem (14) of the subsystems (12), which is not impaired in its functionality with regard to the present scenario (10), is used to train the at least one impaired subsystem (13) for the present scenario (10), whereby the learning system (3) is expanded with regard to the present scenario (10). [2] Method according to claim 1, characterized bythat training data of the subsystems (12) are provided via interfaces (15) with which the subsystems (12) of the learning system (3) are communicatively networked with one another, wherein the other subsystem (14), which is already trained on the present scenario (10), transmits training data suitable for the present scenario (10) to the at least one impaired subsystem (13) via the interfaces (15). [3] Method according to claim 1 or 2, characterized by that after a training process of the at least one impaired subsystem (13) has been carried out, it is checked whether the at least one impaired subsystem (13) has been trained for the present scenario (10), and if not, a new training process of the at least one impaired subsystem (13) is carried out. [4] Method according to one of the preceding claims, characterized bythat after the at least one impaired subsystem (13) has been trained, a functional test of the at least one impaired subsystem (13) is carried out, and if the functional test was carried out successfully, the at least one impaired subsystem (13) is released for use in the learning system (3), and if the functional test was not carried out successfully, the at least one impaired subsystem (13) is trained again. [5] Method according to one of the preceding claims, characterized bythat after the at least one impaired subsystem (13) has been trained, a holistic functional test of the learning system (3) is carried out, and if the functional test was carried out successfully, the learning system (3) is released for at least partial operation of the vehicle (1), and if the functional test was not carried out successfully, an error identification is carried out. [6] Method according to claim 5, characterized by that during error identification it is checked whether the trained at least one subsystem (13) influences a further subsystem (16) of the subsystems (12), and if so, the further subsystem (16) is trained on the basis of the trained at least one subsystem (13). [7] Method according to one of the preceding claims, characterized bythat the at least one impaired subsystem (13) is trained on the basis of machine learning, in particular unsupervised learning, supervised learning, active learning or a knowledge distillation method [8] Method according to one of the preceding claims, characterized by that the learning system (3) is based on a neural network. [9] Electronic vehicle system (2) for an at least partially autonomously operable vehicle (1) with a learning system (3), wherein the electronic vehicle system (2) is designed to carry out a method according to one of the preceding claims. [10] Computer program (7) which can be loaded directly into a memory (8) of a control device (9) of an electronic vehicle system (2) according to claim 9, with program means for carrying out a method according to one of claims 1 to 8 when the program is executed in the control device (9) of the electronic vehicle system (2).

Images