Method, system, and computer program product for training a neural network designed to operate a vehicle and for operating a vehicle with a neural network

Abstract

The invention relates to a method (100), system and a computer program for training (103) a neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) designed to operate a vehicle (10, 11, 12, ... 20, 21, 22), as well as to a method, a system and a computer program for operating a vehicle (10, 11, 12, ... 20, 21, 22) with a neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2). The method (100) comprises training (103), in particular retraining (103'), of a neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) designed to operate (104) a vehicle (10, 11, 12, ... 20, 21, 22), wherein the training (103), in particular retraining (103'), is carried out depending on: - first data relating to one or more first errors (FF10, FF11, FF12) of a vehicle functionality of one or more first vehicles (10, 11, 12); and - second data relating (FF20, FF21, FF22) to a vehicle functionality of one or more first vehicles (20, 21, 22); wherein the training (103) is carried out with respect to an interaction of a first vehicle (10, 11, 12, ...) with a second vehicle (20, 21, 22) in a driving situation.

Description

[0001] The invention relates to a method, a system, and a computer program product for training a neural network designed to operate a vehicle, to such a neural network, and to a method, a system, and a computer program product for operating a vehicle with a neural network. Operating the vehicle can relate in particular to a driver assistance system, an active safety system, a system for at least partially automated driving or parking, and / or a system for signaling to the vehicle's surroundings.

[0002] It is an object of the invention to provide an improved method, a device, a system and a computer program for training a neural network designed to operate a vehicle and / or to operate a vehicle, in particular a driver assistance system, an active safety system, a system for at least partially automated driving or parking, or a system for signaling to the environment of the vehicle.

[0003] According to a first aspect of the invention, a (first) method for training or retraining a neural network designed to operate a vehicle is provided. The training or retraining is carried out depending on: first data in connection with one or more first errors in a vehicle functionality of one or more first vehicles; and second data in connection with one or more second errors in a vehicle functionality of one or more first vehicles. The training or retraining can be carried out with respect to an interaction of a first vehicle with a second vehicle in a (e.g., specific, corresponding to specific features or merely possible or potential) driving situation.

[0004] Furthermore, the method may comprise operating at least one (the same or further) vehicle depending on the neural network trained in this way.

[0005] The first data and / or the second data can (likewise) relate to an interaction between the respective vehicles. For example, the first data and / or the second data are structured and / or processed according to respective interactions and / or taken into account during training. At least part of the first data and / or the second data can (in each case) relate to interactions between the (respective) first vehicles and (any other vehicles, in particular those different from the second vehicles, e.g. third vehicles). In other words, they do not have to be exactly the same vehicles. For example, the data from more than 10 2 , 10 3 , 10 4 , 10 5 10 6 , 10 7 different driving situations and / or interactions in the past are determined and / or taken into account during training (directly or indirectly).

[0006] An interaction can be understood as a unilateral or mutual, particularly multilateral, influence. In particular, the interaction can include or be: - influencing a decision, movement, operating mode of the other vehicle; and / or - participation in the same manoeuvre, in particular an overtaking manoeuvre, an evasive manoeuvre, a shunting manoeuvre; and / or - a significant role of a vehicle (e.g. as a vehicle in front, behind, sideways moving vehicle, an unexpectedly appearing vehicle) in the maneuver of one of the other vehicles; and / or - a disturbance, a jostling, a near miss and / or an increase in a degree of risk for another vehicle or vice versa; and / or - a signal (e.g. the intended turning direction, light signals, sound signals) concerning the other vehicle.

[0007] The respective first data and / or second data can determine and / or process a (possible and / or probable, occurred or not occurred) consequence depending on the respective parameters of the interaction, in particular type, pattern, risk. In particular, this can include the first data and / or second data including or taking into account the latter parameters. For example, the training, in particular retraining, of the at least one neural network can be carried out depending on the one or more parameters of the respective interaction.

[0008] For example, the first data and / or the second data can be assigned to interactions corresponding to certain characteristics and / or parameter value ranges and / or processed depending on these (or on the result of the assignment).

[0009] The method preferably comprises providing the trained, in particular retrained, neural network for operating the first vehicle and / or one or more second vehicles, and / or operating the first vehicle and / or one or more second vehicles (preferably in a coordinated manner) depending on the output layer of the (thus) set up, trained and / or retrained neural network.

[0010] The operation of the (first and / or second) vehicle can include or be a determination or adaptation of the parameters, functional logic, and / or behavior of the vehicle, in particular of the driver assistance system, the active safety system, and / or the at least partially automated driving (e.g., in a similar driving situation or relating to a possible similar interaction), in particular a dynamic and / or temporary (e.g., related to the time interval of a possible driver error). For example, a dynamic operation that can be updated at short time intervals, in particular only or essentially only applicable to certain road sections and / or situations, or road sections and / or situations characterized by certain features, can take place depending on the output layer of the neural network.

[0011] The training of the neural network and / or the operation of the first vehicle and / or the one or more second vehicles can, for example, be designed or optimized to achieve the most anticipatory effect possible (for one or a specific driver). This can create an effect that is pleasing to human perception and practically useful.

[0012] By training and / or operating the at least one neural network, particular advantages (e.g. concerning the effectiveness of the method) can result, since a neural network trained (according to the features of the invention) can handle non-trivial or very complex processes that are otherwise typically not comprehensible or predictable on a rule-based basis.

[0013] When operating the first vehicle and / or the one or more second vehicles (with the neural network fully trained or at least trained to a certain extent), the data can be taken into account based on one or more parameters related to the driving operation. These can be used (e.g., together with other data) to control the input (the input layer) of the neural network.

[0014] The data based on one or more parameters relating to driving operation can (in whole or in part) characterize the same or similar quantities as the respective first data and / or second data and / or have the same or corresponding characteristics with respect to the second vehicle or the second driver of the second vehicle. The data based on one or more parameters relating to driving operation are understood in particular as predicted data (e.g., data valid before and / or during a possible second driver action or for the time interval of the second driver action).

[0015] The data based on one or more parameters relating to driving operation can be used to control the at least one neural network (trained based on the first data). For example, the data based on one or more parameters relating to driving operation are read (with or without a processing step) into the input layer of the at least one neural network.

[0016] One or more similarity measures between the first data and / or the second data, on the one hand, and the data based on one or more parameters relating to driving operation can be determined and taken into account. For example, the input layer of the neural network is (also) trained depending on a similarity measure between the (e.g., respectively current) data based on one or more parameters relating to driving operation with the first data and / or second data (e.g., corresponding at least with respect to one or more features, parameters, patterns, or types of driver action) used to train the at least one neural network.

[0017] Alternatively or additionally, one or more patterns of the parameters in the second data can be determined and compared with one or more patterns in the first data, and the result of the comparison can be taken into account. For example, the input layer of the neural network is (also) controlled depending on the result of the comparison.

[0018] Preferably, the method can be executed entirely or predominantly automatically, dynamically, and / or driver-specifically or driver-group-specifically. In the simplest case, driver groups can be considered that are characteristic of drivers who predominantly drive the vehicle manually or of drivers who predominantly drive the vehicle automatically. In this case, types and / or features and / or (respectively active) operating modes of the first vehicle functionality and / or the second vehicle functionality can be determined and considered.

[0019] In particular, the method (due to the described training of the neural network) can have the objective technical effect that (at least in the result) typical and / or probable errors of vehicle functionalities of different types, which can (at least potentially) lead to (undesirable) consequences, especially in an (e.g. unfavorable) combination, are appropriately taken into account.

[0020] The term "neural network" refers, in particular, to a deep neural network. Alternatively or additionally, the term also refers to another form of artificial intelligence (in particular, one comprising a neural network or a comparable structure). In particular, the training and / or operation of at least one neural network is understood to mean a corresponding application of other types of machine learning.

[0021] For example, the at least one neural network, the training and / or operation thereof can comprise or be an analogous application of one or more of the following conditions or combinations thereof: supervised learning, unsupervised learning, reinforcement learning, multitask learning, classification and regression trees, support vector machines, logical and relational learning, probabilistic graphical models, rule learning, instance-based learning, latent representation, bio-inspired approaches.

[0022] Preferably, the one or more neural networks described in this document (selectively) have one or more features, modules, or variations of a transformer architecture and / or a generative adversarial network (GAN). In particular, the neural network comprises at least one or more modules (e.g., optimized for sequential data) of a transformer and / or a generative adversarial network and / or a generative (application-oriented) model or an LLM (large language model). For example, the at least one model described in this document is a dynamic (application-oriented) model.

[0023] Particularly preferably, the one or more neural networks described in the present document comprise a generative pre-trained transformer and / or a generative, in particular statistical and / or application-oriented, model, or are constructed on the basis of such a model and / or are operated by such a model. For example, the at least one neural network selectively has one or more features of a transformer architecture and / or an LLM (large language model), wherein the neural network or the underlying Foundations model, alternatively or additionally configured to process speech information, is in particular trained to read in reaction data and / or parameters corresponding thereto and / or to operate the first vehicle and / or the second vehicle.

[0024] Preferably, the one or more neural networks underlying the functionalities described in this document are so-called weak artificial intelligence, application-oriented and / or (narrowly) specialized neural networks or models established on their basis.

[0025] The operation of the at least one neural network preferably comprises a generative and / or transformative process. In particular, it can be a generation (generative process) in which the information generated (at the output of the neural network) contains almost no or little (e.g., less than 1, 2, 5, 10, 20, 30%) of the original information (e.g., the information at the input of the neural network with which it is operated) or corresponds to it; and / or comprises a high proportion (e.g., more than 60%, 80%, 90% or a factor of 2, 5, 10, 50, 100) of the information that is new (compared to the information at the input of the neural network) and / or does not correspond to it.

[0026] In the context of this document, the term "generation" is to be understood in particular as the generation of information for generating the respective result or intermediate results (e.g., the control information, adaptation information, and / or setup information), in particular the information underlying the result or intermediate result. For example, the generation of the control information can be understood as the generation of the information from which (e.g., within the scope of an appropriate intermediate step) the respective control information (e.g., converted into a usable form or format) is or can be determined.

[0027] In particular, the at least one neural network is a neural network configured to operate a driver assistance system, an active safety system, a system for at least partially automated driving or parking, or a system for signaling to the surroundings of the vehicle of one or more vehicles or one or more steps of the method. For example, the at least one neural network (in a simplified case) comprises at least 6, 7, 8, and / or at most 16, 18, 30, 100, 300, or 500 layers. An (application-specific) model or generative pre-trained transformer can have a number of approximately 1-100, in particular 5-50 billion parameters. It can comprise one or more convolutional layers.

[0028] A plurality of neural networks, in particular generative pre-trained transformers or models, can be used, in particular those that are trained and / or interconnected independently of one another (e.g., directly or indirectly interconnected to operate one or more (first and / or second) vehicles). Depending on the variant of the method, the features described in this document can relate to one or (selectively or each) several of the neural networks. For example, these can be (each or only) selectively configured to execute certain steps of the method or can relate to neural networks used.

[0029] The term "training" refers in particular to machine learning, non-rule-based learning and / or deep learning, or the execution of machine learning, non-rule-based learning and / or deep learning. Training can include or be supervised or unsupervised training. Training (or setting up) the at least one neural network can include or be end-to-end learning and / or reinforcement learning. In this case, several, in particular a predominant number, or (essentially) all, of the steps necessary for achieving one or more goals or intermediate goals (necessary intermediate steps) can be mapped or integrated into the (at least partially trained) neural network.

[0030] For example, training or retraining may include selecting and / or adjusting the parameters relating to supervision and / or reinforcement, hyperparameter optimization, performing dynamic adaptation of the training data, data augmentation (e.g., computational augmentation of the training data), performing reinforcement learning feedback, rewarding and / or penalizing the neural network, adjusting the feedback and / or interpreting feedback.

[0031] Training the neural network is, in particular, also to be understood as setting up the neural network. Operating the neural network is, in particular, controlling (also: stimulating or prompting) the neural network, in particular the input layer (one or more inputs) of the neural network. Furthermore, changing the operating mode of the at least one neural network can also be considered operating the neural network. For example, operating the neural network can include or be prompting a Foundations model.

[0032] Preferably, the training is carried out with the aim of avoiding a possible undesirable situation, interaction and / or consequence in the context of a (future possible) driving situation caused by a (future possible) first error of a first vehicle functionality of a first vehicle and / or a second error of a second vehicle functionality of a second vehicle.

[0033] For example, the target may include or be a target value. The target or target value may include or characterize a probability value. For example, the target or target value may be selected, determined, or adjusted depending on one or more values specified or conditioned by the functional safety and / or safety in use and / or a selected standard value for the operation of the vehicle, in particular one or more limit values for one or more probability measures and / or consequence measures in connection with the occurrence of a fault.

[0034] For example, the target or target value can be selected depending on a (statistically relevant) size of an occurrence of the comparable interaction during (fully or predominantly) manual driving.

[0035] For example, the corresponding values can be determined or adjusted in particular depending on values specified (e.g. by law and / or according to a standard and / or according to the objective of the manufacturer and / or operator of the vehicle).

[0036] For example, it may be the functional safety and / or safety in use and / or the standard value for the operation of the vehicle and / or the functional safety and / or safety in use and / or the standard value for the operation of a vehicle functionality of the vehicle for at least partially automated driving and / or parking.

[0037] For example, a target value can be selected depending on a parameter relating to functional safety and / or safety in use and / or the standard value of a (corresponding or related) functionality. For example, a standard value can be selectable. For example, the target value can be determined depending on a value predetermined or conditioned by the functional safety and / or safety in use and / or the standard value, in particular depending on limit values for one or more probability measures and / or consequence measures. The probability measures and / or consequence measures can relate to errors of a certain type and / or specific errors. A target value can be adjusted by a certain (e.g. multiplicative, additive or logarithmic) factor from the desired requirement to meet functional safety and / or safety in use and / or the standard value.

[0038] In the context of this document, the term “target value” can always also be understood as a target value range, e.g. a value range that is limited upwards and / or downwards and / or dependent on another variable (such as a parameter). The target value can also be understood as striving for a (suitable) optimum of the respective measure. Such an optimum is in particular a local or global minimum or maximum of the respective measure (in particular to be understood as a suitable approximation to the minimum or maximum). The achievement of one or more target values can therefore also be understood as an optimization. Several target values or an overall optimum can be aimed for and in particular achieved.

[0039] The training can be carried out depending on the data from one or more previous executions of the method (e.g. operating cycles of the method), in particular individual or selected steps of the method and / or depending on data from interactions of the one or more first vehicles or one or more second vehicles and / or the effect or target values achieved thereby.

[0040] In the context of this document, the term "driving situation" can be understood, for example, as a situation (particularly at the level of specific objects) characterized by an arrangement, action, or interaction of road users or by certain driving parameters of road users, or as a specific characteristic (e.g., pattern). In particular, the meaning of the term "driving situation" differs from the frequently colloquially used meaning of the term "traffic situation," which rather corresponds to summary, general, and / or statistical categories such as "free traffic," "heavy traffic," "slow-moving traffic," "traffic jam," "end of a traffic jam," etc.

[0041] A driving situation can be described by driving situation parameters. One or more driving situation parameters can characterize, in particular represent, a specific pattern (also understood as a data pattern), e.g., a pattern characterizing the arrangement and / or speed of objects and / or a pattern of driving situation parameters. The driving situation can also be characterized by a spatial pattern of the so-called open spaces in the vehicle's surroundings or by corresponding parameters.

[0042] Preferably, the at least one driving situation can be characterized by one or more of the following features (or parameters of the driving situation corresponding to these): - a (certain) spatial distribution of road users and / or the movement parameters of road users, in particular an arrangement pattern of road users in the environment of the (real) vehicle; - a (certain) spatial distribution of immobile objects in the vicinity of the vehicle; - a relative position and / or movement parameters to certain types of lane markings, traffic signs, traffic lights (not necessarily to specific traffic lights, etc.); - information about the right of way of the vehicle, in particular vis-à-vis certain road users and / or road users who are actually or at least potentially coming or may come from certain directions, e.g. a crossing road from the right or from the left; - information about an action by a road user in the vicinity of the vehicle, e.g. exceeding a limit value, such as honking, flashing lights, pushing, overtaking the vehicle, attempting to overtake and the like.

[0043] Furthermore, the driving situation may be characterized by one or more parameters related to relevant traffic rules, traffic signs, right of way, traffic lights and / or traffic light phases.

[0044] The driving situation described in this document may include any combination of the described characteristics.

[0045] Preferably, the at least one driving situation may be one or more driving situations that exceed certain limit values, or a driving situation characterized by parameters that exceed certain limit values. For example, the at least one driving situation may be a driving situation involving an undesirable or dangerous approach to an object or road user, an acceleration value that exceeds a limit value, an undesirable arrangement relative to other road users, etc.

[0046] The driving situation may be a special situation (e.g., one that occurs relatively rarely) or a (at least potentially) dangerous driving situation, e.g., a driving situation for which an increased risk is identified or assumed. For example, the method may include recognizing such a driving situation or a driving situation typically preceding such a driving situation and incorporating the recognized information into the method. For example, one or more steps of the method may be initiated depending on the recognized information, or their execution or execution parameters, mode, etc. may be varied.

[0047] As one or more parameters of the driving situation, a temporal and / or spatial change characteristic, in particular a gradient, such as a temporal and / or spatial gradient of the respective parameter, in particular parameter value, can (also) be considered or taken into account.

[0048] For example, the driving situation parameter(s) can be taken into account based on or relating to environmental sensor data and / or information transmitted to the vehicle (e.g., from another road user, via a Car-2-Car or Car-to-X system). The environmental sensor data can be data processed in a specific manner from an environment-detecting sensor, in particular a sensor system, of at least one vehicle.

[0049] For example, the first data (also) characterizes one or more pieces of information about one or more technical parameters of the first vehicle and / or the settings and / or operating logic of the first vehicle active at the time of the driver action, in particular a driver assistance system, active safety system, and / or system for at least partially automated driving of the (respective) first vehicle. The first data (determined for training the neural network) can be acquired, selected, processed, and / or used during training depending on technical parameters of the first vehicle and / or depending on the settings and / or operating logic of the first vehicle active at the time of the driver action.

[0050] For example, the respective first data and / or second data can be determined and / or processed depending on the respective parameters of the driving situation. In particular, this can include the first data and / or second data including or taking into account the latter parameters. For example, the training, in particular retraining, of the at least one neural network can be carried out depending on the one or more parameters of the respective driving situation.

[0051] The existence of a driving situation can generally be assumed by the approach of one or more first vehicles and one or more second vehicles (e.g. to the same intersection, roundabout, entry point, exit point, etc.).

[0052] In the context of this document, the term "data" is understood in particular as dependent, particularly data determined on the basis of the data, especially data sets. The term "characteristic" is to be understood in particular as representing or characterizing. For example, it can refer to encoded data (according to a predetermined dependency).

[0053] A first error and / or a second error can be a (e.g. reproducible or non-reproducible) functional error or a specific error (as an incorrect execution of the vehicle function or one that deviates from the norm and / or from an expectation). The (first and / or second) error in the (respective) vehicle functionality can be a technical error, a technically caused error and / or an error that is at least partially caused by the vehicle's environment and / or at least partially caused by a driving action of the vehicle user, in particular the driver of the vehicle. Preferably, parameters of the (first and / or second) error can be determined and / or taken into account. These can characterize qualitative variables (e.g. type of driver) and / or quantitative variables (e.g. a type, direction, tendency, intensity or degree of error).

[0054] The combination of the first error and the second error can, for example, include or be any sequence, order, and / or accumulation and / or chain of errors in the first and / or second vehicle system. The chain of errors can, in particular, include more than two or more than three chained errors. The method can offer a particular advantage in this case, since conventional or rule-based methods can hardly handle a chain of errors.

[0055] Furthermore, the combinations of one or more first errors of the first vehicle system (one of the first vehicles) and the second error of the second vehicle functionality (one or more second vehicles) that occurred at least partially within a (relatively short) time interval (e.g., within 0.5; 1.2; 5; 8 seconds) or with a temporal overlap (e.g., appeared with regard to their effect) can be understood. Alternatively or additionally, the one or more first errors and one or more second errors can form an error combination if they occur in connection with the same maneuver or a sequence of maneuvers or maneuver phases.

[0056] Preferably, at least one neural network is trained, in particular retrained (primarily): - combinations of errors or combinations of errors corresponding to certain characteristics (e.g. parameter value ranges, types, patterns, time intervals between at least one first error and at least one second error): - to avoid and / or reduce the probability and / or the degree of consequence in the event of their occurrence, in particular in accordance with a predetermined or specifiable objective or target value.

[0057] The training in relation to an interaction between one (or more) first vehicles (on the one hand) and one (or more) second vehicles is to be understood in particular in such a way that: - an unplanned interaction may occur; and / or - the interaction will change significantly (e.g. become more or less critical with regard to its possible risks, consequences, etc.); and / or - the occurrence of one or more first errors changes the significance, risk, consequence level of one or more of the second errors (e.g. with regard to the interaction); and / or - the occurrence of one or more second errors changes the significance, risk, consequence level of one or more of the first errors (e.g. with regard to the interaction).

[0058] The training of the neural network can relate to a possible consequence (also understood as a result or consequence) of an (error) combination of one or more first errors in the first vehicle functionality (also: the first vehicle system) and one or more second errors in the second vehicle functionality (also: the second vehicle system). Different measures of the consequence can be considered. These can, for example, characterize the type and / or intensity of the consequence (e.g., reduced comfort, risk, disadvantage, etc.).

[0059] In particular, the one or more of the first errors and / or the one or more of the second errors relate to (e.g. at least partially automated or assisted) vehicle guidance of the respective (first or second) vehicle.

[0060] First errors and / or second errors can therefore include errors that led to a consequence (e.g., a certain consequence or one that exceeds a certain degree), or errors that could at least potentially have led to a consequence (e.g., with a certain degree of probability). The training data can be determined and / or the training process can be carried out (also) depending on the (first and / or second) errors that led to no or only minor consequences. This allows a sufficient amount of training data to be generated (without having to wait for real disadvantages to occur in the operation of the vehicles).

[0061] For example, first errors and / or second errors may be errors that only under certain circumstances and / or potentially lead to an (e.g. undesirable) consequence, but in particular have not led to it.

[0062] The respective (first) functionality of the one or more first vehicles and / or the respective (second) functionality of the one or more second vehicles may (each) include or be a functionality for executing an assisted or fully or partially automated driver and / or signaling to the surroundings of the respective vehicle (e.g., through light, symbols, and / or sound). These may each be the same, similar, or different functionalities.

[0063] For the purposes of this document, the term "vehicle control" can also include parking or maneuvering (e.g., parking, exiting, maneuvering) as well as the transition between driving and parking. The term "maneuver" can include, for example, a lane change maneuver, evasive maneuver, overtaking maneuver, turning, exiting, or entering (e.g., expressways). The first driver error affecting at least one vehicle control during a first driver action can be a driver error that affects the control of the vehicle in some way, for example, occurs during the control of the vehicle and / or affects the control of the vehicle.

[0064] Preferably, the first errors and / or the second errors are determined from a plurality of vehicles and / or a plurality of journeys and / or a plurality of driving situations and / or (driving) contexts and / or at different locations (i.e. also: positions).

[0065] One or more of the respective data (first data or second data or respective sequences) may refer to a time interval in which the error occurred and / or to a time interval preceding this time interval.

[0066] In particular, the first data and / or second data may be determined depending on statistical clusters and / or patterns of occurrences and / or non-occurrences of the (first and / or second) errors of certain types or may include information about such clusters or patterns.

[0067] Within the scope of the method, first data and / or second data relating to a plurality (e.g. hundreds, thousands, millions, many millions) of first and / or second errors (preferably as corresponding, in particular sequences and / or structured data sets, in particular from sequences) can be determined (e.g. in advance). These can be determined using a backend server, stored and / or processed by this server. The first driver actions can have occurred in one or more different (possibly identical) vehicles. On the basis of the data, frequency information relating to the frequency of a specific driver error in the plurality of first driver actions can be determined.

[0068] A first error and / or a second error or the first data and / or the second data (belonging to a respective data set) may relate to a (preferably the same) interaction of the vehicles in the same driving situation.

[0069] For example, the first errors and / or the second errors can include or be a deviation (e.g. of approximately 20%, more than 30%, 50%) of a parameter relating to the guidance of the vehicle (e.g. speed, acceleration, distance to one another, to other objects, to lane markings, to time parameters, in particular with regard to traffic light phases, etc.) that exceeds a certain limit value or standard value. This can include or be, for example, lateral guidance of the vehicle, e.g. a position within the lane or a departure from the lane or roadway. The driver error can relate to the (e.g. direct or indirect) execution of a driver action (e.g. at least partially manually executed and / or at least partially automatically executed).

[0070] Alternatively or additionally, error-free, in particular within a certain tolerance, error-free and / or precisely executed vehicle control (on the part of the vehicle functionality and / or the driver) can be detected. The first data and / or second data determined (alternatively, preferably additionally to be considered) can relate to error-free, in particular within a certain tolerance, error-free and / or precisely executed driver actions. For example, these data can be processed or considered as comparison data or reference data during training, in particular retraining, of the at least one neural network.

[0071] The detection of a driver action that was executed error-free, in particular within a certain tolerance, error-free and / or precisely, and / or the corresponding first data and / or second data can be taken into account according to one or more features of the method. The one or more first and / or second errors and / or the non-occurrence of the first and / or second errors or the detection of sufficient precision can relate (also only) to the actions of the vehicle functionality, in particular the contribution of the vehicle functionality to the control of the vehicle (e.g., to the overall result of the driver's interaction with the functionality).

[0072] The term "operating" one or more first vehicles and / or second vehicles can be understood in particular as controlling, changing an operating mode, and / or varying one or more operating parameters (in particular also to be understood as initiating corresponding, in particular specific changes). For example, operating the (respective) vehicles comprises controlling, changing the operating modes and / or varying one or more parameters of a vehicle system, in particular to be understood as one (e.g., the same and / or different) vehicle functionality. In particular, operating can include or be effecting or adapting a (e.g., specific or expedient) measure.

[0073] The operation of the first vehicle and / or the second vehicle can comprise or be: operating a vehicle guidance system (e.g. longitudinal guidance, lateral guidance, external signaling) and / or initiating, adapting, changing or suppressing a (arbitrary) measure. This can occur depending on the output layer of the neural network (this is particularly to be understood as: directly or indirectly controlled by the output layer of the neural network). This can comprise or be a selection and / or parameterization of one or more measures, an initiation, a deactivation, in particular a (e.g. temporary) suspension of the measure in a specific case and / or determining or adapting one or more parameters of the measure.

[0074] Preferably, the neural network can be trained, in particular retrained and / or operated in the vehicle for predictive control, changing an operating mode, and / or varying one or more operating parameters or initiating corresponding, in particular specific, changes, in particular a change to one or more vehicle systems.

[0075] The term "neural network" refers to (at least) one or more neural networks. The neural network can comprise or be a portion (e.g., some specific layers) of a larger neural network. In particular, the neural network can be understood as the sum of several neural networks, in particular those that are trained and / or interconnected in a mutually dependent manner (e.g., directly or indirectly interconnected to operate the first vehicle and / or one or more second vehicles).

[0076] The at least one neural network can be a neural network of a driver assistance system, an active safety system, a system for at least partially automated driving or parking, or a system for signaling to the vehicle's surroundings, or a neural network for operating one or more of these systems (or functionalities) of the first vehicle and / or the one or more second vehicles. However, it can also comprise or be a neural network that is (at least partially) separate or decoupled from the vehicle functionalities of these types (intentionally, intentionally).

[0077] Depending on the output layer of the neural network, a (same or different) vehicle functionality, in particular a driver assistance system and / or an active safety system and / or a system for at least partially automated driving or parking and / or a system for signaling to the vehicle's surroundings, can be operated, in particular controlled. Alternatively or in addition to initiating an action (e.g., intervening in the vehicle's chassis, lights, or horn), a property of the vehicle system (e.g., its reaction or response to events that have not yet occurred) can be changed, at least temporarily.

[0078] For example, depending on the output layer of the neural network, a sensitivity and / or reaction of the first vehicle and / or the one or more second vehicles (e.g., corresponding driver assistance systems, active safety systems, systems for at least partially automated driving) and / or a system for signaling the vehicle to its surroundings to certain events of the first vehicle and / or the one or more second vehicles (e.g., in certain situations) can be changed, in particular adapted. This can be done in a predictive manner.

[0079] One or more steps of the method can be carried out, in particular end-to-end, using a correspondingly trained and / or operated neural network. Of course, this does not exclude the use or concurrent use of other means (e.g., one or more computer-implemented calculation formulas, rule-based algorithms, etc.). For example, at least part of one or more steps of the method can be carried out (at least in part) using other means. Preferably, a substantial part of the processing of the data described in the present document and / or a substantial part of the effect of the method can be carried out by means of the at least one neural network.

[0080] The method also does not require direct or indirect control of the first vehicle and / or one or more second vehicles depending on first data and / or second data and / or a result of an aggregation of the first data with the second data.

[0081] Training the neural network is particularly to be understood as setting up the neural network. Operating the neural network is, in particular, controlling the neural network, in particular the input layer (one or more inputs) of the neural network. Furthermore, changing the operating mode of the at least one neural network can also be considered operating.

[0082] The training of the at least one neural network can include or be end-to-end learning and / or reinforcement learning. Several, in particular a majority, or (essentially) all intermediate steps necessary for achieving one or more goals can be mapped or integrated into the (at least partially trained) neural network.

[0083] Training can be carried out for one or more predetermined, specifiable and / or procedurally determined goals.

[0084] The goal of training (especially to be understood as the goal of machine learning) can include or be: - Improvement of a qualitative and / or quantitative indicator that characterises the success of vehicle operation and / or the resulting driving; and / or - improved consistency between the result of vehicle operation and the motivation and / or expectations of the respective driver; and / or - a reduction in the number and / or severity and / or risk of driver errors; and / or - a reduction of a risk measure, e.g. a risk measure resulting from the journey, per journey distance; and / or - a reduction in the consequences of the driver errors (e.g. one or more consequences occurring after the driver error or as a result of a possible, probable and / or predicted first error and / or second error).

[0085] A target value can be specified or can be specified for one or more goals (e.g., the ones mentioned above). For example, a first target value can be specified when training the neural network and / or a second target value can be specified during operation (when using the trained neural network in operation).

[0086] For example, the invention comprises a (first) method or part of the method for training the neural network and / or a (second) method or part of the method in which the trained neural network is operated (for operating the vehicles in commercial mode). The operation comprises, for example, active user operation, wherein the neural network (e.g., trained at least to a certain extent) is used according to a feature of the invention. The method can also comprise both of these methods, which can be carried out, for example, simultaneously, sequentially, or alternately.

[0087] Training can be carried out in multiple phases, over a long period of time, and / or across multiple executions or operating cycles of the process. This can be initiated or continued after the respective vehicle has been delivered to (end) users.

[0088] The method may comprise a change, in particular a long-term, slow update of the neural network encompassing a plurality of operating cycles, depending on a plurality of the parameters described in this document. The training of the at least one neural network may be carried out at least partially, in particular only or predominantly automatically, during the operation of an already at least partially trained neural network and / or in a separate process.

[0089] Training may include moderation. Moderation may be performed depending on the parameters determined in the process. For example, moderation may be performed depending on the degree to which a target value is missed (e.g., for different types of driver actions, contexts, driving situations, and / or by different drivers).

[0090] The method may comprise selecting and / or excluding the sections of the first data to be processed for training the at least one neural network according to one or more predetermined criteria. Alternatively or additionally, the neural network may (also) be trained with other data, wherein the training process is operated, in particular controlled, depending on the first data.

[0091] It may include selecting one or more sections of the first data and / or the second data, excluding one or more sections, and / or dividing the first data and / or the second data into several (different) categories. These (e.g., sections assigned to different categories) can each be used differently for training and / or operating the at least one neural network.

[0092] The selection, exclusion and / or division into several categories can be carried out depending on the achievement of the target or target value corresponding to the first data and / or the second data (which, within the scope of this document, also includes data sections based on the first data or second data) in one or more first driver actions.

[0093] Training (especially to be understood as the process of training) can include or be: - Determining training data, in particular on the basis of the first data sections and / or the second data sections and / or further data; and / or - Determine the backpropagation; and / or - Selecting or adjusting a parameter of the training process.

[0094] The training can be carried out depending on the data from one or more previous executions of the method (e.g. operating cycles of the method), in particular individual or selected steps of the method and / or depending on data from previous uses of the one or more vehicles and / or the goals, intermediate goals or target values achieved in the process.

[0095] One or more aspects of training described above may be varied depending on one or more first errors and / or second errors or depending on the first data and / or second data related to the respective errors.

[0096] Furthermore, the method can consider or take into account a third error (initially possible, probable, or predicted) of a vehicle functionality that occurs or can occur when the vehicle is operated with the neural network (initially possibly, probably, or according to a prediction). This error can have the same characteristics as a first error. Furthermore, the method can consider or take into account a fourth error (initially possible, probable, or predicted) of a driver of the vehicle that occurs or can occur when the vehicle is operated with the neural network (initially possibly, probably, or according to a prediction). This error can have the same characteristics as a second error.

[0097] In the method, a prediction of a (possible and / or probable) third error and / or fourth error and / or a combination of a third error and a fourth error can be made and / or (subsequently) a consequence of the third and / or fourth error can be avoided or reduced. Alternatively or additionally, the method can comprise executing, suppressing and / or varying a measure. The measure can be aimed at improving the interaction of the vehicle (e.g., the vehicle functionality) and the driver of the vehicle. In this case, the measure can be used to reduce a consequence measure and / or probability measure of a third error and / or fourth error and / or a combination of a third error with the fourth error. This can be done, in particular, without explicitly predicting the third error and / or the fourth error or their parameters. For example, the neural network (end-to-end) can be designed to cause (e.g.,(to be understood as parameterizing and / or initiating) the (e.g., most suitable, optimized) measure. For example, training is carried out on determining or adjusting one or more parameters of the measure.

[0098] The method can achieve an objective technical effect by predicting a (possible and / or probable) error and / or executing, suppressing, and / or varying a measure, even without knowledge of (typically arbitrarily diverse and complex) logical or causal connections between specific causes and specific errors being available or enabled. In particular, the logical or causal question regarding the (typically very complex and / or at least partially subjective) causes of a human error need not be answered.

[0099] This can have the advantage of (at least partially) predictive operation, in particular control, of the first vehicle or the second vehicle. For example, the vehicle, which is operated depending on the (fully or at least to a certain extent trained) neural network, can already react to something and / or be appropriately adjusted before the occurrence and / or possible consequence of a specific (second) error is determined or can be determined (e.g., an otherwise recognizable one). Preferably, an error is already predicted (e.g., a few seconds or fractions of a second) before its expected occurrence, in particular before the second driver action. This can gain (valuable) time for determining, selecting, and executing the measure.In particular, a time advantage (a “lead time”, so to speak) can be gained against a possible third error and / or (further possible) fourth error.

[0100] For example, the at least one neural network (e.g., to be used during operation) can be understood as part of a neural network with data based on the at least one (e.g., at least partially trained) neural network. For example, the data is generated during training of the at least one neural network on any computing unit, in particular the computing unit of the backend, e.g., in order to then make it operable (also understood as executable) on one or more other computing units, in particular in (e.g., respective, specific) vehicles and / or in the backend (e.g., for one or more vehicles).

[0101] In particular, data based on the first data and / or second data are used as backpropagation for training and / or as a basis for one of the input variables for operating the at least one neural network (for carrying out the operation of the vehicles). The backpropagation can be determined depending on the parameters achieved in each case in several different time intervals (e.g. under different conditions) or the corresponding data sections, in particular depending on the comparison of the resulting parameters with a target value and / or provided for training the at least one neural network. For example, the backpropagation of the at least one neural network will (also) be determined depending on the degree of one or more deviations from the one or more (respectively selected) target values.

[0102] The method may comprise or be a (first) method for training, in particular retraining, and / or for operating the at least one neural network and / or a (second) method for operating the (first and / or one or more second) vehicles depending on a neural network trained according to one or more features described in the present document.

[0103] According to a further embodiment, the neural network is trained and / or operated to: avoid one or more first errors in the vehicle functionality of a first vehicle; and / or one or more second errors in the vehicle functionality of a second vehicle, in particular those that can form a combination with one another that is specific or corresponds to specific features, and / or to reduce a probability measure and / or consequence measure of a combination of at least one first error and at least one second error.

[0104] For example, this can be done with respect to certain and / or certain features and / or parameter value ranges corresponding interactions and / or (e.g. possible, probable) consequences.

[0105] According to a further embodiment, the method comprises: determining first data depending on the detection and / or evaluation of a second fault of a vehicle functionality of a second vehicle; and / or determining second data depending on the detection and / or evaluation of a first fault of a vehicle functionality of a first vehicle.

[0106] This allows the method to be executed significantly more efficiently. For example, the acquisition and processing of the first data and / or the second data can be carried out more efficiently by a factor of 10, 100, or 1000. Alternatively or additionally, the quality of the training data or training (which is crucial for the performance of the neural network) can be increased.

[0107] According to a further embodiment, the neural network is trained, in particular retrained, and / or operated to: predict (also: execute the prediction) a first error, in particular an undesirable functioning of a vehicle functionality of the first vehicle in response to a driver error, in particular a delayed, insufficient, inappropriate and / or incorrect driver action of the first driver of the first vehicle; and / or predict (also: execute the prediction) a second error, in particular an undesirable functioning of a vehicle functionality of the second vehicle in response to a driver error, in particular a delayed, insufficient, inappropriate and / or incorrect driver action of the second driver of the second vehicle.

[0108] The driver error of the driver (in each case of the same vehicle) can be a driver error, in particular also to be understood as an error made by the driver (in particular also to be understood as: caused or permitted) in connection with the execution of a driver action.

[0109] Preferably, the at least one driver error of the driver of the (respective) vehicle (driver error) can be a human error, e.g. a careless error, an attention lapse, a memory lapse, an error of skill, an error of perception, etc. The driver error can be an action by the driver that is unintentional, incorrect, performed differently than desired and / or in expectation of a different result. In particular, a "driver error" is collectively referred to in this document as a part of a driver action that or the result of which does not correspond to the motivation or expectations of the respective driver. It can be a process that is technically correct or flawlessly detected or executed by the vehicle (e.g. steering, braking, accelerating, performing maneuvers, etc.).

[0110] One or more of the (first and / or second) driver errors are, in particular, errors of the user (responsible for the respective vehicle), in particular the driver, which particularly concerns a specific driving action or a driving action of a specific type. The driver error may be an incorrect, missing (e.g., forgotten), and / or inappropriate driving action for the situation, lane section, etc. The driving action may be an action of the driver in relation to the vehicle or an interaction of the driver with the vehicle.

[0111] Examples of different types of driver error can be: a forgotten indicator, an indicator in the wrong direction, an indicator that has not been switched off; a missing or too late look in a certain direction, e.g. in the direction of an exit, priority road or in the direction of another road user, a missing, incorrect, particularly in terms of intensity exaggerated or understated action by the driver, in particular an inappropriate reaction to a situation, e.g. in the surroundings of the vehicle and / or inside the vehicle, a driving situation and / or an action of another road user.

[0112] The (first and / or second) driver errors may be errors that occur due to the driver's distraction, in particular by another road user, by an event near the vehicle, or due to overlooking or near overlooking or an incorrect assumption about the movement of another road user.

[0113] According to a further embodiment, a determination and / or processing of or the first data from one or more first vehicles and / or a determination and / or processing of or the second data from one or more second vehicles and / or a training, in particular retraining of the neural network; executed depending on: - one or more first parameters and / or first patterns of a first error of a vehicle functionality of the respective first vehicle; and / or - one or more second parameters and / or second patterns of a second fault of a vehicle functionality of the respective second vehicle.

[0114] According to a further refinement: - the first data characterising a first sequence of variables having a temporal and / or causal relationship with the occurrence of the first fault in the vehicle functionality of the one or more first vehicles; and / or - the second data characterising a first sequence of the variables which are temporally and / or causally related to the occurrence of the second fault in the vehicle functionality of the one or more second vehicles.

[0115] Preferably, at least a portion of the first sequence and / or at least a portion of the second sequence may be indicative of the quantities that were valid before the occurrence of the respective error.

[0116] According to a further embodiment, the training, in particular retraining of the neural network is carried out depending on: - one or more first absolute and / or relative first positions at which the respective first error occurred and / or at which the respective first data applied; and / or - one or more second absolute and / or relative second positions at which the respective second error occurred and / or at which the second data was valid.

[0117] In the context of this document, the one or more relative positions are to be understood in particular as parameters that characterise a position in relation to other nearby circumstances, in particular with regard to: - Structural features of the road layout, road markings, or parts of the road structure. For example, this can be a relative position within a lane and / or to a centerline, sidewalk, center of a road layout (intersection, roundabout, center of a curve, exit, etc.); and / or - to the other vehicle. For example, for a first vehicle, this would be one or more second vehicles, and vice versa; and / or - other objects, especially obstacles.

[0118] According to a further embodiment, the training data for training, in particular for retraining the neural network, comprise one or more of the following information and / or are determined depending on the following information: - first sequence of the first position data of the first vehicle before the occurrence of the first error; and / or - second sequence of the second position data of the second vehicle before the occurrence of the second error; and / or - a part of map information from the vicinity of the first positions and / or second positions.

[0119] Alternatively or additionally, these data (sequences) can be used when performing the process of training or retraining the neural network.

[0120] The map information may relate to a section (e.g. a useful excerpt) from a digital map and / or from one or more layers of the digital map (e.g. relating to the surroundings of the driving situation or the (possible, probable interaction).

[0121] According to a further embodiment, the operation of the neural network is carried out during (also: for) the operation of a first vehicle and / or during the operation of a second vehicle: - is carried out depending on one or more variables characterising the driving operation of the first vehicle and / or the second vehicle, in particular the first sequence and / or the second sequence.

[0122] “Driving operation” is to be understood as meaning, in particular, commercial operation in which the operation of at least one of the vehicles is dependent on a neural network that has been trained at least to a certain extent.

[0123] The variables characterizing the driving operation of the first vehicle and / or the second vehicle, in particular the first sequence and / or the second sequence of variables, can include, in particular, a time interval from the recent past (e.g. at least 5, 10, 20, 30, 60 seconds before the (new) driving situation) and / or the current time interval (e.g. the almost current value of said variables).

[0124] According to a further embodiment, the variables (e.g. used during training, retraining and / or operation) include one or more variables characterizing the driving operation of the vehicle: - an absolute and / or relative position where the vehicle in question is located and / or is at least expected to be in the near future; and / or - map information concerning the surroundings of the vehicle, in particular concerning the current and / or upcoming section of the respective vehicle; and / or - at least one parameter of a current driving situation and / or for a driving situation predicted for the near future; and / or - at least one parameter of the current driving context and / or for a driving context predicted in the near future; and / or - at least one parameter of the current traffic sign information.

[0125] The traffic sign information may refer to one or more of the following • an image section comprising at least part of the traffic sign; and / or • a content of the traffic sign; and / or • a part of the vehicle’s surroundings to which the traffic sign refers.

[0126] Preferably, the traffic sign information is information that deviates from and / or goes beyond the content of the traffic sign (as such). For example, the traffic sign information can concern things (e.g., objects, spatial parts, or road users) that acquire a specific significance for the vehicle due to the traffic sign. For example, another road user must be given priority over another road user, or the vehicle itself has priority over another road user, etc.

[0127] The traffic sign can be (or be selected from) any traffic sign or one assigned to any category in accordance with a standard or agreement (e.g. Vienna Convention, European Standard). The traffic sign can, for example, belong to or be a category of warning sign, prohibition sign, regulatory sign, mandatory sign, directional sign, signpost, border crossing sign, etc. In particular, the traffic sign is a traffic sign whose meaning depends on one or more other (optional or variable) conditions, e.g. another road user. The traffic sign can, for example, indicate a right of way regulation (in its surroundings).

[0128] The traffic sign may optionally include a traffic light and / or at least a time-variable display. The traffic sign information can be determined based on sensor data, e.g., camera data from one or more cameras. In particular, the traffic sign information can be determined using a user device carried in the vehicle.

[0129] The method further comprises processing the traffic sign information. In particular, data representing a part of the vehicle's surroundings in connection with the traffic sign can be processed and / or generated during the processing.

[0130] Processing the traffic sign information may include interpreting the meaning of the traffic sign. Processing, in particular interpreting, the traffic sign information may occur particularly when the vehicle has already passed the traffic sign and / or when the traffic sign is no longer visible or no longer clearly visible to the driver of the vehicle.

[0131] Processing may take place depending on data that is determined or predicted (or can be determined or predicted), only after the traffic sign has already been passed by the vehicle and / or when the traffic sign is no longer or no longer clearly visible to the driver of the vehicle.

[0132] Particularly preferably, the processing of the traffic sign information takes place depending on an event (e.g. movement of the vehicle and / or movement of other road users, their arrangement relative to one another, a change in the driving situation or corresponding data) after the traffic sign has been passed by the vehicle and / or when the traffic sign is no longer or no longer clearly visible to the driver of the vehicle.

[0133] The interpretation of the meaning of the traffic sign can be done in connection with, • an expected, in particular predicted, and / or occurred, in particular recognized, driving situation; • one or more structural features of a roadway and / or road layout used by the vehicle; and / or • information about whether the driver of the vehicle could or did see the traffic sign.

[0134] The interpretation of the meaning of the traffic sign in connection with the above-mentioned list entries is to be understood in particular as an interpretation in relation to these list entries and / or in the context of these list entries, for example in relation to the (specific, possibly only later ascertainable) driving situation and / or in relation to structural features of the carriageways or carriageway arrangements or in the context of the driving situation and / or the features of the carriageways or carriageway arrangements.

[0135] According to a further or second aspect of the invention, a (second) method for operating a vehicle is proposed. The (second) method comprises: operating a first vehicle and / or operating a second vehicle with respect to an interaction between the first vehicle and the second vehicle in a driving situation depending on the output layer of a neural network. The neural network can be trained, in particular retrained, depending on: - first data relating to one or more first faults of a vehicle functionality of one or more first vehicles; and - second data in connection with one or more second errors of a vehicle functionality of one or more second vehicles.

[0136] The training, in particular retraining, may relate to the interaction of a vehicle and a respective driver of the vehicle and / or may have one or more features described in this document.

[0137] The method according to the first aspect or steps of the method according to the first aspect can be combined with the method according to the further or second aspect. In the preferred embodiment, these can form the method (overall method). The method (overall method) can comprise: detecting a first driver error relating to vehicle control during a first driver action of a first driver of a first vehicle, and (thereafter or depending on the detection of the driver error) determining first data related to the first driver error; and (thereafter), depending on the first data, training a neural network, and (thereafter) operating the first vehicle and / or one or more second vehicles depending on the output layer of the trained neural network.

[0138] According to a (further) third aspect of the invention, a (first) system for operating a vehicle is proposed, wherein the system is designed to train, in particular retrain, a neural network designed to operate a vehicle, wherein the training, in particular retraining, is executable depending on: - first data relating to one or more first faults of a vehicle functionality of one or more first vehicles; and - second data related to one or more second faults of a vehicle functionality of one or more second vehicles.

[0139] In particular, the operation can be carried out with respect to an (e.g. merely possible, probable and / or predicted) interaction of the one or more first vehicles and / or one or more second vehicles in a driving situation.

[0140] The (first) system may comprise or be a computer program product and / or specialized hardware configured to train a neural network described in this document, in particular, at least primarily specialized and / or optimized for training or retraining such a neural network. The system may comprise or be a correspondingly configured training platform or parts thereof.

[0141] The (first) system may include or be specialized hardware (e.g., in the vehicle and / or in the backend), such as graphics processing units (GPUs) or tensor processing units (TPUs), and / or software libraries and frameworks, for example, according to a selection of features of TensorFlow, PyTorch, Keras, and / or means for storing, processing, and handling the data described in this document (also: datasets or sequences). Alternatively or additionally, this appropriately configured system may include or be an integrated development environment (IDE), visualization tools, a computer program for automating the training process, hyperparameter optimization, and / or scaling.

[0142] According to a (further) fourth aspect of the invention, a (second) system for operating a vehicle is proposed, wherein the system is designed to operate a first vehicle and / or one or more second vehicles depending on the output layer of a neural network, wherein the neural network is trained, in particular retrained, depending on: - first data relating to one or more first faults of a (first) vehicle functionality of one or more first vehicles; and - second data related to one or more second faults of a vehicle functionality of one or more second vehicles.

[0143] The system according to the third aspect or parts of the system according to the third aspect can be combined with the system according to the fourth aspect. In particular, the system according to the third aspect is designed to form a system (as an overall system) together with the system according to the fourth aspect or to be operated as a system (as an overall system).

[0144] The system preferably comprises one or more first vehicles, at least one second vehicle, and / or a backend (also to be understood as a cloud, a server, a portal). Preferably, one or more first vehicles are configured for a direct or indirect operative connection with the at least one second vehicle and / or the backend. The one or more first vehicles can be configured to determine parameters characterizing one or more (first) driver errors, e.g., the type of driver error, the type of corresponding driver actions, etc.

[0145] Furthermore, the one or more first vehicles can be configured to transmit the corresponding data (e.g., when executing the active connection) to the backend. The backend can be configured to process the data from a plurality of the first vehicles, in particular to determine one or more frequency information items (e.g., relating to several different groups of driver actions and driver errors). Based on the processed data (in particular relating to the first data), in particular based on the frequency information, a measure can be initiated (at least partially in the backend and / or at least partially in the second vehicle).

[0146] In particular, the system may also include an operating system of the vehicles and / or data interfaces configured to execute the method, e.g., hardware-related interfaces. The system, e.g., the majority of the first and / or second vehicles and / or the backend, may be operated by the same provider, operator, service provider, and / or a standard or convention designed in accordance with the invention. The one or more first vehicles and the one or more second vehicles may be offered directly or indirectly by the same manufacturer or operator.

[0147] The features of the (second) system relating to the operation of the vehicle or the operation of the neural network and features of the (first) system for training or retraining can be combined in a variety of ways. In particular, the (resulting) system has the features of the first system and features of the second system. Therefore, the features described in this document can each refer mutatis mutandis to the first system and / or the second system. The term "system" is to be understood in particular as the first system, as the second system, or as a system according to a combination of the features of a first system and a second system.

[0148] According to a fifth aspect of the invention, a neural network for operating a vehicle is proposed. The neural network is designed or is trained, in particular retrained, and / or is retrainable, depending on: - first data relating to one or more first faults of a vehicle functionality of one or more first vehicles; and - second data relating to one or more first faults of a vehicle functionality of one or more second vehicles; wherein the training, in particular retraining, relates to an interaction of a first vehicle with a second vehicle in a driving situation.

[0149] In particular, the terms "trained" and "retrained" should also be understood as: fully trained or at least to a certain extent trained or retrained. Training and retraining can be different processes. For the sake of simplicity, this document will occasionally subsume retraining under the term "training" to emphasize that retraining can (selectively, also) exhibit one or more of the described characteristics of training.

[0150] For example, the at least one neural network (e.g. to be used during operation) can be understood as features of or with data based on the at least one (e.g. at least partially trained) neural network.

[0151] For the purposes of this document, the neural network is to be understood as neural network data, in particular specific pre-trained layers or modules of the neural network serving a (specific) purpose, and / or as data for setting up or updating the neural network (i.e., a part, specific layers, or components of the neural network). This may, for example, be data that can be transferred to a vehicle's computing unit.

[0152] In particular, the neural network is configured for training and / or operation in and / or for the vehicle. In particular, it is configured, in particular specialized, trained, and in particular retrainable or retrainable, at least primarily (e.g., through appropriate architecture and / or application orientation) for operating the vehicles according to one or more features described in this document.

[0153] In particular, the training, in particular pre-training and / or operation of the neural network in a computing unit located outside the vehicle (e.g., using the device, remotely) can be carried out depending on the data acquired in the vehicle using the corresponding device (part of the system) and / or trained, retrained and / or operated for the vehicle. In this case, the (first and / or second) system can be configured to acquire and / or receive the data or parameters described in this document or to execute one or more steps of the method entirely or partially remotely.

[0154] For example, the data is generated during training of the at least one neural network on any computing unit, in particular the computing unit of the backend, e.g., in order to then make it operable (also understood as executable) on one or more other computing units, in particular in (e.g., respective, specific) vehicles and / or in the backend (e.g., for one or more vehicles). Training the at least one neural network can include or be setting up the at least one neural network depending on the data from at least one corresponding neural network or one trained for the corresponding use.

[0155] According to a sixth aspect, the invention is characterized by a vehicle, wherein the vehicle is configured to carry out the (first and / or second) method or to operate the (first and / or second) system according to one or more features described in the present document.

[0156] For example, the vehicle can be configured to train, in particular retrain and / or operate, the at least one neural network (located in the vehicle and / or in the mobile unit and / or in the backend). Alternatively or additionally, the vehicle is a vehicle comprising the neural network according to the invention and / or operated dependently on the neural network.

[0157] The vehicle is in particular a motor vehicle, e.g. a passenger car, truck, two-wheeler, motorcycle, scooter, e-bike, pedelec, bus, etc.

[0158] According to a seventh aspect, a computer program product is proposed, wherein the computer program product comprises data of the at least one neural network, and / or a computer program and / or data for setting up, updating, training, in particular retraining, and / or operating the neural network.

[0159] Alternatively or additionally, the computer program product can be designed to execute one or more steps of the (first and / or second) method when executed on at least one computing unit, in particular a computing unit of the (first and / or second) system.

[0160] The computer program product can be embodied as an update of a previous computer program, which, for example, comprises parts of the computer program or the corresponding program code as part of a functional expansion, for example, as part of a so-called "remote software update." The computer program product comprises, in particular, a medium readable by the data processing device on which the program code is stored, or at least an encrypted file. According to a further aspect, a program product is described that comprises an authorized right of access to stored data of the computer program product.

[0161] The invention is described in more detail below, without limiting its generality, using exemplary embodiments and figures. All features of the exemplary embodiments can be combined with one another and / or with all other features described in this document. In the figures: -Fig. 1 shows a flowchart of an exemplary (possibly computer-implemented) method according to the invention, - Fig. 2 a variant of a system according to the invention, - Fig. 3 shows a further variant of a system according to the invention, and - Fig. 4 shows a further variant of a system according to the invention.

[0162] Fig. 1 shows a flowchart of an exemplary (possibly computer-implemented) method 100 according to the invention for operating a vehicle 10, 11, 12, 20, 21, 22.

[0163] The solid arrows indicate a preferred, suggested sequence of steps. Furthermore, the arrows can also mean that the step pointed to by the arrow is executed depending on the result of the step from which the arrow begins. The dash-dotted arrows each indicate an influence during an operating cycle, in particular a control of the execution of a step pointed to by the arrow, depending on the result of one or more steps (from a previous operating cycle) from which the arrow(s) originate. The method can, in particular, comprise only part of the illustrated sequence and / or also further steps.

[0164] The (first) method 100 comprises training 103, in particular retraining 103', of a neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 designed to operate 106, 108 one or more (first) vehicles 10, 11, 12, ... and / or one or more (second) vehicles 20, 21, 22, ... To step 101:

[0165] In step 101, the first data, in particular first sequences, are determined in connection with one or more first errors FF10, FF11, FF12 of a (first) vehicle functionality of one or more first vehicles 10, 11, 12, ... This can occur during (user) operation of the respective vehicles 10, 11, 12, ... or in normal road traffic. This can include detecting the first error (as such). These can be detected as such, in particular retrospectively or post-factum (e.g., in the data collected by the vehicles).

[0166] The determination 101 of first data can be carried out depending on the detection and / or evaluation of a second error FF20, FF21, FF22 of a vehicle functionality of a second vehicle (20, 21, 22). To step 102:

[0167] In step 102, the second data, in particular second sequences, are determined in connection with one or more second errors FF20, FF21, FF22 of a (second) vehicle functionality of one or more second vehicles 20, 21, 22. This can occur during (user) operation of the respective vehicles 10, 11, 12, ... or in normal road traffic. This can include detecting the first error (as such). These can be detected as such, in particular retrospectively or post-factum (e.g., in the data collected by the vehicles).

[0168] The determination 102 of second data is carried out depending on the detection and / or evaluation of a first error FF10, FF11, FF12 of a vehicle functionality of a first vehicle 10, 11, 12. For step 103 or 103':

[0169] In step 103, training 103 or retraining 103' of the at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 takes place.

[0170] The determination of training data for training 103 or retraining 103' and / or the execution of the training or retraining 103' is dependent on the result of steps 101 and / or 102, in particular depending on - first data, in particular first sequences, in connection with one or more first errors FF10, FF11, FF12 of a (first) vehicle functionality of one or more first vehicles 10, 11, 12; and / or - second data, in particular second sequences, in connection with one or more second errors FF20, FF21, FF22 of a (second) vehicle functionality of one or more second vehicles 20, 21, 22.

[0171] The training 103 or retraining 103' can be carried out with respect to an interaction of a first vehicle 10, 11, 12, ... with a second vehicle 20, 21, 22, ... e.g. in a driving situation (in particular also to be understood as: within the framework of a driving situation).

[0172] The training 103 or retraining 103' of the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can be carried out depending on: - one or more first absolute and / or relative first positions at which the respective first error FF10, FF11, FF12 occurred and / or at which the respective first data applied; and / or - one or more second absolute and / or relative second positions at which the second error FF20, FF21, FF22 occurred and / or at which the second data was valid.

[0173] For example, an absolute position (e.g., a specific intersection) and / or a relative position (e.g., a type of intersection, not necessarily the same intersection) can be taken into account. This allows a better result to be achieved when operating 104 the neural network and / or when operating 106 one or more first vehicles 10, 11, 12, ...or when operating 108 one or more second vehicles 20, 21, 22.

[0174] The retraining 103' can (e.g. in contrast to training 103): - vehicle-specific; and / or - user-specific; and / or - context-specific; and / or - driving situation specific; be or be executed.

[0175] Individually tailored support can be provided. To step 104:

[0176] In step 104, the operation of at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 takes place.

[0177] Operating 104 the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can include or be controlling, stimulating, and / or prompting the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2. This occurs at the input layer of the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 (in operational mode).

[0178] The operation 104 of the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can be carried out during or for the operation 106 of a first vehicle 10, 11, 12, ... and / or during or for the operation 108 of a second vehicle 20, 21, 22, ... depending on one or more variables characterizing the driving operation of the first vehicle 10, 11, 12, ... and / or the second vehicle 20, 21, 22, ..., in particular the first sequence and / or the second sequence.

[0179] The one or more variables characterizing the driving operation of the vehicle 10, 11, 12, ... 20, 21, 22 may include or be: - an absolute and / or relative position at which the respective vehicle 10, 11, 12, ... 20, 21, 22 is located (also was located recently) and / or is at least expected to be located in the near future; and / or - map information relating to the surroundings of the respective vehicle 10, 11, 12, ... 20, 21, 22, in particular relating to the current and / or upcoming section of the respective vehicle 10, 11, 12, ... 20, 21, 22; and / or - at least one parameter of a current driving situation and / or for a driving situation predicted for the near future; and / or - at least one parameter of the current driving context and / or for a driving context predicted in the near future; and / or - at least one parameter of the current traffic sign information. For steps 106 and 108:

[0180] In step 106, the operation 106 of a first vehicle 10, 11, 12 takes place. This preferably takes place in relation to a (e.g. possible, expected or predicted) interaction of the first vehicle 10, 11, 12 with a (e.g. possibly existing or already recognized) second vehicle 20, 21, 22 (e.g. in a possible or more probable or already recognized driving situation).

[0181] In step 108, the operation 108 of a second vehicle 20, 21, 22 takes place. This preferably takes place in relation to a (e.g. possible, expected or predicted) interaction of the second vehicle 10, 11, 12 with a (e.g. possibly existing or already recognized) first vehicle 20, 21, 22 (e.g. in a possible or more probable or already recognized driving situation).

[0182] The operation 106 and / or the operation 108 can be carried out by one or the same neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2, depending on the output layer.

[0183] The operation 106 of a first vehicle 10, 11, 12 may include a prediction 105 of a first error, in particular an undesirable functioning of a vehicle functionality of the respective vehicle 10, 11, 12 in response to a (e.g. merely possible, potential) driver error, in particular a delayed, insufficient, inappropriate and / or incorrect driver action of the respective user, in particular driver of the first vehicle 10, 11, 12.

[0184] The operation 108 of a second vehicle 20, 21, 22 may include a prediction 107 of a second error, in particular an undesirable functioning of a vehicle functionality of the respective vehicle 20, 21, 22 in response to a (e.g. merely possible, potential) driver error, in particular a delayed, insufficient, inappropriate and / or incorrect driver action of the respective user, in particular driver of the second vehicle 20, 21, 22.

[0185] In this case, a (e.g., particularly undesirable) combination of the (respective) first errors and the second errors can (also) be predicted. This can be performed entirely or at least partially (also) before a (possible) interaction between vehicles 10, 11, 12, ... 20, 21, 22.

[0186] Subsequently, a particularly predictive operation 106 of the respective first vehicle 10, 11, 12 and / or the operation 108 of the respective second vehicle 20, 21, 22 can be carried out. This can be aimed at avoiding the combination of errors and / or reducing the (possible, expected) degree of consequence.

[0187] Preferably, the operation 106 and / or the operation 108 is or includes effecting (e.g., to be understood as initiating and / or executing) a measure with respect to an interaction (initially possible or to be considered possible) of the one (or more) first vehicles 10, 11, 12 and one (or more) second vehicles 20, 21, 22.

[0188] For example, one or more suitable measures or variants of measures can be generated (unnoticeably for one or more drivers, in the background, so to speak, and / or even with relatively small error probabilities).

[0189] Within the scope of the method 100, one or more parameters of the driving situation and / or boundary conditions and / or the context in connection with the occurrence of the at least one first error FF10, FF11, FF12, and / or one or more second errors FF20, FF21, FF22 can be determined and taken into account.

[0190] The term “parameter” can be understood in particular as: - a temporal and / or spatial change characteristic, in particular a gradient, such as a temporal and / or spatial gradient, of a (respective) parameter or a parameter value; and / or - a pattern formed by one or more parameters or parameter values.

[0191] Preferably, the data relating to the result, in particular consequence measures of specific combinations of one or more first errors FF10, FF11, FF12, and / or one or more second errors FF20, FF21, FF22, can be determined and taken into account during training 103, retraining 103', and / or during operation 104 of the neural network. For example, statistical clusters of occurrences of certain consequences of driver errors of a certain type, characterized by certain parameters, in particular for one or more specific criteria, e.g., patterns of corresponding initial driver actions, driving situations, contexts, etc., can be determined and taken into account.

[0192] Alternatively or additionally, information characterizing the number or frequency of non-occurrence of a combination (specific and / or having specific features) of a first error FF10, FF11, FF12, and one or more second errors FF20, FF21, FF22 and / or a consequence of such a combination can be determined and / or taken into account (in one or more steps of the method 100).

[0193] The at least one measure, in particular a variant and / or a parameter of the measure, can be determined depending on a previously determined input or selection of the driver.

[0194] In particular, the driver may be enabled to (co-)decide (in advance) on an action to be taken if a specific driver error (e.g. a driver error of a specific type) is detected or predicted.

[0195] For example, the driver may be enabled to select and / or specify a measure (which may be configurable and designable by the driver) and / or to compile it from predefined elements.

[0196] For example, the driver may be enabled to select, create or preconfigure a reminder information, e.g. a text, a keyword, an announcement, etc., in particular in the event that a certain driver error is predicted with at least a certain degree of probability and / or consequence.

[0197] In particular, it can be provided that the driver of the vehicle (e.g. by means of a data input and / or user interface) can (co-)decide in advance on the measure (i.e. select, create or compile the measure) with which he can be helped in the event of a possible specific driver error and / or a driver error of a specific type.

[0198] The at least one measure can comprise a change in an operating logic and / or a parameter of a control device for controlling or specifying the guidance of the vehicle (e.g. longitudinal guidance, lateral guidance, maneuver execution, etc.) 10, 11, 12, 20, 21, 22. For example, depending on a (statistical) accumulation of one or more specific driver errors, the behavior of the respective vehicle 10, 11, 12, ... 20, 21, 22, ... can be changed, in particular with regard to the operating logic and / or setting and / or a current operating parameter of the respective driver assistance system FAS, active safety system SAS, system for at least partially automated driving TAF, HAF or parking TAP, HAP, and / or the system SIG for signaling to the environment of the vehicle 10, 11, 12, 20, 21, 22.

[0199] Within the scope of method 100, the at least one measure can comprise outputting driver information with a specific suggestion for changing the behavior of the vehicle (e.g., at least one device or functionality of the vehicle) 10, 11, 12, 20, 21, 22. The driver information can comprise, for example, a suggestion for changing the operating logic and / or a parameter for interpreting an input from the driver of the vehicle 10, 11, 12, 20, 21, 22. The (then possibly effective) change in the behavior of the vehicle 10, 11, 12, 20, 21, 22 can be executed depending on a confirmation from the driver.

[0200] A second vehicle 10, 11, 12, ... 20, 21, 22, ... can be configured to receive data relating to a driver error described in this document directly or indirectly from the one or more first vehicles 10, 11, 12, ... 20, 21, 22, ... In particular, this involves processed data, e.g., in the backend BE or aggregated within the scope of data processing DV.

[0201] Furthermore, this document describes a method which is aimed at determining and / or providing data relating to a driver error (e.g. as part of a first vehicle 10, 11, 12, ... 20, 21, 22, ...), or at receiving data relating to a driver error and / or using it to determine a measure (e.g. as part of a second vehicle 10, 11, 12, ... 20, 21, 22, ...).

[0202] Furthermore, this document describes a system (comprising one or more devices) configured to perform one of the methods 100 described in this document.

[0203] In particular, the system is or comprises one or preferably a plurality of first vehicles 10, 11, 12, ..., one or more second vehicles 20, 21, 22, ..., and optionally a backend BE (in particular with a data processing unit DV for aggregating the data from a plurality of the vehicles 10, 11, 12, 20, 21, 22). It can be a so-called distributed system. Depending on the result of the data processing or the aggregation of the data, the training data can be determined (and used in vehicles F10, F11, F12).

[0204] The system may be configured to execute one or more of the methods 100 described in this document. It may be a distributed system. The system may be configured according to one or more of the features of the method described in the document. The first vehicle 10, 11, 12, ... and the second vehicle 20, 21, 22, ... may be vehicles of different or the same type.

[0205] For example, a list of instructions and / or logic that is characteristic of executing and / or triggering certain actions in the event of certain events can be determined and initiated on the vehicle 10, 11, 12, 20, 21, 22.

[0206] Depending on whether the driver error is likely and, if so, which variant of the first error (FF10, FF11, FF12) or the second error (FF20, FF21, FF22) is to be expected, an action can then be selected and executed (in real time, with minimal resource consumption). A decision regarding an action can be delayed. The number of probable errors and / or actions can, for example, be gradually reduced to the most likely variants. This enables improved, precise, and effective functionality.

[0207] In particular, at least one measure relates to one or more items from the following list (the term “relating” is also to be understood as “being comprised of” or “forming part of”): - support of a driver action by the driver of the first vehicle, - an execution of an influence on the movement of the first vehicle, - generating information for the driver of the first vehicle, - performing an at least partially automated driving maneuver, parking maneuver, parking exit maneuver and / or maneuvering maneuver, - a selection of a parameter or mode of the vehicle lighting of the first vehicle; - a signal from the vehicle to the outside.

[0208] In one example of the method, depending on the output layer of the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 (e.g. as part of a measure), a signaling of the (first and / or second) vehicle 10, 11, 12 to the outside, in particular to the respective other (second or first) vehicle 20, 21, 22, can be proposed, initiated, changed, prevented and / or effected.

[0209] Preferably, the execution, suppression, and / or variation of the at least one measure can be linked to a detection or prediction of a specific driving situation and / or roadway context. For example, the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can be trained and / or operated to cause (e.g., to generate and / or influence a control signal) that the at least one measure (e.g., one that would otherwise be executed) is not executed (e.g., avoided and / or suppressed), or is replaced by an alternative measure, or is executed with a modified first measure parameter.

[0210] Executing, suppressing, and / or varying the at least one measure in a vehicle 20, 21, 22 can be a control, a regulation, or any desired influencing of the first measure. However, the method is not limited to the control or regulation of a variable, for example, in the sense of control engineering. Rather, it can include or be a (direct or indirect) influence on the selection of the type of the first measure, the variant of the first measure, and / or the determination or adjustment of a first measure parameter.

[0211] Varying the measure by the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can also be determining, preferring, and / or limiting a solution space or a tendency to execute, suppress, and / or vary at least one measure.

[0212] Suppressing at least one action (normally executed by the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) can include or be an at least partial suppression of the action, a limitation of the effect, the performance, or a performance characteristic of the action. A action can also be suppressed only partially or to a specific extent (e.g., determined by the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2).

[0213] The invention further includes all functionalities that can be executed in a vehicle 10, 11, 12, ... 20, 21, 22 and that are operated or made operable (also to be understood as: executable) on the basis of the trained neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 - for example for a corresponding driver assistance system FAS, active safety system SAS, system for at least partially automated driving HAF or parking HAP, system SIG for signaling to the surroundings of the vehicle (operated by the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2).

[0214] In the context of this document, the term "vehicle functionality" can also be understood as a performance characteristic (also to be understood as a feature) of a (larger) function, e.g., a user function or equipment of the vehicle 10, 11, 12, ... 20, 21, 22. Furthermore, functionality can be considered, for example, an automatic activation, deactivation, and / or modification of a (further) functionality or driver information that suggests or announces the activation, deactivation, or modification of a functionality. For example, an (automatically generated) suggestion to the driver of the vehicle 10, 11, 12, ... 20, 21, 22 to switch a specific functionality on, off, or to change it in a specific way can also be considered a functionality. A functionality is, in particular, a function (initially at least potentially) available to the user of the vehicle 10, 11, 12, ...20, 21, 22 useful, usable and / or executable functionality.

[0215] Fig. 2 illustrates a variant of the system. In this variant, the training 103 of the at least one neural network DNN1, GPT1 takes place in the same vehicle 10 and / or for the same vehicle 10. In this variant, the first data from the same first vehicle 10, 11, 12 for which the at least one neural network DNN1, GPT is being trained can be used (e.g., only or primarily only). Of course, the first data from other first vehicles and / or second vehicles can also be used (in conjunction).

[0216] The training 103 or retraining 103' can be carried out entirely or partially in the vehicle 10 itself (e.g., without transmitting data to the backend BE or directly or indirectly to other vehicles), or entirely or partially in a backend BE. The initial data from the first vehicle 10 can be transmitted to the backend BE (e.g., step by step, as the first driver errors occur), particularly over a longer period of time (e.g., several days, weeks, months).

[0217] The training 103, 103' of the at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 (in each variant of the method) can include or be a determination (also to be understood as: generation) of training data and / or learning data.

[0218] The training data based on the first data and / or learning data of the at least one neural network DNN1, GPT1 can be determined in the backend BE. These can in turn be transmitted (directly or indirectly) to the vehicle 10 for use in the vehicle 10 (e.g., in response to the transmission of a plurality of first data). The learning data (in particular to be understood as data of the at least one neural network DNN1, GPT1 or for setting up the neural network DNN1, GPT1) can be loaded and / or uploaded to the first vehicle 10 and / or (optionally) also to other vehicles 11, 12, 20, 21, 22 in order to operate the at least one neural network DNN1 in these vehicles depending on the learning data.

[0219] Fig. 3 illustrates a further variant of the system. In this variant, the training 103 or retraining 103' of the at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 takes place depending on the first data from a number of the first vehicles 10, 11, 12. The training can (preferably) be carried out depending on the aggregated first data from multiple vehicles 10, 11, 12, processed in particular according to a predetermined condition, of multiple errors that occurred in different, in particular specific and / or corresponding to specific features: contexts, driving situations, and / or that were caused by multiple different vehicle functionalities and / or drivers.

[0220] Fig. Figure 4 illustrates a further variant of the system. In this variant, a neural network DNN2, GPT2, which is operated here by way of example in a second vehicle 20, can be deployed (in particular also to be understood as loaded and / or updated) depending on the learning data of a neural network DNNZ, GPTZ (generated and / or updated during training). The features or advantages of a method can be used according to the features of the Fig. 1 and features or advantages of a system according to the features of the Fig. 2, Fig. 3 or Fig. 4 can be combined in whole or in part.

[0221] The features of different variants of the (first and / or second) system can be freely combined with one another. For example, the operation of the first vehicle 10, 11, 12, ... and / or the second vehicle 20, 21, 22 (in all variants of the invention) can be dependent on the at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 (physically located in the backend BE or executed there).

[0222] This can be a neural network DNN1, GPT1 (for the one or more first vehicles 10, 11, 12) or DNN2, GPT2 (for the one or more second vehicles 20, 21, 22) intended for the respective vehicle 10, 11, 12, 20, 21, 22 and / or trained for the respective vehicle 11, 12, 20, 21, 22 or a group of similar vehicles 11, 12, 20, 21, 22 and / or a central neural network DNNZ, GPTZ (at least for a number of the vehicles 20, 21, 22). In this case, the aspect of training the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 includes the control signals that are transmitted (e.g. from the backend) for operating the first vehicle 10, 11, 12, ... and / or the second vehicle 20, 21, 22.

[0223] In this case, the first vehicle 10, 11, 12, ... and / or the one or more second vehicles 20, 21, 22 are not controlled during operation depending on the first data, or they do not have to be controlled depending on the first data. Rather, they are controlled depending on the output layer of the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2.

[0224] The input layer of the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can be controlled during operation (e.g. shortly before, during and / or after a second driver action) (again) depending on the second data (e.g. current and / or predicted for a near future of a few seconds and / or local).

[0225] The term “neural network” DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can alternatively or additionally be understood to mean another form of artificial intelligence (in particular comprising a neural network DNN1, DNN2, DNNZ or a comparable structure).

[0226] In particular, the training and / or operation of the at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 is (also) understood to mean a corresponding application of other types of machine learning. For example, the at least one neural network DNN1, DNN2, DNNZ can include or be the training and / or operation of this or a corresponding application of one or more of the following or a combination thereof: supervised learning, unsupervised learning, reinforcement learning, multitask learning, classification and regression trees, support vector machines, logical and relational learning, probabilistic graphical models, rule learning, instance-based learning, latent representation, bio-inspired approaches.

[0227] One or more steps of the method can be carried out, in particular end-to-end, by means of a correspondingly trained and / or operated neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2. Of course, this does not exclude the use or co-use of other means, e.g., one or more computer-implemented calculation formulas, rule-based algorithms, etc. For example, at least part of one or more steps of the method can be carried out (at least in part) by other means. Preferably, a substantial part of the processing of the data described in the present document and / or a substantial part of the effect of the method can be carried out by means of the at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2. The neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can be used as an at least partial update of a (e.g.,previous) neural network DNN1, DNNZ, DDN2, which is updated, for example, as part of a functional extension, for example as part of a functional update.

[0228] A plurality of neural networks DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can be used, in particular neural networks that are trained and / or interconnected independently of one another (e.g., directly or indirectly interconnected for operating the first vehicle 10, 11, 12, ... and / or one or more second vehicles 20, 21, 22). Depending on the variant of the method, the features described in this document can relate to one or (selectively or each) several of the neural networks DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2. For example, these can (each or only) selectively relate to neural networks DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 that are set up or used to carry out certain steps of the method.

[0229] For example, a backpropagation, in particular the error backpropagation, for training 103 the at least one neural network DNN1, DNN2, DNNZ can be determined depending on the processing of the (in particular selected and / or preprocessed) sections on the basis of the first data (e.g. one or more sections assigned to different categories).

[0230] For example, depending on a first category of the first data, the at least one neural network DNN1, DNN2, DNNZ is at least predominantly rewarded. For example, depending on a second category, the at least one neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 is at least predominantly penalized.

[0231] All (e.g. corresponding) features of the method can be combined in all combinations with each other and / or with all (e.g. corresponding) features of the system, in particular one or more variants of the system.

[0232] Furthermore, the respective vehicle 10, 11, 12, 20, 21, 22 can comprise a (control) device configured to operate the vehicle 10, 11, 12, 20, 21, 22. A driver assistance system (FAS), an active safety system (SAS), a system for at least partially automated driving (TAF, HAF) or parking (TAP, HAP), and / or a system (SIG) for signaling to the surroundings of the vehicle 10, 11, 12, 20, 21, 22 are each provided in the vehicles 10, 11, 12, ... 20, 21, 22.

[0233] The neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 can be regarded (e.g. at the same time) as the neural network DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2 of the respective driver assistance system FAS, the active safety system SAS, the system for at least partially automated driving TAF, HAF or parking TAP, HAP, and / or the system SIG for signaling to the environment of the vehicle 10, 11, 12, 20, 21, 22.

Claims

[1] Method (100) for training (103), in particular retraining (103'), a neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) designed to operate (106, 108) a vehicle (10, 11, 12, ... 20, 21, 22), wherein the training (103), in particular retraining (103'), is carried out depending on: - first data relating to one or more first errors (FF10, FF11, FF12) of a vehicle functionality of one or more first vehicles (10, 11, 12); and - second data related to one or more second errors (FF20, FF21, FF22) of a vehicle functionality of one or more first vehicles (20, 21, 22); wherein the training (103) is carried out with respect to an interaction of a first vehicle (10, 11, 12, ...) with a second vehicle (20, 21, 22) in a driving situation. [2] Method (100) according to claim 1, wherein the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) is trained (103) and / or operated (104), - one or more first errors (FF10, FF11, FF12) of the vehicle functionality of a first vehicle (10, 11, 12); and / or - to avoid one or more second faults (FF20, FF21, FF22) of the vehicle functionality of a second vehicle (10, 11, 12), and / or to reduce a probability measure and / or consequence measure of a combination of at least one first fault (FF10, FF11, FF12, ...) and at least one second fault (FF20, FF21, FF22, ...). [3] Method (100) according to claim 1 or 2, comprising: - determining (101) first data depending on the detection and / or evaluation of a second error (FF20, FF21, FF22) of a vehicle functionality of a second vehicle (20, 21, 22); and / or - Determining (102) second data depending on the detection and / or evaluation of a first error (FF10, FF11, FF12) of a vehicle functionality of a first vehicle (10, 11, 12). [4] Method (100) according to one of the preceding claims, wherein the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) is trained (103), in particular retrained (103') on, and / or operated (104) for: - Prediction (105) of a first error (FF10, FF11, FF12), in particular an undesirable functioning of a vehicle functionality of the first vehicle (10, 11, 12) in response to a driver error, in particular a delayed, insufficient, inappropriate and / or erroneous driver action of the first driver of the first vehicle (10, 11, 12); and / or - Prediction (107) of a second error (FF20, FF21, FF22), in particular an undesirable functioning of a vehicle functionality of the second vehicle (20, 21, 22) in response to a driver error, in particular a delayed, insufficient, inappropriate and / or erroneous driver action of the second driver of the second vehicle (20, 21, 22). [5] Method (100) according to one of the preceding claims, wherein: determining (101) and / or processing the first data from one or more first vehicles (10, 11, 12); and / or determining (102) and / or processing the second data from one or more second vehicles (20, 21, 22); and / or training (103), in particular retraining (103') the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2); is executed depending on: - one or more first parameters and / or first patterns of a first error (FF10, FF11, FF12, ...) of a vehicle functionality of the respective first vehicle (10, 11, 12); and / or - one or more second parameters and / or second patterns of a second error (FF20, FF21, FF22, ...) of a vehicle functionality of the respective second vehicle (20, 21, 22). [6] Method (100) according to one of the preceding claims, wherein: - the first data are characteristic of a first sequence of variables that are temporally and / or causally related to the occurrence of the first fault (FF10, FF11, FF12, ...) in the vehicle functionality of the one or more first vehicles (10, 11, 12, ...); and / or - the second data are characteristic of a first sequence of the variables that are temporally and / or causally related to the occurrence of the second fault (FF10, FF11, FF12, ...) of the vehicle functionality of the one or more second vehicles (20, 21, 22, ...); wherein in particular at least one section of the first sequence and / or at least one section of the second sequence characterizes the variables that were valid before the occurrence of the respective fault. [7] Method (100) according to one of the preceding claims, wherein the training (103), in particular retraining (103') of the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) is carried out depending on: - one or more first absolute and / or relative first positions at which the respective first error (FF10, FF11, FF12) occurred and / or at which the respective first data applied; and / or - one or more second absolute and / or relative second positions at which the respective second error (FF20, FF21, FF22) occurred and / or at which the second data applied. [8] Method (100) according to one of the preceding claims, wherein the training data for training (105), in particular retraining (103') of the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) comprise one or more of the following information and / or are determined depending on this information and / or are used in training (103), in particular retraining (103') of the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2): - first sequence of the first position data of the first vehicle (10, 11, 12) before the occurrence of the first error (FF10, FF11, FF12); and / or - second sequence of the second position data of the second vehicle (10, 11, 12) before the occurrence of the second error (FF10, FF11, FF12); and / or - a part of map information from the vicinity of the first positions and / or second positions. [9] Method (100) according to one of the preceding claims, wherein the operation (104) of the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) during operation (106) of a first vehicle (10, 11, 12, ...) and / or during operation (108) of a second vehicle (20, 21, 22, ...) is carried out as a function of one or more variables characterizing the driving operation of the first vehicle (10, 11, 12, ...) and / or the second vehicle (20, 21, 22, ...), in particular the first sequence and / or the second sequence. [10] Method (100) according to claim 9, wherein one or more variables characterizing the driving operation of the vehicle (10, 11, 12, ... 20, 21, 22) include: - an absolute and / or relative position at which the respective vehicle (10, 11, 12, ... 20, 21, 22) is located and / or is at least expected to be located in the near future; and / or - map information relating to the surroundings of the vehicle (10, 11, 12, ... 20, 21, 22), in particular relating to the current and / or upcoming section of the respective vehicle (10, 11, 12, ... 20, 21, 22); and / or - at least one parameter of a current driving situation and / or for a driving situation predicted for the near future; and / or - at least one parameter of the current driving context and / or for a driving context predicted in the near future; and / or - at least one parameter of the current traffic sign information. [11] Method (100) for operating (106, 108) a vehicle (10, 11, 12, 20, 21, 22), wherein the operation (106) of a first vehicle (10, 11, 12) and / or the operation (108) of a second vehicle (20, 21, 22) with respect to an interaction of the first vehicle (10, 11, 12) and the second vehicle (20, 21, 22) in a driving situation, is executed depending on the output layer of a neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2), wherein the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) is or will be trained (103), in particular retrained (103'), depending on: - first data relating to one or more first errors (FF10, FF11, FF12) of a vehicle functionality of one or more first vehicles (10, 11, 12); and - second data relating to one or more second errors (FF20, FF21, FF22) of a vehicle functionality of one or more second vehicles (20, 21, 22). [12] System for operating (106, 108) a vehicle (10, 11, 12, ... 20, 21, 22), wherein the system is designed to train (103), in particular retraining (103'), a neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) designed to operate (106, 108) a vehicle (10, 11, 12, ... 20, 21, 22), wherein the training (103), in particular retraining (103'), is executable depending on: - first data relating to one or more first errors (FF10, FF11, FF12, ...) of a vehicle functionality of one or more first vehicles (10, 11, 12); and - second data related to one or more second errors (FF20, FF21, FF22, ...) of a vehicle functionality of one or more second vehicles (20, 21, 22). [13] System for operating (106, 108) a vehicle (10, 11, 12, ... 20, 21, 22), wherein the system is designed to operate (106, 108) a first vehicle (10, 11, 12) and / or one or more second vehicles (20, 21, 22) depending on the output layer of a neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2), wherein the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) is or will be trained (103), in particular retrained (103'), depending on: - first data relating to one or more first errors (FF10, FF11, FF12) of a vehicle functionality of one or more first vehicles (10, 11, 12); and - second data related to one or more second errors (FF20, FF21, FF22) of a vehicle functionality of one or more second vehicles (20, 21, 22). [14] Neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) for operating (106, 108) a vehicle (10, 11, 12, ... 20, 21, 22), wherein the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) is trained (103), in particular is or will be retrained (103'), and / or is retrainable (103'), depending on: - first data relating to one or more first errors (FF10, FF11, FF12) of a vehicle functionality of one or more first vehicles (10, 11, 12); and - second data in connection with one or more first errors (FF20, FF21, FF22) of a vehicle functionality of one or more second vehicles (20, 21, 22); wherein the training (103), in particular retraining (103'), relates to an interaction of a first vehicle (10, 11, 12) with a second vehicle (10, 11, 12) in a driving situation. [15] Computer program product, wherein the computer program product comprises data of at least one neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) according to claim 14, and / or a computer program and / or data for setting up, updating, training (103), in particular retraining (103'), and / or operating (104) the neural network (DNN1, GPT1, DNNZ, GPTZ, DNN2, GPT2) according to claim 14. [16] Computer program product, wherein the computer program product is designed to execute one or more steps of the method (100) according to one of claims 1 to 11 when executed on at least one computing unit, in particular a computing unit of the system according to claim 13.

Images