Method for operating a driver assistance device for a motor vehicle and corresponding driver assistance device

Abstract

The invention relates to a method for operating a driver assistance system (2) for a motor vehicle (1), wherein an assistance control unit (3) of the driver assistance system (2) is connected electronically to at least one sensor (6) that detects a state variable of the motor vehicle. It is provided that, by means of an artificial neural network running on the assistance control unit (3), sensor data from the at least one sensor (6) describing the state variable of the motor vehicle (1) are evaluated, and an error probability is determined from the sensor data. Depending on the error probability, a communication signal for communication with a user (8) of the motor vehicle (1) is generated and output using the artificial neural network. The invention further relates to a driver assistance system (2) for a motor vehicle (1).

Description

[0001] The invention relates to a method for operating a driver assistance system for a motor vehicle, wherein an assistance control unit of the driver assistance system is connected electronically to at least one sensor that detects a state variable of the motor vehicle. The invention further relates to a driver assistance system for a motor vehicle.

[0002] For example, the prior art is disclosed in US 2021 / 0094582 A1. This discloses a method comprising the following steps: receiving video data from a camera of a driver assistance device; receiving vehicle data via a connection between a vehicle and the driver assistance device; receiving sensor data from one or more sensors connected to the driver assistance device; generating a modified video file containing the video data, at least some of the vehicle data, and at least some of the sensor data; and transmitting the modified video file to an administrative computer located remotely from the driver assistance device.

[0003] Furthermore, patent application US 2019 / 0304208 A1 discloses a method for providing vehicle information to a user via acoustic signals, the method comprising the following steps: receiving diagnostic data from a vehicle computer in a vehicle at a data acquisition and transmission device; transmitting the diagnostic data to a remote diagnostic server on which several preset vehicle conditions are stored; and analyzing the diagnostic data on the remote diagnostic server and generating a warning signal when the diagnostic data meets at least one of the preset vehicle conditions; transmitting the warning signal to a mobile communication device, wherein the mobile communication device is configured to generate the acoustic signal in response to receiving the warning signal, the acoustic signal containing diagnostic information associated with the diagnostic data set received from the vehicle computer.

[0004] The object of the invention is to propose a method for operating a driver assistance system for a motor vehicle which has advantages over known methods, in particular providing a user of the motor vehicle with an easy-to-use system for fault diagnosis.

[0005] According to the invention, this is achieved by a method for operating a driver assistance system for a motor vehicle with the features of claim 1. It is provided that, by means of an artificial neural network running on the assistance control unit, sensor data from at least one sensor describing the state variable of the motor vehicle are evaluated, and an error probability is determined from the sensor data. Depending on the error probability, a communication signal for communication with a user of the motor vehicle is generated and output using the artificial neural network.

[0006] Advantageous embodiments with expedient further developments of the invention are specified in the dependent claims. It should be noted that the exemplary embodiments described in the description are not limiting; rather, any variations of the features disclosed in the description, the claims, and the figures are possible.

[0007] The method serves to operate the driver assistance system. The driver assistance system, or the assistance control unit of the driver assistance system used to carry out the method, is preferably a component of the motor vehicle, but can of course also exist separately from it, particularly until the driver assistance system or the assistance control unit is mounted on or in the motor vehicle. The driver assistance system serves to support a user of the motor vehicle, for example, the driver of the motor vehicle. For this purpose, the driver assistance system uses the assistance control unit to evaluate the sensor data acquired by the sensor, which describes the state variables of the motor vehicle.

[0008] Whenever this description refers to "at least one sensor" or "the sensor," the explanations are always equivalent. Explanations concerning "at least one sensor" apply to "the sensor," and explanations concerning "the sensor" apply to "at least one sensor." In principle, the assistance control unit can be connected to any number of sensors. If multiple sensors are present, the explanations concerning "at least one sensor" or "the sensor" preferably apply to each of the multiple sensors.

[0009] It is known, for example, that the driver assistance control unit receives information from a vehicle component, such as via an OBD interface (OBD: On-Board Diagnostics). Based on this information, a fault diagnosis can then be performed. If the fault diagnosis reveals a fault in the component, this is indicated to the user, for example, by means of a visual display, in particular a warning light and / or a screen. This can only occur, however, if the component's data already indicates the fault. Furthermore, there is insufficient user support provided for fault diagnosis and / or troubleshooting.

[0010] For this reason, according to the invention, the sensor data acquired using the sensor, which describe the state variable of the motor vehicle, are evaluated using the artificial neural network. The error probability is then determined from the sensor data using the artificial neural network. The error probability describes the probability with which a fault or defect of the motor vehicle or a component of the motor vehicle exists. The sensor data can contain only a single measured value for the state variable or for several state variables, measured at a specific time. However, the sensor data can also represent a time course of the measured value, i.e., a time course of the state variable.

[0011] Depending on the probability of an error, the communication signal is generated. This signal is used to communicate with the vehicle's user. For this purpose, it is output after its generation. The output of the communication signal can be, for example, optical and / or acoustic. The optical communication signal is typically output via a screen, while the acoustic communication signal is output via a transducer or loudspeaker.

[0012] The communication signal includes, for example, information and explanations regarding the probability of a fault and / or a vehicle feature related to that probability. The communication signal is preferably generated using generative artificial intelligence, specifically based on the probability of a fault. Preferably, the communication signal is generated using a Large Language Model (LLM). It is preferably provided that the assistance control unit is designed and configured to conduct a dialogue with the user. Thus, the assistance control unit can receive input from the user, in particular feedback on the communication signal, and generate and output a further communication signal in response to the input.

[0013] The described driver assistance system offers a comprehensive and interactive system that the vehicle user can access in case of a problem and which assists them in resolving it. The driver assistance system, or rather its control unit, can access current vehicle data and use the sensor to perform its own analysis and troubleshooting. Ideally, this enables the user to resolve problems independently, thus eliminating the need for a visit to a workshop. Parameters for the artificial neural network are stored in the control unit. These parameters are preferably modifiable via a communication link, allowing for parameter updates, for example, via an over-the-air (OTA) update.

[0014] A further development of the invention provides that the evaluation of the sensor data and the determination of the probability of error are performed upon request by the user, in particular only or exclusively when the user requests it. The evaluation of the sensor data and the determination of the probability of error are thus initiated by the user, for example by a corresponding command, such as a voice command or the like. Preferably, the sensor data are evaluated only when such a request is made by the user in order to determine the probability of error. The described driver assistance system is therefore not intended to perform a continuous diagnosis of the vehicle, but rather such a diagnosis is only carried out upon explicit request by the user. This also serves to realize the interactive nature of the described method.

[0015] A further development of the invention provides that at least one of the following sensors is used as the at least one sensor: a sound sensor, a vibration sensor, and an image sensor. The sound sensor is a transducer that converts sound signals into electrical signals, which ultimately constitute the sensor data or from which the sensor data is derived. The vibration sensor converts vibrations, which occur, for example, in the form of structure-borne sound, into electrical signals. The image sensor is preferably in the form of a camera, in particular a color camera, preferably a true-color camera or a false-color camera. Each of the aforementioned sensors is suitable for a generic diagnosis of the motor vehicle, especially based on symptoms that are also perceptible to the user.A sensor that evaluates a state variable directly perceptible to the user is particularly preferred for determining the sensor data. This allows the driver assistance system to respond very precisely to user requests.

[0016] A further development of the invention provides that the assistance control unit is connected electronically to a system control unit, by means of which a system component of the motor vehicle is controlled, at least temporarily. The system control unit transmits a state value describing the state of the system component to the assistance control unit, and the assistance control unit takes this state value into account when determining the probability of a fault. The motor vehicle has the system component, which in turn includes the system control unit. The system control unit serves to control the system component, for example, to control an actuator of the system component. The system control unit is connected electronically to the assistance control unit, namely via a data transmission connection. The data transmission connection is established, for example, using a data bus.

[0017] The control unit determines the state of the system in the form of a state value. This state value is transmitted from the control unit to the assistance control unit and is additionally considered when determining the probability of a fault. The state value, like the sensor data, thus serves as an input for the artificial neural network. The system could be, for example, a drive system, a safety system, an entertainment system, or similar. Using the state value enables precise system diagnostics and also facilitates the development of a solution through dialogue with the user.

[0018] A further development of the invention provides that, depending on the probability of a fault, the assistance control unit activates at least one additional sensor, different from the at least one primary sensor, to determine further sensor data, with the additional sensor data being taken into account when generating the communication signal. First, the probability of a fault is determined using the sensor data. Depending on the determined probability of a fault, the assistance control unit then activates the additional sensor to determine the further sensor data. For example, the additional sensor is assigned to a component of the vehicle for which the probability of a fault indicates a malfunction.

[0019] The additional sensor data is used to inform the user by being incorporated into the communication signal. For example, if vibrations are detected during driving, particularly when steering, using the vibration sensor, the assistance control unit first evaluates the corresponding sensor data to locate a potential fault. If it is determined that one or more tires may be faulty, an image sensor is activated to determine the condition of the tire(s). This results in a targeted diagnosis of the vehicle.

[0020] A further development of the invention provides that a fault in the motor vehicle is detected as soon as the fault probability corresponds to a probability value indicating the fault. After determining the fault probability, it is evaluated. As long as the fault probability corresponds to an initial probability value indicating proper functioning of the motor vehicle, the fault is not detected.

[0021] However, if the probability of an error corresponds to the second probability value, which indicates the presence of an error, then the error is assumed to exist and detected accordingly. For example, it is assumed that no error exists as long as the probability of an error lies within a specific first probability range. However, if the probability of an error lies outside the first probability range, particularly in a second probability range that differs from the first, then the presence of an error is assumed and detected. This enables a reliable diagnosis of the vehicle.

[0022] A further development of the invention provides that, based on sensor data for several components, a component failure probability is determined, which describes the respective probability that the corresponding component causes the failure. Component data for several components is stored in the assistance control unit. The sensor data is evaluated using the artificial neural network with the component data, and the respective component failure probability is determined separately for each component. The component failure probability describes the probability with which the component to which the component failure probability is assigned exhibits or causes the failure. This also enables particularly targeted diagnostics of the vehicle.

[0023] A further development of the invention provides that an image of the component with the highest probability of defect is recorded by means of the image sensor and the image is displayed to the vehicle user. The image sensor is adjusted to record the component, in particular by being aligned accordingly. The image is preferably displayed to the vehicle user. This significantly facilitates troubleshooting for the user, in particular by drawing their attention to the defective component. Additionally or alternatively, it is provided that the image is used in generating the combined signal, especially in addition to the sensor data.

[0024] A further development of the invention provides that, depending on the probability of error, an actuator influencing the state variable of the motor vehicle is controlled by means of the artificial neural network, and if the value of the state variable changes due to the control of the actuator, further sensor data is acquired using at least one sensor and taken into account when generating the communication signal. The actuator used is one that influences the state variable of the motor vehicle, which is acquired by means of the sensor.

[0025] Activating the actuator results in a change in the sensor data, thus altering the state variable. The actuator is activated taking this state variable, particularly the probability of an error, into account. During or after the actuator activation, which leads to the change in the state variable, the sensor acquires further sensor data that also describe the state variable. This additional sensor data is then used as the basis for the communication signal. This approach is helpful for precisely locating the fault.

[0026] The invention further relates to a driver assistance system for a motor vehicle, in particular for carrying out the method as described in this description, wherein an assistance control unit of the driver assistance system is connected electronically to at least one sensor that detects a state variable of the motor vehicle. The driver assistance system is designed and configured to evaluate sensor data from the at least one sensor describing the state variable of the motor vehicle by means of an artificial neural network running on the assistance control unit and to determine an error probability from the sensor data, wherein, depending on the error probability, a communication signal for communication with a user of the motor vehicle is generated and output using the artificial neural network.

[0027] The advantages of such a procedure or such a design of the driver assistance system have already been mentioned. Both the driver assistance system and the procedure for operating it can be further developed as explained in this description, and reference is made to these explanations in this regard.

[0028] The features and combinations of features described in the description, in particular those described in the following figure description and / or shown in the figures, can be used not only in the combinations specified, but also in other combinations or individually, without departing from the scope of the invention, in particular the scope of the claims. Thus, embodiments that are not explicitly shown or explained in the description and / or the figures, but which emerge from or can be derived from the explained embodiments, particularly within the scope of the claims, are also to be considered as encompassed by the invention.

[0029] The invention is explained in more detail below with reference to the exemplary embodiments shown in the drawing, without limiting the invention. The only embodiment shown is... Fig. 1 a schematic representation of a motor vehicle and a driver assistance device with one or more sensors.

[0030] The Fig. Figure 1 shows a schematic representation of a motor vehicle 1 equipped with a driver assistance system 2. The driver assistance system 2 includes an assistance control unit 3, which is connected to a sensor arrangement 4 and an actuator arrangement 5 via information technology.

[0031] The sensor arrangement 4 has one or more sensors 6, wherein in the embodiment shown here there are two sensors 6, one of which is designed as a sound sensor and the other as an image sensor. The actuator arrangement 5 has one or more actuators, which are not shown here.

[0032] The assistance control unit 3 has a voice interface through which a user 8 of the vehicle 1 can communicate with the driver assistance system 2. An artificial neural network, which implements generative artificial intelligence, is operated on the assistance control unit 3. Using the artificial neural network, sensor data from sensor 6 or sensors 6 are evaluated and an error probability is determined from this data. Depending on the error probability, a communication signal is then generated for communication with the user 8 of the vehicle 1, i.e., for use with the voice interface 7.

[0033] The driver assistance system 2, or its assistance control unit 3, serves, for example, as a voice assistant that supports the user 8 in identifying a fault and preferably also in rectifying the fault. For this purpose, the driver assistance system 2 accesses the sensor data from the sensor(s) 6 and can also control the actuator arrangement 5. This is done, for example, as follows: 1. If unusual engine noises are detected using sensor 6, the driver assistance system 2 uses sensor 6 to analyze the noise and determine the type of fault. If the noises indicate a loose V-belt, instructions are given via the voice interface 7, according to which the user 8 can safely inspect the engine compartment of the vehicle 1 and, if necessary, tighten the V-belt. For further diagnosis, the engine speed is increased using the actuator assembly 5 to determine whether this has an effect on the noise. 2. If vibrations or instability are detected by sensor 6 during the operation of the motor vehicle 1, particularly during steering, the sensor arrangement 4 is used to determine whether the vibrations originate from the vehicle's tires, for example, caused by an imbalance. In this case, the driver assistance system 2 recommends, for example, that the tires be balanced. Additionally, a sensor 6 designed as an image sensor could be used, for example, to visually inspect the tires. 3. If an information or warning light is activated, the driver assistance system 2 could read and evaluate a corresponding fault code from a component of the vehicle 1. A fault associated with the fault code could be rectified together with the user 8 as part of a guided troubleshooting process. If the fault code indicates a more serious problem, the user 8 would be advised to visit a workshop. 4. If the user 8 asks a question about a specific maintenance step via the voice interface 7, the driver assistance system 2 generates and outputs a communication signal containing step-by-step instructions for performing the maintenance step. This preferably includes the correct use of the required tools and materials, including adherence to all safety precautions. Additionally, the driver assistance system 2 uses the image sensor to assist the user 8 in locating the relevant parts of the vehicle 1.

[0034] Overall, the described driver assistance system 2 and the explained procedure enable flexible troubleshooting and, if necessary, troubleshooting. The driver assistance system 2 is designed to enter into a dialogue with the user 8 via the voice interface 7 in order to answer the user's questions about the vehicle's condition and to assist the user in troubleshooting and troubleshooting. REFERENCE MARK LIST: 1 motor vehicle 2 Driver assistance system 3 Assistance control unit 4 Sensor arrangement 5 Actuator arrangement 6 Sensor 7 Voice interface 8 users QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 2021 / 0094582 A1

[0002] US 2019 / 0304208 A1

[0003]

Images