Device and method for operating an assisted, automatic or autonomous driving vehicle

Abstract

A device (100) for operating an assisted, automatic or autonomous driving vehicle (102) comprises a sensor (104, 106) of the vehicle (102) which is configured to acquire at least one sensor information relating to a detection area (108) in the vicinity of the vehicle (102) and to generate sensor data corresponding to the sensor information, and a sensor control unit (112, 114) which is configured to determine, at least on the basis of the sensor data, whether a sensor anomaly of the sensor (104, 106) is present.The device (100) also comprises a context detection unit (116) configured to determine at least one piece of context information and to generate context data corresponding to that context information, and a control unit (126) configured to determine, based on at least the context data, whether the assisted, automatic, or autonomous driving of the vehicle (102) can continue safely if the sensor control unit (112, 114) has determined that a sensor anomaly is present. The control unit (126) is further configured to control at least one functional unit (128) of the vehicle (102) related to the assisted, automatic, or autonomous driving of the vehicle (102) if the control unit (126) has determined that the assisted, automatic, or autonomous driving of the vehicle (102) cannot continue safely.

Description

[0001] The invention relates to a device for operating an assisted, automatic, or autonomously driving vehicle. The device comprises a sensor of the vehicle, which is designed to detect at least one piece of sensor information related to a detection area in the surroundings of the vehicle and to generate sensor data corresponding to the sensor information. The device further comprises a sensor control unit, which is designed to determine, at least based on the sensor data, whether a sensor anomaly is present in the sensor.

[0002] In the current state of the art, the detection of sensor anomalies is performed for each individual sensor using the sensor data processed within a sensor unit consisting of the sensor and the sensor control unit. If a sensor anomaly is detected, a message is sent to a central control unit of the vehicle that the sensor has failed. If certain sensors fail, the assisted, automatic, or autonomous driving of the vehicle is terminated and the driving task is returned to the vehicle driver. The number of sensors that can lead to an impairment and ultimately the deactivation of assisted, automatic, or autonomous driving is very large. Furthermore, for cost reasons, there is no complete redundancy in the sensors. It is therefore very likely that the availability of assisted, automatic, or autonomous driving will be severely reduced due to sensor failures.

[0003] A system for controlling a vehicle is known from DE 10 2018 130 566 A1. The system is designed to detect an assistance event, in particular a sensor anomaly, at least partially based on the sensor data and to determine an autonomous assistance strategy based on the detected sensor anomaly.

[0004] WO 2021 133 727 A1 discloses a system for detecting sensor anomalies in autonomous vehicles. The system selects from various detection techniques and algorithms based on the vehicle's current driving behavior and current environmental conditions.

[0005] A system for detecting sensor anomalies is also known from US Pat. No. 9,274,525 B1. This system is used in autonomous vehicles.

[0006] The object of the invention is to provide a device and a method for operating an assisted, automatic or autonomous vehicle in which the availability of assisted, automatic or autonomous driving is greater than in known devices and methods.

[0007] This object is achieved by a device having the features of claim 1 and by a method having the features of the independent method claim. Advantageous further developments are specified in the dependent claims.

[0008] The proposed device comprises a context detection unit configured to determine at least one piece of context information and to generate context data corresponding to the context information, and a control unit configured to determine, based at least on the context data, whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely if the sensor control unit has determined that a sensor anomaly exists. The control unit of the proposed device is further configured to control at least one functional unit of the vehicle related to the assisted, automatic, or autonomous driving of the vehicle if the control unit has determined that the assisted, automatic, or autonomous driving of the vehicle cannot be continued safely.

[0009] The proposed device solves this problem by considering, in addition to the information regarding whether a sensor anomaly is present, further information, such as the state of the driver, the environment, and / or the vehicle, in addition to the information regarding whether a sensor anomaly is present. Based on this context information, the control unit determines whether a detected sensor anomaly is safety-critical, i.e., whether assisted, automatic, or autonomous driving of the vehicle must be terminated to avoid endangering road safety. In other words, based on the context information, the control unit determines a strategy for dealing with the sensor anomaly, rather than simply terminating assisted, automatic, or autonomous driving upon a detected sensor anomaly. This increases the availability of assisted, automatic, or autonomous driving.When the assisted, automatic, or autonomous driving of the vehicle needs to be terminated, the control unit controls the vehicle's functional unit. In particular, the control unit controls an output unit of the vehicle to inform the driver that the assisted, automatic, or autonomous driving of the vehicle is being terminated. The control unit can further control an actuator of the vehicle as the functional unit of the vehicle to terminate the assisted, automatic, or autonomous driving. For example, the control unit can control a drive system, a braking system, and / or a steering system.

[0010] In this document, assisted driving refers to automation levels 1 and 2 according to the Federal Highway Research Institute. Automated driving refers to automation levels 2, 2+, and 3 according to the Federal Highway Research Institute. Autonomous driving refers to automation levels 4 and 5 according to the Federal Highway Research Institute. Driver assistance systems within the meaning of this document include, for example, adaptive cruise control, lane keeping assist, parking assist, longitudinal guidance assist, or a combination of the aforementioned driver assistance systems, such as traffic jam assist.

[0011] A sensor anomaly is, in particular, a sensor degradation or a sensor failure. The sensor anomaly can be detected, in particular, using known methods. The sensor control unit is, in particular, part of a sensor unit that comprises the sensor and the sensor control unit and forms a structural unit. For example, the sensor and the sensor control unit are arranged on an electrical circuit board or otherwise spatially close to one another. Alternatively, the sensor control unit can also be part of the central control unit (ECU) of the vehicle.

[0012] In one embodiment, the sensor is an image capture unit of the vehicle, which is designed to capture an image with a representation of the detection area as the sensor information and to generate image data corresponding to the image as the sensor data. The image capture unit is in particular a camera of the vehicle, for example a front camera, a side camera, or a rear camera. The image capture unit is in particular designed to capture light in the optical spectrum and / or in the infrared spectrum. The capture of light in the infrared spectrum has the additional advantage that the image capture unit can capture the image even in poor lighting conditions, for example at night. Alternatively, the image capture unit can also be a time-of-flight camera, a RADAR system, or a LIDAR system in order to generate a three-dimensional image of the detection area, in particular in the form of a point cloud.Furthermore, the image capture unit can be configured to capture an image sequence consisting of consecutive images with an image of the capture area. In such an embodiment, the image data corresponds to the image sequence.

[0013] In a further embodiment, the sensor is a distance sensor of the vehicle, which is configured to determine a distance between the distance sensor and an object located in the detection range as the sensor information and to generate distance data corresponding to the distance as the sensor data. The distance sensor is, in particular, an ultrasonic sensor. The distance sensor can also be an active or passive sonar system. Furthermore, the distance sensor can also be a RADAR system or a LIDAR system.

[0014] Image capture units and distance sensors are the most important sensors in assisted, automatic, or autonomous vehicles, as they capture information about the vehicle's immediate surroundings. However, other sensors can also be part of the device. The proposed device monitors the proper functioning of the image capture unit and / or the distance sensor and detects a sensor anomaly if they are not functioning or are no longer functioning properly. Since each sensor in the vehicle has its own detection range, the failure of the image capture unit and / or the distance sensor means that the vehicle is, so to speak, blind within this detection range. Based on the context information, the control unit therefore determines, in particular, whether the detection range must currently be monitored by the sensor for safe assisted, automatic, or autonomous driving of the vehicle.

[0015] In a further embodiment, the context detection unit comprises an occupant detection unit configured to detect at least one piece of driver information relating to the vehicle driver as the context information and to generate driver data corresponding to the driver information as the context data. At levels 3 and 4 according to the Federal Highway Research Institute, the vehicle driver must assume the driving task under certain circumstances. This generally means that the vehicle driver is already attentively observing the vehicle's surroundings and the traffic situation. Depending on what the driver's attention is currently focused on, a sensor anomaly is not safety-critical, and assisted, automatic, or autonomous driving does not necessarily have to be terminated.Based on the driver information, the determination of whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely can thus be carried out more reliably. Examples of driver information and how the driver information can be used to determine whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely are described below using specific embodiments.

[0016] In a further embodiment, the occupant detection unit comprises an image capture unit configured to capture at least one driver image containing an image of the vehicle driver as the driver information and to generate driver image data corresponding to the driver image as the driver data. The image capture unit is, in particular, an interior camera of the vehicle and is, in particular, configured to capture a two-dimensional image of the vehicle interior. Furthermore, the image capture unit can be configured to capture a plurality of temporally successive images, each containing an image of the vehicle driver, to generate an image sequence. In such an embodiment, the driver image data corresponds to the image sequence. The image capture unit, in particular, captures light in the optical spectrum and / or in the infrared spectrum.The detection of light in the infrared spectrum has the additional advantage that the image capture unit can capture the image or image sequence of the driver even in poor lighting conditions, for example, at night. As an alternative to an optical camera, the image capture unit can also be a time-of-flight camera, which generates a three-dimensional image of the vehicle interior, particularly in the form of a point cloud. A series of features can be reliably determined from the image or image sequence, allowing conclusions to be drawn as to whether the driver is paying attention and / or what the driver's attention is focused on. Examples of such features are described below using specific embodiments.

[0017] In a further embodiment, the control unit is designed to determine, based on the driver image data, whether the driver's gaze is directed at the detection area, and to determine that the assisted, automatic, or autonomous driving of the vehicle can continue safely if the driver's gaze is directed at the detection area. If the driver's gaze is directed at the detection area, a sensor anomaly is not necessarily safety-critical. While the vehicle is, so to speak, blind to the detection area, the detection area is monitored by the driver. If, for example, a front camera of the vehicle fails but the driver is attentively monitoring the area in front of the vehicle, the failure may not be immediately safety-critical, and the assisted, automatic, or autonomous driving of the vehicle can continue for the time being.

[0018] Based on the driver image data, the control unit can determine, in particular, the driver's gaze direction and / or head pose in order to determine whether the driver's gaze is directed towards the detection area. The gaze direction and / or head pose, i.e. the orientation and position of the driver's head, can be determined from the driver image data, in particular using a suitable machine learning method. The gaze direction can be determined, in particular, from the driver's pupil orientation. Both the driver's gaze direction and head pose make it possible to determine very precisely what the driver's gaze is directed towards. This makes it possible to reliably determine whether the driver's gaze is directed towards the detection area.

[0019] In a further embodiment, the occupant detection unit is configured to detect at least one physiological variable of the vehicle driver as the driver information and to generate physiological data corresponding to the physiological variable as the driver data. Based on the physiological data, it is possible, for example, to very reliably determine whether the vehicle driver is currently inattentive, for example because they are asleep. Based on this information, it is then possible to more reliably determine whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely. To detect the physiological variable, the occupant detection unit can comprise a series of sensors, for example an ultra-wideband sensor (UWB sensor) or sensors in the steering wheel of the vehicle, which can determine, for example, the skin conductivity or the pulse of the vehicle driver.Alternatively or additionally, the occupant detection unit can also include a mobile device worn on the driver's body, such as a smartwatch. Using the mobile device, the physiological parameters can be measured directly on the driver's body.

[0020] In a further embodiment, the control unit is configured to determine, based on the driver data, whether the driver is attentive, and to determine that the assisted, automatic, or autonomous driving of the vehicle cannot be continued safely if the driver is not attentive. If the driver is not attentive, a sensor anomaly is assessed differently than if the driver is attentive. This makes it possible to determine more reliably overall whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely. The driver's attentiveness can be determined, for example, based on the driver image data. For example, based on the driver's heart rate, it can be determined whether the driver is tired or at risk of microsleep. The driver's attentiveness can also be determined based on the physiological data.

[0021] In a further embodiment, the context detection unit comprises an environment detection unit configured to detect at least one piece of environmental information relating to the vehicle's surroundings as the context information and to generate environmental data corresponding to the environmental information as the context data. The environment detection unit comprises, in particular, further sensors of the vehicle, such as, for example, a further image detection unit or a further distance sensor. However, the environment detection unit can also be configured to determine and / or receive navigation information, such as maps of the vehicle's surroundings and / or a current position of the vehicle.Based on the situation in the vehicle's surroundings, for example, the traffic situation or the road conditions in the vehicle's surroundings, the control unit can reliably assess how safety-critical a detected sensor anomaly is. Examples of the environmental information and how the environmental information can be used to determine whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely are described below using specific embodiments.

[0022] In one embodiment, the environment detection unit is configured to detect and / or receive a road topology in the surroundings of the vehicle as the environmental information. The control unit can, for example, determine the road topology from images of the surroundings or extract it from a map of the surroundings provided by a navigation system. Based on the road topology, the control unit can reliably decide whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely. For example, based on the road topology, the control unit can determine that a failure of a sensor monitoring the right or left side of the vehicle is not safety-critical and that the assisted, automatic, or autonomous driving of the vehicle can be continued safely.

[0023] In a further embodiment, the control unit is designed to determine at least one object in the surroundings of the vehicle based on the environmental data and to generate object data corresponding to the determination. The control unit is further designed to carry out the determination as to whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely, taking into account the object data. The object in the surroundings of the vehicle can be, for example, another road user, an animal, a road barrier, or similar objects whose presence in the surroundings of the vehicle influences the safe driving of the vehicle. The presence of the object can be determined, in particular, taking into account sensor data from image acquisition units and / or distance sensors.Based on the object data, the control unit can reliably decide whether the vehicle's assisted, automatic, or autonomous driving can continue safely. In one example, the control unit determines, based on image data from a side camera of the vehicle, that there are no objects to the side of the vehicle. A failure of a distance sensor monitoring the side area of ​​the vehicle is therefore not considered safety-critical. The vehicle's assisted, automatic, or autonomous driving can continue for the time being.

[0024] In a further embodiment, the control unit is designed to determine a current behavior and / or intention of at least one other road user based on the environmental data and to generate behavior data corresponding to the determination. The control unit is further designed to determine whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely, taking into account the behavior data. The behavior and / or intention of the other road user can be determined, in particular, taking into account sensor data from image capture units and / or distance sensors. Based on the behavior data, the control unit can reliably decide whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely.In one example, the control unit determines, based on sensor data from a distance sensor, that another vehicle is traveling to the side of the vehicle. The failure of a side camera monitoring the side on which the other vehicle is traveling is therefore assessed as safety-critical. The vehicle's assisted, automatic, or autonomous driving is therefore terminated.

[0025] In a further embodiment, the context detection unit is configured to determine and / or receive the current speed of the vehicle as context information. The control unit is configured, in particular, to determine whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely if the value of the current speed of the vehicle is less than or equal to a predetermined value. The control unit can receive the current speed of the vehicle, for example, from an odometer unit of the vehicle or determine it based on navigation data. The lower the speed of the vehicle, the less safety-critical a sensor anomaly is.By observing the current speed of the vehicle, a sensor anomaly can be better assessed by the control unit and there are fewer failures in assisted, automatic or autonomous driving.

[0026] In a further embodiment, the control unit is designed to control the functional unit of the vehicle in order to change the level of assisted, automatic, or autonomous driving of the vehicle if the control unit has determined that the assisted, automatic, or autonomous driving of the vehicle cannot be continued in a roadworthy manner. In particular, the control unit is designed to reduce the level of assisted, automatic, or autonomous driving of the vehicle if the control unit has determined that the assisted, automatic, or autonomous driving of the vehicle cannot be continued in a roadworthy manner. For example, the control unit can control the functional unit of the vehicle in order to switch the vehicle from an automatic driving mode, for example, level 2+ according to the Federal Highway Research Institute, to assisted driving mode, for example, level 2 according to the Federal Highway Research Institute.The lower the level of assisted, automatic, or autonomous driving of the vehicle, the more driving tasks the driver must perform themselves. Lower levels also place lower demands on sensor availability, so that assisted, automatic, or autonomous driving of the vehicle can potentially continue at a lower level even in the event of a sensor anomaly. This further increases the availability of assisted, automatic, or autonomous driving.

[0027] The invention further relates to a method for operating an assisted, automatic, or autonomously driving vehicle. In the method, a sensor of the vehicle detects at least one piece of sensor information related to a detection area in the surroundings of the vehicle and generates sensor data corresponding to the sensor information. It is determined, at least on the basis of the sensor data, whether a sensor anomaly of the sensor is present. At least one piece of context information is determined, and context data corresponding to the context information is generated. Based at least on the context data, it is determined whether the assisted, automatic, or autonomous driving of the vehicle can be continued safely.At least one functional unit of the vehicle related to the assisted, automatic or autonomous driving of the vehicle is activated if it has been determined that the assisted, automatic or autonomous driving of the vehicle cannot be continued in a roadworthy manner.

[0028] The method has the same advantages as the claimed device. In particular, the method can be further developed with the features of the dependent claims directed to the device. Furthermore, the device described above can be further developed with the features described in this document in connection with the method.

[0029] Embodiments of the invention are explained in more detail below with reference to the figures, in which: Fig. 1 shows a schematic representation of a device for operating an assisted, automatic or autonomous vehicle according to an embodiment; and Fig. 2 shows a flowchart of a method for operating an assisted, automatic or autonomous vehicle according to an embodiment.

[0030] Fig. 1 shows a schematic representation of a device 100 for operating an assisted, automatic or autonomous vehicle 102 according to an embodiment.

[0031] The device 100 is used to evaluate whether sensor anomalies of sensors 104, 106 of the vehicle 102 are safety-critical, i.e., whether the assisted, automatic, or autonomous driving of the vehicle 102 can be continued safely. Within a detection range 108, the sensors 104, 106 detect at least one piece of sensor information that the vehicle 102 requires for assisted, automatic, or autonomous driving. If one of the sensors 104, 106 exhibits a sensor anomaly, i.e., no longer functions properly or fails, the assisted, automatic, or autonomously driving vehicle 102 essentially loses its ability to perceive the detection range 108. Whether this is safety-critical depends on further circumstances, for example, on what the attention of a driver 110 of the vehicle 102 is focused on.The device 100 therefore evaluates a sensor anomaly taking these additional circumstances into account in order not to unnecessarily terminate the assisted, automatic or autonomous driving.

[0032] In the Fig. In the exemplary embodiment shown in Figure 1, one of the sensors 104, 106 is embodied, for example, as a first image capture unit 104, which is arranged and configured to capture an image of the detection area 108, which is located, purely by way of example, in front of the vehicle 102. The first image capture unit 104 can, for example, be a front camera of the vehicle 102. Alternatively, the first image capture unit 104 can also be a forward-facing RADAR system or LIDAR system that captures a three-dimensional image of the detection area 108. In particular, the first image capture unit 104 is configured to capture an image sequence of a plurality of temporally successive images of the detection area 108. The first image capture unit 104 is further configured to generate and provide sensor data in the form of image data corresponding to the image or image sequence.

[0033] A second of the sensors 104, 106 is embodied, for example, as a distance sensor 106 arranged at the front of the vehicle 102, for example, as an ultrasonic sensor. The distance sensor 106 is arranged and configured to determine a distance between the distance sensor 106 and an object in the detection zone 108, for example, another road user. The distance sensor 106 is further configured to generate and provide distance data corresponding to the distance as sensor data.

[0034] Each of the sensors 104, 106 is assigned a sensor control unit 112, 114, which in the illustrated embodiment is arranged next to the associated sensor 104, 106. The sensor control units 112, 114 are configured to determine whether a sensor anomaly exists based on the sensor data of the respective associated sensor 104, 106. For this purpose, the respective sensor control unit 112, 114 can perform known methods for determining a sensor anomaly.

[0035] The device 100 also includes a context detection unit 116 for determining the aforementioned additional circumstances, which are also referred to in this document as context information. The context information can, for example, be a state of the vehicle driver 110, the environment, and / or the vehicle 102. The context detection unit 116 generates and provides context data corresponding to the context information.

[0036] To detect the state of the vehicle driver 110, the context detection unit 116 includes an occupant detection unit 118. The driver's state is detected by the occupant detection unit 118 in the form of at least one piece of driver information and provided as driver data corresponding to the driver information. The driver data is part of the context data.

[0037] In the exemplary embodiment shown, the occupant detection unit 118 comprises a second image acquisition unit 120, which is embodied purely by way of example as an interior camera of the vehicle 102. The second image acquisition unit 120 is arranged and configured to capture a two-dimensional image of the vehicle interior and in particular of the vehicle driver 110, which is also referred to below as the driver image. Alternatively, the second image acquisition unit 120 can also be a time-of-flight camera that captures a three-dimensional image of the vehicle interior. In particular, the second image acquisition unit 120 can also be configured to capture a driver image sequence of several temporally successive driver images. The second image acquisition unit 120 is further configured to generate and provide driver data in the form of driver image data corresponding to the driver image or the driver image sequence.

[0038] The occupant detection unit 118 of the illustrated embodiment further comprises a mobile device 122, which is shown purely as an example as a smartwatch. The mobile device 122 measures at least one physiological parameter directly on the body of the vehicle driver 110 and provides the measured parameter in the form of physiological data. The physiological data are part of the driver data.

[0039] In order to detect the state of the surroundings of the vehicle 102, the context detection unit 116 further comprises an environment detection unit 124. The state of the environment or a situation in the surroundings of the vehicle 102 is detected by the environment detection unit 124 in the form of at least one piece of environment information and provided as environment data corresponding to the environment information. The environment data are part of the context data. In order to detect the surroundings of the vehicle 102, the environment detection unit 124 is connected, purely by way of example, to the sensors 104, 106 of the device 100. In the exemplary embodiment shown, the sensor data generated by the first image detection unit 104 and the distance sensor 106 are thus also environment data. In other embodiments, the device 100 can also have its own sensors, for example image detection units and / or distance sensors.

[0040] As further environmental information, the environmental detection unit 124 can, for example, determine a road topology in the surroundings of the vehicle 102. For example, the environmental detection unit 124 can determine the road topology from a map of the surroundings of the vehicle 102 provided by a navigation unit. The navigation unit can be a navigation unit of the vehicle 102 itself and / or a navigation unit of another mobile device, for example a smartphone or a tablet computer, that the vehicle driver 110 has brought into the vehicle 102.

[0041] Finally, to be able to detect the state of the vehicle 102, the context detection unit 116 can receive and / or determine vehicle information related to the vehicle 102 in the form of vehicle data. For example, the context detection unit 116 can receive movement data related to the current movement of the vehicle 102 from an odometer unit of the vehicle 102. In particular, the context detection unit 116 can receive the current speed of the vehicle 102 from a speedometer of the vehicle 102.

[0042] The device 100 further comprises a control unit 126. The control unit 126 is connected to the sensor control units 112, 114 and can receive information from the sensor control units 112, 114 as to whether a sensor anomaly has been detected in the sensor 104, 106 assigned to the respective sensor control unit 112, 114. The control unit 126 is further connected to the context detection unit 116 and can receive the context data from the context detection unit 116. The control unit 126 is designed to carry out a method for operating the assisted, automatic, or autonomously driving vehicle 102 in order to evaluate a detected sensor anomaly with regard to road safety. In the method, the control unit 126 processes the context data and determines whether a sensor anomaly is safety-critical, i.e., whether the assisted, automatic, or autonomous driving of the vehicle 102 can be continued in a roadworthy manner or not.If the control unit 126 has determined that the assisted, automatic, or autonomous driving of the vehicle 102 cannot be continued in a roadworthy manner, the control unit 126 controls at least one functional unit 128 of the vehicle 102. For example, the control unit 126 can control the functional unit 128 of the vehicle 102 to terminate the assisted, automatic, or autonomous driving of the vehicle 102. Likewise, the control unit 126 can control the functional unit 128 of the vehicle 102 to reduce a level of assisted, automatic, or autonomous driving of the vehicle 102. Alternatively or additionally, the control unit 126 can control a display unit of the vehicle 102 as the functional unit 128 to inform the driver 110 that the assisted, automatic, or autonomous driving of the vehicle 102 cannot be continued in a roadworthy manner.The method for operating the assisted, automatic or autonomous vehicle 102 is described below with reference to . Fig. 2 is described in more detail.

[0043] Fig. 2 shows a flowchart of the method for operating the assisted, automatic or autonomous vehicle 102 according to an embodiment.

[0044] The method is started in step S200. In step S202, the sensors 104, 106 acquire the sensor information. The sensors 104, 106 can acquire the sensor information automatically. However, the sensors 104, 106 can also be controlled by the control unit 126 of the device 100 or a central control unit of the vehicle 102 (ECU) to acquire the sensor information. The sensors 104, 106 provide the respectively acquired sensor information as the sensor data. In one example, the first image acquisition unit 104 acquires the image of the detection area 108 in front of the vehicle 102 in step S202 and provides this image as the image data.

[0045] In step S204, the sensor control units 112, 114 determine whether a sensor anomaly exists in the sensor 104, 106 assigned to the respective sensor control unit 112, 114. In the aforementioned example, the sensor control unit 112, 114 assigned to the first image acquisition unit 104 determines from the image data that the first image acquisition unit 104 is dirty and that the image captured by the first image acquisition unit 104 does not depict the entire detection area 108. The sensor control unit 112, 114 then determines that a sensor anomaly exists in the first image acquisition unit 104.

[0046] In step S206, the context detection unit 116 detects the context information and provides it as context data. The context detection unit 116 can detect the context information automatically or be controlled by the control unit 126 of the device 100 to detect the context information, for example, when the control unit 126 receives information from the sensor control unit 112, 114 that a sensor anomaly exists. In the aforementioned example, the second image detection unit 120 detects the driver image and provides it as driver image data. The distance sensor 106 detects the detection area 108 and determines the distances between the distance sensor 106 and the objects in the detection area 108. The context detection unit 116 also receives the current speed of the vehicle 102 from the speedometer of the vehicle 102.

[0047] If at least one sensor anomaly was detected in step S204, the control unit 126 determines in step S208, based on the context data, whether the assisted, automatic, or autonomous driving of the vehicle 102 can be continued safely. To this end, the control unit 126 determines whether the sensor anomaly is safety-critical, taking into account the state of the driver, the environment of the vehicle 102, and / or the vehicle 102.

[0048] In the aforementioned example, the control unit 126 determines from the driver image data that the driver's 110 gaze is directed toward the area in front of the vehicle 102, i.e., the detection range 108 of the first image acquisition unit 104. Based on the distance data from the distance sensor 106, the control unit 126 determines that there are no objects in front of the vehicle 102. Finally, the control unit 126 determines that the speed of the vehicle 102 is less than 30 km / h. Based on all of this information, the control unit 126 determines that the contamination of the first image generation unit is not safety-critical and that the assisted, automatic, or autonomous driving of the vehicle 102 can continue for the time being.

[0049] In another example, the control unit 126 determines, based on the image data from the first image generation unit, that a cyclist is located in front of the vehicle 102. The control unit 126 further determines, based on the image data from the second image acquisition unit 120, that the vehicle driver is tired and therefore inattentive. In this example, the control unit 126 determines that a sensor anomaly of the distance sensor 106 is safety-critical and that the assisted, automatic, or autonomous driving of the vehicle 102 cannot continue.

[0050] If the control unit 126 has determined in step S208 that the assisted, automatic, or autonomous driving of the vehicle 102 cannot be continued in a roadworthy manner, the control unit 126 controls the functional unit 128 of the vehicle 102 in step S210. In one example, the control unit 126 controls actuators of the vehicle 102 to bring the vehicle 102 to a roadworthy stop after the control unit 126 has determined that a sensor anomaly of the first image acquisition unit 104 exists and the driver is not paying attention. In another example, the control unit 126 instead controls the functional unit 128 to end the automatic driving (level 3 of the Federal Highway Research Institute) of the vehicle 102 and switch to an assisted driving mode (level 2 of the Federal Highway Research Institute). The method is then terminated in step S212.

[0051] In the case of the Fig. 1 and Fig. 2, at least one of the sensors 104, 106, one of the sensor control units 112, 114, the context detection unit 116 and the control unit 126 form the device 100 for operating the assisted, automatic or autonomously driving vehicle 102. Further in the Fig. 1 and Fig. Elements and features shown in Figure 2 and mentioned in the preceding description may be part of the claimed device 100. Likewise, method steps described with reference to the device 100 may be part of the claimed method for operating the assisted, automatic, or autonomously driving vehicle 102. List of reference symbols 100 device 102 vehicles 104 Image acquisition unit 106 Distance sensor 108 Detection area 110 drivers 112, 114 Sensor control unit 116 Context detection unit 118 Occupant detection unit 120 image acquisition unit 122 mobile devices 124 Environmental detection unit 126 Control unit 128 functional unit QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2018 130 566 A1

[0003] WO 2021 133 727 A1

[0004] US 9,274,525 B1

[0005]

Claims

[1] Device (100) for operating an assisted, automatic or autonomous vehicle (102), with a sensor (104, 106) of the vehicle (102) which is designed to detect at least one piece of sensor information related to a detection area (108) in the surroundings of the vehicle (102) and to generate sensor data corresponding to the sensor information; a sensor control unit (112, 114) configured to determine, at least on the basis of the sensor data, whether a sensor anomaly of the sensor (104, 106) exists; a context detection unit (116) designed to determine at least one piece of context information and to generate context data corresponding to the context information; and a control unit (126) which is designed to determine, on the basis of at least the context data, whether the assisted, automatic or autonomous driving of the vehicle (102) can be continued in a traffic-safe manner if the sensor control unit (112, 114) has determined that a sensor anomaly is present, and to control at least one functional unit (128) of the vehicle (102) related to the assisted, automatic or autonomous driving of the vehicle (102) if the control unit (126) has determined that the assisted, automatic or autonomous driving of the vehicle (102) cannot be continued in a traffic-safe manner. [2] Device (100) according to claim 1, wherein the sensor is an image acquisition unit (104) of the vehicle (102) which is designed to acquire an image with a depiction of the detection area (108) as the sensor information and to generate image data corresponding to the image as the sensor data. [3] Device (100) according to claim 2, wherein the sensor is a distance sensor (106) of the vehicle (102) which is designed to determine, as the sensor information, a distance between the distance sensor (106) and an object located in the detection area (108), and to generate, as the sensor data, distance data corresponding to the distance. [4] Device (100) according to one of the preceding claims, wherein the context detection unit (116) comprises an occupant detection unit (118) which is designed to detect at least one piece of driver information relating to the vehicle driver (110) as the context information and to generate driver data corresponding to the driver information as the context data. [5] Device (100) according to claim 4, wherein the occupant detection unit (118) comprises an image acquisition unit (120) which is designed to acquire at least one driver image with an image of the vehicle driver (110) as the driver information and to generate driver image data corresponding to the driver image as the driver data. [6] Device (100) according to claim 5, wherein the control unit (126) is designed to determine, on the basis of the driver image data, whether the driver's (110) gaze is directed at the detection area (108), and to determine that the assisted, automatic or autonomous driving of the vehicle (102) can be continued in a traffic-safe manner if the driver's (110) gaze is directed at the detection area (108). [7] Device (100) according to one of claims 4 to 6, wherein the occupant detection unit (118) is designed to detect at least one physiological variable of the vehicle driver (110) as the driver information and to generate physiological data corresponding to the physiological variable as the driver data. [8] Device (100) according to one of claims 4 to 7, wherein the control unit (126) is designed to determine, on the basis of the driver data, whether the vehicle driver (110) is attentive, and to determine that the assisted, automatic or autonomous driving of the vehicle (102) cannot be continued in a roadworthy manner if the vehicle driver (110) is not attentive. [9] Device (100) according to one of the preceding claims, wherein the context detection unit (116) comprises an environment detection unit (124) which is designed to detect at least one item of environment information relating to the environment of the vehicle (102) as the context information and to generate environment data corresponding to the environment information as the context data. [10] Device (100) according to claim 9, wherein the environment detection unit (124) is designed to detect and / or receive a road topology in the environment of the vehicle (102) as the environment information. [11] Device (100) according to claim 9 or 10, wherein the control unit (126) is designed to determine at least one object in the surroundings of the vehicle (102) on the basis of the environmental data and to generate object data corresponding to the determination; and wherein the control unit (126) is designed to carry out the determination as to whether the assisted, automatic or autonomous driving of the vehicle (102) can be continued in a traffic-safe manner, taking into account the object data. [12] Device (100) according to one of claims 9 to 11, wherein the control unit (126) is designed to determine a current behavior and / or an intention of at least one other road user on the basis of the environmental data and to generate behavior data corresponding to the determination; and wherein the control unit (126) is designed to carry out the determination as to whether the assisted, automatic or autonomous driving of the vehicle (102) can be continued in a traffic-safe manner, taking into account the behavior data. [13] Device (100) according to one of the preceding claims, wherein the context detection unit (116) is designed to determine and / or receive the current speed of the vehicle (102) as the context information; and wherein the control unit (126) is designed in particular to determine that the assisted, automatic or autonomous driving of the vehicle (102) can be continued in a traffic-safe manner if the value of the current speed of the vehicle (102) is less than or equal to a predetermined value. [14] Device (100) according to one of the preceding claims, wherein the control unit (126) is designed to control the functional unit (128) of the vehicle (102) in order to change the level of assisted, automatic or autonomous driving of the vehicle (102) when the control unit (126) has determined that the assisted, automatic or autonomous driving of the vehicle (102) cannot be continued in a roadworthy manner. [15] Method for operating an assisted, automatic or autonomous vehicle (102), in which a sensor (104, 106) of the vehicle (102) detects at least one piece of sensor information related to a detection area (108) in the surroundings of the vehicle (102) and generates sensor data corresponding to the sensor information; determining at least on the basis of the sensor data whether there is a sensor anomaly in the sensor (104, 106); at least one piece of context information is determined and context data corresponding to the context information is generated; on the basis of at least the context data, it is determined whether the assisted, automatic or autonomous driving of the vehicle (102) can be continued safely if it has been determined that a sensor anomaly exists; and at least one functional unit (128) of the vehicle (102) relating to the assisted, automatic or autonomous driving of the vehicle (102) is activated if it has been determined that the assisted, automatic or autonomous driving of the vehicle (102) cannot be continued in a traffic-safe manner.

Images