Method and system for improving perception in collision avoidance systems

Abstract

Method for object determination in the environment of a vehicle using driver observation (100), comprising: - detecting (110) a sudden increase in the driver's attention by evaluating data from at least one sensor for detecting physiological parameters of the driver; - determining (120) a direction of gaze of the driver by evaluating data from at least one further sensor at the time of the sudden increase in attention; and - Determining (130) an object in the surroundings of the vehicle (131) in the direction of view of the driver, taking into account the suddenly increased attention.

Description

[0001] The invention is generally in the field of collision avoidance for vehicles, in particular in improving the perception of collision avoidance systems.

[0002] Generally speaking, a collision avoidance system (hereinafter referred to as a system) involves automatic intervention in a vehicle's driving behavior. Furthermore, such a system also issues warnings to the driver, alerting them to detected hazardous situations. The driver's assessment of the collision avoidance system's response is crucial. If a number of system interventions or warnings are deemed unjustified by the driver, this can significantly undermine the driver's confidence in the system's effectiveness.

[0003] Perception in collision avoidance systems refers to the ability to identify objects in the vehicle's surroundings. This involves an assessment of the actual presence of the identified object, also known as the probability of existence. Accordingly, improving perception can serve to improve object detection. In addition or alternatively, improving perception can also serve to increase the probability of the detected object's existence.

[0004] Objects include, for example, people (e.g. pedestrians), and / or animals and / or vehicles and / or buildings.

[0005] To minimize the risk of an intervention or warning that is perceived as unnecessary or even incorrect, such systems are parameterized conservatively, meaning that a warning or intervention is triggered as late as possible. However, this contradicts the fact that early intervention can significantly reduce the potential consequences of a collision, since the kinetic energy during a collision decreases quadratically with the time interval between collisions, or a collision can be completely prevented by early braking and / or evasive maneuvers.

[0006] DE 43 38 244 A1 describes a hazard avoidance system that is capable of preventing a vehicle from colliding with obstacles and reducing the effect of a collision on a vehicle.

[0007] DE 101 35 742 A1 shows a motor vehicle with a system for automatic line of sight detection and a system for automatic obstacle detection and automatic vehicle control and / or a system for distance control.

[0008] CN 105083291 A teaches a driver assistance system based on line-of-sight detection. The system can detect how a driver verifies lines of sight of all positions of a vehicle using a visual detection unit.

[0009] EP 2 977 975 A1 discloses a heart rate sensor that detects a driver's heart rate. A concentration level calculation unit estimates the driver's concentration level.

[0010] DE 10 2007 012 955 A1 teaches a method that includes identifying a stationary object adjacent to a vehicle at a specific time and measuring the time until a measuring vehicle passes the object. The distance from the measuring vehicle to the neighboring vehicle is calculated based on the measured time.

[0011] EP 2 546 819 A1 discloses a device for monitoring the surroundings of a vehicle. The device comprises a device for detecting objects in the vicinity of the vehicle based on an image; a display means for displaying display images on display screens; and a warning when an object is detected.

[0012] DE 10 2014 004 395 B3 teaches a method for operating a driver assistance system with an environment detection device for detecting a motor vehicle environment and a processing device for determining a dangerous situation in the motor vehicle environment.

[0013] DE 10 2007 044 578 A1 discloses a device for providing information, which comprises a device for detecting an observed object and an information output unit. The device outputs information about an observed object, which was detected by the device for detecting an observed object, from the information output unit.

[0014] DE 10 2015 204 284 A1 provides a method for monitoring the driving behavior of a motor vehicle driver. The method comprises: monitoring the behavior of the driver of a motor vehicle using a sensor system; comparing the driver's behavior with a predetermined behavior pattern; and assisting the driver depending on the result of the comparison.

[0015] It is known from US Pat. No. 7,206,697 B2 that modeling and observation of the driver can be used to adapt the parameters of a driver assistance system to warn of or avoid a collision. Furthermore, it is known from EP 1 779 355 B1 that collision avoidance systems can incorporate the driver's attention into the calculation of the trigger time. According to the prior art, this can be achieved by recording steering activity, pedal actuation, blinking observations using interior cameras, or by measuring the pulse at the steering wheel.

[0016] However, the known determination and evaluation of driver-specific data using sensors installed in the vehicle is limited to the triggering point of the measure identified as necessary by the system. However, the driver-specific data are not taken into account when determining and assessing the safety situation itself. Accordingly, the determination and assessment can only be based on the usual routine measures for determining the hazardous situation, which can be carried out continuously. Thus, the driver's heightened alertness is not taken into account when determining and assessing the hazardous situation.

[0017] There is therefore a need to further improve the identification and assessment of the hazard situation.

[0018] The problem is solved by a method and a system having the features of the respective independent patent claims.

[0019] In the exemplary embodiments of the present invention, a sudden increase in the driver's attention is registered by suitable sensors. In addition, the driver's line of sight is registered in such a situation, also by suitable sensors. Thus, a hazard detected by the driver, including the driver's line of sight, is detected by sensors. The information detected by the driver is then taken into account when determining the dangerous situation. This determination includes identifying objects in the vicinity of the vehicle and their position, as well as determining whether these objects are sufficiently certain to exist. Interventions in the vehicle's driving behavior or warning measures for the driver are only justified if the existence of the objects is sufficiently certain. The degree of certainty of detecting the object is referred to as the probability of existence.A measure of a sufficient probability of existence is also referred to as evidence. In this case, evidence is understood as that which appears to be unequivocally recognizable or also as knowledge determined with particular certainty. Accordingly, if evidence is present, the system assumes the presence of detected objects determined with particular certainty. Only when evidence is present does the system stop its investigation and subsequently makes its secured perception available for a system reaction. Driver observation can now improve object recognition and / or increase the probability of the detected object's existence. Both measures can shorten the system's processing time and / or justify more drastic interventions in driving behavior or more drastic warnings to the driver. Earlier system interventions can also be triggered.

[0020] Objects refer to the detection of potentially present objects, people, and arrangements in a vehicle's surroundings that may be relevant to the vehicle's driving behavior. These are, in particular, objects that, if ignored, could endanger the vehicle, or the objects themselves. Accordingly, objects to be identified include, for example, people (e.g., pedestrians), and / or animals, and / or vehicles, and / or buildings, and / or roadway features, such as the roadway topology. See, for example, Töpfer, On Compositional Hierarchical Models for Holistic Lane and Road Perception in Intelligent Vehicles, 2014.

[0021] According to a first aspect of the present invention, a method for object detection in the surroundings of a vehicle using driver observation is proposed. This method comprises detecting a sudden increase in the driver's attention by evaluating data from at least one sensor configured to detect the driver's physiological parameters. Furthermore, the method comprises determining the driver's gaze direction by evaluating data from at least one further sensor at the time of the sudden increase in attention. Furthermore, the method comprises determining an object in the surroundings of the vehicle, taking into account the sudden increase in attention and the gaze direction.

[0022] Object recognition is the identification of an object by comparing data from an image of the vehicle's surroundings with known stored objects or their elements. Object recognition can be carried out in stages. For example, lines can first be identified from the data of the vehicle's surroundings. These lines can have various shapes, so-called weak features, which are also referred to below as features. In a further step, these lines can then be combined to form more complex object elements, for example a person's head, arm or torso, so-called strong features. These are also referred to below as object elements. A sufficient number of matching object elements leads to a recognized object. Other objects can also characterize the roadway topology, which can be understood as the course of the road in the landscape.Each stage requires evidence for the statement it makes. Even after object formation, the object's existence probability can still be changed, for example, if driver observation allows for a corresponding increase beyond the evidence. Known methods for detecting such features and object elements are published in "Sigal et al., Loose-limbed People: Estimating 3D Human Pose and Motion Using Nonparametric Belief Propagation, 2011" and "Sudderth et al., Graphical Models for Visual Object Recognition and Tracking, 2006."

[0023] Physiological parameters characterize the driver's condition. Thus, the recording of individual parameters, such as heart rate or pulse rate, can provide information about the driver's physiological condition. If various physiological parameters can be measured, the information about the driver's condition naturally increases. Such parameters can be recorded from the driver, for example, using measuring tapes or clothing equipped with sensors, so-called wearables or smart watches, or corresponding glasses. They can be recorded by observing the driver, for example, with an optical interior camera in the passenger compartment of a vehicle or with an infrared camera to measure skin surface temperature. They can also be recorded from the body using internal sensors, so-called smart pills.

[0024] Sudden changes in the parameters indicate that the driver is startled or startled, as is common when the driver detects a hazard. Accordingly, a degree of change can be defined for each observed parameter; exceeding this value can be interpreted as a startle in the driver. For this purpose, the parameters of the physiological variables are scanned for characteristic deviations from normal behavior. If the estimated attention exceeds a threshold, this value, along with a gaze direction vector, is passed on to the sensor data processing unit of the sensor device. Accordingly, the continuous recording of these parameters can be used to detect a sudden increase in the driver's attention, for example, when startled.

[0025] The driver's gaze direction is recorded using gaze direction vectors, which are evaluated in a computing unit using coordinate transformations so that the gaze direction vector and specific objects can be viewed in the same coordinate system. Consequently, it is known which point in the vehicle's surroundings the driver's gaze falls on.

[0026] Specifically, this step could look like this: Determination of the gaze direction vector using an indoor camera, first individually for both eyes, point of the gaze direction line in the pupils, then averaging of the two lines to x → _Viewing direction. The coordinate transformation into the sensor coordinate system (vehicle KOS) results in a straight line for the driver’s line of sight to x_→(gaze direction,vehicle−cos)=x_(eye position,vehicle−cos)+λ x_→gaze direction

[0027] Points or objects in the vehicle's surroundings that are hit by the driver's gaze can now be determined by checking the straight line from equation 0 for intersection with relevant objects.

[0028] In addition to the gaze direction, the temporal course of the gaze direction vector (fixation duration of the gaze on the object above a threshold value of, for example, Δt Fix , for example, 300 milliseconds). Taking into account the temporal progression and the direction of view in combination with the measured physiological variables, a probability P Fahrer (k) (11) can be used to estimate the time at which the driver detected an object. The variable k denotes the current time step.

[0029] Using the gaze direction vector, a spatial region of interest (ROI) is defined. A distance metric is used to check whether objects and / or object elements and / or features are located within the spatial region of interest.

[0030] The vehicle's surroundings are the immediate area around the vehicle, which is detected and evaluated by the system. Its orientation is primarily in the direction of travel, although rear and side traffic is also considered. In particular, collision avoidance by the active system operates primarily in a range from 0 m to 200 m in front of the vehicle. In addition to the collision avoidance area, there are numerous other areas in the vehicle's surroundings that are used for lane changing or other functions.

[0031] Sensors for systems according to the invention include, for example, radar systems, LIDAR (light detection and ranging) systems with optical distance and speed measurement, stereoscopic optical camera systems, and ultrasound systems. Information about the vehicle's surroundings can also be provided through communication with the vehicle's surroundings, particularly with other vehicles or stationary information providers. In particular, such additional information can be used to expand the observable surroundings of the vehicle beyond what is immediately detectable by the vehicle's own sensors.

[0032] This allows the driver's hazard detection to be incorporated into the system's perception of objects.

[0033] Optionally, the object to be detected in the vehicle's surroundings can comprise object elements. These, in turn, can comprise features. Features, object elements, and objects thus form a hierarchy whose levels can be processed sequentially. Detection of the object elements and / or features is based on a search of an image of the vehicle's surroundings using data at a first resolution. Furthermore, in the driver's line of sight, detection of the object elements and / or features is based on a search of the image using data at a second resolution. The second resolution is finer than the first resolution.

[0034] Object elements characterize an object. If a sufficient number of unique object elements are identified, the object can be verified. Object elements can include more complex shapes, such as arms, legs, head, and torso. The object elements are arranged in a specific geographical arrangement and position relative to one another. For example, the head element sits on the torso element in an object of a person who is, for example, a pedestrian.

[0035] The object elements can, in turn, be based on features. Features are simpler geometric shapes, which can be formed as lines, for example. Several features arranged in a suitable manner can result in an object element. The features themselves can be determined directly from the image of the vehicle's surroundings. Because they comprise simpler geometric shapes, less computational effort is required than when determining object elements.

[0036] The system now strives to recognize features in one stage. In a subsequent stage, object elements can be formed from the recognized features. In a further stage, it strives to create objects from the recognized object elements.

[0037] An image of the vehicle's surroundings is provided by a volume of data recorded by the available sensors. The sensors will primarily be sensors located in or on the vehicle. However, data for the image of the surroundings can also be provided by external sensors, for example from other vehicles, which is also referred to as Car2Car communication. This refers to the exchange of information and data between motor vehicles with the aim of alerting the driver to critical and dangerous situations at an early stage. Furthermore, exchange with fixed stations, for example at the side of the road, can also be provided. This is also known as Car-2-X. Communication with the surroundings can take place, for example, via the vehicle's own or existing infrastructure.The collection of data is intended to enable early warnings of black ice, traffic jams or other obstacles, thereby making road traffic safer and faster.

[0038] The image can be divided into sensor-specific images, where, for example, the radar and ultrasound images are captured, stored, and evaluated separately. However, a common image is also conceivable, in which all sensor data is captured, stored, and evaluated in a shared data cloud. It is also conceivable that a separate image is maintained for the active system and a separate image is maintained for other systems in the vehicle, such as the lane keeping assistant.

[0039] The data of the image of the vehicle's surroundings is available at a resolution that primarily depends on the individual sensors used. However, the data volume can already reach such volumes that it is not possible to fully process the image data within the time available until the next image. The typical image refresh frequency can be sensor-specific and is usually between 10 and 50 milliseconds. For this reason, for routine data analysis, the data volume is reduced by reducing the image resolution. Known compression methods can be used for this purpose. With reduced resolution, the recognizability of features and object elements is naturally reduced to a certain extent.

[0040] This makes it possible to examine the image of the vehicle's surroundings with particular precision in the area of ​​suspected danger.

[0041] Optionally, the data in the first resolution can be searched using a first processor speed. The data in the second resolution can be searched using a second processor speed, with the second processor speed being higher than the first processor speed. The first and second processor speeds can differ by a factor of 2 or 4 (depending on the approach).

[0042] Processor performance can be influenced by various measures. For example, different clock speeds can be made available to the processor, which specify a different processing speed. Due to the increased power consumption and / or increased thermal load, this time is limited. Furthermore, additional processor tasks can be blocked for the duration of the examination with a higher resolution. This allows the processor to devote its entire computing power to searching the image of the vehicle's surroundings. In a multi-processor system, several processors can search the image of the vehicle's surroundings in parallel or quasi-parallel. Accordingly, processor performance can be increased through individual measures or by combining the aforementioned measures.

[0043] Advantageously, the increased processing effort can be at least partially compensated for in terms of time by increased computing power.

[0044] In embodiments, object elements identified in the driver's line of sight can be supplemented by simulated object elements that are assigned to an object hypothesis.

[0045] The identification of object elements can be based on identified features, for example, identified lines. For example, there may be some identified object elements in the driver's line of sight, but these are insufficient to identify an object. For example, object elements have been identified that include the image of an arm, a leg, and a head fragment. However, since the identified object elements can be ambiguous, the presence of an object, for example, a person, cannot yet be reliably detected. During normal processing, the system would not be able to identify an object in this situation. However, since the object probability is significantly increased in the driver's line of sight, the system supplements the recognized object elements with presumably matching simulated object elements of an object hypothesis. The latter include all object elements of an object.

[0046] The basis for identifying object elements can be the weak features used to form object elements, as already mentioned. Such features can be, for example, lane markings or abstract features such as edges at grayscale or color transitions. Furthermore, features such as extremities, head poses, or similar features can be extracted from image data using known methods. Such methods are also referred to as "bottom-up" methods.

[0047] This advantageously significantly lowers the threshold for object detection in the likely critical area of ​​the vehicle's surroundings.

[0048] Optionally, a check is performed to determine whether the simulated object elements can be matched with the identified object elements. If the simulated object elements match the identified object elements, the object is determined taking into account both the identified object elements and the simulated object elements.

[0049] Because of the required high degree of certainty of the existence of the recognized objects, also called evidence, the agreement between the identified object elements and the corresponding simulated object elements is a necessary prerequisite for the supplementation by the simulated object elements.

[0050] The correspondence between simulated and identified object elements is determined by comparing typical properties of the object element. For example, extent, shape, location, and relative position to other object elements can be considered. For example, an object element of an arm can be compared for correspondence with respect to a person's assumed body pose, also known as posture, and the person's perceived movement sequence. However, correspondence does not mean an identical image. Rather, the correspondence can be limited to the recognizability of essential elements of the object elements and their interrelationships.

[0051] The usual identification mechanism of the objects can thus be used advantageously.

[0052] Further optionally, a plurality of types of object hypotheses can be stored. The type of object hypothesis used can depend on data describing a driver reaction.

[0053] For example, the degree of startle the driver is experiencing can be determined. In cases of moderate startle, object hypotheses that correspond to expected objects in the vehicle's surroundings can be favored. In cases of severe startle, possibly combined with other driver reactions, such as unexpected steering responses, unexpected objects are more likely to be inferred. For example, in the first case, a vehicle might narrowly overtake the driver's own vehicle. In the second case, a child might run into the road.

[0054] This can advantageously support the selection of appropriate object hypotheses.

[0055] In embodiments, the object hypothesis can only be supplemented if a test step shows that the object formation has not yet led to an identifiable object in the area of ​​the driver's line of sight.

[0056] Accordingly, before supplementing the identified object elements with the simulated object elements, a check is carried out to determine whether the identified object elements already allow the formation of an object hypothesis. To do this, the object formation process is run, which aims to convert the object elements into an object hypothesis. Only if this process fails are the simulated object elements supplemented.

[0057] In other words, a distinction can be made between two cases. In the first case, recognized objects can have an existence probability P Sensor (k), where k denotes the current time step of a corresponding calculation method. Using the gaze direction vector and the driver's reaction, an existence probability for an object was estimated, as already mentioned. Now, the existence probability of the object is calculated according to the formula Pobj(k)=PSensor(k)δS⋅PDriver(k)δF where δS+δF=1 changed if P Fahrer (k) > P senzor (k). The variables δ S and δ F are parameters with which the weighting of the driver's reaction can be determined.

[0058] Due to the increased probability of existence, due to the additional security provided by driver observation, intervention by the driver assistance system can be permitted, which would not have been possible due to the uncertain knowledge of the vehicle's sensors alone. The response of the driver assistance function is thus improved.

[0059] In the second case, the area in the driver's line of sight can first be searched for object elements. Since the area in the driver's line of sight represents only a portion of the vehicle's entire surroundings, this search can be performed at a higher resolution (e.g., while maintaining the same search time or computational complexity). Furthermore, it is conceivable to release increased computing power for the search based on the clues provided by the driver's reaction. This can be achieved, for example, by temporarily utilizing additional computing time available in the vehicle. After the search with higher computing power or higher resolution in image processing methods, a new set of additional object elements may result. This set of additional object elements is then passed on to a processing chain. This can lead to a new set of objects based on all of the object elements now detected.This allows objects to be detected that would not have been detectable without the driver's reaction, for example, because the resolution was too low due to the size of the search area. A method for object formation or feature extraction is known, for example, from WO 2009019250 A2. If no new object element is found in the driver's line of sight, the area of ​​the driver's line of sight is searched for relevant features, which can lead to an increased number of features. These can then be passed on to a processing chain, thus generating new objects via the intermediate step of new object elements.

[0060] To determine the existence probability, dense stereo data generated by a stereo camera array and independent of temporal filtering can be used. Temporal filtering can be performed, for example, using a Kalman filter.

[0061] This can advantageously increase the quality of object formation and at the same time limit the system's additional investigative effort to unclear cases.

[0062] In exemplary embodiments, at least the content or characteristics of a predefined geometric shape in the image of the vehicle's surroundings can alternatively be examined for predefined object elements. Furthermore, the identified object elements and / or the content or characteristics of the predefined geometric shape in the image of the vehicle's surroundings can be examined for predefined features.

[0063] Geometric shapes can be designed as rectangles, for example, for detecting vehicles or pedestrians. Alternatively, the geometric shapes can also be designed as clothoids, which are particularly suitable for detecting the course of the road. A clothoid is a special flat curve that can be uniquely identified in the plane, except for similarity, by its properties. It is used as a transition curve in curves in road and railway construction. Its curvature increases linearly and ensures smooth driving dynamics.

[0064] As an alternative to the bottom-up method, the top-down method described here can also be used. The choice of method may depend on the design of the sensor data processing system.

[0065] Optionally, determining the object can include an object assessment with an existence probability to indicate the probability that the object actually exists. In this case, the existence probability of an object in the driver's line of sight, determined in a first step without taking the line of sight into account, is increased in a second step. Additionally or alternatively, the existence probability of the object outside the driver's line of sight, determined in a first step without taking the line of sight into account, can be decreased in a second step.

[0066] The probability of existence can be understood as an expression or measure of certainty that the detected object actually exists. High certainty is also referred to as evidence, as already explained. The probability of existence can at least assume a value that provides the evidence necessary for detecting the object. Furthermore, the probability of existence can be further increased beyond the evidence. In other words, the certainty of existence can be further increased. For example, for objects that have already been detected, the driver's gaze can increase the certainty of the object's existence.

[0067] However, the data collected from the recorded physiological parameters can also reduce the determined probability of existence. This can be particularly the case if an object with a clear probability of existence is not in the driver's line of sight, especially if it is located adjacent to the driver's line of sight.

[0068] Advantageously, with an increased probability of existence, more drastic interventions in the vehicle's driving behavior can be made, and / or the measures can be initiated earlier and / or be more extensive. Alternatively, if the driver focuses on other objects, a reduced probability of existence of objects can lead to the deletion of unsafe objects.

[0069] Further optionally, the evaluation of the data corresponding to a physiological parameter of the driver can indicate that the driver is startled.

[0070] A sudden increase in alertness can be detected in varying degrees. The most significant change in physiological parameters can be interpreted as a driver being startled. Less significant changes can also be attributed to a different level of alertness change, such as surprise. Accordingly, by evaluating the physiological parameters, a gradation of the observed driver's state can be determined.

[0071] This allows the system to intervene drastically in the driving behavior when the driver is startled, which is the highest level of driver attention.

[0072] Further optionally, data for detecting at least one physiological parameter of the driver can be determined with the aid of at least one sensor worn on the body and / or located in the body and / or located in the passenger compartment of a vehicle.

[0073] This advantageously allows for flexible adaptation to specific circumstances. For example, if the driver wears sensors on or in their body, the corresponding data can be evaluated. However, even without such body-mounted sensors, the driver's physiological parameters can be determined.

[0074] In embodiments, the data of the driver's physiological parameters may reveal dilation of the pupils and / or increase in heart rate and / or perspiration and / or changes in the blinking of the eye.

[0075] Advantageously, the driver’s attention can be reliably determined by combining as many different parameters as possible.

[0076] Optionally, an embodiment of the sensor arranged in the passenger compartment of a vehicle may include an interior camera and / or glasses worn by the driver.

[0077] This makes it possible to determine the driver’s viewing direction precisely.

[0078] Body-worn sensors can be designed as items of clothing or jewelry with integrated sensors. For example, wristbands can be used to record blood pressure and pulse rate. These are usually worn directly on the driver's skin. Sweating can also be measured using such so-called wearables. Additionally or alternatively, specific glasses worn by the driver can record the driver's physiological parameters. For example, the driver's line of sight can be recorded and communicated to the vehicle wirelessly. Skin temperature measurement or sweating detection can also be carried out in this way. Sensors located in the body can be pill-shaped and can be swallowed. They can detect chemical reactions in the body and, for example, also communicate corresponding values ​​to the vehicle wirelessly.Sensors arranged in the passenger compartment of a vehicle can include an optical camera and / or an infrared camera. These, in turn, can be used to determine the driver's line of sight. Furthermore, fatigue, inattention, or microsleep can be detected. Furthermore, the pulse rate, for example, from the steering wheel, can be detected. Preferably, the data from all available sensors is evaluated together.

[0079] A further embodiment of the invention comprises a computer program for performing at least one step of the above-mentioned methods. The computer program runs on a programmable hardware component.

[0080] This makes it advantageous to easily change the function, as the programmable hardware component does not need to be changed in many cases.

[0081] A further embodiment of the invention relates to a system for influencing active vehicle safety. It comprises a sensor unit for detecting the surroundings of a vehicle. It further comprises a detection processor for detecting a sudden increase in the driver's attention by evaluating at least one physiological parameter of the driver. It further comprises a determination processor for determining the driver's gaze direction at the time of the sudden increase in attention. It further comprises a determination processor for determining an object in the surroundings of the vehicle, taking into account the sudden increase in attention and the gaze direction.

[0082] The driver’s hazard detection can thus be advantageously incorporated into the determination of objects in the system.

[0083] Optionally, the recognition processor can be coupled to at least one detection device that is fixed to the vehicle and / or arranged on the driver and / or arranged in the driver.

[0084] These detection devices can be worn on the body, for example in the form of glasses or a bracelet, and / or be located inside the body.

[0085] The data from the vehicle-mounted recording devices can advantageously be supplemented by further data that describe the physiognomic state of the driver.

[0086] Some exemplary embodiments of the present invention are explained in more detail below with reference to the accompanying figures. They show: Fig. 1 a method according to an embodiment of the invention Fig. 2 a form of an embodiment of the invention in hierarchical form Fig. 3 a form of an embodiment of the invention with simulation Fig. 4 a further embodiment of the invention with simulation Fig. 5 a form of an embodiment of the invention with respect to the probability of existence Fig. 6 an embodiment of the invention implemented as a system as a basic circuit diagram Fig. 7 an embodiment of the invention implemented in more detail as a system

[0087] Various embodiments will now be described in more detail with reference to the accompanying drawings, in which some embodiments are illustrated. In the figures, the thickness dimensions of lines, layers, and / or regions may be exaggerated for clarity.

[0088] In the following description of the attached figures, which show only a few exemplary embodiments, identical reference numerals may designate identical or comparable components. Furthermore, collective reference numerals may be used for components and objects that appear multiple times in an exemplary embodiment or in a drawing, but are described together with respect to one or more features. Components or objects described with identical or collective reference numerals may be identical with respect to individual, several, or all features, for example, their dimensions, but may also be different, unless the description explicitly or implicitly indicates otherwise.

[0089] Although embodiments are susceptible to various modifications and variations, embodiments are illustrated in the figures as examples and will be described in detail herein. It should be understood, however, that embodiments are not intended to limit the specific forms disclosed, but rather, embodiments are intended to cover all functional and / or structural modifications, equivalents, and alternatives within the scope of the invention. Like reference numerals designate like or similar elements throughout the description of the figures.

[0090] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments. As used herein, the singular forms "a," "an," "another," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. Further, it is to be understood that terms such as "includes," "including," "comprises," and / or "having," as used herein, indicate the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more features, integers, steps, operations, elements, components, and / or groups thereof.

[0091] Fig. 1 shows a method according to an embodiment of the invention in a basic circuit diagram. Fig. 1 shows a method for object detection in the environment of a vehicle using driver observation 100. A step 110 of this method, represented as a rectangle, comprises detecting a sudden increase in the driver's attention by evaluating data from at least one sensor for recording the driver's physiological parameters. This can include detecting the sudden dilation of the pupils. A second step of the method 120, also represented as a rectangle, comprises determining the driver's gaze direction by evaluating data from at least one further sensor at the time of the sudden increase in attention. This determination can be supported by an interior camera or suitable driver glasses.In step 130, also depicted as a rectangle, the detection of the suddenly increased attention 110 and the determination of the driver's gaze direction 120 are taken into account, which is represented by arrows from step 110 to step 130 and from step 120 to step 130. Furthermore, an arrow points to the representation of step 130, symbolizing data from the vehicle's surroundings. In step 130, an object in the vehicle's surroundings is now determined, taking the three input variables into account.

[0092] Fig. Figure 2 shows a form of an embodiment of the invention in hierarchical form 140. A rectangle represents an image of the vehicle's surroundings with data at a first resolution 142. A rectangle 143 is placed within this rectangle, representing the data in the driver's line of sight and the corresponding data from the vehicle's surroundings at a second resolution. All data is examined for features 134 represented as ovals, which comprise simple geometric shapes. A further oval 133 encompassing the oval 134 represents object elements that may be composed of features 134. Object elements can also be extracted from the data. An oval 132 encompassing the oval 133 represents objects that may be composed of object elements 133. The ovals are placed, by way of example, in the area of ​​the data with the first resolution 142. The data with the second resolution 143 is also searched in the same way (not shown).

[0093] Fig. 3 shows a form of an embodiment of the invention with simulation. Three rectangles are shown containing object elements. The left rectangle 145 shows object elements that were determined from the search of the image of the vehicle's surroundings, for example, by composing corresponding features. The object elements shown refer to a person and are represented as a head fragment, a left arm, a torso, and a right leg. These object elements are not sufficient to recognize the object, in this case the person. If the recognized object elements are in the field of view of the driver, simulated object elements are added that correspond to a known object hypothesis. This object hypothesis can be selected depending on the driver's reaction and the recognized object elements.The simulated object elements are represented by dashed lines in rectangle 146 and include the complete head, both arms, both legs, and the torso. A "plus" sign between rectangles 145 and 146 represents the combination of both types of object elements. An arrow points to rectangle 147, which shows the sum of the identified and simulated object elements. Identified object elements are represented as solid lines. Simulated object elements are represented as dashed lines. According to the invention, it can be checked before additions whether the simulated object elements match the corresponding identified object elements, as already described.

[0094] Fig. 4 shows a further embodiment of the invention with simulation. A decision diamond indicates a test step 150. This checks whether the object formation in the area of ​​the driver's line of sight has not yet led to an identified object. An arrow points to the diamond, representing data that includes identified object elements and objects identified from them in the area of ​​the driver's line of sight. In the case of objects that have already been identified, an arrow from the right-hand diamond tip leads to further processing 154. In the case of objects that have not yet been identified, an arrow from the left-hand diamond tip leads to rectangle 152, which represents the addition of the simulated object elements to the identified object elements. An arrow leads to rectangle 152, which originates from a plurality of object hypotheses 132 represented as rectangles. This represents the selection of the appropriate object hypothesis from the plurality of available object hypotheses.In exemplary embodiments, process 152 may also include object formation based on the supplemented object elements, so that subsequently identified objects are available. An arrow from rectangle 152 to further processing 154 indicates the further processing of the supplemented object elements. This further processing 154 may result in intervention in the vehicle's behavior or include a warning to the driver.

[0095] Fig. 5 shows an embodiment of the invention with respect to existence probability. Rectangle 180 shows a determination of the existence probability of a detected object, as already described. An arrow leads from rectangle 180 to the tip of a diamond. This symbolizes the further processing of the existence probability. If the object is located in the driver's line of sight, an arrow marked with "+" leads from the left diamond tip to process 185. In process 185, the value of the existence probability is increased. If the object is located outside the driver's line of sight, an arrow marked with "-" leads from the right diamond tip to process 190. In process 190, the value of the existence probability is decreased.

[0096] Fig. 6 shows an exemplary embodiment of the invention embodied as a system as a basic circuit diagram. Various sensor and processor units are arranged in the system 200, which is designed as a rectangle. A sensor unit 210 receives sensor data, which is represented as an arrow pointing toward the sensor unit 210. This sensor data represents the vehicle's surroundings. The system 200 further comprises a detection processor 220. This detects a sudden increase in the driver's attention by evaluating the data of at least one of the driver's physiological parameters. The data flow is represented as an arrow pointing toward the detection processor 220. The system 200 further comprises a determination processor 230. This determines the driver's direction of gaze at the time of the sudden increase in attention.An arrow pointing to the detection processor 230 symbolizes the associated data, which can be generated, for example, by an interior camera or special driver glasses. The system 200 further includes a determination processor 240. A combined arrow leads from the sensor unit 210, the recognition processor 220, and the detection processor 230 to the determination processor 240. This arrow symbolizes a data flow to the determination processor 240. The latter uses this data to determine an object in the vehicle's surroundings, taking into account the suddenly increased attention and the direction of gaze.

[0097] Fig.Figure 7 shows the embodiment of the invention implemented in more detail as a system. Here, the sensor 301 acquires data from the vehicle's surroundings. An arrow to the sensor data processing 302 marks the data transport. The sensor data processing 302 includes the feature extraction 303 for recognizing features from the data from the vehicle's surroundings. The feature extraction 303 sends the recognized features to form an object hypothesis 304. This, in turn, sends the object hypotheses to the object update 305, indicated by an arrow. As an output of the sensor data processing, a signal is fed to a driver assistance function 312 at a decision diamond 313. An arrow leads from this decision diamond 313 to the object update 305. The figure also includes a sensor for gaze direction detection 306, which is connected to a coordinate transformation into the sensor coordinate system 307. The data from this sensor data flows to an object recognition detection 309.Furthermore, a sensor unit for physiological variables 310 is shown with a data flow for data processing and further to object recognition detection 309. A separate summary of the coordinate transformation into the sensor coordinate system 307 and the object recognition detection 309 leads to the decision diamond 313 in the driver assistance function 312. An output of the decision diamond 313 leads from the driver assistance function 312 to the function block 314 for further processing in the case of evident objects.

[0098] Embodiments of the invention can be summarized as follows: In research projects for the development of active safety systems, the problem often arises that the driver's assessment of a system reaction cannot be predicted. However, this purely subjective assessment by the driver is of paramount importance, since the intervention of an active safety system can lead to a loss of confidence in the system if the driver perceives the intervention as unjustified.

[0099] To minimize the risk of an intervention or warning that is perceived as unnecessary or even incorrect, such systems are parameterized conservatively, meaning that a warning or intervention is triggered as late as possible. However, this contradicts the fact that early intervention can significantly reduce the potential consequences of a collision, since the kinetic energy during a collision decreases quadratically with the time interval between collisions, or a collision can be completely avoided by early braking or evasive maneuvering.

[0100] It is known that modeling and observation of the driver can be used to adapt the parameters of a driver assistance system to warn of or avoid a collision. Furthermore, it is known that collision avoidance systems can incorporate the driver's attention into the calculation of the trigger point. According to the state of the art, this can be achieved by recording steering activity, pedal actuation, blinking observations using interior cameras, or by measuring the pulse at the steering wheel.

[0101] Existing solutions determine driver-specific data using sensors installed in the vehicle. Determining physiological data, such as heart rate, can be particularly problematic, as existing systems measure heart rate at the steering wheel. However, if the driver only lightly touches the steering wheel, a robust heart rate estimate cannot be made. This is particularly problematic in automated driving, as this value cannot be measured, as hands-off driving may also be permitted. Heart rate measurement using sensors installed in the seat can also fail if excessively thick clothing prevents a measurement.

[0102] According to the invention, physiological data is collected via wearables (e.g., smart watches, glasses) or body-mounted sensors (e.g., smart pills) to determine the driver's state. In particular, a unit is proposed that determines whether a driver jumps when a hazard is detected. Any data measurable by the wearable or body-internal sensors can be used (heart rate, chemical indication, etc.). Via a communication interface between the wearable or body-internal sensors and the vehicle (e.g., Bluetooth, WLAN (Wireless Local Area Network), NFC (Near Field Communication), etc.), information about the driver's state (in particular, the moment of startle, but also fatigue / secondary sleep, inattention) can be provided to the computing unit of a collision avoidance or collision warning system.

[0103] The sensors for determining driver status do not need to be permanently installed in the vehicle. This means there are no additional costs for the vehicle itself. Since the driver has purchased and used the required sensors themselves, the acceptance of using the data in the vehicle is expected to be high. The sensors used originate from the consumer electronics sector.

[0104] Furthermore, the detection of whether a driver considers a situation dangerous (startle) has not yet been directly implemented in a collision avoidance system. This information can be used to ensure that an intervention is in the driver's best interest. This means that, under certain circumstances, an intervention can be carried out earlier – with the driver's "consent" – without the risk of false triggering due to the driver's subjective assessment. The use of wearables or body-internal sensors as sensors for driver condition monitoring can be used in automated driving when contact measurement via a steering wheel is not possible or the driver's body posture does not allow observation by an in-vehicle camera (for example, due to variably rotatable seats in an automated vehicle).

[0105] The preferred design uses a smartwatch to determine the pulse. This data is transmitted to the processing unit of a collision avoidance system via a communication interface between the smartwatch and the vehicle. This processing unit calculates the probability that the driver considers emergency intervention justified, for example, based on startle detection, which is to be carried out using the measured pulse.

[0106] If the computer assesses that the driver considers possible intervention necessary, and the collision avoidance system's situation detection also identifies a critical situation, the system can intervene. This intervention is carried out with the driver's consent.

[0107] In addition to heart rate, other physiological variables can also be used to assess the driver's condition. Furthermore, the information can be provided via other conceivable wearables (e.g., a wristband, T-shirt, etc.) or via internal body sensors.

[0108] Another embodiment uses startle detection to precondition the perception of a collision avoidance system. The driver's gaze direction during the startle moment can also be observed. This information can then be used to re-scan a specific area of ​​the environment for hazards with increased computing power.

[0109] In contrast to known solutions, the proposed method is robust to the driver's pose (i.e., the driver's pose within the vehicle interior). Furthermore, the robustness of the driver state estimation is increased by the fact that wearables are worn directly on the skin. This makes the method suitable, among other things, for monitoring the driver during automated driving (hands-off). Acceptance of this type of data acquisition is likely to be high, as the sensors are purchased and used by the driver themselves and originate from the consumer electronics sector.

[0110] Furthermore, the following can be said regarding improved perception using driver state monitoring: The idea can be used to improve the sensory perception of a vehicle. The vehicle is equipped with various sensors, including a camera with subsequent image processing.

[0111] In a first step, the driver's physiological parameters are measured. Wearables or vehicle-mounted sensors can be used for this purpose. Through continuous observation, moments of shock or times with particular characteristics in the observed parameters can be identified (e.g., dilation of the pupils, increased heart rate, etc.).

[0112] In the second step, the direction of view during the above-mentioned time is determined using an interior camera (or special glasses).

[0113] This information can be used to improve the perception / object recognition of a system as follows. 1st Possibility: (Object formation of perception already completed) If the driver reacts as described above, the presence measure for an object perceived by the sensory system can be increased if the gaze points to the presumed object position. The driver's reaction, together with the direction of gaze toward the position where the sensory system sees the object, then provides additional evidence for the presence of an object. Advantages here are: Better quality of the objects seen → fewer false alarms when used in an active vehicle safety system 2nd possibility: (Object formation not yet complete) The vehicle's perception system has not yet formed an object hypothesis due to a lack of features at the object formation level. However, the driver's reaction suggests that an object is present. Two possibilities arise: Based on the evidence from the driver's observation and the weak features found in the image, an object hypothesis can be formulated. For this purpose, an object hypothesis could also be suggested based on the driver's reaction. Subsequently, it is checked whether the features recognized in the image fit the proposed object hypothesis. Put simply, the object hypothesis suggestion based on the driver's reaction allows a re-evaluation of the features already found, but which are not sufficient on their own.

[0114] The driver's gaze direction indicates an area in the image where an object is expected based on the driver's reaction. The subsequent search can then focus on the smaller portion of the image. The image can be searched again there without prior downscaling (often common when searching for features in a large image). This makes it possible to search the relevant area of ​​the image and apply increased computing power there, in a way that would not be possible for the entire image.

[0115] In summary, a vehicle is provided with at least one first sensor device (301) including sensor data processing (302) and at least one second sensor device (306), as well as an evaluation unit (309). The first sensor device, including sensor data processing, provides information about objects in the vehicle's surroundings in the form of a list of sensor objects. Sensor objects represent detected objects (305), and the sensor objects include at least the position and the probability of existence of the detected object as attributes. Sensor data processing (302) is a step-by-step process that represents a directed processing flow that starts with the raw data from the sensors (e.g., 3D LIDAR reflection points in space, radar reflections, features extracted from camera images) and ends with an environment image (sensor objects).Methods for sensor data processing that generate sensor objects with the attributes described above from the raw data are known.

[0116] Examples of features extracted from camera images can include lane markings and abstract features such as edges at grayscale or color transitions. Furthermore, features such as extremities, head poses, or similar can be extracted from image data using known methods (see [Suddert], [Sigal]). These individual features are referred to below as object elements, since several of these object elements can usually be combined to form a single object. (For pedestrians, see [Sigal]; for roadway topology, see [Töpfer]). This process is understood as a bottom-up process, since relatively abstract environmental features or object elements are combined to form complex objects.

[0117] Other approaches to object detection use a top-down method. Object hypotheses (e.g., boxes for vehicles or pedestrians, clothoids for lane markings) are directly fitted, i.e., merged, with environmental features.

[0118] If there is sufficient similarity between object hypotheses and raw data / features, an object is created.

[0119] Accordingly, both bottom-up and top-down methods use objects (c) as well as intermediate results at different levels of abstraction (for example, features (a), object elements (b) (set of features that abstractly represent information in order to minimize, for example, memory or bus load).

[0120] In the proposed method, the intermediate results of this sensor data processing are stored such that (a) describe the set of abstract features, (b) the set of object elements, and (c) the set of objects.

[0121] The second sensor device (306) is used to detect the driver's gaze direction vector (known methods), which is evaluated in a computing unit (307) using coordinate transformations so that the gaze direction vector and the sensor objects can be viewed in the same coordinate system. Consequently, according to (307), it is known which point in the vehicle's surroundings the driver's gaze falls on. Specifically, this step could look like this: Determination of the gaze direction vector using an indoor camera, initially individually for both eyes, point of the gaze direction line in the pupils, subsequent averaging of the two lines to x→Viewing direction. The coordinate transformation into the sensor coordinate system (vehicle KOS) results in a straight line for the driver’s line of sight to x→Viewing direction.Fag.−KGS=xEye position,Vehicle−KOS+λ x→Viewing direction

[0122] Points or objects in the vehicle's surroundings that are hit by the driver's gaze can now be determined by checking the straight line from equation 0 for intersection with relevant objects.

[0123] The third sensor device (310) serves to measure physiological variables. Generally, the third sensor unit is different from the second sensor device; however, in particular, physiological characteristics can also be measured via the second sensor device (e.g., pupil dilation via an interior camera), so that the third sensor unit is omitted. In the selected embodiment, the third sensor device is a wearable that can measure the heart rate.

[0124] Due to the temporal course of the gaze direction vector (fixation duration of the gaze on the object above a threshold value Δt Fix,for example 300 milliseconds) in combination with the measured physiological variables of the third sensor device, a probability P Fahrer (k) (311) can be used to estimate the time with which the driver detected an object. The variable k denotes the current time step.

[0125] For this purpose, characteristic deviations from normal behavior are searched for in the signal of the physiological variables (known methods).

[0126] If the estimated probability (309) exceeds a limit value, this and the viewing direction vector (308) are passed on to the sensor data processing of the first sensor device.

[0127] With the help of the viewing direction vector (308) a spatial region of interest (ROI) is defined.

[0128] A distance metric is used to check whether elements of the sets (a), (b), or (c) are located within the ROI (association).

[0129] The following is a case differentiation: If elements of set (c) are located in the ROI, go to case 1. If this is not the case, go to case 2. The cases differ in that in the first variant, plausible objects (often referred to as object candidates) were identified during sensor data processing. In the second variant, no plausible objects or no objects at all were found. The term plausible here makes it clear that different strategies can be applied during sensor data processing. For example, implausible hypotheses can be excluded at each calculation step for performance reasons. This can result in object elements being excluded and thus also objects. In this case, it can happen that no objects are detected. If these hard decisions are not made, then even implausible object elements are used to form objects.In this case, implausible object hypotheses arise.

[0130] The invention described here is basically applicable to both processing strategies. However, the explanation assumes processing with hard decisions, as this corresponds more closely to the real application case.

[0131] Case 1: An element of set (c) was found in the ROI. An element of set (c) is a sensor object, as described above. It therefore has an existence probability P as an attribute. Sensor (k), where k denotes the current time step of the calculation method. Using the gaze direction vector and the driver's reaction, an existence probability for an object was estimated. Now, the existence probability of the object is calculated according to the formula Pobj(k)=Psensor(k)δs⋅PDriver(k)δF where δS+δF=1 changed if P Fahrer (k) > P Sensor(k). The variables δ S and δ F are parameters with which the weighting of the driver's reaction can be determined.

[0132] The increased probability of driver observation allows for intervention by the driver assistance system, which would not have been possible due to the uncertain knowledge of the vehicle's sensors alone. The response of the driver assistance function is thus improved.

[0133] If no object (element of set (c)) is found, case 2 applies. In case 2, the ROI is first searched for elements of set (b). Since the ROI represents only a subset of the entire environment image, this search can be performed at a higher resolution (e.g., while maintaining the same search time or computational complexity).

[0134] It is also conceivable that, based on the driver's reaction, increased computing power could be released for the search. This could be achieved, for example, by temporarily utilizing additional computing time available in the vehicle.

[0135] After searching with higher computing power or higher resolution in image processing methods, a new set (b') is obtained, which contains additional object elements.

[0136] This new set (b') is then passed on to the processing chain. A new set (c') results from (b') according to the processing chain. This allows objects to be detected that could not have been detected without the driver's reaction (e.g., because the resolution was too low due to the size of the search area). It can be assumed here that calculating (c') from (b') does not require more computing power than calculating (c) from (b). However, if (b, b') were to be calculated without any reason (triggered by the driver model), i.e., permanently from (c) and (c'), more computing power would be required.

[0137] A method for object formation or feature extraction is known, for example, from WO 2009019250 A2.

[0138] If no new object element is found in the ROI, i.e., (b) = (b'), the ROI is searched for relevant environmental features, resulting in a new set (a'), which is then passed on to the processing chain. The result is (c'').

[0139] Another embodiment provides not only a bottom-up search for new features in the ROI, but also a top-down search of a set of objects (c*) in the ROI, and an attempt is made to align the found features (a) with (c*). Furthermore, (c*) can also be aligned with previously calculated object elements (b*). Furthermore, it is also conceivable to decompose (c*) into object elements (b*) using spatial relations in order to align (b*) with (a). The chosen variant depends on the sensor data processing configuration.

[0140] Furthermore, if the driver fails to respond, it is possible to reduce the plausibility of objects from (c) in the ROI. This can be done, for example, indirectly proportional to the estimated driver attention. This is useful to reduce false triggers. For example, an invalid object may be passed to the function, and the function may react to it. If the driver is very attentive and does not react to the object that triggers the function response, the function response can be suppressed or delayed, for example, by changing the object plausibility.

[0141] Although some aspects have been described in connection with a device, it is understood that these aspects also represent a description of the corresponding method, so that a block or component of a device can also be understood as a corresponding method step or as a feature of a method step. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block, detail, or feature of a corresponding device.

[0142] A programmable hardware component can be formed by a processor, in particular by a computer processor (CPU = Central Processing Unit), a graphics processor (GPU = Graphics Processing Unit), a computer, a computer system, an application-specific integrated circuit (ASIC = Application-Specific Integrated Circuit), an integrated circuit (IC = Integrated Circuit), a single-chip system (SOC = System on Chip), a programmable logic element or a field-programmable gate array with a microprocessor (FPGA = Field Programmable Gate Array).

[0143] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. Therefore, it is intended that the invention be limited only by the scope of the following claims and not by the specific details presented in the description and explanation of the embodiments herein. List of reference symbols 100 methods for object recognition 110 Recognizing sudden increased attention 120 Determining the driver's line of sight 130 Identifying an object 131 (identifiable) object in the vicinity of a vehicle 132 Object hypothesis 133 (identified) object element 134 simulated data in a first resolution 135 feature 140 Image of the surroundings of a vehicle 142 data in a first resolution 143 data in a second resolution 145 identified object elements in the driver’s line of sight 146 simulated object elements 147 identified object elements supplemented by simulated object elements 150 test steps 152 Supplementing the identified object elements with an object hypothesis 154 Further processing 180 Object valuation with a probability of existence 185 Increasing the probability of existence 190 Lowering the probability of existence 200 System for influencing active vehicle safety 210 Sensor unit 220 recognition processor 230 Investigation Processor 240 Determination Processor 301 Sensor 302 Sensor data processing 303 Feature extraction 304 Formation of object hypothesis 305 Object update 306 Gaze direction detection sensor 307 Data processing 309 Object recognition detection 310 Physiological size sensor 312 Driver assistance function 313 object records available 314 Vehicle influence / warning

Claims

[1] Method for determining objects in the environment of a vehicle using driver observation (100), comprising: - detecting (110) a sudden increase in the driver's attention by evaluating data from at least one sensor for detecting physiological parameters of the driver; - determining (120) a direction of gaze of the driver by evaluating data from at least one further sensor at the time of the sudden increase in attention; and - Determining (130) an object in the surroundings of the vehicle (131) in the direction of view of the driver, taking into account the suddenly increased attention. [2] Method according to claim 1, wherein the object (131) to be determined in the surroundings of a vehicle comprises object elements (133) which in turn comprise features (135), wherein an identification of the object elements (133) and / or the features (135) is based on a search of an image of the surroundings of the vehicle (140) with data in a first resolution of the image (142), wherein in the area of ​​the driver's line of sight the identification of the object elements (133) and / or the features (135) is based on a search of the image with data in a second resolution of the image (143), and wherein the second resolution (143) is finer than the first resolution (142). [3] The method of claim 2, wherein the data in the first resolution is searched with a first processor power, and wherein the data in the second resolution is searched with a second processor power, the second processor power being higher than the first processor power. [4] Method according to one of the preceding claims, wherein object elements (145) identified in the area of ​​the driver's line of sight are supplemented by simulated object elements (146) which are assigned to an object hypothesis (132), wherein the object hypothesis (132) comprises all object elements of an object. [5] Method according to claim 4, wherein it is checked whether the corresponding simulated object elements (146) can be matched with the identified object elements (145) by comparing typical properties of the object elements, wherein in the case of a match between the corresponding simulated object elements (146) and the identified object elements (145), the determination of the object (131) takes place taking into account the identified object elements (145) and the simulated object elements (146) of the object hypothesis. [6] Method according to one of claims 4 or 5, wherein a plurality of types of object hypotheses (132) are stored, wherein the types of the object hypotheses correspond to an expectability of objects in the vehicle environment, wherein the type of object hypothesis used depends on data describing a driver reaction; and / or wherein the object hypothesis (132) is only supplemented (152) if a test step (150) shows that object formation in the area of ​​the driver's line of sight has not yet led to a determinable object (131), wherein the object formation has the aim of converting the object elements into an object hypothesis. [7] Method according to one of claims 1 to 3, wherein the image of the surroundings of the vehicle is examined for predetermined object elements or for at least one manifestation of a predetermined geometric shape in order to identify object elements, and wherein the identified object elements or the manifestation of the predetermined geometric shape in the image of the surroundings of the vehicle is examined for predetermined features (135). [8] Method according to one of the preceding claims, wherein the determination of the object comprises an object evaluation of a probability of existence to indicate the probability that the object actually exists, wherein the object evaluation of the probability of existence of an object (180) determined in a first step without taking the line of sight into account is increased (185) in a second step for an object in the area of ​​the driver's line of sight, and / or wherein the object evaluation of the probability of existence of the object determined in a first step without taking the line of sight into account is decreased (190) in a second step for an object outside the area of ​​the driver's line of sight. [9] Method according to one of the preceding claims, wherein the evaluation of the data corresponding to a physiological parameter of the driver comprises an evaluation with regard to a fright of the driver, wherein the fright of the driver is determined when a degree of change of the observed parameter is exceeded. [10] Method according to one of the preceding claims, wherein the data for detecting at least one physiological parameter of the driver are determined with the aid of at least one sensor worn on the driver's body and / or located in the driver's body and / or located in a vehicle in the passenger compartment. [11] Method according to one of the preceding claims, wherein the evaluation of the data of the physiological parameter of the driver comprises the dilation of the pupils and / or the increase in pulse and / or perspiration and / or the changed blinking of the eye by evaluating data from at least one sensor for detecting physiological parameters of the driver. [12] Method according to one of the preceding claims, wherein an embodiment of the sensors arranged in the passenger compartment of a vehicle comprises an interior camera and / or glasses worn by the driver. [13] System for influencing active vehicle safety by automatically intervening in the driving behavior of a vehicle or by warning the driver (200), comprising - A first sensor unit (210, 301) for detecting data from the surroundings of the vehicle; - A second sensor unit (310) for detecting data of at least one physiological parameter of the driver; - A detection processor (220) designed as a programmable hardware component, which is connected to the second sensor unit for detecting a sudden increase in the driver's attention by evaluating the data of at least one physiological parameter of the driver; - A third sensor unit (306) for detecting data of the driver's line of sight; - A determination processor (230, 309) designed as a programmable hardware component, which is connected to the third sensor unit for determining the driver's line of sight by evaluating the data of the detected driver's line of sight at the time of the sudden increase in attention; - A determination processor (240, 302 / 312) designed as a programmable hardware component, which is connected to the first sensor unit (210, 301), the recognition processor (220) and the determination processor (230, 309) for determining an object in the surroundings of the vehicle in the direction of view of the driver, taking into account the suddenly increased attention and the data of the surroundings of the vehicle. [14] The system of claim 13, wherein the detection processor is coupled to at least one detection device fixed to the vehicle or arranged on the driver or arranged in the driver.

Images