Method for adapting a recognition function to determine properties of at least one object and method for determining properties of at least one object, at least one living being and / or at least one person

Abstract

A method (30) is proposed for adapting a recognition function for determining properties of at least one object (58), at least one living being and / or at least one person (24), in particular in a vehicle (20), comprising the steps, Receiving observation data of at least one object (58), at least one living being and / or at least one person (24) from an observation device (22), Determining the brightness and stability of the observation data and verifying the stability of the brightness, checking whether the stability of the brightness meets a predetermined minimum stability, and / or determining the quality of the observation data and verifying the determined quality, checking whether the determined quality meets a predetermined minimum quality, adjusting the detection function depending on the stability of the brightness of the observation data and / or the quality of the observation data if the stability of the brightness is worse than the predetermined minimum stability and / or if the determined quality is worse than the predetermined minimum quality.

Description

State of the art

[0001] From DE 10 2014 214 352 A1 a method and an arrangement for operating an occupant observation system are known. Disclosure of the invention

[0002] A method is proposed for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person, in particular in a vehicle, comprising the steps of receiving observation data of the at least one object, the at least one living being and / or the at least one person from an observation device, Determining a brightness of the observation data and a stability of the brightness of the observation data and checking the stability of the brightness, checking whether the stability of the brightness corresponds to a predetermined minimum stability, and / or determining a quality of the observation data and checking the determined quality, checking whether the determined quality corresponds to a predetermined minimum quality, Adapting the detection function depending on the stability of the brightness of the observation data and / or the quality of the observation data if the stability of the brightness is worse than the specified minimum stability and / or if the determined quality is worse than the specified minimum quality.

[0003] The advantage of the present invention is that the recognition function can be adapted, in particular safely and / or reliably, to the current conditions, the quality of the observation data, and / or the brightness. This allows the recognition function to be used to recognize properties, in particular safely and / or reliably. Advantageously, the quality of the results of the recognition function can be improved or optimized. This allows for better determination of properties of the at least one object, the at least one living being, and / or the at least one person. Advantageously, the brightness and / or the quality of the observation data can be analyzed in such a way that the recognition function can be improved accordingly.This allows for reliable observation of the at least one object, the at least one living being, and / or the at least one person. If the at least one object, the at least one living being, and / or the at least one person is or are located in a vehicle, monitoring of the vehicle interior, including the occupants and / or objects, can be ensured. This can, in particular, increase road safety.

[0004] In a further development, the method comprises a further step of observing the at least one object, the at least one living being, and / or the at least one person using the observation device to generate observation data. This allows, in particular, observation data to be generated safely and / or reliably using the observation device. This allows for adaptation of the recognition function, in particular safely and / or reliably.

[0005] In an exemplary embodiment, individual images of the observation data can be analyzed to determine the brightness of the observation data and / or the stability of the brightness of the observation data and / or to determine the quality of the observation data. Advantageously, the brightness, sharpness and / or contrast of the observation data can be analyzed to determine the quality of the observation data. This analysis can relate to the entire image or to individual regions of the image. These regions can, for example, be defined in advance and relate only to specific detected objects and / or detected body parts of one or more persons or living beings and / or the background of the individual image or specific parts of the background, such as the roof lining or a window area. These regions can also be determined automatically by a learning system.Analyzed areas of the individual images can be stable or variable over time.

[0006] Alternatively or additionally, one or more of the following measures can be carried out to determine the quality of the observation data: - Analysis of a signal-to-noise ratio, - Analysis of dynamic range and entropy.

[0007] Alternatively or additionally, one or more of the following measures may be taken to determine the stability of the brightness of the observation data: - Determining a dispersion measure of an average image brightness of individual images in a certain period of time, - Determining an average brightness change in given areas of the individual images, - Identification of a background of the at least one object, the at least one living being and / or the at least one person and determining a dispersion measure of an image brightness in the background, - Identification of shadow areas and detection of a movement of a shadow across an image and / or in one or more specified areas, - Performing a histogram analysis, - Analysis of a distribution of gray values, - Analysis of a dynamic range, - Calculation of the Kullback-Leibler divergence, - Performing a self-monitoring machine learning process with reference data, - Determining changes in an optical flow, - Determining a normalized cross correlation, - Perform a frequency analysis or a Fourier transform and analyze the distribution in the frequency domain.

[0008] In this way, the quality of the observation data and / or the stability of the brightness can be determined in a particularly safe and / or reliable manner, which advantageously enables an adaptation of the detection function in a particularly safe and / or reliable manner.

[0009] In a further development, the detection function can be adapted by performing one or more of the following measures: - Discarding at least part of the observation data, - Calculation of quality measures and / or confidence measures for estimation results and / or at least part of the observation data, - Determining one or more weights for a fusion with observation data from other periods, - Determining one or more weights for fusion with observation data from another observation device and / or other observation data from other sensors.

[0010] Alternatively or additionally, the detection function can be adapted by controlling an illumination unit of the observation device accordingly, depending on the stability of the brightness of the observation data and / or the quality of the observation data.

[0011] As a result, an adaptation of the detection function can be carried out in a particularly safe and / or reliable manner, whereby an observation of the at least one object, the at least one living being and / or the at least one person can be ensured in a particularly reliable manner.

[0012] Furthermore, a method is proposed for determining properties of at least one object, at least one living being and / or at least one person, in particular in a vehicle, comprising the steps of Receiving observation data of the at least one object, the at least one living being and / or the at least one person from an observation device, Evaluating the observation data by means of the adapted recognition function according to one of the preceding claims to determine properties of the at least one object, the at least one living being and / or the at least one person.

[0013] This allows properties of the at least one object, the at least one living being, and / or the at least one person to be determined in a particularly safe and / or reliable manner, thereby ensuring reliable observation of these objects. For example, an appropriate response can be made in an emergency. If the at least one object, the at least one living being, and / or the at least one person are located in a vehicle, the vehicle interior can be observed in a particularly safe and / or reliable manner. This can increase safety in the vehicle and thus in road traffic.

[0014] Advantageously, the method can comprise a further step of observing the at least one object, the at least one living being, and / or the at least one person using the observation device to generate observation data. This allows observation data to be generated in a particularly safe and / or reliable manner using the observation device. This enables the application of the recognition function and thus the determination of properties in a particularly safe and / or reliable manner.

[0015] In an advantageous embodiment, the detection function can be used to determine or ascertain behavior, conditions, and / or vital parameters of the at least one object, the at least one living being, and / or the at least one person. Advantageously, the following properties can be determined or ascertained using the detection function: - A recognition and / or classification of the at least one object, the at least one living being and / or the at least one person, - Physical characteristics of the at least one living being and / or the at least one person, for example a height, a weight, an age and / or a gender, - An identification of the at least one object, the at least one living being and / or the at least one person, - behaviors, a gaze behavior, a body pose and / or a body movement of the at least one living being and / or the at least one person, - Fatigue, distraction and / or driving ability of at least one person, - Vital parameters of the at least one living being and / or the at least one person, for example heart rate, respiration, blood pressure, oxygen saturation and / or one or more body temperatures, - A change in properties over time.

[0016] This allows properties of the at least one object, the at least one living being, and / or the at least one person to be determined safely and / or reliably, thereby ensuring reliable observation of these objects. For example, appropriate action can be taken in an emergency.

[0017] Furthermore, a system for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person is proposed, comprising a computing unit which is designed to carry out a method for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person.

[0018] Furthermore, a system for determining properties of at least one object, at least one living being and / or at least one person is proposed, comprising a computing unit which is designed to carry out a method for determining properties of at least one object, at least one living being and / or at least one person.

[0019] In an advantageous embodiment, the system can comprise an observation device for observing the at least one object, the at least one living being, and / or the at least one person to generate observation data. This allows observation data to be generated in a particularly safe and / or reliable manner using the observation device. This enables the application of the recognition function and thus the determination of properties in a particularly safe and / or reliable manner. Short description of the drawings

[0020] Embodiments of the invention are illustrated in the drawings and explained in more detail in the following descriptions. The same reference numerals are used for the elements shown in the various figures and have a similar effect, and a repeated description of the elements is omitted. They show: Fig. 1 is a schematic representation of a vehicle with a device for observing a vehicle occupant; Fig. 2 a schematic representation of a method according to an embodiment of the present invention; Fig. 3 is a schematic representation of a method according to an embodiment of the present invention; Fig. 4 a schematic representation of a method according to an embodiment of the present invention; Fig. 5 is a schematic representation of a method according to an embodiment of the present invention; Fig. 6 is a schematic diagram of a system according to an embodiment of the present invention; Fig. 7 a schematic representation of a method according to an embodiment of the present invention. Embodiments of the invention:

[0021] Fig. 1 shows a schematic representation of a vehicle 20, for example a motor vehicle, for example a car, with an observation device 22 for observing a vehicle interior, in particular one or more vehicle occupants 24 or passengers 24, other living beings, for example animals, and / or objects. The vehicle 20 has, in particular, an interior 26 or a vehicle interior 26, wherein, in particular, one or more seats 27 for one or more vehicle occupants 24 can be arranged in the vehicle interior 26. Furthermore, the vehicle 20 has an observation device 22 for observing the vehicle interior 26 and / or one or more vehicle occupants 24, for example a driver and / or a front passenger and / or one or more other passengers, and / or other living beings and / or objects.The observation device 22 can also be referred to, for example, as an observation system, driver monitoring system, interior monitoring system, interior sensing system, occupant observation system, and / or a surveillance system. The observation device 22 can, for example, be configured to detect a direction of gaze, a posture, and / or the position of the head or face or eyes of the vehicle occupant 24, or the state of fatigue and / or other vital signs of the vehicle occupant 24. Furthermore, an identity or physical characteristics such as height, weight, age, and / or gender of the vehicle occupant 24 can be detected.

[0022] The observation device 22 can be arranged in particular in a dashboard, in an instrument panel, in a steering wheel, on a vehicle roof, on a window, on a rearview mirror or on a pillar, for example on an A-pillar and / or a B-pillar of the vehicle 20, or at another location in the vehicle 20.

[0023] For observing the vehicle occupant 24, other living beings, and / or objects, the observation device 22 can have a recording unit. In a further development, the observation device 22 can have an illumination unit for emitting light rays, in particular infrared rays. Alternatively, the observation device 22 can not have a separate illumination unit, and the vehicle interior or the vehicle occupants, other living beings, and / or objects can be illuminated by light from an environment.

[0024] The lighting unit can be directed in particular toward the interior, for example toward the vehicle occupant 24, the other living being and / or object, and thus toward the vehicle seat 27, in order to illuminate the vehicle occupant 24 with light rays, in particular infrared rays. In other words, the lighting unit can emit light rays, in particular infrared rays, toward the vehicle occupant 24, the other living being and / or object. The lighting unit can illuminate parts of or the entire interior of the vehicle 20. Thus, individual areas of the interior or the entire interior can be illuminated by the lighting unit. The lighting unit can be designed, for example, as a lighting unit, lighting element, light-emitting diode, LED, OLED and / or laser diode and / or comprise a lighting unit, a lighting element, a light-emitting diode, an LED, OLED and / or a laser diode.

[0025] The recording unit can be designed, for example, as an image recording unit, for example as a sensor or a camera, wherein the recording unit is directed towards the interior, for example the entire interior or a defined area of ​​the interior, the vehicle occupant 24, the other living being and / or object and thus towards the vehicle seat 27, the other living being and / or object, in order to visually capture the vehicle occupant 24, the other living being and / or objects. The camera can be designed as an infrared camera module. By designing it as an infrared camera module, it is possible to carry out the observation even at night, without the vehicle occupant 24 being brightly illuminated and thus, in particular, disturbed or blinded.

[0026] In other words, the observation device 22 is designed to observe a vehicle interior. The observation device 22 can have a lighting unit for emitting light rays, for example, infrared rays, toward the vehicle interior and a receiving unit for receiving light rays, for example, infrared rays. Preferably, the light rays emitted by the lighting unit can be reflected on or in the vehicle interior, with the reflected light rays being steerable toward the receiving unit.

[0027] Furthermore, the observation device 22 has a control unit 28 or an evaluation unit 28 or a computing unit 28 for controlling the lighting unit and / or the recording unit and / or for processing the data recorded by means of the recording unit.

[0028] In an alternative embodiment, the observation device can be arranged in a room, for example, a building. The at least one object, the at least one living being, and / or the at least one person can be located in a vehicle or in a room and observed there.

[0029] Fig. Figure 2 shows a schematic representation of a method 30 for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person according to an embodiment of the present invention. The method 30 can be implemented, for example, by means of a system according to Fig. 6. The method 30 may, for example, be part of the method according to Fig. 7. The at least one object, the at least one living being and / or the at least one person can be located in a vehicle or in a room.

[0030] In a first step 32 of the method 30, observation data of the at least one object, the at least one living being and / or the at least one person are received by an observation device. The observation device has, in particular, a camera or is designed as a camera. The observation data can, for example, be processed according to Fig. 3 can be generated.

[0031] In a second step 34 of method 30, the brightness of the observation data and the stability of the brightness of the observation data are determined, and the stability of the brightness is checked. It is checked whether the stability of the brightness corresponds to a specified minimum stability.

[0032] This allows the brightness or brightness distribution of an individual image to be analyzed. For example, inhomogeneous lighting can result in areas on the individual image with different brightness. This can change the stability of the brightness. If there are large differences in the brightness distribution in areas within an individual image, resulting in an inhomogeneous brightness distribution or lighting, then a lower stability of the brightness is determined, for example. If there are small differences in the brightness distribution in areas within an individual image, resulting in a more homogeneous brightness distribution or lighting, then a greater stability of the brightness is determined, for example. A predetermined minimum stability can, for example, be a value for a brightness difference between individual areas within an individual image.

[0033] Alternatively or additionally, brightness or brightness distribution can be analyzed across multiple individual images. For example, changing lighting can result in individual images with different brightness levels. This can change the stability of the brightness. If there are large differences in the brightness distribution across the individual images, resulting in an inhomogeneous brightness distribution or changing lighting, then a lower stability of the brightness is determined, for example. If there are small differences in the brightness distribution across the individual images, resulting in a more homogeneous brightness distribution or lighting, then a greater stability of the brightness is determined, for example. A predetermined minimum stability can, for example, be a value of a brightness difference between individual consecutive individual images.

[0034] In an advantageous embodiment, one or more of the following measures can be carried out to determine the stability of the brightness of the observation data: - Determining a dispersion measure of an average image brightness of individual images in a certain period of time, - Determining an average brightness change in given areas of the individual images, - Identification of a background of the at least one object, the at least one living being and / or the at least one person and determining a dispersion measure of an image brightness in the background, - Identification of shadow areas and detection of a movement of a shadow across an image and / or in one or more specified areas, - Performing a histogram analysis, - Analysis of a distribution of gray values, - Analysis of a dynamic range, - Calculation of a Kullback-Leibler divergence, - Performing a self-monitoring machine learning process with reference data, - Determining changes in an optical flow, - Determine one of the normalized cross correlations, - Performing a frequency analysis or a Fourier transform and an analysis of a distribution in the frequency domain.

[0035] For example, if different results are obtained by using several measures, an average of the results, the best result or the worst result can be used.

[0036] In a third step 36 of the method 30, the quality of the observation data is determined. Furthermore, the determined quality is checked, whereby it is verified whether the determined quality corresponds to a predetermined minimum quality.

[0037] The method 30 can comprise the second step 34 and / or the third step 36. In other words, the method 30 can comprise either the second step 34 or the third step 36, or the method 30 can comprise the second step 34 and the third step 36. The method 30 can thus comprise either both steps, thus the second step 34 and the third step 36, or only one step of each. The second step 34 and the third step 36 can, in particular, be performed independently of one another, in parallel, or in different orders.

[0038] In an exemplary embodiment, individual images of the observation data can be analyzed to determine the brightness of the observation data and / or a stability of the brightness of the observation data and / or the quality of the observation data. Advantageously, a brightness, a sharpness, a noise and / or a contrast of the observation data can be analyzed to determine a quality of the observation data. If low sharpness, a high brightness value, a high noise and / or a low contrast is determined, low quality can be concluded. If a low brightness value, high sharpness, low noise and / or a high contrast is determined, high quality can be concluded. A predetermined minimum quality can, for example, be a sharpness value, a contrast value and / or a noise value.

[0039] Alternatively or additionally, one or more of the following measures can be carried out to determine the quality of the observation data: - Analysis of a signal-to-noise ratio, - Analysis of a dynamic range and an entropy.

[0040] In particular, time-varying quality characteristics of the camera recording or image data are analyzed.

[0041] For example, if different results are obtained by using several measures, an average of the results, the best result or the worst result can be used.

[0042] In a fourth step 38 of the method, the recognition function is adapted depending on the stability of the brightness of the observation data and / or the quality of the observation data if the stability of the brightness is worse than the specified minimum stability and / or if the determined quality is worse than the specified minimum quality.

[0043] In an advantageous embodiment, the detection function can be adapted by performing one or more of the following measures: - Discarding at least part of the observation data, - Calculation of quality measures and / or confidence measures for estimation results and / or at least part of the observation data, - Determining one or more weights for a fusion with observation data from other periods, - Determining one or more weights for fusion with observation data from another observation device and / or other observation data from other sensors.

[0044] Alternatively or additionally, the detection function can be adapted by controlling an illumination unit of the observation device accordingly, depending on the stability of the brightness of the observation data and / or the quality of the observation data.

[0045] For example, if different results are obtained by using several measures, an average of the results, the best result or the worst result can be used.

[0046] The time required to determine the stability of the brightness or the current quality of the images may depend on the requirements of the respective detection function that may need to be modified, for example the window size required for detection.

[0047] Fig. 3 shows a schematic representation of a method 30 for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person according to an embodiment of the present invention. The method 30 can be implemented, for example, by means of a system according to Fig. 6. The method 30 may, for example, be part of the method according to Fig. 7. The procedure 30 according to Fig. 3 can, for example, be carried out according to the method 30 according to Fig. 2. The method 30 can thus comprise the first step 32, the second step 34, the third step 36, and the fourth step 38.

[0048] Procedure 30 according to Fig. 3 comprises a further fifth step 40. The fifth step 40 can in particular be carried out before the first step 32. In the fifth step 40 of the method 30, the at least one object, the at least one living being and / or the at least one person is observed by means of the observation device for generating observation data. The observation device in particular comprises a camera or is designed as a camera. The observation device can be carried out according to the observation device according to Fig. 1 and / or arranged in a vehicle.

[0049] Fig. Fig. 4 shows a schematic representation of a method 42 for determining properties of at least one object, at least one living being and / or at least one person according to an embodiment of the present invention. The method 42 can be implemented, for example, by means of a system according to Fig. 6. The method 42 may, for example, be part of the method according to Fig. 7 be.

[0050] In a first step 44 of the method 42, observation data of the at least one object, the at least one living being and / or the at least one person are received by an observation device. The observation device has, in particular, a camera or is designed as a camera. The observation data can, for example, be processed according to Fig. 5 can be generated.

[0051] In a second step 46 of the method 42, the observation data are evaluated by means of the adapted recognition function to determine properties of the at least one object, the at least one living being and / or the at least one person. The recognition function can be implemented in particular according to the method according to Fig. 2 and / or Fig. 3. In other words, the adapted recognition function, which is determined according to the method according to Fig. 2 and / or Fig. 3 is used to determine properties.

[0052] In an advantageous embodiment, the detection function can be used to detect or determine behavior, conditions, and / or vital parameters of the at least one object, the at least one living being, and / or the at least one person. In other words, one or more objects or living beings, properties, behavior, condition(s), or vital parameters can be captured using one or more cameras and evaluated or determined using the detection function. Preferably, properties that are stable over time and / or properties that change over time can be determined. Advantageously, the following properties can be detected or determined using the detection function: - A recognition and / or classification of the at least one object, the at least one living being and / or the at least one person, - Physical characteristics of the at least one living being and / or the at least one person, for example a height, a weight, an age and / or a gender, - An identification of the at least one object, the at least one living being and / or the at least one person, - behaviors, a gaze behavior, a body pose and / or a body movement of the at least one living being and / or the at least one person, - Fatigue, distraction and / or driving ability of at least one person, - Vital parameters of the at least one living being and / or the at least one person, for example heart rate, respiration, blood pressure, oxygen saturation and / or one or more body temperatures, - A change in properties over time.

[0053] For example, if different results are obtained by using several measures, an average of the results, the best result or the worst result can be used.

[0054] In a further development, the method 42 can have a further step. In the further step of the method 42, a function can be triggered or changed depending on the determined property of the at least one object, the at least one living being and / or the at least one person. For example, information or a warning can be issued or changed, information or a warning can be sent and / or a driving safety function can be triggered or changed. Information or a warning can be issued, for example, by means of an acoustic and / or visual signal, for example by means of a loudspeaker and / or a display. Alternatively or additionally, information or a warning can be sent to an external unit, for example to an external server. This can, for example, result in an emergency message being issued.If the person is in a vehicle, a safety function can be triggered or modified, for example. For example, an autonomous driving state can be established and / or the vehicle can be guided to a safe stop. Furthermore, the functioning of comfort or driving safety functions which partially or independently take over longitudinal and / or lateral guidance can be adapted. Furthermore, the triggering conditions and / or type of warning of warning functions which affect the longitudinal and / or lateral guidance of the vehicle, such as a distance warning system or a lane departure warning system, can be modified. Depending on the properties determined, warnings or vehicle interventions can preferably be triggered earlier, for example at greater distances to the front or to the lane markings.In the case of warning functions, for example, the warning modalities can be modified and / or expanded depending on the detected characteristics to increase the likelihood that the warning will be registered by the driver in a timely manner. This can, in particular, increase road safety.

[0055] Fig. Fig. 5 shows a schematic representation of a method 42 for determining properties of at least one object, at least one living being and / or at least one person according to an embodiment of the present invention. The method 42 can be implemented, for example, by means of a system according to Fig. 6. The method 42 may, for example, be part of the method according to Fig. 7. The procedure 42 according to Fig. 5 can, for example, be carried out according to the method 42 according to Fig. 4. The method 42 can thus comprise the first step 44 and the second step 46.

[0056] Procedure 42 according to Fig. 5 has a further third step 48. The third step 48 can in particular be carried out before the first step 44. In the third step 48 of the method 42, the at least one object, the at least one living being and / or the at least one person is observed by means of the observation device for generating observation data. The observation device in particular has a camera or is designed as a camera. The observation device can according to the observation device according to Fig. 1 and / or arranged in a vehicle.

[0057] Fig. 6 shows a schematic representation of a system 50 for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person according to an embodiment of the present invention. The system 50 has a computing unit 52. The computing unit 52 is designed to carry out a method for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person according to the method according to Fig. 2 and / or Fig. 3. In this advantageous embodiment, the computing unit 52 is arranged, for example, outside a vehicle 20. For example, the computing unit 52 can be arranged on an external server or be part of an external server. In an alternative embodiment, the computing unit 52 can be arranged inside the vehicle 20. The vehicle 20 can, for example, according to the vehicle 20 according to Fig. 1. The vehicle preferably has an observation device 22. The observation device 22 can be designed to observe the at least one object, the at least one living being and / or the at least one person to generate observation data. The observation device 22 can, for example, be designed according to Fig. 1 and / or arranged in a vehicle.

[0058] Fig. 6 shows a schematic representation of a system for determining properties of at least one object, at least one living being and / or at least one person according to an embodiment of the present invention. The system has a computing unit 52. The computing unit 52 is designed to carry out a method for determining properties of at least one object, at least one living being and / or at least one person according to Fig. 4 and / or Fig. 5. In this advantageous embodiment, the computing unit 52 is arranged, for example, outside a vehicle 20. For example, the computing unit 52 can be arranged on an external server or be part of an external server. In an alternative embodiment, the computing unit 52 can be arranged inside the vehicle 20. The vehicle 20 can, for example, according to the vehicle 20 according to Fig. 1. The vehicle preferably has an observation device 22. The observation device 22 can be designed to observe the at least one object, the at least one living being and / or the at least one person to generate observation data. The observation device 22 can, for example, be designed according to Fig. 1 and / or arranged in a vehicle.

[0059] Fig. Fig. 7 shows a schematic representation of a method 56 for adapting a recognition function and for determining properties of at least one object, at least one living being and / or at least one person by means of the recognition function according to an embodiment of the present invention. The method 56 can be implemented, for example, by means of the systems according to Fig. 6. The vehicle 20 according to Fig. 7 may according to Fig. 1. The vehicle 20 has, in particular, at least one observation device 22. A person 24 and an object 58 are arranged or located in the vehicle 20. Changing environmental conditions 60 prevail in the surroundings of the vehicle 20. The changing environmental conditions 60 result, in particular, in different brightness levels in the vehicle 20 and / or lead to different and possibly temporally variable qualities of the camera recordings. The observation device 22 thus records observation data under different environmental conditions 60 or different brightness levels, resulting in qualitatively different observation data.

[0060] The observation device 22 can also be referred to as an interior camera. By means of the observation device 22, various detection tasks can be performed in the area monitored by it, for example, object detection, detection and in particular identification of persons or other living beings as well as properties, states and / or vital parameters. In the future, an observation device in the vehicle will be used for tasks in the comfort and / or safety areas. For example, driver condition monitoring can be used to determine the extent to which a driver can, in principle, perform the driving task without impairment and whether the driver actually does so. For this purpose, certain contents of the camera image can be identified by an algorithm, for example, edges, details of the eyes, the head, certain landmarks, for example facial landmarks, a specific skin area and others.In many cases, the identification of such content depends on the lighting. Sudden, gradual, and / or frequent changes in the lighting of an area can affect the recognition results. This can impair the quality of the results. The same applies, in particular, to the quality of the camera recording; for example, increased noise can make certain content less easy to identify.

[0061] Using the proposed method 56, optical recognition quality can be improved, for example, for recognizing objects, living beings, properties, behavior, states, or vital parameters, by determining the stability of brightness and / or the current, time-varying quality of the camera image and taking this into account either in the recognition function and / or in the use of its recognition results. Advantageously, the quality of the results of one or more recognition functions can be improved under conditions of variable brightness.

[0062] One example is the detection of a person's vital signs, such as respiration or changes in skin light absorption, which can allow conclusions to be drawn about the person's heart rate. Recording and evaluating the stability of brightness helps identify potentially erroneous detections of the actual quantity of interest, thereby increasing the accuracy of the detection of the actual quantity of interest.

[0063] The image data recorded by the observation device 22 are divided into individual images in a first step 62. Advantageously, the individual images are then windowed in a further second step 64. The data or individual images processed in this way are analyzed in a third step 66 such that an exposure or brightness and in particular a stability of the brightness is analyzed. If the stability of the brightness does not correspond to the desired stability, measures for further processing of the data with regard to an exposure of the images are carried out in a fourth step 68, in particular depending on the desired result. Alternatively, the exposure of the interior can be changed or adjusted, in particular an emission of light rays for illumination. The measures can be carried out, for example, according to the method according to Fig. 2. Alternatively or additionally, in a parallel process, the thus processed data or individual images can be analyzed in a fifth step 70 in such a way that the quality of the observation data is analyzed. If the quality does not correspond to the desired quality, measures to correct the quality are carried out in a sixth step 72 depending on the desired result. The measures can, for example, be carried out according to the method according to Fig. 2 and / or Fig. 3. The second step 64, the third step 66, the fourth step 68, the fifth step 70 and the sixth step 72 can be carried out in particular within the system 50 for adapting a recognition function for determining properties of at least one object, at least one living being and / or at least one person. The system 50 can be carried out in particular according to Fig. 6 be trained.

[0064] The data recorded by the observation device 22 are analyzed in a seventh step 74 using the recognition function. This allows, in particular, properties of the at least one object, the at least one living being and / or the at least one person to be determined. This results in initial results of the recognition function in an eighth step 76. If the recognition function is now adapted using the system 50 and thus the method for adapting the recognition function, the observation data are evaluated using the adapted recognition function. This results in modified results of the recognition function in a ninth step 78. This allows, in particular, properties of the at least one object, the at least one living being and / or the at least one person to be determined. This can be done, for example, according to the method according to Fig. 4 and / or Fig. 5. The seventh step 74, the eighth step 76 and the ninth step 78 can be carried out in particular within the system 54 for determining properties of at least one object, at least one living being and / or at least one person. The system 50 can be carried out in particular according to Fig. 6 be trained.

[0065] Some examples of the procedure are described below.

[0066] In a first example, the detection function is adapted such that an estimation of a parameter is suspended, using a heart rate estimation as an example. Prevailing brightness is continuously examined, for example, using one of the following methods: - Analysis of the average brightness in given areas, for example in the vehicle background, - histogram analysis, - Analysis of a normalized cross correlation (Normalized Cross Correlation), - Machine learning methods.

[0067] At the same time, an algorithm evaluates brightness changes in specified areas, for example in the driver's face, within a specified time window and uses this to estimate an average heart rate for the time window. Analysis of the brightness values ​​for the same time window reveals strong periodic fluctuations, for example caused by frequent changes in light and shadow. The frequency and amplitude of these fluctuations are compared with defined limit values, for example those dependent on fixed and / or other conditions. An example of a dependency on other conditions can be determined by comparing the average amplitude of the brightness changes of the target person's skin. For example, for people with smaller brightness changes, more stable brightness is required for heart rate detection.Comparing the frequency and amplitude of the fluctuations with defined thresholds reveals that the light fluctuations have a high amplitude and are within the range of the target frequency, for example, the heart rate estimate. The heart rate estimate is discarded in all time windows where comparison with the thresholds reveals that the amplitude and / or frequency of the light fluctuations leads to relevant, particularly intense, interference with heart rate detection.

[0068] In a second example, time windows of the detection function are adapted to the stability of the brightness using the example of a heart rate estimate. In this embodiment, the focus is on estimating a heart rate. As in the first example, the brightness is continuously examined; see the first example for this. The course of the brightness shows that after a phase of stable conditions, an intense change occurs, with the brightness then becoming stable again. This can be determined, for example, by analyzing several small time windows and the course of a measure of change across the time windows. The heart rate estimate is then discarded in all time windows in which the brightness changes significantly, for example, within the window with a high degree of scatter. As soon as the brightness is stable again, the heart rate estimate is reinitialized orThe position of time windows for heart rate estimation is determined retrospectively so that the stability of the brightness in these time windows is maximally high.

[0069] In a further development, additional data such as location, navigation data, time of day, position of the sun, current weather, and other data can be used. This can enable forward-looking planning, for example, if changes in brightness are expected in the near future before entering a tunnel or before turning.

[0070] A third example concerns the selection of a sensor or channel or the weighting in a sensor data fusion using the example of a heart rate estimation. In this embodiment, the focus is on estimating a heart rate. As in the first and second examples, the brightness is continuously examined; see the first or second example for this. Furthermore, the quality of the camera images is determined, for example via a signal-to-noise ratio, the aperture opening, the gain and / or the dynamic range or entropy. Depending on the results, for example by comparison with limit values, one of several possible sensors or channels is selected for a detection function or, in the case of sensor data fusion, the degree to which the measurement from its sensor and / or a channel affects the overall result is adjusted, for example by weighting factors.If the camera images are sufficiently well lit during a specific time window, for example, during the day, are not overexposed, and the illumination fluctuation in the RGB channel is below a defined value, the heart rate is estimated using the RGB channel. If one of the conditions is violated, the heart rate is estimated from an NIR channel or rejected if the requirements for stable brightness and quality are not met for NIR either. Depending on the conditions, the heart rate values ​​estimated from the RGB or NIR channel can be combined to varying degrees with other sensor data, such as an indoor radar.

[0071] A fourth example involves adjusting the confidence of values ​​estimated from camera recordings. In this embodiment, the quality of the camera recording is continuously evaluated, see especially the third example. Depending on the quality, a confidence measure of values ​​estimated from the camera recordings can be adjusted, for example, parameters for gaze detection, a head pose and / or body pose, eye characteristics such as eye opening, or vital data parameters. Other functions that further process this estimated data can be configured differently based on the changed confidence value. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2014 214 352 A1

[0001]

Claims

[1] Method (30) for adapting a recognition function for determining properties of at least one object (58), at least one living being and / or at least one person (24), in particular in a vehicle (20), comprising the steps of Receiving observation data of the at least one object (58), the at least one living being and / or the at least one person (24) from an observation device (22), Determining a brightness of the observation data and a stability of the brightness of the observation data and checking the stability of the brightness, checking whether the stability of the brightness is greater than a predetermined minimum stability, and / or determining a quality of the observation data and checking the determined quality, checking whether the determined quality is greater than a predetermined minimum quality, Adapting the detection function depending on the stability of the brightness of the observation data and / or the quality of the observation data if the stability of the brightness is less than the specified minimum stability and / or if the determined quality is less than the specified minimum quality. [2] Method (30) according to claim 1, comprising a further step of observing the at least one object (58), the at least one living being and / or the at least one person (24) by means of the observation device (22) to generate observation data. [3] Method (30) according to one of the preceding claims, wherein individual images of the observation data are analyzed to determine the brightness of the observation data and / or a stability of the brightness of the observation data and / or to determine the quality of the observation data. [4] Method (30) according to one of the preceding claims, wherein a brightness, a sharpness and / or a contrast and / or a noise of the observation data is analyzed to determine a quality of the observation data. [5] Method (30) according to one of the preceding claims, wherein one or more of the following measures are carried out to determine a quality of the observation data: - Analysis of a signal-to-noise ratio, - Analysis of a dynamic range and an entropy. [6] Method (30) according to one of the preceding claims, wherein one or more of the following measures are carried out to determine the stability of the brightness of the observation data: - Determining a dispersion measure of an average image brightness of individual images in a certain period of time, - Determining an average brightness change in given areas of the individual images, - identifying a background of the at least one object (58), the at least one living being and / or the at least one person (24) and determining a dispersion measure of an image brightness in the background, - Identification of shadow areas and detection of a movement of a shadow across an image and / or in one or more specified areas, - Performing a histogram analysis, - Analysis of a distribution of gray values, - Analysis of a dynamic range, - Calculation of a Kullback-Leibler divergence, - Performing a self-monitoring machine learning process with reference data, - Determining changes in an optical flow, - Determining a normalized cross correlation, - Performing a frequency analysis or a Fourier transform and an analysis of a distribution in the frequency domain. [7] Method (30) according to one of the preceding claims, wherein the recognition function is adapted by performing one or more of the following measures: - Discarding at least part of the observation data, - Calculation of quality measures and / or confidence measures for estimation results and / or at least part of the observation data, - Determining one or more weights for a fusion with observation data from other periods, - Determining one or more weightings for a fusion with observation data from another observation device (22) and / or other observation data from other sensors. [8] Method (30) according to one of the preceding claims, wherein the recognition function is adapted by controlling a lighting unit of the observation device (22) accordingly depending on the stability of the brightness of the observation data and / or the quality of the observation data. [9] Method (42) for determining properties of at least one object (58), at least one living being and / or at least one person (24), in particular in a vehicle (20), comprising the steps of receiving observation data of the at least one object (58), the at least one living being and / or the at least one person (24) from an observation device (22), evaluating the observation data by means of the adapted recognition function according to one of the preceding claims for determining properties of the at least one object (58), the at least one living being and / or the at least one person (24). [10] Method (42) according to claim 9, comprising a further step of observing the at least one object (58), the at least one living being and / or the at least one person (24) by means of the observation device (22) to generate observation data. [11] Method (42) according to one of the preceding claims 9 to 10, wherein a behavior, states and / or vital parameters of the at least one object (58), the at least one living being and / or the at least one person (24) can be determined or are determined by means of the recognition function. [12] Method (42) according to one of the preceding claims 9 to 11, wherein the following properties can be determined or are determined by means of the recognition function: - A recognition and / or classification of the at least one object (58), the at least one living being and / or the at least one person (24), - Physical characteristics of the at least one living being and / or the at least one person (24), for example a height, a weight, an age and / or a gender, - An identification of the at least one object, the at least one living being and / or the at least one person, - behaviors, a gaze behavior, a body pose and / or a body movement of the at least one living being and / or the at least one person (24), - fatigue, distraction and / or driving ability of at least one person (24), - vital parameters of the at least one living being and / or the at least one person (24), for example a heartbeat activity, respiration, blood pressure, oxygen saturation and / or one or more body temperatures, - A change in properties over time. [13] System (50) for adapting a recognition function for determining properties of at least one object (58), at least one living being and / or at least one person (24) with a computing unit (52) which is designed to carry out a method (30) according to one of the preceding claims 1 to 8. [14] System (54) for determining properties of at least one object (58), at least one living being and / or at least one person (24) with a computing unit (52) which is designed to carry out a method (42) according to one of the preceding claims 9 to 12. [15] System (54) according to claim 14, wherein the system comprises an observation device (22) for observing the at least one object (58), the at least one living being and / or the at least one person (24) to generate observation data.

Images