Interior monitoring system and its operating method, and vehicle equipped with such an interior monitoring system - Patents.com

Abstract

The invention relates to a method for operating a camera-based interior monitoring system (1), in which at least one camera (1.1) as at least one sensor records camera images, a computing unit (1.2) evaluates the sensor signals and, depending on the evaluated sensor signals, configures parameters for setting the interior monitoring system (1). The computing unit (1.2) serves at least two applications (APP1, APP2, APP3), each of which requires a different configuration value for at least one parameter in a specific operating situation, the computing unit (1.2) calculates a current application priority (2) for the current operating situation and sets the configuration value required by the higher-ranking application (APP1) as a target configuration for a first parameter, and for each application (APP1, APP2, APP3), one importance is assigned to the configuration value of the parameter, respectively, and if the importance of the configuration value of the target configuration of another parameter of the lower-ranking application (APP2, APP3) in the application priority (2) is greater than the importance of the configuration value of the target configuration of the parameter of the higher-ranking application (APP1), at least one other parameter is configured according to the target configuration requested by the lower-ranking application (APP2, APP3).

Description

[Technical field]

[0001] The invention relates to a method for operating a camera-based interior monitoring system of the kind as defined in the preamble of claim 1, to an interior monitoring system of the kind as defined in the preamble of claim 10, as well as to a vehicle equipped with such an interior monitoring system. [Background technology]

[0002] Vehicles are increasingly being equipped with interior monitoring systems to provide various comfort and / or safety functions. Using seat occupancy sensors and seat belt buckle sensors, it can be detected, for example, whether a vehicle occupant has fastened his or her seat belt while driving. In such a case, a warning can be issued. In addition to this, by using cameras, corresponding interior monitoring systems can also monitor the vehicle occupants. This makes it possible to analyze whether the person driving the vehicle is paying sufficient attention to the driving event, is tired, or even to identify the person using biometric information.

[0003] Such camera-based tasks or functions for vehicle interior monitoring are referred to below as applications. Depending on the application, different requirements are set for the optical detection of the vehicle interior. For example, if only the driver of the vehicle is present in the vehicle, optical monitoring of the rear of the vehicle is irrelevant. If the driver of the vehicle wears infrared-resistant glasses, recording of camera images in the infrared spectrum is unnecessary, since the blinking of the person as well as the direction of the gaze cannot be detected in the infrared light spectrum. In such a case, detection by color images of the camera may be advantageous for other applications. A relatively small camera resolution is required to detect a person sitting in a vehicle seat. However, when identifying a person, a higher resolution is required to recognize biometric information.

[0004] Therefore, a new question arises as to how the in-vehicle monitoring system should be operated in relation to all current operating conditions so that the various applications can optimally perform their respective tasks.

[0005] From DE 10 2005 13 2007 A1 a method and a device for monitoring at least one vehicle occupant and a method for activating at least one assistance device are known, which method provides for camera-based monitoring of a vehicle occupant in order to detect his / her vital signs. If a brightness change occurs around the vehicle, for example due to the vehicle entering a tunnel, the vehicle can detect this and automatically switch the image recording mode of the camera used for monitoring the vehicle occupant from color image recording to image recording in the infrared spectrum and vice versa. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] DE 102011110486 Summary of the Invention [Problem to be solved by the invention]

[0007] The invention is based on the problem of providing an improved method for operating a camera-based interior monitoring system, with the help of which the interior monitoring system is respectively able to optimally perform the tasks related to the provision of at least two different applications. [Means for solving the problem]

[0008] According to the invention, this problem is solved by a method for operating a camera-based interior monitoring system with the features of claim 1, as well as a corresponding interior monitoring system with the features of claim 10 and a vehicle equipped with such an interior monitoring system with the features of claim 11. Advantageous embodiments and developments are evident from the dependent claims.

[0009] In a general operating method of a camera-based interior monitoring system, at least one camera is used as a sensor and records camera images. A computing unit evaluates the corresponding sensor signals and calculates configuration values ​​as a function of the evaluated sensor signals and configures parameters for configuring the interior monitoring system. For example, the calculation of the configuration values ​​of the parameters is performed by calculation, iterative methods or rules so that a camera image quality required for the application can be achieved. According to the invention, this general method is improved in that the computing unit provides at least two applications, which preferably simultaneously evaluate the same sensor signals of the camera and each request different configuration values ​​for at least one first parameter in a specific operating situation at the same time. The computing unit then calculates a current application priority for the current operating situation and sets the configuration value requested by the higher-ranking application as a target configuration for the first parameter. Simultaneous evaluation of the same sensor signals means that the applications work in parallel based on the sensor signals identified and received by the cameras.

[0010] In other words, the method according to the invention classifies the applications provided by the computing unit according to their "importance" and the in-car monitoring system finally sets the configuration values ​​of the camera in such a way that the application with the highest importance works optimally. Configuration values ​​according to a target configuration in this context refer to the minimum configuration values ​​of the parameters required for optimal functioning of the application in a certain operating situation without limiting it. Through a suitable configuration of the parameters, the camera image generated by the camera and, accordingly, the information derived from the computing unit by analysis of the camera image or the sensor signal optimally meet the requirements of the most important application in the respective operating situation. However, this also means that the functions simultaneously provided by the less important application are only provided with limited or low accuracy. However, this can be accepted since the other applications are less important for the current operating situation. At least two applications simultaneously receive the same sensor signal and evaluate and process it in parallel, so that information loss due to switching the camera parameters can be avoided.

[0011] The term operating conditions includes ambient conditions such as lighting, vehicle seat occupancy, driving conditions, object shadows, or solar radiation.

[0012] The evaluation of the current operating situation is then performed as a function of the evaluated sensor signals, which in this case include camera images generated by the camera as well as sensor data from other sensors. Since the method according to the invention can be applied to a vehicle, sensors or sensor data can be used, for example seat belt sensors, seat occupancy sensors, light sensors, etc., as well as vehicle interior information, such as the vehicle's travel speed or acceleration. In the case of different operating situations, for example a vehicle occupant is seated in a particular seat, the vehicle is travelling at a particular travel speed, there are particular light conditions, etc., it is necessary to reevaluate the operating situation that is relevant at the time and to reconfigure the parameters for setting the interior monitoring system accordingly.

[0013] Furthermore, according to the present invention, for each application, a weight is assigned to each of the configuration values ​​of the application, and if the weight of the configuration value of the target configuration of another parameter of a lower application in the application priority order is greater than the weight of the configuration value of the target configuration of a parameter of a higher application, then at least one other parameter is configured according to the target configuration requested by the lower application.

[0014] As already mentioned, less important applications operate simultaneously with more important applications outside their respective optimal operating parameters. In order to avoid unnecessarily large degradation or deterioration of the respective functionality provided by the respective less important applications, the less important parameters for the more important applications can be configured according to the target configuration of the less important applications, provided that they are equal to or greater than the minimum configuration values ​​of the more important applications. In other words, the parameters of the higher-order application are overwritten by the configuration values ​​of the lower-order application's parameters if the latter is better, i.e. has a higher importance, in the camera settings, otherwise the configuration values ​​of the higher-order application are used. The importance of the configuration values ​​is due to technical characteristics, e.g. high resolution is more important than low resolution, low signal-to-noise ratio is more important than high signal-to-noise ratio. Furthermore, it is also possible to define arbitrarily and freely that certain configuration values ​​are higher than other configuration values, e.g. IR images are higher than RGB images. For this purpose, it is also possible to consider, for example, only two applications or to consider more than two applications. For example, the parameters for setting up an in-car monitoring system are configured according to the target configuration requested by the higher-level application, unless the configuration value of the target configuration of the lower-level application has a higher importance. If the characteristics are conflicting, different importance cannot be assigned. For example, if the higher-level application requests a short exposure time and the lower-level application requests a long exposure time, the exposure time parameter is always set to the value requested by the higher-level application by specifying the same importance.

[0015] In the following, the top application will be referred to as the first application, and the applications lower in the application priority order will be referred to as the second, third, etc. applications accordingly.

[0016] A first parameter that is important for providing a full range of functionality of a first application is then configured to a configuration value of this parameter according to a target configuration of the first application. A second parameter configuration value having a correspondingly lower value with respect to the target configuration can then be configured according to the target configuration of the second application in order to optimally function the first application compared to the second application. A third parameter having a lower configuration value with respect to the target configuration can then be set to a configuration value corresponding to the target configuration of the third application in order to optimally function the first and second applications compared to the third application. Advantageously, the method determines the configuration of the camera parameters according to the priority of the respective application, so that functionality is optimized for each application.

[0017] According to a further advantageous embodiment of the method, the computing unit dynamically determines the application requirements and / or application qualities required for each application for each current or future operating situation that changes from time to time, and determines the application priorities and / or importance of configuration parameters depending on the application requirements and / or application qualities of the application at the time.

[0018] By defining new application priorities for each operating situation, the importance of applications can be evaluated according to the operating situation. In this case, for one application, only the application quality or the application requirements, or both can be changed at the same time according to the operating situation.

[0019] Application quality refers to the quality of the camera image or information obtained therefrom that is necessary for operation in the current operating situation. The quality of the camera image is a measure of whether the application can operate in the current operating situation.

[0020] For example, if the resolution of a particular camera image is too low, applications that recognize certain features, such as gaze direction applications, cannot work reliably. As a function of the application quality preset for the application requirements, optimal configuration values ​​for the current application are calculated by evaluating the recorded camera images, for example by measuring the current application quality, and the camera configuration values ​​are calculated according to rules, tables, iterations or calculation methods such that the application quality preset for the current application is achieved, i.e. the measured application quality matches the preset application quality. In other words, an optimal configuration for each application, i.e. a target configuration, is calculated. As a function of the application priority and the importance of the configuration values, the camera settings are calculated, which are then referenced to all applications simultaneously.

[0021] Application requirements are understood to be the requirements set for a given application in a given operating situation, meaning in particular what parameters of the camera must be configured with what values ​​or settings under what boundary or ambient conditions or in what operating situation of the application, preferably according to their rank.

[0022] In an advantageous development of the method, the identification of the application requirements comprises the following work steps: - reading predefined application rules; - evaluating sensor signals or assistance systems to predict future operating conditions; - reading user settings; - analyzing the camera images to detect obstacles in the camera's field of view; - analyzing the camera images to evaluate the lighting conditions of the scene recorded by the camera; - analyzing the camera images to detect objects and / or people; - evaluating the sensor signals and performing sensor fusion; - Reading application specific priorities It includes at least one of the steps.

[0023] In this case, it is particularly preferred that all of the listed work steps are carried out, in particular maintaining the listed order of use.

[0024] For example, reading predefined application rules for a "driver monitoring" application or operating situation may include application requirements that the eyes and mouth of the person operating the vehicle be visible in the camera image and that the signal-to-noise ratio not degrade below a certain level.

[0025] For example, by analyzing the vehicle's travel speed and acceleration based on the sensor signals, it is possible to recognize that the vehicle will brake before a traffic light, etc., and to predict that the vehicle will stop before the traffic light as an application requirement. If the vehicle is stopped at a traffic light, in this situation, monitoring the driver's drowsiness or attention state is not very important in terms of application priority. Similarly, by recognizing the current situation with a camera as a sensor, future situations can be inferred and the corresponding application requirements can be derived. For example, the state of an unoccupied seat will remain for a predictable time frame, so the future monitoring priority can be lowered accordingly. In another example, if driving through a tunnel is read from the assistance system for a previous time frame, then for example color images will be lowered in priority in this time frame accordingly.

[0026] The possibility of individually presetting user settings allows the in-car monitoring system to be adapted to the preferences of the individual vehicle user and to determine application requirements, e.g. to protect the privacy of the vehicle user, face recognition can be deactivated, in which case face recognition is given a particularly low priority.

[0027] If there are obstacles in the camera's field of view, the camera may not be able to see the eyes of the person driving the vehicle, in which case the "driver monitoring" application is ranked particularly low.

[0028] If the scene recorded by the camera is particularly dark, for example, an application calling for the recording of a camera image in the infrared spectrum is given a high ranking, whereas an application calling for the recording of a color camera image is given a low ranking, and vice versa for a particularly bright scene.

[0029] Object and / or person detection can, for example, identify the location of vehicle occupants within the vehicle. If no one is in the rear seat of the vehicle, then tracking the head position of the seat in the rear seat of the vehicle is of lower priority.

[0030] As already mentioned, sensor data of further sensors can also be taken into account to determine the application priority. For example, the vehicle speed can be used and if the driving speed is below 10 km / h, there is no need to monitor the alertness and / or fatigue of the person driving the vehicle. Therefore, the ranking of this application in this operating situation will be particularly low.

[0031] For each application, application-specific priorities can also be defined: for example, vital signs monitoring and / or fatigue recognition can be given a particularly high priority, while a selfie application can be given a lower priority.

[0032] Preferably, the prediction of future operating conditions includes predicting the time when the person will not blink or the time when the observed person will blink again. If the observed person, for example the driver, blinks, there is a high probability that this person will not blink for the next 300 ms. This means that for example, if the camera takes camera images at a frequency of 60 per second, for the next 18 camera images, it is less important to execute the driver monitoring application, because there is no need to recognize blinks. Therefore, for the next 18 camera images, the in-car monitoring system is configured to have more suitable operating conditions for performing vital signs monitoring. This could for example involve taking color images instead of camera images in the infrared spectrum and / or evaluating other camera coverages, especially at other resolutions. In other words, advantageously, the priority of the application for driver monitoring is lowered in the time frame of future operating situations, whereby other applications are given a higher priority compared to driver monitoring, and the settings of their parameters allow optimized functioning in the best of cases without restrictions, especially when the parameters only allow conflicting settings, e.g. only long or short exposure times.

[0033] In a further advantageous embodiment of the method, the following image quality metrics, known as KPIs (Key Performance Index), are furthermore used to characterize the application quality: - pixel saturation, -histogram, -S / N ratio, - Structural similarity index, - a measure of complex wavelet structural similarity, and / or - Trust value calculated using artificial intelligence At least one of the cameras is designated and evaluated to inspect at least one camera image.

[0034] Artificial neural networks, among others, can be used to determine the confidence values. The enumerated image quality metrics of the camera images vary depending on the selected parameter settings or the settings of the interior monitoring system. For example, the operating conditions may change, causing the lighting of the scene recorded by the camera to change. If the parameters (such as exposure time or aperture) with which the camera generates the camera images remain constant, this will affect the corresponding quality of the recorded camera images. For example, the images may be overexposed or underexposed. If the application quality cannot be influenced by appropriate measures, the corresponding application may be down-ranked.

[0035] According to another advantageous embodiment of the invention, with the setting of the target configuration, i.e. for setting the target configuration, the following variables are used: -Select which camera among multiple cameras to use for recording camera images, - the choice of lighting devices used to illuminate the scene to be recorded by the camera, -Light distribution characteristics of lighting devices, - camera exposure time, ISO and / or aperture, -focal plane, -Zoom level, - the wavelength spectrum recorded by the camera, - camera frame rate, -resolution, - the image portion of the recorded camera image that is to be evaluated, and / or -How to post-process the camera images after recording At least one of these is set.

[0036] In order to adapt the operating mode of the in-car monitoring system to the current operating situation, various settings can be made using the enumerated values. Depending on which application has a higher priority, i.e. is more important, in the current operating situation, the corresponding parameters are configured according to the target configuration required by this application. The respective settings of the camera parameters can generate a corresponding application quality. For example, if the scene recorded by the camera is dark, a lighting device can be used to illuminate this scene and / or the exposure time, ISO sensitivity or aperture of the camera can be adjusted to obtain a more recognizable camera image. In case of monitoring several people and / or if the camera's view is obstructed, a second or more cameras can be used to record the scene.

[0037] The light distribution characteristics of the lighting device can determine, for example, whether the lighting device is operated in a pulsed or continuous mode, and also determine at what wavelength or in what wavelength band the lighting device emits light at any given time. For example, the lighting device can emit white, warm white, colored light and / or infrared light.

[0038] Depending on the application, it may be necessary to focus on a different focal plane. Depending on which application has a higher priority in the current operating situation, the appropriate focal plane is focused on.

[0039] Similarly, depending on the application, it may be necessary to evaluate a particular image portion at a particular zoom level or at a particular resolution in order to be able to recognize patterns, objects, structures, etc. that are relevant to the application.

[0040] For post-processing of the camera images, for example filtering, image sharpening, tone mapping and / or demosaicing can be applied to the corresponding camera images. In this way, the target configuration can be used to adjust the post-processing of the camera images accordingly.

[0041] In a further advantageous embodiment of the method, at least one variable that can be set furthermore with the target configuration is also used as an application requirement. For example, for some applications, it may be necessary to be able to recognize certain features in a certain image portion. For this, it may also be necessary to set the focal plane of the camera image, the zoom level, the image resolution and / or the image portion to be evaluated to standard values ​​that are most suitable for the respective operating situation. For example, an application that requires a relatively high image resolution is given a correspondingly high application priority. This is also applicable to the further listed variables that can be set with the target configuration. A further example may be related to the camera used to record the scene. For example, if a first and a second camera are recording the scene, and the first camera is integrated in the dashboard of the vehicle and is directed towards the person driving the vehicle, this camera may record part of the rear seat together. The second camera is for example integrated in the B-pillar of the vehicle and is directed towards the rear seat. Thereby, since the "person detection" application is already covered by the recording of the rear seat of the vehicle by the second camera, the first camera is configured according to the target configuration required by the attention analysis, which requires, for example, the recording of camera images in particularly high frequencies and in the infrared spectrum.

[0042] According to a further advantageous embodiment of the method, the application quality of at least one application for a future operating situation is predicted from at least one image quality metric or image quality characteristic calculated for current and / or past operating situations, preferably when the application requirements are constant, or from at least one value derived therefrom. Determining the application quality for a future operating situation makes it possible to clarify how extensively the functionality of a particular application can be provided without possibly changing the configuration of the camera or, advantageously, without maintaining a parameter configuration of the functionality that is not limited. The future operating situations can be calculated, for example, by using a navigation system, by previewing on the travel route, for example changing light conditions when entering or leaving a tunnel, stopping at traffic lights, position relative to the sun, etc. Target configuration values ​​of the parameters of the various applications can be predicted as a function of the required application quality. The camera settings are then pre-configured as a function of the order of the applications and / or the importance of the configuration values ​​of the parameters, so that new operating conditions can also be used for the applications without delay when they arise, and the applications can work without a configuration process.

[0043] According to a further advantageous embodiment of the method according to the invention, at least one application is - Identifying the direction of gaze, -Fatigue detection, - Seat occupancy detection, -Facial recognition, -Vital signs monitoring, -Photo and / or video capture capabilities - Gesture recognition It is formed from.

[0044] The photo and / or video capture function can be, for example, an application for carrying out video calls. The listed applications each set individual requirements for the interior monitoring system. The respective target configurations of the individual parameters therefore differ. Some of them may even be mutually exclusive. For example, the determination of the gaze direction requires the recording of camera images in the infrared spectrum, while the photo and / or video capture function requires the recording of color camera images. Seat occupancy detection as well as gesture recognition require camera images with a relatively low resolution in order to function correctly. In contrast, high-resolution camera images are required to carry out face recognition.

[0045] In the case of an interior monitoring system with at least one camera and a computing unit, the camera and the computing unit are adapted according to the invention to carry out the above-mentioned method. The interior monitoring system may have several cameras, as well as one or several lighting devices for illuminating the scene recorded by the one or several cameras. In this case, the cameras may be differently constructed. For example, some cameras may have a fixed aperture and a fixed focus, while others may be equipped with a zoom lens, i.e. a variable focus, as well as an adjustable aperture. Furthermore, there may be cameras that record light in the infrared spectrum and cameras that record light in the visible range. There may also be cameras that can record both light in the infrared spectrum and in the visible range. Similarly, the individual lighting devices may only emit light of a certain wavelength or wavelength band. Furthermore, the lighting devices may be operated continuously or intermittently. To control the lighting devices in an intermittent operating mode, these devices may be controlled with any modulation.

[0046] In this case, the computing unit may be formed by one individual computing device or may be divided into several computing devices, for example a first computing device for controlling the camera, a second computing device for evaluating the camera images or the sensor signals and a third computing device for determining application requirements, application quality, importance of configuration values ​​and / or application priority.

[0047] In a vehicle with an interior monitoring system, according to the invention, at least one camera is integrated in the instrument cluster, head unit, interior mirror and / or roof control unit, in particular between the driver's side and the passenger's side with respect to the transverse axis of the vehicle and preferably at least at the height of the steering wheel with respect to the longitudinal axis of the vehicle. A camera that is mounted in the center of the vehicle, i.e. between the driver's side and the passenger's side and at least at the height of the steering wheel, can record a particularly wide field of view inside the vehicle. By arranging it in the front area of ​​the vehicle and pointing towards the rear, it is possible to record the vehicle occupants from the front. This makes it possible to provide a large number of different applications. It also makes it possible to dispense with further cameras, which requires a configuration suitable for the particular operating situation and the particular application, as well as the application of the method according to the invention for finding the parameters used for setting the interior monitoring system.

[0048] Further advantageous embodiments of the inventive operating method of a camera-based interior monitoring system, the interior monitoring system as well as the vehicle will become apparent from the exemplary embodiments which are explained in greater detail below with reference to the figures. [Brief description of the drawings]

[0049] [Figure 1] 1 is a schematic plan view of a vehicle according to the present invention; [Diagram 2] 2 is a flow chart of a method according to the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0050] 1 shows a vehicle 3 according to the invention. The vehicle includes an interior monitoring system 1 according to the invention, which comprises at least one camera 1.1, a computing unit 1.2 as well as a number of lighting devices 1.3. The camera 1.1 is arranged between the driver's side and the passenger's side with respect to the transverse vehicle axis Ax and at least at the height of the steering wheel with respect to the longitudinal vehicle axis Az. This allows a particularly large space inside the vehicle to be recorded, so that not only people sitting in the driver's seat 5.1 or the passenger seat 5.2, but also people sitting in the rear seat 5.3 at the rear of the vehicle 3 can be recorded. For example, the camera 1.1 is integrated in the head unit 4 of the vehicle 3.

[0051] Furthermore, the vehicle 3 can also have further cameras, which are not shown in detail. These cameras can be integrated, for example, in the instrument cluster, the headliner, the A-, B- or C-pillars, the seats, the dashboard, etc. The same applies for the lighting device 1.3, which can, for example, be arranged laterally with a relatively small distance to the camera 1.1. The provision of several lighting devices 1.3 allows a particularly large area to be illuminated inside the vehicle, so that the risk of missing relevant objects in too dark parts of the vehicle interior is reduced.

[0052] 2 is a flow chart of the method according to the invention. In method step 201, camera images are generated by the camera 1.1. These camera images are read in by the computing unit 1.2, processed and provided simultaneously to a number of applications APP1, APP2, APP3. For each application APP1, APP2, APP3, application requirements are specified in method step 202. For this purpose, for example, a predefined target configuration of the interior monitoring system 1 is read from the memory of the computing unit 1.2, future operating situations are predicted, user settings are read and the camera images are analyzed, so as to detect, for example, obstructed camera views, evaluate the lighting situation inside the vehicle and detect objects and / or people. For each application APP1, APP2, APP3, predefined application-specific priority values ​​can also be read.

[0053] In method step 203, the computing unit 1.2 individually determines an application quality to be achieved for each application APP1, APP2, APP3. The application quality represents image quality metrics of the camera images, such as pixel saturation, histogram, S / N value, structural similarity index, complex wavelet structural similarity index, and / or confidence value calculated using artificial intelligence. In this case, each application APP1, APP2, APP3 can request an individual minimum value for a particular image quality metric, so that each application APP1, APP2, APP3 can provide its respective functionality to the maximum.

[0054] In method step 204, for each application APP1, APP2, APP3, a ranking is identified taking into account application requirements and / or application quality for an operating situation. The rankings of the applications APP1, APP2, APP3 are sorted in method step 205, from which an application priority order 2 is formed. For example, in Fig. 2, the first application APP1 has the highest ranking, followed by the second application APP2, the third application APP3, and so on.

[0055] Furthermore, in method step 204, the computing unit 1.2 can assign a respective importance to the configuration values ​​of the parameters for APP1, APP2, APP3. If one or more parameters for configuring the interior monitoring system 1 are configured according to a target configuration required by the first application APP1, one or more further parameters are also configured according to the target configuration required by the subordinate applications APP2, APP3, i.e. the target configuration required for optimal operation of the applications, if the importance of the configuration values ​​of the further parameters according to the target configuration of the subordinate applications APP2, APP3 is greater than the importance of the parameters according to the target configuration of the superior application APP1. In other words, the configuration values ​​according to the target configuration values ​​of the further parameters of the subordinate application APP2, for example, are then qualitatively higher with respect to the superior application APP1, so that the functionality of the application APP1 is fully provided and the functionality is ensured. In this case, the first application APP1 is placed above the second application APP2, which is placed above the third application APP3.

[0056] In method step 206, the operating mode manager 6, which may for example be a program running on the computing unit 1.2, checks whether the changing operating situation and the associated changed application requirements and / or application qualities have caused a change in the application priority 2 or a new assignment of importance to the configuration values ​​of the parameters of the applications APP1, APP2, APP3. If this is not the case, no changes are made to the settings of the interior monitoring system 1 and camera images continue to be taken.

[0057] On the other hand, if the application priority 2 is changed due to a change in the operating situation and / or the importance of the configuration values ​​of the parameters of the respective applications APP1, APP2, APP3 is changed, target configuration values ​​of the individual parameters for configuring the interior monitoring system 1 are calculated in method step 207. These target configuration values ​​are then set in method step 208, whereby the interior monitoring system 1 records and evaluates the camera images in accordance with the new settings.

[0058] Method step 207 can also include the prediction of future operating situations. Here, the expected application quality can be predicted for future operating situations of the individual applications APP1, APP2, APP3. This makes it possible to check whether a previously determined target configuration parameter set is suitable for providing the functions of the individual applications APP1, APP2, APP3 to a greater extent or more reliably. This makes it possible to find a target configuration parameter set that is optimal for the current operating situation, so that as many applications APP1, APP2, APP3 as possible provided by the computing unit 1.2 are optimally operated in the current operating situation.

Claims

1. A method for operating a camera-based interior monitoring system (1), comprising: at least one camera (1.1) as at least one sensor, which records camera images; a computing unit (1.2) evaluates sensor signals; and, depending on the evaluated sensor signals, configures parameters for setting the interior monitoring system (1), comprising: said computing unit (1.2) serves at least two applications (APP1, APP2, APP3), each of which simultaneously requires different configuration values ​​for at least one parameter in a specific operating situation, said computing unit (1.2) calculates a current application priority (2) for a current operating situation and sets said first parameter configuration value required by said application (APP1) with a higher priority as a target configuration; A method characterized in that for each application (APP1, APP2, APP3), a configuration value of a parameter is assigned an importance, and if the importance of the configuration value of the target configuration of another parameter of the lower application (APP2, APP3) in the application priority (2) is greater than the importance of the configuration value of the target configuration of the parameter of the higher application (APP1), at least one other parameter is configured in accordance with the target configuration requested by the lower application (APP2, APP3).

2. 2. The method according to claim 1, characterized in that the computing unit (1.2) determines application requirements and / or application qualities for each application (APP1, APP2, APP3) for each current or future operating situation and determines the application priority (2) depending on the application requirements and / or the application qualities of each application (APP1, APP2, APP3).

3. Identifying application requirements involves the following steps: - reading predefined application rules; - evaluating sensor signals or assistance systems to predict future operating conditions; - reading user settings; - analyzing the camera images to detect obstructed camera views; - analyzing the camera images to evaluate the lighting conditions of the scene recorded by said camera (1.1); - analysing the camera images to detect objects and / or people; - evaluating the sensor signals and performing sensor fusion; - reading application specific priorities; 3. The method according to claim 2, further comprising at least one of the steps:

4. 4. The method of claim 2 or 3, wherein predicting the future operating state includes predicting how long it will take for the observed person to blink again.

5. To determine the application quality, the following image quality metrics are calculated for at least one camera image: - pixel saturation, -histogram, -S / N ratio, - Structural similarity index, - a measure of complex wavelet structure similarity, and / or - Trust value calculated using artificial intelligence The method according to claim 2 , characterized in that at least one of the following is evaluated:

6. Using the target configuration, the following values ​​are obtained: - selection of which camera (1.1) among several cameras is to be used for recording the camera image; - the selection of the lighting device (1.3) used to illuminate the scene recorded by said camera (1.1); - the light distribution characteristic of said lighting device (1.3), the exposure time, ISO speed and / or aperture of said camera (1.1), -focal plane, - zoom level, - the wavelength spectrum recorded by said camera (1.1), the frame rate of said camera (1.1), -resolution, - image portions of the recorded camera images to be evaluated, and / or - How to post-process the camera images after recording The method according to claim 1 , characterized in that at least one of the following is set:

7. 7. The method of claim 6, wherein at least one numerical value configurable with the target configuration is used as an application requirement.

8. 8. A method according to claim 2, characterized in that the application quality of at least one application (APP1, APP2, APP3) for a future operating situation is predicted from at least one image quality metric calculated in a current and / or past operating situation or from at least one value derived from a selection of image quality metrics.

9. At least one application (APP1, APP2, APP3) - Identifying the direction of gaze, - Fatigue detection, - Seat occupancy detection, -Facial recognition, - monitoring vital signs, - Photo and / or video capture functionality - Gesture recognition 9. The method according to claim 1, wherein the polymer is formed from

10. An interior monitoring system (1) comprising at least one camera (1.1) and a computing unit (1.2), An interior-vehicle monitoring system (1), characterized in that the camera (1.1) and the computing unit (1.2) are configured to perform the method according to any one of claims 1 to 9.

11. A vehicle (3) equipped with an in-vehicle monitoring system (1) according to claim 10, A vehicle (3) characterized in that at least one camera (1.1) is integrated in an instrument cluster, a head unit (4), a rear-view mirror and / or a roof control unit, in particular between the driver's side and the passenger's side with respect to the vehicle's transverse axis (Ax) and preferably at least at the height of the steering wheel with respect to the vehicle's longitudinal axis (Az).

Images