Method of operating a vehicle interior monitoring system, control device, vehicle interior monitoring system and vehicle

Abstract

The invention relates to a method for operating a vehicle interior monitoring system, comprising at least one camera unit. By means of a control device, adaptable camera parameters of the camera unit are adjusted in the camera unit by means of at least one camera-specific control command, and at least one image taken by the camera unit is received and evaluated, wherein the result of the evaluation is output as camera-specific result data. It is provided that at least one generic control command is generated by the control device for adjusting the adaptable camera parameters of the camera unit, the generic control command is converted into the camera-specific control command in accordance with a camera configuration of the camera unit stored in the control device, the camera-specific result data is converted into generic result data in accordance with the camera configuration of the camera unit stored in the control device, and the generic result data of the camera unit is provided to a data fusion device.

Description

Technical Field

[0001] This invention relates to a method, control device, vehicle interior space monitoring system, and vehicle. Background Technology

[0002] Vehicles are equipped with interior space monitoring systems designed to monitor the vehicle's interior space and the occupants present within it. Monitoring allows for the determination of occupant presence and / or location. For example, the vehicle interior space monitoring system can detect the driver's gaze direction or occupant gestures. According to current technology, vehicle interior space monitoring systems typically have two camera units that monitor corresponding detection areas within the vehicle's interior space. The camera units capture single or multiple consecutive images of the corresponding detection areas and provide them to a control unit for evaluation, which performs a desired assessment of the images.

[0003] The evaluation of the detected images and the control (or driving) of the corresponding camera units via the vehicle interior space monitoring system must be adapted to those camera units. This requires programming the control unit of the vehicle interior space monitoring system to manipulate the corresponding camera units, optimize camera parameters, and thus ensure that the captured images meet the image specifications of the corresponding evaluation algorithm. Because camera units and / or their camera drivers (or camera drivers) can differ from one another, the corresponding algorithms must be adapted to each camera unit and / or its camera driver. This results in development costs increasing with the number of types of camera units used, as there is no unified and universal control method for each camera unit.

[0004] The evaluation algorithm used to assess images provided by the camera unit must also be adapted to the camera unit's camera parameters, such as resolution or the camera unit's installation location within the vehicle's interior space. When stitching together measurement data based on images from the respective camera units, the unique characteristics of each camera unit must be taken into account.

[0005] Due to the anticipated increase in camera units in vehicle interior space monitoring systems, as well as the variations in camera units and their increased placement within the vehicle interior space, the following issues arise: a significant amount of software adaptation / adjustment is required to ensure the operation of the vehicle interior space monitoring system.

[0006] WO 2020 / 161610 A2 describes an adapted application for monitoring a train carriage using a single 2D camera and monitoring algorithms. It specifies that a monitoring algorithm is selected from a list of monitoring algorithms based on the occupant's state, determined based on the first set of images in an image sequence of the carriage. The monitoring algorithm is applied to the second set of images in the image sequence, which includes images following the first set of images, in order to enable monitoring of the vehicle.

[0007] US 2020 / 0143560 A1 describes a vehicle monitoring system comprising: an illumination source emitting invisible light that illuminates at least a portion of a vehicle driver; a reflector disposed on the vehicle and within the driver's line of sight of the illuminated portion; a camera disposed within the vehicle and having a field of view including the reflector; and a control device having an image processor that processes image data detected by the camera. Summary of the Invention

[0008] The purpose of this invention is to provide a solution that reduces the cost of adapting vehicle interior space monitoring to different systems.

[0009] This objective is achieved through the subject matter of the independent claims. Advantageous improvements of the invention are disclosed through the features of the dependent claims, the following description, and the accompanying drawings.

[0010] A first aspect of the invention relates to a method for operating a vehicle interior space monitoring system. The vehicle interior space monitoring system includes at least one camera unit configured to monitor a corresponding monitoring area within the vehicle interior space. The method is executed via a control device of the vehicle interior space monitoring system.

[0011] In the first step of the method, adaptable camera parameters of the camera unit are adjusted by a control device using at least one camera-specific control command to adapt for image capture of at least one image by the camera unit, according to the image specifications of the selected evaluation algorithm. The capture of at least one image may also involve capturing a sequence of images. In other words, at least one camera unit is operated by the control device. The camera unit is operated to adapt at least one adaptable camera parameter. For example, at least one adaptable camera parameter may involve adjustable exposure time during image capture, the image area to be captured during image capture, or color adjustment. The adjustment of at least one adaptable camera parameter is achieved through at least one camera-specific control command. The camera-specific control command can be preset by the corresponding camera unit for adapting at least one adaptable camera parameter. The camera-specific control command may be related to the machine type of the camera unit and / or the camera driver used to operate the camera unit. At least one adaptable camera parameter is adapted by the control device such that the image captured by the camera unit during image capture meets the image specifications of the image. The image specifications may, for example, specify the required brightness, the required contrast value, or a specific image area. Here, the image specifications are related to the selected evaluation algorithm. The selected evaluation algorithm could be, for example, an algorithm for detecting motion and / or the driver's gaze direction. By adapting at least one adaptable camera parameter by the control device, it is ensured that at least one image captured by the camera unit meets the image specifications defined by the selected evaluation algorithm.

[0012] In the second step, at least one image captured by a camera unit is received via a control device. The image is evaluated by the control device according to a selected evaluation algorithm, wherein the evaluation result is output as camera-specific result data. In other words, the image is evaluated by the control device as preset by the selected evaluation algorithm. The evaluation algorithm may include computer vision methods. Here, for example, the driver's gaze direction or image points may be detected in at least one captured image. The evaluation result is output as camera-specific result data. The result data may be associated with the corresponding camera unit. For example, it may be specified that the detected image points are output in image coordinates, which may be associated with the orientation of the camera unit in monitoring the vehicle's interior space.

[0013] This invention specifies that a control device generates general control commands according to a predetermined adaptation algorithm to adjust the adaptable camera parameters of a camera unit based on the image specifications of a selected evaluation algorithm. The general control commands differ from camera-specific control commands in that they are camera-independent. In particular, the general control commands can be based on a reference system provided for the adaptation algorithm and can be used solely for further internal processing by the control device. In other words, this invention specifies that a control device generates general control commands to adjust the adaptable camera parameters of a camera unit so that the image specifications of the selected evaluation algorithm are satisfied. The generation of the general control commands can be achieved by the control device according to a predetermined adaptation algorithm. The predetermined adaptation algorithm can, for example, be based on general camera data or on images pre-captured by the camera unit to determine the general control commands. For example, it can be specified that the image specifications of the selected evaluation algorithm specify a specific contrast value. The evaluation algorithm can, for example, determine, based on already captured images, what exposure time must be set as the adaptable camera parameter of the camera unit to satisfy the image specifications. The general control commands can describe the adaptation of the adaptable camera parameters in a general system. A general system may include predetermined level numbers. General control commands are described to adapt adaptable camera parameters according to a reference system. Because the general control commands are not related to the camera unit to be manipulated, the following advantage is obtained: the adaptation algorithm does not need to be adapted to the camera-specific control commands and capabilities of the corresponding camera unit.

[0014] In another step, it is specified that the general control commands are converted into camera-specific control commands by the control device according to the camera configuration stored in the control device for the corresponding camera unit, using a predetermined conversion method. In other words, the general control commands are not used to directly control the corresponding camera unit by the control device.

[0015] Therefore, in order to adapt to compatible camera parameters in the corresponding camera unit, it is necessary to convert general control commands into camera-specific control commands. To enable this conversion, a corresponding camera configuration is stored in the control device for each camera unit. This camera configuration may, for example, include a dictionary for translating general control commands into camera-specific control commands. The corresponding camera configuration may also include specifications for transforming allowed value ranges or values ​​that can be set in the camera unit. The camera configuration of the corresponding camera unit is read in a predetermined conversion method to convert the general control commands into camera-specific control commands.

[0016] In another step, it is specified that camera-specific result data is converted into generalized result data according to a predetermined generalization method based on the camera configuration stored in the control device of the camera unit. In other words, camera-specific result data is output by the camera unit, wherein the camera-specific result data is related to the camera unit itself. For example, detected motion can be referenced to the camera unit. However, in order to further evaluate the result data and, for example, to fuse the result data with data from other camera units, it is necessary to convert the result data into generalized result data independent of the respective camera unit. For example, it can be specified that the detected motion is transformed from coordinates based on the camera unit to coordinates based on a reference point in the vehicle's interior space. Subsequently, the generalized result data of the camera unit is provided by the control device to the data fusion device. The data fusion device can be designed to merge generalized result data from different camera units.

[0017] The improved embodiment of the present invention specifies that the vehicle interior space monitoring system includes at least two camera units. In other words, the present invention specifies that the method is implemented on the vehicle interior space monitoring system via a control device, the vehicle interior space monitoring system including at least two camera units for monitoring corresponding monitoring areas within the vehicle interior space. This improved embodiment offers the advantage that common result data from at least two camera units can be provided to a data fusion device.

[0018] An improved embodiment of the present invention specifies that the selected evaluation algorithm is chosen based on the camera configuration. In other words, the selected evaluation algorithm is chosen from multiple evaluation algorithms based on the camera configuration of the corresponding camera unit. For example, it can be specified that a specific evaluation can be performed using different evaluation algorithms. It is possible that the evaluation algorithms are different from each other, wherein the evaluation algorithm can be optimized for corresponding camera parameters. For example, it can be specified that the evaluation algorithm can be optimized for different contrast values ​​and / or resolutions. The control device can select the evaluation algorithm most suitable for the parameters stored in the camera configuration. This provides the advantage that the evaluation algorithm to be executed can be determined by the control device based on the camera configuration.

[0019] An improved embodiment of the present invention specifies that corresponding general result data from at least two camera units are integrated / merged into fused data by a data fusion device according to a predetermined data fusion method. In other words, general data results from at least two camera units are fused by the data fusion device according to a predetermined data fusion method to generate fused data. For example, it can be specified that the corresponding general result data includes motion detected by the corresponding camera units, and the motion detected by the corresponding camera units is fused by the data fusion device to generate fused data describing three-dimensional motion.

[0020] An improved embodiment of the present invention specifies that at least one camera parameter of the camera unit is retrieved from the camera unit via a control device and stored in the camera configuration. In other words, the present invention specifies that at least one configuration parameter is read from the camera unit via a control device. The configuration parameter may, for example, describe the resolution of the camera unit. The control device stores at least one read configuration parameter in the camera configuration. This provides the advantage that at least one configuration parameter of the camera unit does not need to be manually stored in the control device. An improved embodiment of the present invention specifies that the camera configuration includes the orientation (i.e., direction and position) of the corresponding camera unit within the vehicle's interior space. In other words, the orientation of the corresponding camera unit is stored in the corresponding camera configuration as a configuration parameter of the corresponding camera unit.

[0021] An improved embodiment of the present invention specifies that, according to the corresponding camera configuration, a control device assigns corresponding weighting values ​​to the corresponding general result data of at least two camera units. The weighting values, along with the general result data, are provided to a data fusion device via the control device. The general result data is then synthesized into fused data according to the corresponding weighting values ​​and a predetermined data fusion method.

[0022] In other words, this invention specifies that general result data is fused using a data fusion device, wherein the general result data of corresponding camera units are weighted differently, and the weighting of the general result data of the corresponding camera units is based on corresponding weighting values. Here, the corresponding weighting values ​​are assigned to the corresponding camera units by a control device according to the corresponding camera configuration, and are forwarded to the data fusion device together with the general result data. The weighting values ​​can, for example, describe the accuracy or reliability of the general result data established from the images of the corresponding camera units. For example, it can be specified that a camera unit with a higher resolution than other camera units is assigned a higher weighting value than a camera unit with a lower resolution. When fusing the general result data of the corresponding camera units, the weighting performed by the data fusion device can be used, for example, to give more consideration to the result data of more reliable camera units when the result data is biased.

[0023] For applications or situations that are available in the method but not explicitly described herein, it may be specified that the method outputs fault messages and / or requests for user feedback and / or adjusts to standard settings and / or a predetermined initial state.

[0024] A second aspect of the invention relates to a control device for a vehicle interior space monitoring system. The control device is designed to adjust adaptable camera parameters of at least one camera unit in the vehicle interior space monitoring system using at least one camera-specific control command, to adapt image capture for at least one image captured by the camera unit according to an image specification of a selected evaluation algorithm. The control device is designed to receive and evaluate at least one image captured by the camera unit according to the selected evaluation algorithm. The control device is designed to output the evaluation result as camera-specific result data. Specifically, the control device is designed to generate at least one general control command according to a predetermined adaptation algorithm for adjusting the adaptable camera parameters of the camera unit according to the image specification of the selected evaluation algorithm. The control device is designed to convert the general control command into camera-specific control commands according to a predetermined conversion method based on a camera configuration of the camera unit stored in the control device. The control device is designed to convert camera-specific result data into general result data according to a predetermined generalization method based on a camera configuration of the camera unit stored in the control device. The control device is designed to provide the general result data of the camera unit to a data fusion device.

[0025] The control device may include a data processing apparatus or a processor apparatus designed to execute embodiments of the method according to the invention. The processor apparatus may therefore include at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field-Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). Furthermore, the processor apparatus may have program code designed to execute embodiments of the method according to the invention when implemented by the processor apparatus. The program code may be stored in the data memory of the processor apparatus.

[0026] A third aspect of the invention relates to a vehicle interior space monitoring system having a control device according to a second aspect of the invention. The vehicle interior space monitoring system has at least one camera unit designed to monitor a corresponding monitoring area within the vehicle interior space.

[0027] The fourth aspect of the invention relates to a vehicle having an interior space monitoring system according to the third aspect of the invention.

[0028] The vehicle according to the invention is preferably designed as an automobile, especially as a passenger car or van, or as a bus or motorcycle.

[0029] The present invention also includes a control device according to the invention, a vehicle interior space monitoring system according to the invention, and an improved version of a vehicle according to the invention, which have features already described in conjunction with the improved version of the method according to the invention. For this reason, the corresponding improved versions of the control device according to the invention, the vehicle interior space monitoring system according to the invention, and the vehicle according to the invention will not be described again here.

[0030] As another solution, the invention also includes a computer-readable storage medium comprising instructions that, when implemented by a computer or computer complex, cause the computer or computer complex to implement an embodiment of the method according to the invention. The storage medium may, for example, be at least partially designed as non-volatile data memory (e.g., flash memory and / or SSD (solid-state drive)) and / or at least partially designed as volatile data memory (e.g., RAM (random access memory)). A processor circuit having at least one microprocessor may be provided by the computer or computer complex. The instructions may be provided as binary code or an assembler, and / or as source code in a programming language (e.g., C).

[0031] The present invention also includes combinations of features of the described embodiments. Therefore, the present invention also includes implementations having combinations of features of multiple embodiments described herein, provided that these embodiments are not described as mutually exclusive. Attached Figure Description

[0032] Embodiments of the present invention are described below. Therefore:

[0033] Figure 1 A schematic diagram of a vehicle equipped with an interior space monitoring system is shown.

[0034] Figure 2 Another schematic diagram of a vehicle equipped with an interior space monitoring system is shown;

[0035] Figure 3 A schematic diagram is shown illustrating different levels for executing the method via a control device; and

[0036] Figure 4 A schematic diagram of the process for operating a vehicle interior space monitoring system is shown. Detailed Implementation

[0037] The embodiments described below are preferred embodiments of the invention. In the embodiments, the components described in the embodiments represent individual features of the invention that can be considered independent of each other, and these features also independently improve the invention. Therefore, this disclosure should also include combinations different from the combinations of features shown in the embodiments. Furthermore, the described embodiments can also be supplemented by other features of the invention that have already been described.

[0038] In the accompanying drawings, the same reference numerals denote elements that have the same function.

[0039] Figure 1 A schematic diagram of a vehicle equipped with an interior space monitoring system is shown.

[0040] The vehicle interior space monitoring system 2 may have multiple camera units 3, each designed to monitor a corresponding monitoring area 4 within the vehicle compartment. Each camera unit 3 may have a corresponding monitoring area 4. The vehicle interior space monitoring system 2 may have a control device 5, designed to operate the corresponding camera units 3. Each camera unit 3 may have a corresponding camera driver 6 and camera parameters 7 and 25, which may be fixed or adjustable. The control device 5 may transmit camera-specific control commands 8 to the corresponding camera unit 3 to adapt the adjustable camera parameters 7 so that the captured image 13 meets predetermined image specifications 9 and 10. The corresponding image specifications 9 and 10 may be associated with a selected evaluation algorithm 12 if multiple possible evaluation algorithms exist. Image specifications 9 and 10 may, for example, specify predetermined brightness or contrast values. This ensures that the captured image 13 meets image specifications 9 and 10, so that the image 13 can be evaluated according to the selected evaluation algorithm 12. The selected evaluation algorithm 12 may, for example, be an algorithm for identifying motion or line-of-sight directions in the captured image 13. The selected evaluation algorithm 12 can output the result as camera-specific result data 14. The control device 5 is designed to compensate for differences between camera units 3, such as those attributable to different camera drivers 6. For example, it is possible that a camera-specific control command 8 can be generated by the control device 5 from a general control command 15, where the general control command 15 can be a predetermined adaptation algorithm 16, 17 from multiple adaptation algorithms. The general control command 15 can be non-camera-specific and specifies preset adaptation values ​​for the corresponding camera unit 3. Adaptation algorithms 16, 17 can output the general control command 15 independently of the characteristics of the corresponding camera unit 3. Therefore, it is not necessary to adapt the evaluation algorithm to the corresponding camera unit 3 or the corresponding camera driver 6. The adaptation of the general control command 15 to the camera-specific control command 8 can be performed according to the corresponding stored camera configuration 18 using a predetermined conversion method 19. The conversion method 19 can, for example, have a table that can assign the general control command 15 to the corresponding camera-specific control command 8. To avoid compatibility issues between the evaluation algorithm and the corresponding camera unit 3, the control device 5 is designed to execute a predetermined generalization method 20. The predetermined generalization method 20 may be designed to convert the corresponding camera-specific result data 14 into general result data 21. The camera-specific result data may, for example, describe the coordinates of the detected motion relative to the orientation 26 of the camera unit 3, while the general result data is independent of the corresponding camera unit 3.Advantageously, the general result data 14, 21 can be input into the data fusion device 22, which can convert the general result data 14, 21 into fused data 24 according to a predetermined data fusion method 23. It is possible that the camera units 3 may differ from each other in terms of accuracy. To take this into account when fusing the corresponding general result data 21, it can be stipulated that a corresponding weighting value 27 is assigned to the corresponding general result data 21 according to the corresponding camera configuration 18. The weighting value 27 can be taken into account by the data fusion device 22 when executing the predetermined data fusion method 23.

[0041] Figure 2 Another schematic diagram of a vehicle 1 with an interior space monitoring system 2 is shown.

[0042] Figure 3 A schematic diagram is shown of different levels for executing the method via control device 5.

[0043] A camera driver layer L0 at one end of a multi-layered system may include a corresponding camera driver 6, which enables the camera unit 3 to operate. Each camera unit 3 and its camera driver 6 may differ from one another, thus potentially requiring the transmission of corresponding camera-specific control commands 8 to the corresponding camera driver 6 to manipulate the corresponding camera unit 3. Furthermore, the fixed camera parameters 25 and variable camera parameters 7 of each camera unit 3 may differ from one another. To eliminate the adaptation of the corresponding evaluation algorithms 11, 12 and the corresponding adaptation algorithms 16, 17 to the corresponding camera unit 3, a predetermined layer may be provided, which can undertake the adaptation to the corresponding camera unit 3.

[0044] For example, it can be specified that the camera abstraction layer L1 can be arranged between the adaptation layer L2, which includes the adaptation algorithm 16, and the camera driver layer L0. The camera abstraction layer L1 can be configured to convert general control commands 15 into camera-specific control commands 8. In other words, the camera abstraction layer L1 accepts general control commands 15 regardless of the type of a single camera unit 3, and converts the general control commands 15 determined for the corresponding camera unit 3 into camera-specific control commands 8 according to a predetermined conversion method 19. This provides the advantage that the corresponding adaptation algorithm 16 can output general control commands 15 that are independent of the type of the corresponding camera unit 3.

[0045] The adaptation layer L2 may include different adaptation algorithms 16, which can be implemented to adapt the captured image 13 to the corresponding image specifications 9, 10. The corresponding adaptation algorithm 17 may be associated with the camera unit 3 and / or the selected evaluation algorithm 12.

[0046] The selected evaluation algorithm 11 can be chosen from multiple evaluation algorithms 12 in the evaluation layer L3, and is configured, for example, to detect head position or movement in the captured image 13. It is also possible that camera-specific result data 14 is associated with the corresponding camera unit 3 during the evaluation. It can be specified that the camera-specific result data 14 must be transmitted to the data fusion device 22 as general result data 21 in a predetermined format. To avoid adaptation within a single evaluation algorithm 12, a generalization layer L4 can be arranged between the evaluation layer L3 of the evaluation algorithm 12 and the data fusion layer L5 of the data fusion device 22. This generalization layer can be designed to convert the camera-specific result data 14 into general result data 21 according to a predetermined generalization method 20 and provide it to the data fusion device 22.

[0047] It can be stipulated that the general result data 21 is forwarded to the data fusion device 22 in the data fusion layer L5 to execute the predetermined data fusion method 23.

[0048] The control unit 5 of the vehicle interior space monitoring system 2 in vehicle 1 can be adapted to the camera units 3 arranged in the vehicle interior space via software configuration. For this purpose, each camera unit 3 reports camera parameters 7, 25 to the control unit 5. These camera parameters are important for evaluating the images 13 of the corresponding camera unit 3 to generate camera-specific result data 14. Camera parameters 7, 25 may include technically immutable camera parameters 25, such as the corresponding vertical and horizontal field of view, resolution, number of images 13 provided per second, orientation within the vehicle interior space, and RGB / IR capability. Camera parameters 7, 25 may also include adaptable camera parameters 7, such as the number of illumination levels and adjustable image segments.

[0049] The vehicle interior space monitoring system 2 has a camera configuration 18 with available camera units 3. The camera units 3 may include a driver-oriented camera unit 3, a centrally oriented camera unit 3, and other camera units 3. The camera configuration 18 describes the capabilities and characteristics of the respective camera units 3. Therefore, the control device 5 is designed to determine, based on the camera configuration 18, which evaluation algorithms 12 are applied to the images 13 originating from the respective camera units 3. The camera configuration 18 can be preset to determine which weighting values ​​27 should be assigned to the general result data 21 of the respective camera units 3 during data fusion. The camera configuration 18 can also be preset to determine how to convert camera-specific result data 14 into general result data 21.

[0050] The vehicle interior space monitoring system 2 provides a software architecture that includes an adaptation layer between the camera unit 3 and the data fusion device 22. The camera-hardware-abstraction layer is the interface between the software modules used for the adaptation algorithms and the corresponding camera driver 6. Its purpose is to convert the general control commands 15 provided by the software modules used for the adaptation algorithms 16, 17 into camera-specific control commands 8 required by the camera driver 6, according to the camera configuration 18 of the corresponding camera unit 3 and a predetermined conversion method 19.

[0051] The adaptation layer contains software modules for adaptation algorithms, which aim to adapt the adaptable camera parameters 7 of the corresponding camera unit 3 using general control commands 15, so as to optimize the image 13 for computer image processing according to the image specifications 9, 10 of the selected evaluation algorithm 12. The implementation of software modules for outputting adaptation algorithms 16, 17 that are independent of the corresponding camera unit 3 and camera driver 6 enables the standardization of control over different options provided by different camera drivers 6. Since the general control commands 15 are independent of the corresponding camera unit 3 and camera driver 6, it is not necessary to adapt the adaptation algorithms 16, 17 to the corresponding camera unit 3 and camera driver 6. The adaptable camera parameters 7 can be changed, for example, by the general control commands 15 used for lighting adaptation or calibration. This allows for changes to the camera unit 3 and / or camera driver 6, while preserving the capability of the adaptation algorithms 16, 17 for adapting the adaptable camera parameters 7 of the corresponding camera unit 3. Coordination with the corresponding camera unit 3 and camera driver 6 is achieved through a conversion algorithm of camera-hardware-abstraction layer, which converts general control commands 15 into corresponding camera-specific control commands 8 according to the corresponding camera configuration 18.

[0052] Evaluation algorithms 11 and 12 evaluate the image 13 received by camera unit 3 and output the driver's head position or the driver's eye opening state as camera-specific result data 14. Based on the fixed, technical camera parameters 7 and 25 of camera unit 3, the perception system determines which evaluation algorithm of evaluation algorithms 11 and 12 should be selected as evaluation algorithm 11 for evaluating the corresponding image 13, and which weighting value 27 should be assigned to the corresponding result data 21 of the corresponding camera unit 3 for the data fusion method 23 performed by data fusion device 22.

[0053] The adaptation layer serves as the interface between the evaluation layer and the fusion layer. Its purpose is to convert camera-specific result data 14 into general result data 21 so that the results of the evaluation algorithm 11, which are independent of the camera parameters 7 and 25 of the corresponding camera unit 3, can be interpreted / understood.

[0054] Figure 4 A schematic diagram of the process for operating the vehicle interior space monitoring system 2 is shown.

[0055] Control device 5 can generate at least one general control command 15 according to predetermined adaptation algorithms 16 and 17, for adjusting the adaptable camera parameters 7 of camera unit 3 according to the image specifications 9 and 10 of the selected evaluation algorithm 12. According to the camera configuration 18 of camera unit 3 stored in control device 5, the general control command 15 can be converted into camera-specific control commands 8 and 15 according to predetermined conversion method 19.

[0056] The control device 5 allows for the adjustment of adaptable camera parameters 7 in the camera unit 3 using at least one camera-specific control command 8, so as to adapt S3 for image capture of at least one image 13 by the camera unit 3 according to the image specifications 9, 10 of the selected evaluation algorithm 12.

[0057] The camera unit 3 can capture at least one image 13 according to at least one adjusted and adaptable camera parameter 7, and send it to the control device 5 via S4.

[0058] The control device 5 can receive at least one image 13 captured by the camera unit 3 and evaluate the image according to the selected evaluation algorithm 12, wherein the evaluation result can be output as camera-specific result data 14 S5.

[0059] Using the control device 5, camera-specific result data 14 can be converted into general result data 21 according to the camera configuration 18 stored in the control device 5 and a predetermined generalization method 20. The general result data 21 of the camera unit 3 can then be provided to the data fusion device 22 via the control device 5 (S6).

[0060] Given the available space in the vehicle's cockpit, the price range in which different models compete, the laws, regulations, and standards of different geographic markets, the design principles followed by specific brands, and the fact that new models developed in the next 10-15 years will follow the same architecture, camera sensor arrays must constantly evolve, despite further advancements in optical and computer-aided technologies, as well as other influencing factors. The solution to this increasing sensor variability lies in developing visual perception systems that can adapt to available sensors and describe their technological limitations, thereby maintaining architectural compatibility.

[0061] Overall, the examples illustrate how solutions can be provided for the increasing variability of sensors.

Claims

1. A method for operating a vehicle interior space monitoring system (2), the vehicle interior space monitoring system comprising at least one camera unit (3), wherein, Through the control device (5) - In the camera unit (3), the adaptable camera parameters (7) of the camera unit (3) are adjusted by means of at least one camera-specific control command (8) so as to adapt to image capture for at least one image (13) to be captured by the camera unit (3) according to the image specifications (9, 10) of the evaluation algorithm (12). - Receive and evaluate at least one image (13) captured by the camera unit (3) according to the evaluation algorithm (12), wherein the evaluation result is output as camera-specific result data (14). Its characteristic is that it is controlled by a control device (5). - Generate at least one general control command (15) according to a predetermined adaptation algorithm (16, 17) to adjust the adaptable camera parameters (7) of the camera unit (3) according to the image specifications (9, 10) of the selected evaluation algorithm (12). The general control command describes the adaptation of the adaptable camera parameters in the general system. - Based on the camera configuration (18) stored in the control device (5) of the camera unit (3), the general control command (15) is converted into a camera-specific control command (8) according to a predetermined conversion method (19), wherein the camera configuration includes the orientation of the corresponding camera unit in the vehicle interior space. - Based on the camera configuration (18) stored in the control device (5) of the camera unit (3), the camera-specific result data (14) is converted into general result data (21) according to a predetermined generalization method (20). - Provide the general result data (21) of the camera unit (3) to the data fusion device (22).

2. The method according to claim 1, characterized in that, The vehicle interior space monitoring system (2) includes at least two camera units (3).

3. The method according to claim 1 or 2, characterized in that, The selected evaluation algorithm (12) is chosen based on the camera configuration (18).

4. The method according to claim 2, characterized in that, The corresponding general result data (21) of at least two camera units (3) are merged into fused data (24) by the data fusion device (22) according to a predetermined data fusion method (23).

5. The method according to claim 1 or 2, characterized in that, At least one camera parameter of the camera unit (3) is retrieved from the camera unit (3) by the control device (5) and stored in the camera configuration (18).

6. The method according to claim 1 or 2, characterized in that, The camera configuration (18) includes the orientation (26) of the corresponding camera unit (3) in the interior space of the vehicle.

7. The method according to claim 2, characterized in that, According to the corresponding camera configuration (18), the corresponding weighted values ​​(27) are assigned to the corresponding general result data (21) of at least two camera units (3) by the control device (5). - The weighted values ​​(27) and general result data (21) are provided to the data fusion device (22) via the control device (5). - Based on the corresponding weighting value (27), the general result data (21) is merged into fused data (24) by the data fusion device (22) according to the predetermined data fusion method (23).

8. A control device (5) for a vehicle interior space monitoring system (2), wherein, The control device (5) is designed to be used for, - In the camera unit (3), the adaptable camera parameters (7) of at least one camera unit (3) of the vehicle interior space monitoring system (2) are adjusted by means of at least one camera-specific control command (8) in order to adapt the image capture for at least one image (13) to be captured by the camera unit (3) according to the image specifications (9, 10) of the selected evaluation algorithm (12). - Receive and evaluate at least one image (13) captured by the camera unit (3) according to the selected evaluation algorithm (12), and output the evaluation result as camera-specific result data (14). Its features are, The control device (5) is designed to be used for, - Generate at least one general control command (15) according to a predetermined adaptation algorithm (16, 17) to adjust the adaptable camera parameters (7) of the camera unit (3) according to the image specifications (9, 10) of the selected evaluation algorithm (12). The general control command describes the adaptation of the adaptable camera parameters in the general system. - Based on the camera configuration (18) of the camera unit (3) stored in the control device (5), the general control command (15) is converted into a camera-specific control command (8) according to a predetermined conversion method (19), wherein the camera configuration includes the orientation of the corresponding camera unit in the vehicle interior space. - Based on the camera configuration (18) stored in the control device (5) of the camera unit (3), the camera-specific result data (14) is converted into general result data (21) according to a predetermined generalization method (20), and - Provide the general result data (21) of the camera unit (3) to the data fusion device (22).

9. A vehicle interior space monitoring system (2), the vehicle interior space monitoring system comprising the control device (5) according to claim 8.

10. A vehicle (1) comprising the vehicle interior space monitoring system (2) according to claim 9.

Images