Method for operating a multi-camera system with at least two cameras for a motor vehicle by means of an electronic computing device, computer program product, computer-readable storage medium

Abstract

The invention relates to a method for operating a multi-camera system (3) with at least two cameras (5, 6, 7, 8) for a motor vehicle (1) by means of an electronic computing device (4) of the multi-camera system (3), comprising the steps: determining at least one current first control value (10) for at least a first camera (5) of the multi-camera system (3) depending on a current environmental parameter by means of the electronic computing device (4); determining at least one current second control value (11) for at least a second camera (6) of the multi-camera system (3) depending on a historical control value (12) for the second camera (6) by means of the electronic computing device (4); and controlling the first camera (5) by means of the first control value (10) and controlling the second camera (6) by means of the second control value (11) by means of the electronic computing device (4). Further, the invention relates to a computer program product, to a computer-readable storage medium as well as to an electronic computing device (4).

Description

[0001] Method for operating a multi-camera system with at least two cameras for a motor vehicle by means of an electronic computing device, computer program product, computer- readable storage medium

[0002] The invention relates to a method for operating a multi-camera system with at least two cameras for a motor vehicle by means of an electronic computing device of the multicamera system according to the applicable claim 1. Further, the invention relates to a corresponding computer program product, to a corresponding computer-readable storage medium as well as to a corresponding electronic computing device.

[0003] From the motor vehicle construction, multi-camera systems are already known, which can for example correspondingly capture an environment of the motor vehicle. Hereto, it is for example provided that at least two cameras, in particular for example at least four cameras, are connected to a common electronic computing device. They can for example be used for the environmental capture, for example for at least partially automated or fully automated drive functions. However, the problem herein is in that the images of multiple cameras are processed in particular in a so-called ADAS (Advanced Driver Assistance) function. Therein, a problem is in maintaining a high image quality and stability despite of for example a limited processing power and bandwidth of the electronic computing device and in particular in varying light conditions.

[0004] Thereto, it is known from the prior art that the use of a basic scheduler is used in existing camera systems, which alternatingly processes some cameras and skips other ones, which is also referred to as skipping. However, this can result in problems like abrupt changes or fluctuations in the image quality, in particular in high-speed scenarios with fast scene / lighting changes.

[0005] CN1 15965926 A relates to a road marking inspection system mounted on vehicle, which is associated with the technical field of the machine image processing and includes a WEB server, a PC-smart image processing module and a mobile phone app, wherein the WEB server is used for maintaining machine contact data of traffic signs and traffic marking lines, the aggregation of data, which has been uploaded from the PC-smart image processing module and the mobile phone app, and the display and the export of a comprehensive recognition result; the smart PC module is used to fast collect markings and marked lines, to analyze the markings or marked lines in real time and to upload the recognition results to the WEB server via a 4G network in real time; the mobile phone app is used to input traffic sign information, to upload the traffic sign information to the WEB server and then to output the traffic sign information to the smart PC vision module in a standing book mode.

[0006] KR 100 887 075 B1 describes a method for automatic exposure control of an image sensor using a PID technology, which is capable of fast determining a moderate exposure to determine the analog gain and the exposure time of the image sensor and to control the brightness value of the image captured by the image sensor. An exposure time reference table, an analog gain reference table and a stage reference table are created. The target value is compared to the brightness value of a video image from a plurality of images, which are input from the image sensor one after the other. The error rate corresponding to the error is calculated. The defined error rate is calculated and it is determined if it becomes substantially 0. The error size showing the difference of the error rate about the video frame and previous frame is calculated with a statement. The PID index of the reference table is set by the control.

[0007] It is the object of the present invention to provide a method, a corresponding computer program product and a corresponding computer-readable storage medium as well as a corresponding electronic computing device, by means of which a multi-camera system for a motor vehicle can be operated in improved manner.

[0008] The object is solved by a method, a corresponding computer program product, a corresponding computer-readable storage medium as well as a corresponding electronic computing device according to the independent claims. Advantageous forms of configuration are specified in the dependent claims.

[0009] An aspect of the invention relates to a method for operating a multi-camera system with at least two cameras for a motor vehicle by means of an electronic computing device of the multi-camera system. Determining at least one current first control value for at least a first camera of the camera system depending on a current environmental parameter by means of the electronic computing device is effected. At least one current second control value for at least a second camera of the camera system is determined depending on a historical control value for the second camera by means of the electronic computing device, and controlling the first camera by means of the first control value and controlling the second camera by means of the second control value by means of the electronic computing device are effected. Thus, it is allowed that the control is effected by means of the electronic computing device based on different dependencies. Thus, the control value for the first camera can in particular be determined depending on the environmental parameter and the control of the second camera can be determined depending on the historical control value in the present embodiment. For example, this has the advantage, if the electronic computing device should have a correspondingly low computing capacity, thus, an evaluation of the environmental parameter does not have to be effected for the second camera, but the current control value is instead ascertained based on the historical control value, whereby corresponding computing capacities can in particular be saved.

[0010] The multi-camera system in the following embodiment comprises at least two cameras, in particular at least three, preferably four cameras. Therein, it can for example be provided that the control values of three cameras are determined based on the environmental parameter and the control value is in turn ascertained based on the historical control value in a fourth camera.

[0011] Thus, the invention in particular takes advantage of the fact that a smart scheduler is used, which employs a predictive controller to for example adapt the control values for each camera of the multi-camera system based on a previous course and the current trend even if for example some cameras are excluded from the processing due to the computing capacity of the electronic computing device.

[0012] Thus, in the invention, a smart scheduler is in particular proposed, which is also referred to as intelligent scheduler, for the multi-camera system for motor vehicles, in which the image processing technologies are for example used for ADAS functions. Based on the used scheduler, the image quality and stability can in turn be improved even if in particular at least one camera is excluded from the processing due to the increasing number of cameras and the limited processing power and bandwidth of the electronic computing device. Therein, the smart scheduler also uses a predictive controller, to for example generate a proposal for the next cycle based on an adaptive prediction, which anticipates and adapts the future change of the light conditions.

[0013] Therein, the proposed solution in particular uses the implementation approach of the predictive scheduler, which can generate a proposal value for a next cycle based on an adaptive prediction, which can anticipate future changes for example of the lighting and adapt to them. Therein, the predictive controller is a feedforward control mechanism, which uses a model of the multi-camera system and of the environment to predict the future behavior of the multi-camera system and to generate an optimum control measure.

[0014] Thus, the smart scheduler is in particular a component, which manages the allocation of resources and tasks between the different cameras and the image processing unit. The smart scheduler decides based on the priority, the availability and the requirement of the multi-camera system, which cameras are to be processed and which ones are to be skipped. The smart scheduler can use different methods to decide, which camera is to be skipped, for example a systematic schedule or a schedule based on priority. The smart scheduler also coordinates the communication and synchronization between the cameras and the image processing unit.

[0015] Thus, a gist of the invention is in particular the predictive scheduler, which adapts the control values for each camera based on the previous course and the current trend instead of using a fixed or random schedule, which alternatingly processes some cameras or not at all.

[0016] The advantage of this invention compared to the prior art is in particular in that an improved image quality and stability can be realized in that abrupt changes or fluctuations of the gain and exposure values are avoided. Furthermore, the method can adapt to different light conditions in that it predicts and adapts future changes based on the current trend and error. Furthermore, the processing power and the bandwidth can be optimized in that it omits only those cameras, which have minimum errors or deviations at the gain and exposure values. Further, the safety and efficiency of the ADAS function can also be increased in that it provides consistent and accurate images of all of the cameras.

[0017] According to an advantageous form of configuration, a prediction value for the second control value is predicted by means of a prediction module of the electronic computing device for determining the second control value. Thus, the second control value can be reliably ascertained. Further, it can be provided that a mathematical model for at least the second camera is provided for the prediction. Thus, corresponding parameters of the camera itself can in particular also be taken into account.

[0018] A further advantageous form of configuration provides that a mathematical equation for the second camera or a statistical model for the second camera or a neural network for the second camera is provided as the mathematical model. Thus, the prediction model within the predictive scheduler can in particular comprise a model of the multi-camera system and the environment to predict the future behavior of the multi-camera system and for example of the lighting conditions. Therein, the model can be mathematical equations, a statistical formula, a neural network or another method, which can capture the dynamics and the features of the multi-camera system and of the environment. The prediction model can use the past data and the current data as inputs into the model and output a prediction value for the gain-exposure values for each camera for the next cycle.

[0019] It is also advantageous if the prediction value is optimized by means of a cost function. Thus, an optimization step can in particular be used within the electronic computing device or the predictive controller, which for example uses also the sensor restrictions to optimize the predicted value. The cost function is a measure of the quality of the system power based on the deviation from the target values and the control effort. The cost function can be a linear function, a square function, a weighted function or any other function, which can quantify the power of the multi-camera system. The sensor restrictions are the physical restrictions of the sensor, which measures the gain and exposure values. The sensor restrictions can also be the minimum and maximum values, the range, the resolution, the accuracy or any other parameter, which defines the capabilities and limits of the sensor. The predictive controller uses an optimizer to find the optimum prediction value, which minimizes the cost function without considering the sensor restriction. The optimizer can be a gradient descent, a genetic algorithm, a particle swarm or any other algorithm, which can solve the optimization problem.

[0020] A further advantageous form of configuration provides that a gain value and / or exposure value for a respective camera are determined as a respective control value. In particular, both the gain value and the exposure value can be correspondingly determined. The cameras can in particular so-called automatic cameras, which in turn autonomously adjust themselves based on the gain value and / or the exposure value such that an improved environmental perception and capture can be performed. Thus, an improved environmental capture can be realized based on the gain value and / or exposure value.

[0021] In a further advantageous form of configuration, it is provided that the method is performed in case of a computing capacity shortage and / or a bandwidth shortage of the electronic computing device. In particular in the motor vehicle construction, it has turned out that more and more cameras are connected to the electronic computing device. Now, if corresponding shortages should occur, thus, the electronic computing device can be formed not to perform or to skip a corresponding processing of some cameras. According to the invention, at least an automated adaptation or the control values for these cameras is now performed. In other words, if correspondingly sufficient computing power should be present in the electronic computing device, thus, an automatic adaptation of the control values based on the current environmental parameters is effected. But if a reduced computing power or not sufficient computing power should be present, thus, the method can be correspondingly performed, whereby an improved evaluation of the environment can nevertheless be realized in case of too low computing capacity.

[0022] Further, it has proven advantageous if it is decided by means of the electronic computing device, in which camera based on current environmental parameters and in which camera based on the historical control value the respective current control value is determined. Thus, the electronic computing device can in particular be formed to decide in various manners, which cameras are skipped, or for example a systematic schedule or a schedule based on priority is performed. Therein, the electronic computing device also controls the communication and the synchronization between the cameras and image processing units.

[0023] Further, it has proven advantageous if the first control value and the second control value are additionally determined by means of a proportional-integral-derivative controller. The proportional-integral-derivative controller is in particular a so-called PID controller. Thus, the PID controller can in particular be used after the processing phase, which can adapt the gain and exposure values for each camera based on the previous course and the current trend. The PID controller is a feedback mechanism, which uses proportional, integral and derivative terms to calculate an output value, which minimizes the error between a desired target value and a measured process variable. In this case, the target value is the optimum gain and exposure value for each camera and the process variable is the actual gain and exposure value, which is measured by a sensor. The PID controller can then generate a corresponding output value, it corrects the error and brings the process variable closer to the target value.

[0024] According to a further advantageous form of configuration, an evaluation of the proportional-integral-derivative controller is performed before an evaluation of the prediction module for determining the second control value. In other words, the proportional-integral-derivative controller is formed before the prediction module in terms of processing. Thus, the prediction module uses the evaluation of the proportional- integral-derivative controller to be able to make a corresponding prediction. Thus, the control value for a camera can be determined in improved manner. It is also advantageous that a current trend in the environment is determined by means of the proportional-integral-derivative controller and the current trend is taken into account in determining the control values. For example, an exposure trend, for example if it gets brighter or darker, can be correspondingly used, to be able to reliably determine the control value.

[0025] It has further proven advantageous if the current first control value is determined depending on the current environmental parameter and the current second control value is determined depending on the historical second control value in a first time step, and the current second control value is determined depending on the current environmental parameter and the current first control value is determined depending on a historical first control value in a second time step following the first time step. In other words, it is provided that the first control value is determined based on the environmental parameter and the second control value is determined based on the historical data in the first time step. This can be inversely effected in a second time step following the first time step. If the multi-camera system should for example comprise four cameras, thus, the second camera can first be controlled based on historical data in a first time step, the third camera can be controlled based on historical data in a second time step, the fourth camera can be controlled based on the historical data in a third time step, and the first camera can be controlled based on the historical data in a fourth time step. Thus, it is a linear sequence, which cameras are "skipped", in this embodiment. Alternatively, it can for example also be controlled related to priority, for example a rearwards directed camera can basically be correspondingly skipped in a forwards directed parking maneuver, while the forwards directed cameras correspondingly evaluate their environment based on the current environmental condition.

[0026] The presented methods are in particular a computer-implemented method. Therefore, a further aspect of the invention relates to a computer program product with program code means, which cause an electronic computing device, when the program code means are run by the electronic computing device, to perform a method according to the preceding aspect.

[0027] Therefore, the invention further also relates to a computer-readable storage medium with at least one computer program product according to the preceding aspect.

[0028] A still further aspect of the invention also relates to an electronic computing device of a multi-camera system with at least two cameras for a motor vehicle, wherein the electronic computing device is formed for performing the method according to the preceding aspect.

[0029] In particular, the method is performed by means of the electronic computing device.

[0030] Further, the invention also relates to a multi-camera system with at least two cameras and with an electronic computing device according to the preceding aspect.

[0031] The invention also relates to a motor vehicle with a multi-camera system according to the preceding aspect.

[0032] Advantageous forms of configuration of the method are to be regarded as advantageous forms of configuration of the computer program product, of the computer-readable storage medium, of the electronic computing device, of the multi-camera system as well as of the motor vehicle. Hereto, the electronic computing device, the multi-camera system as well as the motor vehicle comprise concrete features to be able to perform corresponding method steps.

[0033] An electronic system can be understood by an electronic vehicle guidance system, which is configured to guide a vehicle in fully automated or fully autonomous manner, in particular without an intervention in a control by a driver being necessary. The vehicle automatically performs all of the required functions, such as for example steering, braking and / or acceleration maneuvers, the observation and capture of the road traffic as well as corresponding reactions. In particular, the electronic vehicle guidance system can implement a fully automatic or fully autonomous driving mode of the motor vehicle according to level 5 of the classification according to SAE J3016. A driver assistance system (advanced driver assistance system, ADAS) can also be understood by an electronic vehicle guidance system, which assists the driver in partially automated or partially autonomous driving. In particular, the electronic vehicle guidance system can implement a partially automated or partially autonomous driving mode according to the levels 1 to 4 according to the SAE J3016 classification. Here and in the following, "SAE J3016" refers to the corresponding standard in the version of April 2021.

[0034] Therefore, the at least partially automatic vehicle guidance can include guiding the vehicle according to a fully automatic or fully autonomous driving mode of the level 5 according to SAE J3016. The at least partially automatic vehicle guidance can also include guiding the vehicle according to a partially automated or partially autonomous driving mode according to the levels 1 to 4 according to SAE J3016. The at least one control signal can for example be provided to one or more actuators of the motor vehicle, among them for example one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the motor vehicle. The one or more actuators can influence a longitudinal and / or lateral control of the motor vehicle to at least partially automatically guide the motor vehicle.

[0035] The assistance information can be output via an output device of the motor vehicle, for example a display and / or an audio output system and / or a haptic output system.

[0036] Here and in the following, an artificial neural network can be understood as a software code, which is stored on a computer-readable storage medium and represents one or more linked artificial neurons or can emulate their function. Therein, the software code can also include multiple software code components, which can for example have different functions. In particular, an artificial neural network can implement a non-linear model or a non-linear algorithm, which maps an input to an output, wherein the input is given by an input feature vector or an input sequence and the output can for example include an output category for a classification task, one or more predicated values or a predicated sequence.

[0037] In the present disclosure, a computing unit / electronic computing device can for example be understood as a data processing device with processing circuits. Thus, a computing unit can perform computing operations to process data. The computing operations can also include indexed accesses to a data structure, for example a look-up table, LUT.

[0038] In particular, a computing unit can include one or more computers, one or more microcontrollers and / or one or more integrated circuits, for example one or more application-specific integrated circuits, ASIC, one or more field-programmable gate arrays, FPGA, and / or one or more systems on a chip, SoC. The computing unit can also include one or more processors, for example one or more microprocessors, one or more central processing units, CPU, one or more graphics processing units, GPU, and / or one or more signal processors, in particular one or more digital signal processors, DSP. The computing unit can also include a physical or virtual cluster of computers or others of the mentioned units.

[0039] A computing unit can also include one or more hardware and / or software interfaces and / or one or more storage units. Therein, a storage unit can be designed as a volatile data memory, for example as a dynamic random access memory, DRAM, or static random access memory, SRAM, or as a non-volatile data memory, for example as a read-only memory, ROM, as a programmable read-only memory, PROM, as an erasable programmable read-only memory, EPROM, as an electrically erasable programmable read-only memory, EEPROM, as a flash memory or flash EEPROM, as a ferroelectric random access memory, FRAM, as a magnetoresistive random access memory, MRAM, or as a phase-change random access memory, PCRAM.

[0040] For application cases or application situations, which can arise in a method according to the invention and which are not explicitly described herein, it can be provided that an error message and / or a request for inputting a user feedback are output and / or a default setting and / or a predetermined initial state are adjusted according to the method.

[0041] Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and / or shown in the figures can be encompassed by the invention not only in the respectively specified combination, but also in other combinations. In particular, implementations and feature combinations can also be encompassed by the invention, which do not comprise all of the features of an originally formulated claim. Moreover, implementations and feature combinations can be encompassed by the invention, which extend beyond or deviate from the feature combinations set forth in the relations of the claims.

[0042] There show:

[0043] Fig. 1 a schematic top view to a motor vehicle with an embodiment of a multi-camera system with an embodiment of an electronic computing device;

[0044] Fig. 2 a schematic block diagram according to an embodiment of a multi-camera system;

[0045] Fig. 3 a schematic block diagram according to an embodiment of an electronic computing device;

[0046] Fig. 4 a still further schematic block diagram according to an embodiment of an electronic computing device; and Fig. 5 a temporal flow diagram according to an embodiment of the method.

[0047] In the following, the invention is explained in more detail based on specific embodiments and associated schematic drawings. In the figures, identical or functionally identical elements can be provided with the same reference characters. Optionally, the description of identical or functionally identical elements is not necessarily repeated with respect to different figures.

[0048] Fig. 1 shows a schematic side view of an embodiment of a motor vehicle 1 . In the present embodiment, the motor vehicle 1 comprises at least one assistance system 2, which in particular performs so-called ADAS functions. These ADAS functions can in particular be functions, which are formed for an at least partially automated operated or fully automated operating mode of the motor vehicle 1 . Hereto, the motor vehicle 1 comprises at least one multi-camera system 3 in the following embodiment. The multi-camera system 3 comprises at least one electronic computing device 4, a first camera 5, a second camera 6, a third camera 7 as well as a fourth camera 8 (Fig. 2). The cameras 5, 6, 7, 8 are in particular formed for capturing an environment 9 of the motor vehicle 1 .

[0049] Fig. 2 in turn shows a schematic block diagram of the multi-camera system 3. In the present embodiment, it is in particular shown that the four cameras 5, 6, 7, 8 are coupled to the electronic computing device 4.

[0050] Thus, a method for operating the multi-camera system 3 with the at least two cameras 5, 6, 7, 8 can in particular be proposed. Therein, determining at least one current first control value 10 for at least the first camera 5 depending on a current environmental parameter is effected by means of the electronic computing device 4. Determining at least one current second control value 11 for at least the second camera 6 of the multi-camera system 3 depending on a historical control value 12 (Fig. 3) and controlling the first camera 5 by means of the first control value 10 and controlling the second camera 6 by means of the second control value 11 are effected.

[0051] Fig. 3 in turn shows a schematical block diagram according to an embodiment of the electronic computing device 4. In the following embodiment, it is in particular shown that the historical control value 12 serves as an input for a PID controller, which is also referred to as proportional-integral-derivative controller 13. Thereafter, a summation module 14 follows. A predicted control value 15 as well as a prediction module 16 are also shown.

[0052] Therein, Fig. 3 in particular shows an implementation approach, wherein the PID controller is in turn used at the end of the processing phase, which adapts the gain and exposure values, in particular as the control values 10, 11 for each camera 5, 6, 7, 8 based on the previous course and the current trend. Therein, the PID controller is a feedback mechanism, which uses proportional, integral and derivative terms to calculate an output value, which minimizes the error between a desired target value and a measured process variable. In this case, the target value is the optimum gain and exposure value for each camera 5, 6, 7, 8 and the process variable is the actual gain and exposure value, which is measured by each camera 5, 6, 7, 8. Then, the PID controller can generate an output value, it corrects the error and brings the process variable closer to the target value.

[0053] Therein, the implementation approach shown now also uses the prediction module 16, which can generate a proposal value for the next cycle based on an adaptive prediction, which can anticipate future changes of the lighting conditions and adapt to them. Therein, the predictive controller or the prediction module 16 is a feedforward control mechanism, which uses a model of the system and of the environment 9 to predict the future behavior of the system, in particular of the multi-camera system, and to generate an optimum control measure. Therein, the predictive controller or the prediction module 16 operates after the PID phase.

[0054] Fig. 4 shows a schematic block diagram according to an embodiment of the prediction module 16. Therein, an optimizer 17, a mathematical model 18, a process module 19 as well as an output signal 20 are in particular shown. A future error 21, a cost function 22 as well as sensor restrictions 23 in turn serve as an input of the optimizer 17. Future inputs 24 as well as the output signal 20 are in turn shown as the input signal for the mathematical model 18. A predicted output signal 25 is in turn fed back from the mathematical model 18.

[0055] Therein, it is in particular shown that the prediction module 16 in turn comprises the main steps of prediction, optimization, adaptation and generation. In the step of the prediction, the prediction module 16 uses a model of the system and of the environment 9 to predict the future behavior of the multi-camera system 3 and of the exposure conditions. The mathematical model 18 can be a mathematical equation, a statistical formula, a neural network or another model, which can capture the dynamics and the features of the multi-camera system 3 and of the environment 9. The prediction module 16 can use the past data and the current data as inputs into the mathematical model 18 to output a prediction value for the gain and exposure values for each camera 5, 6, 7, 8 for the next cycle.

[0056] The optimizer 17 in turn uses a cost function 22 and the sensor restrictions 23 to optimize the predicted value. The cost function is a measure of the quality and the system power based on the deviation from the target values and the control effort. The cost function 22 can be a linear function, a square function, a weighted function or any other function, which can quantify the power of the multi-camera system 3. The sensor restrictions 23 are the physical restrictions of the sensor or of the camera 5, 6, 7, 8, which measures the gain and exposure values. The sensor restrictions 23 can be the minimum and maximum values, the range, the resolution, the accuracy or any other parameter, which defines the capabilities and limits of the sensor. The prediction module 16 uses the optimizer 17 to find the optimum prediction value, which minimizes the cost function considering the sensor restrictions 23. The optimizer 17 can be a gradient descent, a genetic algorithm, a particle swarm or any other algorithm, which can solve the optimization problem.

[0057] In the step of the adaptation, the prediction module 16 uses the adaptive prediction to update the mathematical model 18 and the cost function 22 based on the feedback and learning. The prediction module 16 can obtain feedbacks from the PID controller, the sensor, the user interface or the multi-camera system 3 and use these feedbacks to correspondingly adapt the mathematical model 18 and the cost function 22. The prediction module 16 can also use learning methods like reinforcement learning, supervised learning, unsupervised learning and any other method, which can improve the accuracy and efficiency of the prediction and the optimization. The prediction module 16 can use the adaptation to be able to process uncertainties and the changes in the multi-camera system 3 and in the environment 9. The process module 19 in turn generates a proposal value for the gain and exposure values for each camera 5, 6, 7, 8 for the next cycle based on the optimized prediction value. The proposed value is then sent to the PID smart scheduler, the sensor, the user interface and the multi-camera system 3 according to configuration and communication. The proposed value is used to adapt the gain and exposure values for each camera 5, 6, 7, 8 for the next cycle, in particular for the omitted / skipped camera 5, 6, 7, 8.

[0058] Fig. 5 in turn shows a temporal flow diagram according to an embodiment of the method. In particular, four time steps ti , t2, ta and t4 are shown. In the first time step ti , the processing for the first camera 5, the second camera 6 as well as the third camera 7 is in turn switched to active, while the processing of the fourth camera 8 is skipped. In particular, the determination of the control values based on the historical control value 12 is then in turn effected for the fourth camera 8. In the second time step t2, it is in turn shown that the data of the first camera 5 is not processed, but the data of the second camera 6, the third camera 7 and the fourth camera 8. In the third time step ts, the processing of the second camera 6 is in turn skipped, while the other three cameras 5, 7, 8 are switched to active. In the fourth time step t4, the third camera 7 is in turn inactive and is skipped, while the other three cameras 5, 6, 8 are active.

[0059] Thus, the scheduler is in particular also provided as a component of the electronic computing device 4, which manages the allocation of resources and tasks between the different cameras 5, 6, 7, 8 and the image processing unit. The smart scheduler decides based on the priority, the availability and the requirement of the multi-camera system 3, which cameras 5, 6, 7, 8 are to be processed and which ones are to be skipped. The smart scheduler can use various methods to decide, which camera 5, 6, 7, 8 is to be skipped, for example a systematic schedule as shown in Fig. 5, or schedule based on priority. The smart scheduler also coordinates the communication and synchronization between the cameras 5, 6, 7, 8 and the image processing unit. The typical linear scheme of the smart scheduler for active and skipped cameras 5, 6, 7, 8 is in turn shown in Fig. 5.

Claims

Claims1. A method for operating a multi-camera system (3) with at least two cameras (5, 6, 7, 8) for a motor vehicle (1) by means of an electronic computing device (4) of the multi-camera system (3), comprising the steps:- determining at least one current first control value (10) for at least a first camera (5) of the multi-camera system (3) depending on a current environmental parameter by means of the electronic computing device (4);- determining at least one current second control value (11) for at least a second camera (6) of the multi-camera system (3) depending on a historical control value (12) for the second camera (6) by means of the electronic computing device (4); and- controlling the first camera (5) by means of the first control value (10) and controlling the second camera (6) by means of the second control value (11) by means of the electronic computing device (4).

2. The method according to claim 1, characterized in that a prediction value for the second control value (11) is predicted by means of a prediction module (16) of the electronic computing device (4) for determining the second control value (11).

3. The method according to claim 2, characterized in that a mathematical model (18) for at least the second camera (6) is provided for the prediction.

4. The method according to claim 3, characterized in that a mathematical equation for the second camera (6) or a statistical model for the second camera (6) or a neural network for the second camera (6) is provided as the mathematical model (18).

5. The method according to any one of the preceding claims 2 to 4, characterized in that the prediction value is optimized by means of a cost function (22).

6. The method according to any one of the preceding claims, characterized in that a gain value and / or an exposure value for a respective camera (5, 6, 7, 8) are determined as the respective control value (10, 11).

7. The method according to any one of the preceding claims, characterized in that the method is performed in case of a computing capacity shortage and / or a bandwidth shortage of the electronic computing device (4).

8. The method according to any one of the preceding claims, characterized in that it is decided by means of the electronic computing device (4), in which camera (5, 6, 7, 8) based on the current environmental parameter and in which camera (5, 6, 7, 8) based on the historical control value (12) the respective current control value (10, 11) is determined.

9. The method according to any one of the preceding claims, characterized in that the first control value (10) and the second control value (11) are additionally determined by means of a proportional-integral-derivative controller (13).

10. The method according to any one of claims 2 to 8 and 9, characterized in that for determining the second control value (11), an evaluation of the proportional- integral-derivative controller (13) is performed before an evaluation of the prediction module (16).

11. The method according to any one of claims 9 or 10, characterized in thatby means of the proportional-integral-derivative controller (13), a current trend in the environment (9) is determined and the current trend is taken into account in determining the control values (11 , 12).

12. The method according to any one of the preceding claims, characterized in that in a first time step (h), the current first control value (10) is determined depending on the current environmental parameter and the current second control value (11) is determined depending on the historical second control value (12), and in a second time step (t2) following the first time step (h), the current second control value (11) is determined depending on the current environmental parameter and the current first control value (10) is determined depending on a historical first control value.

13. A computer program product with program code means, which cause an electronic computing device (4), when the program code means are run by the electronic computing device (4), to perform a method according to any one of claims 1 to 12.

14. A computer-readable storage medium with at least one computer program product according to claim 13.

15. An electronic computing device (4) of a multi-camera system (3) with at least two cameras (5, 6, 7, 8) for a motor vehicle (1), wherein the electronic computing device (4) is formed for performing a method according to any one of claims 1 to 12.

Images