Automatic exposure setting of a motor vehicle camera

Abstract

For automatic exposure setting of a camera (3) of a motor vehicle (1), a first image is received from an imager of the camera (3), and the first exposure settings used to generate the first image are received, the first exposure settings including a first exposure time. An exposure time adjustment value is determined based on a predefined target image brightness and based on at least one of the first exposure settings or an image brightness of the first image using a trained generative neural network module (5). Second exposure settings are determined, with a second exposure time being determined by adjusting the first exposure time according to the exposure time adjustment value. The second exposure settings are transmitted to the imager to generate a second image.

Description

[0001] The present invention is directed to a computer-implemented method for automatically adjusting the exposure of a camera of a motor vehicle. The invention further relates to a method for at least partially automatically guiding a motor vehicle, wherein such a computer-implemented method is performed. The invention also relates to a data processing device for carrying out such a computer-implemented method, an electronic vehicle guidance system with such a data processing device, and corresponding computer program products.

[0002] Common camera auto-exposure systems use simple metering methods or PID controllers, which may not work well in complex or dynamic lighting conditions. In particular, flickering or other distracting effects may occur if, for example, the auto-exposure settings are adjusted too slowly or the control method is prone to overshoot. Manual tuning or calibration for different camera sensors or user preferences may also be required. Common metering methods use predefined areas or regions of interest in the image to measure the brightness of the scene and then adjust the exposure setting for the subsequent image accordingly.PID controllers use feedback loops to compare the current brightness of the output image to a target value and then adjust the exposure settings using proportional, integral, and derivative control.

[0003] Generative neural networks can generate artificial images that are virtually indistinguishable from real camera images. For example, generative neural networks can take a camera image and generate an artificial image with the same or nearly the same content in a different domain or with a different style. In this way, domain translations between different domains can be achieved, for example, from daytime to nighttime, between different weather conditions, between different people's hairstyles, and so on. An example of generative neural networks is generative adversarial networks (GANs), presented in the paper by I.J. Goodfellow et al.: "Generative Adversarial Networks" (arXiv:1406.2661v1). In the paper by Y. Choi et al.: "StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation" (arXiv:1711).09020v3) describes a multi-domain image-to-image translation using a GAN architecture. The publication P. Isola et al.: "Image-to-Image Translation with Conditional Adversarial Networks" (arXiv:1611.07004v3) describes the use of conditional adversarial networks for image-to-image translation.

[0004] Vision transformer networks, introduced in the paper A. Dosovitskiy et al.: “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” (arXiv:2010.11929v2), can also be considered as generative neural networks, as described, for example, in the papers Y. Jiang et al.: “TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up” (arXiv:2102.07074v4) and B. Zhang “StyleSwin: Transformer-based GAN for High-Resolution Image Generation” (arXiv:2112.10762v2).

[0005] It is an object of the present invention to improve the automatic exposure adjustment of a camera of a motor vehicle to make the adjustment faster and more accurate.

[0006] This object is achieved by the subject matter of the independent claim. Preferred embodiments and further implementations are subject matter of the dependent claims.

[0007] The invention is based on the idea of ​​using a trained generative neural network to determine an exposure time adjustment value depending on the current exposure settings or an image brightness of the current image and depending on a predefined target image brightness.

[0008] According to one aspect of the invention, a computer-implemented method for automatically adjusting the exposure of a camera of a motor vehicle is provided. A first image is obtained from an imager of the camera, and first exposure settings used by the camera, in particular by the imager, to generate the first image are obtained, wherein the first exposure settings include a first exposure time. An exposure time adjustment value is determined using a trained generative neural network module. The exposure time adjustment value is determined as a function of a predefined target image brightness. The exposure time adjustment value is also determined as a function of the first exposure settings or an image brightness of the first image.Second exposure settings are determined, with a second exposure time of the second exposure settings being determined by adjusting the first exposure time according to the exposure time adjustment value. The second exposure settings are provided to the imager to generate a second image.

[0009] Unless otherwise stated, all steps of the computer-implemented method can be performed by a data processing device that includes at least one computing unit, in particular a data processing device of the motor vehicle. In particular, the at least one computing unit is configured or adapted to execute the steps of the computer-implemented method. For this purpose, the at least one computing unit can, for example, store a computer program that includes instructions that, when executed by the at least one computing unit, cause the at least one computing unit to execute the computer-implemented method.

[0010] All computing units of the at least one computing unit can be contained in the motor vehicle. However, it is also possible for all computing units of the at least one computing unit to be part of an external computing system outside the motor vehicle, for example a backend server or a cloud computing system. It is also possible for the at least one computing unit to have at least one vehicle computing unit of the motor vehicle and at least one external computing unit contained in the external computing system. The at least one vehicle computing unit can, for example, be contained in one or more electronic control units (ECUs) and / or one or more zone control units (ZCUs) and / or one or more domain control units (DCUs) of the vehicle and / or the camera.

[0011] For each embodiment of the computer-implemented method according to the invention, a non-purely computer-implemented implementation of a method for automatically adjusting the exposure of a camera of a motor vehicle is directly obtained by including corresponding steps for generating the first image by the imager using the first exposure settings and / or for generating the second image by the imager using the second exposure settings.

[0012] An imager can be understood as an electronic device that includes an image sensor with an array of optical detectors. The image sensor can be embodied as a CCD image sensor, a CMOS image sensor, in particular as an active pixel sensor or a passive pixel sensor, or as another image sensor that is sensitive to visible light. In addition to the image sensor, the imager can also include other electronic components, for example for reading out the image sensor and / or for preprocessing, etc. The imager can be implemented on a single semiconductor chip and is therefore also referred to as an imager chip. Alternatively, the image sensor and the other electronic components can be implemented on different semiconductor chips, which can be mounted, for example, on a common circuit board.

[0013] Depending on the actual design of the imager, particularly the image sensor, the exposure settings may include other parameters besides the exposure time. For example, the exposure settings may include an analog gain value and / or a digital gain value for the imager, especially if the image sensor is a CMOS image sensor.

[0014] The exposure time adjustment value is a value that defines how the second exposure time is determined, for example, based on the first exposure time. The exposure time adjustment value can, for example, be a time difference between the first exposure time and the second exposure time, which is added to or subtracted from the first exposure time to determine the second exposure time. The exposure time adjustment value can also be another value that allows the second exposure time to be calculated from the first exposure time. However, the exposure time adjustment value can also be given directly by the second exposure time.

[0015] In the present disclosure, the term "neural network" refers to an artificial neural network. The generative neural network module can be constructed according to various known architectures, for example, as a GAN, in particular as a trained generator module of a GAN, or as a transformer-based architecture.

[0016] Specifically, the generative neural network module is trained to reconstruct an input camera image such that the reconstructed artificial image matches the input camera image except for the image brightness, which in the artificial image is given by or approximately given by the target image brightness. This can be considered a domain translation problem, where the target domain is a domain with the target image brightness.If the generative neural network module, for example the GAN, is provided with a large number of training images that have the predefined target image brightness and the generative neural network module is trained according to known training approaches based on these training images, the generative neural network module learns the distribution of the training images, for example in a latent space, and is therefore able to reconstruct camera images with an image brightness that is adapted to the target image brightness after training.

[0017] Image brightness can be defined in different ways. For example, image brightness can be determined by averaging the color values ​​according to different color channels of the respective image or by using a corresponding weighted average. Image brightness can also be determined directly based on the raw image obtained from the image sensor.

[0018] By determining the exposure time adjustment value as described according to the computer-implemented method of the invention, the second exposure settings can be obtained, particularly in a single iteration, such that the resulting image brightness of the second image approximately corresponds to the target image brightness. Consequently, the desired image brightness can be achieved more quickly and / or more accurately than when using conventional controllers such as PID controllers, without increasing the risk of overshoot or flicker.

[0019] According to several embodiments, an artificial image with the target image brightness is generated by applying the trained generative neural network module to the first image. The exposure time adjustment value is determined depending on the first exposure settings, the first image, and the artificial image.

[0020] Consequently, a particularly high accuracy of the determined second exposure settings is achieved since the exposure time adjustment value is determined based on an artificial image created on the basis of the first image itself.

[0021] According to several embodiments, the exposure time adjustment value is determined by applying a trained further neural network module to input data that depends on the first exposure settings, the first image, and the artificial image.

[0022] The additional neural network module can be designed according to known architectures and trained according to known training methods to predict the exposure time adjustment value based on the input data. For example, the additional neural network module can be based on a convolutional neural network architecture for regressing the exposure time adjustment value. During the training phase, corresponding training data, which includes a plurality of training images with different image brightnesses and the corresponding exposure settings, as well as corresponding artificial training images with the target image brightness, are used to regress the corresponding exposure time adjustment values, which transform the training images into the corresponding artificial training images.Furthermore, for each pair of a training image and a corresponding artificial training image, a corresponding exposure time adjustment reference value is provided. A regression loss function can then be evaluated depending on the regressed exposure time adjustment values ​​and the corresponding ground truth values.

[0023] According to several embodiments, the exposure time adjustment value is determined depending on the image brightness of the first image and the artificial image.

[0024] In other words, instead of using the first exposure settings and the first image to estimate the exposure time adjustment value, the image brightness of the first image is extracted from the first image and used together with the artificial image to estimate the exposure time adjustment value.

[0025] Even in such embodiments, a particularly high accuracy of the determined second exposure settings is achieved, since the exposure time adjustment value is determined based on an artificial image which is generated, for example, on the basis of the first image itself.

[0026] According to several embodiments, the exposure time adjustment value is determined by applying the trained further neural network module to input data that depends on the image brightness of the first image and the artificial image.

[0027] The above statements regarding the further neural network module can be transferred analogously to such embodiments, wherein the training data contain the respective image brightnesses of the training images instead of the exposure settings.

[0028] According to several embodiments, reference data is obtained that includes a plurality of reference images and corresponding exposure settings used to generate the respective reference image, wherein the exposure settings include a corresponding exposure time. For each reference image of the plurality of reference images, (i) an artificial reference image with the target image brightness is generated by applying the trained generative neural network module to the respective reference image, (ii) an exposure adjustment reference value is determined depending on the exposure settings of the respective reference image, the respective reference image, and the artificial reference image, and (iii) the exposure settings and the exposure adjustment reference value are stored in a lookup table. The exposure time adjustment value is determined depending on the first exposure settings according to the lookup table.

[0029] In such embodiments, the generative neural network module is not applied directly to the first image; rather, the lookup table is pre-calculated by applying the generative neural network module to the reference images. The resulting lookup table maps the exposure settings, in particular the exposure times, to the corresponding exposure adjustment values ​​required to achieve the target image brightness. The lookup table can also be represented, for example, as a corresponding curve mapping the exposure settings to the exposure adjustment values ​​and can therefore also be understood as a camera response model. The lookup table is then used during camera operation to determine the second exposure settings based on the first exposure settings of the first image.

[0030] This can reduce the computing time for determining the second exposure settings based on the first exposure settings during camera operation.

[0031] According to several embodiments, the exposure adjustment reference value is determined by applying the trained further neural network module to input data that depends on the exposure settings of the respective reference image, the respective reference image, and the artificial reference image.

[0032] The above statements regarding the further neural network module can be transferred analogously to such embodiments.

[0033] According to several embodiments, reference data is obtained according to a plurality of reference images and the respective exposure settings used to generate the respective reference image, wherein the exposure settings include a respective exposure time. For each reference image of the plurality of reference images, (i) an artificial reference image with the target image brightness is generated by applying the trained generative neural network module to the respective reference image, (ii) an exposure adjustment reference value is determined depending on an image brightness of the respective reference image and the artificial image, and (iii) the image brightness of the respective reference image and the exposure adjustment reference value are stored in a lookup table. The exposure time adjustment value is determined depending on the image brightness of the first image according to the lookup table.

[0034] The above statements can be applied analogously to such embodiments.

[0035] According to several embodiments, the exposure adjustment reference value is determined by applying a trained further neural network module to input data that is dependent on an image brightness of the respective reference image and the artificial image.

[0036] The above statements can be adopted analogously to such embodiments.

[0037] According to several embodiments, the first exposure settings include at least one first gain value for the imager, wherein the image sensor of the imager is, in particular, a CMOS image sensor. The second exposure settings include at least one second gain value for the image sensor. The at least one second gain value is determined depending on the second exposure time and the target image brightness.

[0038] For example, a further lookup table may be provided that maps the second exposure time to the at least one second gain value to achieve the desired image brightness. Since the required gain settings for a given target image brightness and a given exposure time may depend on the detailed design of the imager, the accuracy of the second exposure settings may be improved.

[0039] According to several embodiments, the at least one first gain value includes a first analog gain value and the at least one second gain value includes a second analog gain value.

[0040] The image processing pipeline includes an analog-to-digital conversion (ADC) of the image sensor's pixel values. The analog gain values ​​correspond to the respective gain before the ADC is performed.

[0041] According to several embodiments, the at least one first gain value includes a first digital gain value and the at least one second gain value includes a second digital gain value.

[0042] The digital gain values ​​correspond to a respective gain after the ADC has been performed.

[0043] According to several embodiments, the first image corresponds to a first camera frame of the camera and the second image corresponds to a second camera frame of the camera, wherein the second camera frame directly follows the first camera frame.

[0044] In other words, the first camera frame and the second camera frame are immediately consecutive. Consequently, the exposure adjustment to the second exposure settings occurs within a very short period of time.

[0045] For applications or application situations that may occur in a method according to the invention and are not explicitly described here, it can be provided that an error message and / or a request for user feedback is output according to the method and / or a standard setting and / or a predetermined initial state is set.

[0046] According to a further aspect of the invention, a method for at least partially automatically driving a motor vehicle is provided. In this case, a computer-implemented method or a method for automatic exposure adjustment according to the invention is carried out, wherein the first image and the second image are generated by an imager of a camera of the motor vehicle. At least one control signal for at least partially automatically driving the motor vehicle is generated by means of at least one computing unit, for example at least one computing unit of the motor vehicle, depending on the first image and / or the second image, and / or driver assistance information for supporting a driver of the motor vehicle in driving the motor vehicle is generated by means of the at least one computing unit depending on the first image and / or the second image.

[0047] The motor vehicle includes, for example, an electronic vehicle guidance system for carrying out the computer-implemented method or the method for automatic exposure adjustment according to the invention and / or for generating the at least one control signal and / or the driver assistance information.

[0048] An electronic vehicle guidance system can be understood as an electronic system designed to guide a vehicle fully automatically or autonomously, in particular without requiring driver intervention. The vehicle automatically performs all required functions, such as steering, braking, and / or acceleration maneuvers, monitoring and detecting road traffic, and responding accordingly. In particular, the electronic vehicle guidance system can implement a fully automatic or fully autonomous driving mode of the motor vehicle according to Level 5 of the SAE J3016 classification. An electronic vehicle guidance system can also be understood as an advanced driver assistance system (ADAS), which supports the driver in partially automated or semi-autonomous driving.In particular, the electronic vehicle guidance system can implement a partially automated or semi-autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and below, "SAE J3016" refers to the corresponding standard in the April 2021 version.

[0049] The at least partially automated vehicle guidance may therefore include driving the vehicle according to a fully automated or fully autonomous driving mode of Level 5 according to SAE J3016. The at least partially automated vehicle guidance may also include driving the vehicle according to a partially automated or semi-autonomous driving mode according to Levels 1 to 4 according to SAE J3016.

[0050] The at least one control signal can, for example, be provided to one or more actuators of the motor vehicle, including, for example, one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the motor vehicle. The one or more actuators can influence a longitudinal and / or lateral control of the motor vehicle in order to guide the motor vehicle at least partially automatically.

[0051] The driver assistance information can be output via an output device of the motor vehicle, for example a display and / or an audio output system and / or a haptic output system.

[0052] According to a further aspect of the invention, a data processing device is provided that includes at least one computing unit. The at least one computing unit is adapted to execute a computer-implemented method for automatic exposure adjustment according to the invention.

[0053] In the present disclosure, a computing unit can be understood, for example, as a data processing device with processing circuits. A computing unit can therefore perform computing operations to process data. The computing operations can also include indexed accesses to a data structure, for example, a look-up table (LUT).

[0054] A computing unit can in particular comprise one or more computers, one or more microcontrollers and / or one or more integrated circuits, for example one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more single-chip systems (SoCs). The computing unit can also contain one or more processors, for example one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The computing unit can also comprise a physical or virtual cluster of computers or other of the aforementioned units.

[0055] A computing unit may also include one or more hardware and / or software interfaces and / or one or more memory units. A memory unit may be embodied as a volatile data memory, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM), or as a non-volatile data memory, for example, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or flash EEPROM, a ferroelectric random access memory (FRAM),a magnetoresistive random access memory (MRAM) or a phase-change random access memory (PCRAM).

[0056] According to a further aspect of the invention, an electronic vehicle guidance system comprising a data processing device according to the invention is provided. The at least one computing unit is configured to generate at least one control signal for at least partially automatically guiding a motor vehicle based on the first image and / or the second image and / or to generate driver assistance information to assist a driver of the motor vehicle in guiding the motor vehicle based on the first image and / or the second image.

[0057] According to several embodiments, the electronic vehicle guidance system includes the camera and the imager is configured to generate the first image and the second image.

[0058] According to several embodiments of the electronic vehicle guidance system, the imager includes a CMOS image sensor.

[0059] Further embodiments of the electronic vehicle guidance system according to the invention result directly from the various embodiments of the computer-implemented method or the method according to the invention, and vice versa. In particular, individual features and corresponding explanations as well as advantages of the various embodiments of the computer-implemented method or method according to the invention can be transferred analogously to corresponding embodiments of the electronic vehicle guidance system according to the invention. In particular, the electronic vehicle guidance system according to the invention is designed or programmed to execute a computer-implemented method or method according to the invention.

[0060] In particular, the electronic vehicle guidance system according to the invention carries out a computer-implemented method or method according to the invention.

[0061] According to a further aspect of the invention, a computer program is provided that includes instructions. When the instructions are executed by a data processing device, in particular by a data processing device according to the invention, the instructions cause the data processing device to execute a computer-implemented method for automatic exposure adjustment according to the invention.

[0062] The instructions can be provided, for example, as program code. The program code can be provided, for example, as binary code or assembly code and / or as source code of a programming language, for example, C, and / or as a program script, for example, Python.

[0063] According to a further aspect of the invention, a further computer program is provided that contains further instructions. When executed by an electronic vehicle guidance system according to the invention, the further instructions cause the electronic vehicle guidance system to execute a method according to the invention for at least partially automatically guiding a motor vehicle.

[0064] The further instructions can be provided, for example, as program code. The program code can be provided, for example, as binary code or assembly code and / or as source code of a programming language, for example, C, and / or as a program script, for example, Python.

[0065] According to a further aspect of the invention, a computer-readable storage medium is provided which stores a computer program and / or another computer program according to the invention.

[0066] The computer program, the further computer program and the computer-readable storage medium are each computer program products that contain the instructions and / or the further instructions.

[0067] Further features of the invention emerge from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of the figures and / or shown in the figures can be encompassed by the invention not only in the respectively specified combination, but also in other combinations. In particular, the invention can also encompass embodiments and combinations of features that do not have all the features of an originally formulated claim. Furthermore, the invention can encompass embodiments and combinations of features that go beyond the combinations of features set out in the backreferences to the claims or deviate from them.

[0068] The invention is explained in more detail below using specific embodiments and associated schematic drawings. In the figures, identical or functionally equivalent elements may be provided with the same reference numerals. The description of identical or functionally equivalent elements may not necessarily be repeated for different figures.

[0069] The figures show Fig. 1 is a schematic representation of a motor vehicle with an exemplary embodiment of an electronic vehicle guidance system according to the invention; Fig. 2 is a schematic flow diagram of an exemplary embodiment of a computer-implemented method for automatic exposure adjustment according to the invention; Fig. 3 is a schematic flow diagram of another exemplary embodiment of a computer-implemented method for automatic exposure adjustment according to the invention; and Fig. 4 is a schematic flow diagram of another exemplary embodiment of a computer-implemented method for automatic exposure adjustment according to the invention.

[0070] Fig. 1 schematically shows a motor vehicle 1 with an exemplary embodiment of an electronic vehicle guidance system 2 according to the invention.

[0071] The motor vehicle 1, for example the electronic vehicle guidance system 2, includes a camera 2 mounted on the motor vehicle 1. The electronic vehicle guidance system 2 includes a data processing device with at least one computing unit 4. The at least one computing unit 4 can be configured to execute a computer-implemented method for automatic exposure adjustment according to the invention. Alternatively, one or more steps of such a computer-implemented method can be executed by means of at least one further computing unit of the data processing device, wherein the at least one further computing unit is arranged outside the motor vehicle 1.

[0072] The camera 2, in particular an imager of the camera 2, generates a sequence of images comprising a first image followed by a second image. To generate the first image, the camera 2 uses first exposure settings including a first exposure time. According to the computer-implemented method, second exposure settings are provided to the imager of the camera 2, and the camera 2 then generates the second image using the second exposure settings.

[0073] The at least one computing unit 4 generates at least one control signal for at least partially automatically guiding the motor vehicle, depending on the first image and / or the second image. Alternatively or additionally, at least one computing unit 4 generates driver assistance information to support a driver of the motor vehicle 1 in guiding the motor vehicle 1, depending on the first image and / or the second image.

[0074] Fig. 2 shows a schematic flow diagram of an exemplary embodiment of a computer-implemented method for automatic exposure adjustment according to the invention.

[0075] In step 200, the first image is obtained from the imager of camera 2, and the first exposure settings, including the first exposure time used to generate the first image, are obtained from camera 2. In step 220, an exposure time adjustment value is determined depending on a predefined target image brightness and depending on at least one of the first exposure settings or an image brightness of the first image using a trained generative neural network module 5. In step 240, the second exposure settings are determined, wherein a second exposure time of the second exposure settings is determined by adjusting the first exposure time according to the exposure time adjustment value. In step 260, the second exposure settings are provided to the imager for generating the second image.

[0076] Fig. 3 shows a schematic flow diagram of an exemplary embodiment of a computer-implemented method for automatic exposure adjustment according to the invention, which is based on the Fig. 2 described implementation.

[0077] In step 220, an artificial image with the target image brightness is generated by applying the trained generative neural network module 5 to the first image. The exposure time adjustment value is determined depending on the first exposure settings, the first image, and the artificial image, or depending on the image brightness of the first image and the artificial image.

[0078] In particular, the exposure time adjustment value is determined by applying a trained additional neural network module 6 to the input data. The input data depends on the first exposure settings, the first image, and the artificial image, or the input data depends on the image brightness of the first image and the artificial image.

[0079] Fig. 4 shows a schematic flow diagram of another exemplary embodiment of a computer-implemented method for automatic exposure adjustment according to the invention, which is based on the Fig. 2 described implementation.

[0080] Therein, reference data 7 are obtained, which include a plurality of reference images and the respective exposure settings for generating the respective reference image, wherein the exposure settings include a respective exposure time.

[0081] The following steps are performed for each of the plurality of reference images. An artificial reference image with the brightness of the target image is generated by applying the trained generative neural network module to the respective reference image. An exposure adjustment reference value is determined, for example, depending on the exposure settings of the respective reference image, the respective reference image, and the artificial image. For example, the exposure adjustment reference value is determined by applying the trained further neural network module 6 to input data that depends on the exposure settings of the respective reference image, the respective reference image, and the artificial image. The exposure settings and the exposure adjustment reference value are stored in a lookup table 8.The further neural network module 6 can also be referred to as a curve generator module, which generates the data to be stored in the lookup table 8 and thus generates an effective response curve of the camera.

[0082] In step 220, the exposure time adjustment value is determined depending on the first exposure settings according to the lookup table 8.

[0083] According to several embodiments, the invention implements an advanced exposure control system that can automatically adjust the exposure time and, in some embodiments, the sensor gain of the imager based on the brightness of the scene.

[0084] In several embodiments, the system includes three components, namely a generative neural network module, a curve generator, in particular the further neural network module, and an exposure controller that applies the exposure settings to the imager.

[0085] In several embodiments, a fully convolutional neural network may take an image captured by the camera as input and output a set of features representing the characteristics of the scene.

[0086] In several embodiments, the generative neural network module may be trained using image statistics and embedded line information as input features, as well as light box data, lux meter data, test tables, and / or light source information to create different lighting conditions for the training data.

[0087] In several embodiments, the generative neural network module can also adapt to different scenarios while the camera is in use by performing online training. The embedded line information can also include information about the current exposure time and sensor gains, which can be used to monitor oscillations during training. If these values ​​are exceeded or undershot, it can be an indication that the curve parameters need to be adjusted accordingly. The curve generator module can design a curve that can be used to calculate the optimal exposure parameters for the next image. The curve can be, for example, linear, quadratic, or exponential, depending on the relationship between the input and output variables.The curve generator can also allow the user to choose the type of curve they prefer or let the algorithm automatically select the best curve based on predefined criteria.

[0088] In several embodiments, the exposure controller can use the curve generated by the curve generator to determine the exposure time and sensor gain for the next image. The exposure controller can also apply defined constraints, ensuring that the exposure settings remain within predefined ranges and do not cause over- or underexposure. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] IJ Goodfellow et al.: “Generative Adversarial Networks” (arXiv:1406.2661v1

[0003] Y. Choi et al: “StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation” (arXiv:1711.09020v3

[0003] P. Isola et al: “Image-to-Image Translation with Conditional Adversarial Networks” (arXiv:1611.07004v3

[0003] A. Dosovitskiy et al: “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” (arXiv:2010.11929v2

[0004] Y. Jiang et al.: “TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up” (arXiv:2102.07074v4

[0004] B. Zhang “StyleSwin: Transformer-based GAN for high-resolution image generation.” (arXiv:2112.10762v2

[0004]

Claims

[1] Computer-implemented method for automatically adjusting the exposure of a camera (3) of a motor vehicle (1), wherein - a first image is received from an imager of the camera (2) and first exposure settings used to generate the first image are received, the first exposure settings comprising a first exposure time; - an exposure time adjustment value is determined depending on the first exposure settings or an image brightness of the first image and depending on a predefined target image brightness using a trained generative neural network module (5); - second exposure settings are determined, wherein a second exposure time of the second exposure settings is determined by adjusting the first exposure time according to the exposure time adjustment value; and - the second exposure settings are made available to the imager to generate a second image. [2] The computer-implemented method of claim 1, wherein an artificial image having the target image brightness is generated by applying the trained generative neural network module (5) to the first image, and - the exposure time adjustment value is determined depending on the first exposure settings, the first image and the artificial image; or - the exposure time adjustment value is determined depending on the artificial image and the image brightness of the first image. [3] A computer-implemented method according to claim 2, wherein - the exposure time adjustment value is determined by applying a trained further neural network module (6) to input data which depend on the first exposure settings, the first image and the artificial image; or - the exposure time adjustment value is determined by applying a trained further neural network module (6) to input data which depends on the artificial image and the image brightness of the first image. [4] A computer-implemented method according to claim 1, wherein - reference data (7) are received which comprise a plurality of reference images and respective exposure settings used to generate the respective reference image, the exposure settings comprising a respective exposure time; - for each of the multitude of reference images, i) an artificial reference image with the target image brightness is generated by applying the trained generative neural network module (5) to the respective reference image; (ii) an exposure adjustment reference value is determined depending on the exposure settings of the respective reference image, the respective reference image and the artificial reference image; and iii) the exposure settings and the exposure adjustment reference value are stored in a look-up table (8); and - the exposure time adjustment value is determined depending on the first exposure settings according to the look-up table (8). [5] A computer-implemented method according to claim 4, wherein the exposure adjustment reference value is determined by applying a trained further neural network module (6) to input data which depend on the exposure settings of the respective reference image, the respective reference image and the artificial reference image. [6] A computer-implemented method according to claim 1, wherein - reference data (7) are received which comprise a plurality of reference images and respective exposure settings used to generate the respective reference image, the exposure settings comprising a respective exposure time; - for each of the multitude of reference images, i) an artificial reference image with the target image brightness is generated by applying the trained generative neural network module (5) to the respective reference image; ii) an exposure adjustment reference value is determined depending on the artificial reference image and an image brightness of the respective reference image; and iii) the image brightness of the respective reference image and the exposure adjustment reference value are stored in a look-up table (8); and - the exposure time adjustment value is determined as a function of the image brightness of the first image according to the look-up table (8). [7] A computer-implemented method according to claim 5, wherein the exposure adjustment reference value is determined by applying a trained further neural network module (6) to input data which depend on the artificial reference image and the image brightness of the respective reference image. [8] Computer-implemented method according to one of the preceding claims, wherein - the first exposure settings include at least a first gain value for the imager; - the second exposure settings include at least a second gain value for the imager; and - the at least one second gain value is determined as a function of the second exposure time and the target image brightness. [9] A computer-implemented method according to claim 8, wherein - the at least one first gain value comprises a first analog gain value and the at least one second gain value comprises a second analog gain value; and / or - the at least one first gain value comprises a first digital gain value and the at least one second gain value comprises a second digital gain value. [10] A computer-implemented method according to any one of the preceding claims, wherein the first image corresponds to a first camera image of the camera and the second image corresponds to a second camera image of the camera, the second camera image directly following the first camera image. [11] Method for at least partially automatically driving a motor vehicle (1), wherein a computer-implemented method according to one of the preceding claims is carried out, wherein the first image and the second image are generated by an imager of a camera (3) of the motor vehicle (1) and - at least one control signal for at least partially automatically guiding the motor vehicle (1) is generated by at least one computing unit (4) as a function of the first image and / or the second image; and / or - driver assistance information for supporting a driver of the motor vehicle (1) in driving the motor vehicle (1) is generated by the at least one computing unit (4) as a function of the first image and / or the second image. [12] Data processing device with at least one computing unit (4) adapted to carry out a computer-implemented method according to one of claims 1 to 10. [13] Electronic vehicle guidance system (2) with a data processing device according to claim 12, wherein the at least one computing unit (4) is configured to - to generate at least one control signal for at least partially automatically guiding a motor vehicle (1) in dependence on the first image and / or the second image; and / or - to generate driver assistance information to assist a driver of the motor vehicle (1) when driving the motor vehicle (1) as a function of the first image and / or the second image. [14] Electronic vehicle guidance system (2) according to claim 13, comprising the camera (3), wherein the imager is arranged to generate the first image and the second image. [15] Electronic vehicle guidance system (2) according to claim 14, wherein the imager comprises a CMOS image sensor. [16] A computer program product containing instructions which, when executed by a data processing device, cause the data processing device to carry out a computer-implemented method according to any one of claims 1 to 10. [17] A computer program product comprising instructions which, when executed by an electronic vehicle guidance system (2) according to any one of claims 13 to 15, cause the electronic vehicle guidance system (2) to carry out a method according to claim 11.

Images