Adapted image capture

Abstract

A method for adapted image capture comprises capturing a first image of a target area by illuminating the target area with an illuminator and performing image recording of the target area with an image sensor unit, the illuminator and the image sensor unit being controlled according to a first image capture setting. The method further comprises analyzing the first image to determine whether it meets a predefined criterion. A further step of the method is generating control information for a second image capture setting if the predefined criterion is not met on the basis of input information relating to the first image, the second image capture setting comprising a modification compared to the first image capture setting in order to provide an improvement in terms of meeting the predefined criterion. The method further comprises capturing a second image of the target area by illuminating the target area with the illuminator and performing image recording of the target area with the image sensor unit, the illuminator and the image sensor unit being controlled according to the control information generated on the basis of the input information relating to the first image and thus according to the second image capture setting. Further described is a system for adapted image capture.

Description

[0001] ADAPTED IMAGE CAPTURE

[0002] DESCRIPTION

[0003] The present invention relates to a method for adapted image capture. The invention further relates to a system for adapted image capture.

[0004] This patent application claims priority to German patent application 10 2024 103 262.6, the disclosure of which is hereby incorporated by reference.

[0005] The capture and generation of optical images of an illuminated target area is employed in a variety of different application areas. These include image recording and camera systems, systems for fingerprint recognition, systems for object tracking, wherein these systems may e.g. be integrated in handheld devices such as mobile phones, the field of microscopy and numerous other applications. Depending on the conditions present during image acquisition and the components used for this purpose, the captured images may deviate from demands such as high image sharpness or specified color balance. In this respect, the following issues may be involved.

[0006] Fixed illumination may limit the spatial-frequencies space. Fixed illumination angles may result in a fixed number of spatial frequencies, and therefore a low resolution. In order to get around this problem, methods such as Fourier ptychography may be applied. In Fourier ptychography, a virtual larger field of illumination (FOI) is created from a series of images acquired under multiple different illumination angles. The images are combined using an iterative phase retrieval algorithm such that a high-resolution image, and thus a high space-bandwidth product, may be provided. This is at the expense of complex and time-consuming image capture and image reconstruction. Furthermore, a delicate calibration is required to precisely relate the k-space (spatial frequencies) with the illumination angles. The obj ect of the present invention is to speci fy a solution for an improved image capture .

[0007] This obj ect is achieved by the features of the independent patent claims . Further advantageous embodiments of the invention are speci fied in the dependent claims .

[0008] According to one aspect of the invention, a method for adapted image capture is proposed . The method comprises capturing a first image of a target area by illuminating the target area with an illuminator and performing image recording of the target area with an image sensor unit . In this step, the illuminator and the image sensor unit are controlled according to a first image capture setting . The method further comprises analyzing the first image to determine whether it meets a predefined criterion . A further step of the method involves generating control information for a second image capture setting i f the predefined criterion is not met on the basis of input information relating to the first image . The second image capture setting comprises a modi fication compared to the first image capture setting in order to provide an improvement in terms of meeting the predefined criterion . The method further comprises capturing a second image of the target area by illuminating the target area with the illuminator and performing image recording of the target area with the image sensor unit . In this step, the illuminator and the image sensor unit are controlled according to the control information generated on the basis of the input information relating to the first image and thus according to the second image capture setting .

[0009] The proposed method of fers the possibility of fast capture and generation of an improved optical image . Instead of capturing a large number of images and synthesi zing them in a complex process , as is the case with Fourier ptychography, the method utili zes an upstream optimi zation for image enhancement . In this respect , for the case that the f irst image that is acquired using the illuminator and image sensor unit , hereby being controlled according to the first image capture setting, does not ful fil the predetermined criterion, an adaption of the image capture setting is performed . To this end, provision of control information for the second image capture setting is performed on the basis of input information relating to the first image . The second image capture setting is modi fied from the first image capture setting in order to allow an improvement or optimi zation with regard to meeting the predefined criterion . The control information and thus the second image capture setting are accordingly applied to capture the second image using the illuminator and the image sensor unit , which are thereby controlled according to the second image capture setting . The second image acquired in this way may therefore comply with the predefined criterion, or may at least come closer to meeting the predefined criterion than is the case with the first image .

[0010] Further advantages are that performing the method does not need complex calibration, and no expensive equipment is required to carry out the method . The method may also be performed with small computational time . Moreover, external influence on the image capture such as influence of external light sources may be reliably reduced or suppressed .

[0011] In the following, further possible details and embodiments are described which may be considered for the method .

[0012] An optical image captured of the target area using the illuminator and the image sensor unit such as the first and second image may be provided in the form of a digital image or, respectively, in the form of image data or image information of the respective image . The image data may be provided by the image sensor unit , and may have already undergone data processing or pre-processing performed by the image sensor unit . Moreover, the step of analyzing the first image may be performed using the image data of the first image . Within the scope of the analyzing or before the actual analyzing, data processing may ( also ) take place , e . g . to achieve f iltering and / or resi zing or rescaling of the respective image or image data .

[0013] In a similar way, the input information relating to the first image on the basis of which the control information is generated may comprise image data or processed image data of the first image , wherein in the latter case ( again) e . g . filtering and / or resi zing or rescaling of the respective image or image data may be provided . The input information may, i f applicable , also comprise information in relation to the predefined criterion or in relation to not meeting the predefined criterion .

[0014] The method steps such as controlling the illuminator and the image sensor unit for image capture , image analyzing and generating of control information may be performed in a computer-implemented manner . In this regard, the controll ing of the illuminator and image sensor unit may be carried out using a controller, the analyzing may be carried out using a processing module , and the generating of control information may be carried out using a generating module . These and other components that may be used in the method may be realized in the form of hardware of a computer or computer system and / or software running on such a computer or computer system. In order to allow for transmission and exchange of information and data between the components and modules , appropriate communication devices and interfaces may further be present . In this respect , reference is also made to the possibi lity to implement at least one of the components in the form of a cloud-based application, which may be hosted on another computer or server for this purpose . The exchange of information and data for the use of such a component may take place i . a . via the Internet . A cloud-based configuration may e . g . be considered for the aforementioned generating module . Moreover, a configuration in the form of or comprising an arti ficial neural network may be provided for the generating module . In this context , the following may be considered . In a further embodiment , the generating of the control information is carried out using an arti ficial neural network to which the input information is fed . The arti ficial neural network is trained to generate the control information on the basis of the input information . In this way, the method may be performed with a high reliability .

[0015] The arti ficial neural network may comprise a plural ity of interconnected neurons or nodes . The arti ficial neural network may further comprise an input layer, at least one hidden layer and an output layer . Each of the layers may comprise a plurality of neurons . The input information may be fed to the input layer . Upon receipt of the input information, the artificial neural network may provide or propagate respective output information in the form of the control information that may be outputted via the output layer . The training of the arti ficial neural network may be performed by feeding the input layer with ground truths representing a plurality of reference images and performing a backpropagation algorithm for tuning and setting parameters of the arti ficial neural network such as its weights .

[0016] For the illuminator, various di f ferent configurations may be considered . In order to illuminate the target area, the illuminator may be configured to generate and emit electromagnetic radiation or light radiation . For this purpose , the illuminator may comprise a radiation-generating device . This may be reali zed in such a way that the illuminator comprises a plurality of separately controllable light-emitting areas or light-emitting pixels via which a respective light radiation may be emitted . The illuminator may thus comprise a structured and addressable light source . In this respect , the illuminator may comprise an array of emitters such as lightemitting diodes ( LED) , organic light-emitting dioses (OLED) or laser diodes . The illuminator may also comprise or be integrated in a display such as a LED, OLED or liquid crystal display ( LCD) . Moreover, the illuminator may not only com- prise a radiation-generating device but also , downstream thereof , a device for radiation shaping and / or radiation directing . Examples for this may include optics and a micromirror device or digital micromirror device ( DMD) .

[0017] The image sensor unit via which an optical image of the target area may be recorded, may be configured to detect and record electromagnetic radiation or light radiation for this purpose . The image sensor unit may operate or may be sensitive to radiation at least in a wavelength region o f light radiation emitted by the illuminator . Similar to the illuminator, the image sensor unit may comprise a plurality of light-sensing areas or light-sensing pixels via which light radiation may be detected . For this purpose , the image sensor unit may be or may comprise a pixelated image sensor, e . g . an active-pixel image sensor such as a complementary metal- oxide-semiconductor ( CMOS ) image sensor, or a charge-coupled device ( CCD) image sensor . Moreover, the image sensor unit may comprise , upstream of the image sensor, a component for radiation shaping such as receiving optics .

[0018] The modi fication of the image capture setting may comprise a modi fication of an operation setting of the illuminator and / or of the image sensor unit . This may include at least modi fying an operating mode of the illuminator, such that the modi fication relates only to the operating mode of the illuminator or to the operating modes of both the illuminator and the image sensor unit . In this respect , the following method variants , which may also be used in combination, may be considered .

[0019] In a further embodiment , the modi fication of the image capture setting comprises changing an illumination pattern provided by the illuminator . In this respect , the illuminator is configured to provide di f ferent illumination patterns with which the target area may be illuminated . The illumination patterns may di f fer from each other in that the target area is irradiated spatially di f ferent with light radiation . Ac- cordingly, changing the illumination pattern may relate to irradiating the target area or subareas of the same with light radiation in a spatially different manner. The difference applied here may refer to intensity (including an intensity of zero) and / or wavelength or light color. For this purpose, a change may be made to the operation of a radiationgenerating device and / or a radiation shaping / direct ing device (if present) of the illuminator. In this regard, variants as described in the following may be applied.

[0020] In a further embodiment, the modification of the image capture setting comprises changing a spatial distribution of subareas of the target area that may be illuminated by the illuminator. The subareas may also be referred to as spots. With regard to this, the illuminator may be configured to selectively illuminate (or not illuminate) different subareas of the target area. For changing the spatial distribution, the illumination of at least one subarea may be changed by ending the illumination of the relevant subarea or illuminating the (previously unlit) subarea. For this purpose, the illuminator may e.g. comprise a plurality of light-emitting pixels, and the controlling of at least one light-emitting pixel may be changed accordingly. Alternatively or in addition, variants as described in the following may be considered .

[0021] In a further embodiment, the modification of the image capture setting comprises changing an angle of emission of light radiation that may be emitted from the illuminator. In this way, a spatially different illumination of the target area or of subareas of the same may be provided.

[0022] In a further embodiment, the illuminator comprises a plurality of emitters configured to emit coherent light radiation, and the modification of the image capture setting comprises changing a phase relationship between coherent light radiations emitted from the emitters. The emitters applied here may e.g. be laser diodes. Changing the phase relationship, which may relate to at least two of the emitters , may result in changing an angle of emission of light radiation emitted from the illuminator . Changing the phase relationship may be effected by di f ferent on and of f switching of the emitters .

[0023] In a further embodiment , the illuminator comprises a micromirror device , and the modi fication of the image capture setting comprises changing a position or rotational position of at least one micromirror of the micromirror device . In this way, an angle of emission of light radiation emitted from the illuminator may be changed, as well . The micromirror device may be a DMD .

[0024] In a further embodiment , the modi fication of the image capture setting comprises changing a color or color di stribution of light radiation that may be emitted from the illuminator . For this purpose , the illuminator may comprise a light source or a plurality of emitters or light-emitting pixels configured to provide di f ferent-colored light radiation ( s ) . A possible configuration is described in the following .

[0025] In a further embodiment , the illuminator comprises lightemitting pixels each comprising subpixels configured to emit dif ferent-colored light radiations . As an example , the pixels may each comprise three subpixels configured to emit a red, a green and a blue light radiation, and thus an RGB subpixel arrangement . This may be reali zed e . g . using a LED array or OLED array . The emission of light radiation from the pixels and subpixels is carried out on the basis of a PWM (pulsewidth modulation) control scheme . The modi fication of the image capture setting comprises changing the PWM control scheme . In this way, the color or color distribution of light radiation emitted from the illuminator may be changed . With respect to the PWM control scheme , the subpixels may be periodically powered for certain times and remain of f in between such times . The ratio of durations of the on and of f times and thus the duty cycle may define the perceivable average intensity and brightness of a light radiation generated by a subpixel . Therefore , the intensities of light radiations generated by the subpixels , and thus the color of a total light radiation emitted from a pixel , may be set and changed by respectively setting and changing the duty cycles .

[0026] In a further embodiment , the modi fication of the image capture setting comprises changing an image recording setting of the image sensor unit . In this respect , the image sensor unit is configured to allow it to be operated with di f ferent image recording settings . The changing of the image recording setting may be employed alternatively or together with changing an illumination pattern provided by the illuminator , as described above . The changing of the image recording setting may relate to an image recording time .

[0027] In a further embodiment , which may be considered in this context , the modi fication of the image capture setting comprises changing a coding of exposure of the image sensor unit . In this respect , the image sensor unit is configured to perform image recording with di f ferent exposure times . The changing of the coding of exposure makes it e . g . possible to achieve defocusing . As described above , the image sensor unit may be or may comprise an image sensor such as a CMOS or CCD image sensor . Changing the coding of exposure may comprise changing a shutter or exposure time of the image sensor .

[0028] For the case that the image sensor unit comprises , in addition to an image sensor, an upstream component for radiation shaping such as receiving optics , and the image sensor unit is also configured to provide di f ferent settings or positions of this component , the changing of the image recording setting of the image sensor unit may also comprise changing a setting or position of the radiation shaping component .

[0029] With respect to the predefined criterion used in the method, the following may apply . In a further embodiment , the predefined criterion i s a predefined balance of spatial frequency content , and may thus relate to image resolution . The term " spatial frequency" applied with respect to an image means how quickly the brightness changes over a certain distance . The spatial frequency thus defines the number or amplitude of brightness changes per spatial unit in a part of the respective image . Meeting the predefined criterion here may involve or mean whether the spatial frequency content of the analyzed first image complies with a predefined balance requirement . In order to veri fy this , the analyzing of the first image may comprise performing the Fourier trans form of the first image in order to provide its spatial frequency content , and performing an evaluation based on the occurrence of a proportion or percentage of preselected spatial frequencies . Here , the existing proportion of these spatial frequencies may be compared with a predefined threshold, and ful filment of the predefined criterion may be determined as to whether the threshold is undercut or exceeded . The preselected spatial frequencies may be low or lowest frequencies . In this context the following may further come into play .

[0030] For the case that the first image comprises a large amount of low spatial frequencies , this may mean that the image is too blurry . In this embodiment , analyzing the first image may comprise performing the Fourier trans form of the first image in order to provide its spatial frequency content , averaging a portion of low or lowest spatial frequencies of the first image , and checking i f the averaged portion exceeds a predefined threshold . I f this is the case , this may mean that the image is too blurry, and therefore the predefined criterion is not met . In this regard, changing the image capture setting makes it possible to capture an image , i . e . the second image , which may be less blurred compared to the first image .

[0031] In a further similar embodiment , the predefined criterion is a predefined blur level or refers to not exceeding a predefined blur level , and may thus relate to image sharpness . In this case , meeting the predefined criterion may involve or mean whether the blur level of the analyzed first image complies with a predefined blur level requirement . In order to check this , the analyzing of the first image may comprise performing the Fourier trans form of the first image in order to provide its spatial frequency content , and performing an evaluation based on the occurrence of a proportion or percentage of preselected spatial frequencies . Here too , the existing proportion of these spatial frequencies may be compared with a predefined threshold, and compliance with the predefined criterion may be determined as to whether the threshold is undercut or exceeded . The preselected spatial frequencies may be high or highest frequencies . In this context the following may further apply .

[0032] For the case that the first image comprises a small or insufficient amount of high spatial frequencies , this may again mean that the image is too blurry . In this embodiment , analyzing the first image may comprise performing the Fourier trans form of the first image in order to provide its spatial frequency content , averaging a portion of high or highest spatial frequencies of the first image , and performing a check as to whether the averaged portion exceeds a predefined threshold . I f this is not the case , this may mean that the image is too blurry, and therefore the predefined criterion is not met . In this regard, changing the image capture setting makes it possible to capture an image , i . e . the second image , which may be sharper compared to the first image .

[0033] In a further embodiment , the predefined criterion i s a predefined color balance . Meeting the predefined criterion here may involve or mean whether a color content of the analyzed first image complies with a predefined color balance requirement . In order to veri fy this , the analyzing of the first image may comprise generating a color histogram of the first image , and evaluating the color histogram . Generating the color histogram may comprise classi fying colors or color ranges of the first image into bins in accordance with their frequency of occurrence in the image. In the context of evaluating the color histogram, a check may be performed if certain or preselected histogram bins of the color histogram have enough intensity or frequency. Performing the check may include comparing the color histogram of the first image with a reference color histogram. If the considered histogram bins of the color histogram have a too low intensity or frequency, this may mean that the first image comprises an insufficient color balance. In this respect, changing the image capture setting allows to capture an image, i.e. the second image, which may comprise an enhanced color balance compared to the first image.

[0034] In a further embodiment, the predefined criterion is a predefined recognition level of an object. This embodiment may be considered if the method is applied in the context of classifying, tracking or recognizing an object. The object may e.g. be a finger, a fingerprint of a finger, a face or an iris of an eye. Meeting the predefined criterion here may involve or mean whether a degree or percentage of recognition of the respective object complies with a predefined score or likelihood. For this purpose, the first image may at first be processed to determine the degree of recognition of the object in question, the object being reproduced by the first image. Then, it may be checked if the determined degree of recognition is below a predefined threshold. If this is the case, this may mean that the recognition level is too low. In this regard, changing the image capture setting makes it possible to capture an image, i.e. the second image, via which an increased level of likelihood may be provided.

[0035] With reference to the foregoing embodiment, it is noted that the object in question is not an individual or unique object such as a unique fingerprint of a person, but a general or arbitrary object of a predetermined object class or object group, and therefore e.g. any finger, any fingerprint, any face or any iris of an eye. In this regard, the generating of the control information for the purpose of capturing the sec- ond image serves to achieve an improvement in relation to the image capture setting in order to increase the recognition level of the obj ect of the respective obj ect class .

[0036] With regard to the method, there is the option of carrying it out with a plurality of di f ferent predefined criteria, for example two or more of the above-mentioned predefined criteria . The employment of a plurality of predefined criteria may be implemented in connection with a plurality of arti ficial neural networks , as is the case with the following method variant .

[0037] In a further embodiment , the method comprises selecting the predefined criterion from a plurality of predefined criteria, and selecting an arti ficial neural network from a plurality of arti ficial neural networks depending on the selected predefined criterion . Each of the arti ficial neural networks is associated with a respective one of the plurality o f predefined criteria . Moreover, the generating of the control information is performed by using the selected arti ficial neural network and feeding the input information to the selected arti ficial neural network . Upon receipt of the input information, the arti ficial neural network may provide the control information .

[0038] The above method variant may be applied with regard to multiple tasks , each task being related to a respective predefined criterion . For each task, a di f ferent criterion and associated arti ficial neural network are selected and used in the method . With regard to the plurality of arti ficial neural networks , details as described above may apply . Therefore , the arti ficial neural networks are trained to generate the control information on the basis of the input information, wherein the training may be performed in relation to the respective task and predefined criterion . Moreover, reference is made to the possibility of providing a cloud-based configuration for each of the arti ficial neural networks . The first image capture setting applied in the method for capturing the first image may be a default image capture setting . This may mean that the first image capture setting enables satis factory image capture on average under most conditions .

[0039] In a further embodiment , the first image capture setting is selected from a plurality of di f ferent default image capture settings . In this way, a preoptimi zation may be provided in relation to the first image capture setting such that the method may reliably converge to an improved or optimal image capture . The plurality of default image capture settings may enable highlighting di f ferent aspects or characteri stics of an image of the target area, e . g . high spatial frequencies or low spatial frequencies . The selecting of the first image capture setting from the plurality of default image capture settings may, previous to capturing the first image , comprise successively capturing images of the target area us ing the illuminator and the image sensor unit , hereby being controlled in a successive manner according to each of the di fferent default image capture settings , analyzing the captured images in relation to ful filling a predefined pre-criterion, and then setting one of the default image capture settings as the first image capture setting depending on which of the captured images best meets the predefined pre-criterion or comes closest to meeting the predefined pre-criterion . The predefined pre-criterion may correspond to the predefined criterion used in the analysis of the first image , but its ful filling may be subj ect to a less stringent requirement .

[0040] As indicated above , the captured second image may not meet the predefined criterion, but may come closer to meeting the predefined criterion than is the case with the first image . In this regard, the following procedure may be used .

[0041] In a further embodiment , the second image is analyzed to determine whether it meets the predefined criterion . I f this is not the case , it is provided to repeat the method ( or respec- tive method steps ) iteratively ( i . e . at least once ) until the predefined criterion is met . This may be implemented in the following manner .

[0042] After capturing and analyzing the second image , one or a plurality of successive iteration stages may be applied . Each iteration stage may comprise the following steps : Generating control information for a modi fied image capture setting on the basis of input information relating to the previous image not ful filling the predefined criterion, the modi fied image capture setting comprising a modi fication compared to the previous image capture setting in order to provide an improvement in terms of meeting the predefined criterion; capturing an image of the target area by illuminating the target area with the illuminator and performing image recording of the target area with the image sensor unit , the illuminator and the image sensor unit being controlled according to the control information generated on the basis of the input information relating to the previous image and thus according to the modi fied image capture setting; and analyzing the image captured in accordance with the modi fied image capture setting to determine whether it meets the predefined criterion .

[0043] The analysis of the second image and, i f applicable , one or a plurality of further images captured in one or a plurality of further iteration stages , may be carried out as described above for the first image . Moreover, above-described details may correspondingly apply to features such as modi fying an image capture setting in relation to capturing a further or further images in one or a plurality of further iteration stages . In the event that the second image , or a subsequent image captured in at least one successive iteration stage , meets the predefined criterion, the method may further comprise generating a respective output and / or performing further processing in relation to the second image or subsequent image . According to a further aspect of the invention, a system for adapted image capture is proposed . The system comprises an illuminator, an image sensor unit , a controller for controlling the illuminator and the image sensor unit , a processing module and a generating module . The system is configured to carry out capturing a first image of a target area by illuminating the target area with the illuminator and performing image recording of the target area with the image sensor unit , the illuminator and the image sensor unit being controlled by the controller according to a first image capture setting for this purpose . The system is further configured to perform analyzing the first image to determine whether it meets a predefined criterion using the processing module , and generating control information for a second image capture setting i f the predefined criterion is not met using the generating module to which input information relating to the first image is fed . The second image capture setting comprises a modi fication compared to the first image capture setting in order to provide an improvement in terms of meeting the predefined criterion . The generating module is conf igured or trained to generate the control information on the basis of the input information . The system is further configured to perform capturing a second image of the target area by illuminating the target area with the illuminator and performing image recording of the target area with the image sensor unit , the illuminator and the image sensor unit being controlled by the controller according to the control information generated by the generating module on the basis of the input information relating to the first image and thus according to the second image capture setting for thi s purpose .

[0044] Corresponding to the above-described method, the proposed system enables a fast capture and generation of an improved optical image . In this respect , the second second image acquired using the system may ful fil the predefined criterion, or may at least come closer to ful filling the prede fined criterion than is the case with the first image . The system may be used to carry out the above-described method or individual or several variants of the method described above . In a corresponding manner, aspects and detai ls mentioned above with reference to the method may also be employed for the system . For the system, and likewise for the method, di f ferent applications may be considered . As an example , the system may be integrated on a device such as a portable or wearable device , e . g . a mobile phone or smart watch, a smart display, or on AR ( augmented reality) or VR (virtual reality) glasses .

[0045] The advantageous configurations and developments of the invention explained above and / or presented in the dependent claims may - apart from, for example , in cases of clear dependencies or incompatible alternatives - be employed individually or else in any desired combination with one another .

[0046] The above-described properties , features and advantages of this invention and the way in which they are achieved will become clearer and more clearly understood in association with the following description of exemplary embodiments which are explained in greater detail in association with the schematic drawings , in which :

[0047] Figure 1 shows an illustration of a system for adapted image capture ;

[0048] Figure 2 shows a flow chart of a method for adapted image capture ;

[0049] Figure 3 shows an exemplary sequence of images to i llustrate an adapted image capture ;

[0050] Figure 4 shows an illustration of an arti ficial neural network used in the system and method; Figures 5 to 10 illustrate possible variants for modi fying an image capture setting that may be applied in the system and method;

[0051] Figure 11 shows a flow chart of a further method for adapted image capture ; and

[0052] Figures 12 to 15 illustrate possible use cases for the system and method .

[0053] Possible configurations and variants of a method and system 100 for adapted and dynamic image capture are described with reference to the following schematic figures . It is pointed out that the schematic figures may not be true to scale . Therefore , components , elements and structures shown in the figures may be illustrated with exaggerated si ze or size reduction in order to af ford a better understanding . In addition, it is pointed out that features and details which are described in relation to one configuration may also be used in relation to other configurations , and that several configurations and their features may be combined with one another . Matching features may therefore only be described in detail in relation to one configuration .

[0054] The method and system 100 may feature advantages such as fast capture of enhanced optical images , small computational time , and reliable reduction or suppression of influence of external light af fecting image capture . The captured images may e . g . comprise a high resolution . Moreover, no expensive equipment is required to carry out the method and reali ze the system 100 , and by implementing the method and system 100 , tasks such as obj ect recognition, classi fication and tracking may be reali zed with a high reliability and accuracy .

[0055] Figure 1 shows an illustration of a system 100 for adapted image capture , which may also be understood as general scheme of the invention . Possible applications for the system 100 may include integrating it on a portable or wearable device such as a mobile phone or smart watch, a smart display, AR ( augmented reality) or VR (virtual reality) glasses , etc . This is outlined in more detail below . The system 100 comprises an illuminator 110 for illuminating a target area 160 and an image sensor unit 120 for performing image recording in order to capture an optical image of the illuminated target area 160 . The target area 160 may also be referred to as scene . With regard to the illuminating and image recording, the illuminator 110 is configured to generate and emit light radiation 170 in the direction of the target area 160 , and the image sensor unit 120 is configured to sense or detect light radiation 170 coming ( i . a . ) from the target area 160 . The light radiation 170 that is sensed and recorded by the image sensor unit 120 may comprise a radiation portion produced by the illuminator 110 and, i f applicable , al so a radiation portion from the environment . These radiation portions may be partially reflected or scattered at the target area 160 in the direction of the image sensor unit 120 . The image sensor unit 120 may thereupon provide an optical image in digital form and thus respective image data 220 . The image data 220 may be subj ected to data processing or preprocessing by the image sensor unit 120 prior to being provided .

[0056] The image sensor unit 120 may operate or may be sensitive to radiation at least in a wavelength region of light radiation 170 emitted by the illuminator 110 . The illuminator 110 may be configured to produce and emit light radiation 170 in the visible wavelength range or another wavelength range such as the infrared wavelength range .

[0057] As explained in more detail below, both the illuminator 110 and image sensor unit 120 may comprise a pixelated configuration . In this respect , the illuminator 110 may comprise a plurality of separately addressable light-emitting pixels 112 configured for emitting light radiation 170 ( see e . g . figure 5 ) . The image sensor unit 120 may comprise a plural ity of light-sensing pixels 122 configured for detecting l ight radiation 170 ( see figure 10 ) .

[0058] The system 100 further comprises a controller 130 configured to control the illuminator 110 and the image sensor unit 120 . The controller 130 may be also referred to as driver . The controlling is performed via control signals 211 , 212 which are generated by the controller 130 and transmitted from the controller 130 to the illuminator 110 ( control signals 211 ) and to the image sensor unit 120 ( control signals 212 ) . Via the control signals 211 , 212 , the controller 130 specifies an operation setting of the illuminator 110 and image sensor unit 120 according to which the image capture takes place . This setting is also referred to as image capture setting in the following .

[0059] A further component part of the system 100 is a processing module 140 configured to analyze an image of the target area 160 generated with the aid of the illuminator 110 and the image sensor unit 120 . The analysis relates to as to whether the respective image meets a predefined criterion . The image analysis is performed by the processing module 140 using the image data 220 of the respective image provided by the image sensor unit 120 , which is therefore transmitted from the image sensor unit 120 to the processing module 140 . Within the framework of the analyzing or prior to the actual analyzing, the processing module 140 may ( also ) carry out data processing, e . g . for the purpose of filtering and / or resizing or rescaling the image or image data 220 in question .

[0060] For the case that the result of the image analysis consists in that the image complies with the predefined criterion ("y" , yes ) , the processing module 140 is configured to provide a respective output 225 . In this respect , further processing may take place beforehand . Depending on the task or use case , the output 225 may e . g . comprise the image or corresponding image data 220 ( i f applicable , in proces sed form) , or another information . In the opposite case , i f the result of the analysis is that the image does not ful fill the predefined criterion ("n" , no ) , the system 100 is configured to trigger a procedure in order to obtain an enhancement with regard to the image capture . Here , a further component part of the system 100 , i . e . a generating module 150 is used to which input information

[0061] 221 relating to the image that failed to meet the predefined criterion is transmitted from the processing module 140 . The generating module 150 is configured or trained, upon receipt of the input information 221 , to generate control information

[0062] 222 for a modi fied or optimal image capture setting on the basis of the input information 221 . The modi fied image capture setting comprises a modi fication compared to the previously applied image capture setting (with the use o f which the failed image was acquired) in order to provide an improvement in terms of meeting the predefined criterion .

[0063] The input information 221 fed to the generating module 150 may comprise image data 220 of the failed image or processed image data of that image . In the latter case , e . g . filtering and / or resi zing or rescaling of the respective image or image data 220 may be performed by the processing module 140 prior to transmitting such data to the generating module 150 . Moreover, the input information 221 may also or additionally comprise information with respect to the predefined criterion or with respect to the failed image not complying with the predefined criterion .

[0064] The control information 222 generated by the generating module 150 on the basis of the input information 221 relating to the failed image is transmitted from the generating module 150 to the controller 130 , which thereupon controls the illuminator 110 and image sensor unit 120 via respective control signals 211 , 212 transmitted to these components 110 , 120 in order to capture a further image of the target area 160 . In this regard, the illuminator 110 and image sensor unit 120 are controlled by the controller 130 according to the control information 222 generated by the generating module 150 and thus according to the modi fied image capture setting . The modi fication applied here may relate to the operating mode of the illuminator 110 and / or of the image sensor unit 120 .

[0065] The image acquired by applying the modi fied image capture setting, for which respective image date 220 may again be provided by the image sensor unit 120 , may comply with the predefined criterion, or may at least come closer to fulfilling the predefined criterion then is the case with the previous image . This may be determined in the above-described manner by the processing module 140 performing an analysis of the respective image using the corresponding image data 220 provided by the image sensor unit 120 . For the case that the predefined criterion is ( again) not ful filled and thus ( another ) fail is present , the system 100 is configured to repeat the above-described procedure of obtaining enhanced image capture , i . e . generating control information 222 , capturing an image and performing image analysis . This may also be done multiple times , i f applicable , until the prede fined criterion is met .

[0066] Components of the system 100 such as the controller 130 , the processing module 140 and the generating module 150 may be reali zed in the form of hardware of a computer or computer system and / or software running on such a computer or computer system . Moreover, further components such as communication devices and interfaces may be provided via which the abovedescribed transmission and exchange of information and data, including transmitting control signals 211 , 212 from the controller 130 to the illuminator 110 and image sensor unit 120 , may be reali zed . It is also possible to implement one or more of these components in the form of a cloud-based application, e . g . to reduce power consumption, which for this purpose may be hosted on another computer or server . In this regard, exchanging information and data may take place i . a . via the Internet (not depicted) . This may e . g . apply to the generating module 150 . Moreover, with respect to the generating module 150 , a configuration comprising an arti ficial neural network

[0067] 151 may be considered, as described below with reference to figure 4 .

[0068] In order to further illustrate the functionality of the system 100 and also the repetition procedure indicated above , a method for adapted image capture is described in the following with reference to figure 2 . Figure 2 illustrates the method in the form of a flowchart of method steps 201 to 205 . The system 100 shown in figure 1 may be operated according to the method shown in figure 2 . For the following description, reference is therefore also made to features of figure 1 .

[0069] In the method, an initial or first image of a target area 160 is captured ( step 201 ) . This is done by illuminating the target area 160 with the illuminator 110 and performing image recording of the target area 160 with the image sensor unit 120 . In this regard, the illuminator 110 and image sensor unit 120 are controlled according to an initial or first image capture setting . The controlling of the illuminator 110 and image sensor unit 120 is performed using the controller 130 . The image sensor unit 120 provides respective image data 220 of the first image .

[0070] Subsequently, the first image is analyzed to determine whether it meets a predefined criterion ( step 202 ) . This step is performed by the processing module 140 , and by using the image data 220 of the first image provided by the image sensor unit 120 .

[0071] In the event that the first image ful fills the predetermined criterion ("y" , yes ) , a respective output 225 is provided and / or further processing is performed in relation to the first image ( step 205 ) . In this regard, the first image or its image data 220 may be output , e . g . for the purpose of being displayed on a display device , or the image data 220 may be subj ect to further processing in order to output other information . I f , on the contrary, the first image does not ful fi ll the predefined criterion ("n" , no ) , control information 222 for a subsequent or second image capture setting is generated on the basis of input information 221 relating to the first image ( step 203 ) . The second image capture setting comprises a modi fication compared to the first image capture setting in order to provide an improvement in terms of meeting the predefined criterion . The generating of the control information 222 is performed using the generating module 150 . The input information 221 may comprise the image data 220 of the first image , or processed ( e . g . filtered and / or rescaled) image data of the same . The input information 221 may, i f applicable , also comprise information with regard to the predef ined criterion or with regard to the first image not meeting the predefined criterion .

[0072] This is followed by capturing a subsequent or second image of the target area 160 by illuminating the target area 160 with the illuminator 110 and performing image recording of the target area 160 with the image sensor unit 120 ( step 204 ) . In this step, the illuminator 110 and image sensor unit 120 are controlled according to the control information 222 generated on the basis of the input information 221 relating to the first image , and thus according to the second image capture setting . Similar to capturing the first image , the illuminator 110 and image sensor unit 120 are controlled by the controller 130 , which hereby performs the controlling using the control information 222 and thus according to the second image capture setting . The image sensor unit 120 provides respective image data 220 of the second image .

[0073] The second image captured in this way is then analyzed to determine whether it meets the predefined criterion ( step 202 ) . Similar to the analysis of the previous first image , the analysis of the second image is performed via the processing module 140 , and by using the image data 220 of the second image provided by the image sensor unit 120 . The application of the second image capture setting may have the effect that the second image fulfills the predefined criterion. In this case, a respective output 225 and / or further processing is performed in relation to the second image (step 205) .

[0074] It is also possible that the second image, although being captured by utilizing the modified second image capture setting, does not fulfill the predefined criterion. In this case, since the second image capture setting is enhanced compared to the first image capture setting, the second image may come closer to meeting the predefined criterion than the first image. In this case, the method, i.e. the sequence of steps 203, 204 and then 202, is repeated iteratively until the predefined criterion is met. If the predefined criterion is then fulfilled, step 205 follows in relation to the respective image.

[0075] With regard to this, after capturing and analyzing the second image, at least one or a plurality of successive iteration stages are applied. Each iteration stage comprises a step 203, i.e. generating (further) control information 222 for a (further) modified image capture setting on the basis of input information 221 relating to the previous failed image not fulfilling the predefined criterion, the (further) modified image capture setting comprising a modification compared to the previous image capture setting in order to provide an improvement in terms of meeting the predefined criterion, a step 204, i.e. capturing an image (further image) of the target area 160 by illuminating the target area 160 with the illuminator 110 and performing image recording of the target area 160 with the image sensor unit 120, the illuminator 110 and image sensor unit 120 being controlled according to the (further) control information 222 generated on the basis of the input information 221 relating to the previous (failed) image and thus according to the (further) modified image capture setting, and a step 202, i.e. analyzing the (further) image captured in accordance with the ( further ) modi fied image capture setting to determine whether it meets the predefined criterion . For these steps , aforementioned details such as the use of the controller 130 , processing module 140 and generating module 150 apply in a corresponding manner .

[0076] Figure 3 shows , in the manner of a simpli fied representation for further illustration of the method, an exemplary sequence of images that may be captured during execution of the method . Here , an obj ect in the form of a triangle is present in an illuminated target area 160 . As illustrated, initially a first image 191 of the target area 160 is captured using the illuminator 110 and image sensor unit 120 . The first image

[0077] 191 reproduces the obj ect in a very blurred condition . The first image 191 therefore fails to meet the predefined criterion, which is determined through a respective analysis of its image data . In this regard, the applied predefined criterion may relate to a blur level . Thereafter, a second image

[0078] 192 of the target area 160 is captured using the il luminator 110 and image sensor unit 120 , hereby being control led according to a modi fied second image capture setting . The second image 192 reproduces the obj ect in a sharper, less blurred condition compared to the first image 191 . In this respect , the second image 192 comes closer to ful fi lling the predefined criterion, but still does not meet the predefined criterion . This is determined through a respective analysis of the image data of the second image 192 . Subsequently, a third image 193 of the target area 160 is captured using the illuminator 110 and image sensor unit 120 , hereby being controlled according to a further modi fied third image capture setting . The third image 193 reproduces the obj ect in a sharp condition . In this case , the third image 193 complies with the predefined criterion, which results from a respective analysis of its image data .

[0079] As indicated above , the generating module 150 applied in the system 100 and method for the purpose of generating control information 222 for the controller 130 may comprise an arti- ficial neural network 151 , which optionally may be reali zed as a cloud- implemented component . The structure of such an arti ficial neural network 151 is shown in figure 4 for further illustration . The arti ficial neural network 151 comprises a plurality of interconnected neurons (not depicted) which are arranged or combined in layers of the arti ficial neural network 151 . In this regard, the arti ficial neural network 151 comprises an input layer 152 , at least one or a plurality of hidden layers 153 downstream of the input layer 152 , and an output layer 154 downstream of the hidden layer ( s ) 153 . Each of the layers 152 , 153 , 154 comprises a plural ity of neurons .

[0080] The arti ficial neural network 151 is trained to generate control information 222 for an improved image capture setting on the basis of input information 221 relating to a respective image . The input information 221 is transmitted to the artificial neural network 151 i f , as described above , the processing module 140 determines through analysis that the respective image does not ful fill the predefined criterion . The input information 221 may comprise image data 220 o f the respective image provided by the image sensor unit 120 , or processed image data of the same , and additionally, i f the case may be , information in relation to the applied predefined criterion or in relation to the image not meeting the predefined criterion . Upon being fed with the input information 221 , the arti ficial neural network 151 provides or propagates an output in the form of the control information 222 that is outputted via the output layer 154 . The control information 222 is then transmitted to the controller 130 which thereupon initiates and controls the capturing of a further image based on this .

[0081] Depending on the respective use case and configuration of the system 100 and method, including the applied predef ined criterion, a di f ferent configuration of the arti ficial neural network 151 may apply . Therefore , features such as the number of the hidden layers 153 and the number of neurons of the layers 152 , 153 , 154 may vary depending on the speci fic application . In this regard, features related to the image sensor unit 120 such as pixel sensor si ze ( total amount of pixels ) may decree the configuration or si ze of the input layer 152 , and features related to the illuminator 110 such as number of addressable pixels or light spots times other characteristics such as intensity, angle of emission, wavelength etc . of the emitted light radiation 170 may constitute the configuration of the output layer 154 .

[0082] The training of the arti ficial neural network 151 may be done by feeding the input layer 152 with input or input data in the form of ground truths representing a plurality of reference images . The reference images may comprise images fulfilling the applied predefined criterion as well as images not ful filling the same . The ground truths may, i f applicable , also comprise information in relation to the applied predefined criterion or in relation to the reference images ful filling or not ful filling the predefined criterion . The reference images may be associated with a known or predetermined output . By comparing the output provided by the arti ficial neural network 151 upon being fed with the ground truths to the known or predetermined output , i . e . basically the di fference between the same , a cost function reproducing an error may be evaluated . By using this error and performing a backpropagation algorithm, parameters of the arti ficial neural network 151 such as its weights may be tuned for each reference image , i f necessary, with a multiple repetition of this operation ( called epochs ) , until the resulting error is below an acceptable level or predetermined threshold .

[0083] In the following, possible configurations of the il luminator 110 and image sensor unit 120 as well as variants for modi fying the image capture setting that may be applied in the system 100 and method are described . The modi fication of the image capture setting, which, as described above , is performed by corresponding control by the controller 130 , may comprise modi fying an operating mode of the illuminator 110 so that an illumination pattern provided by the illuminator 110 is changed. For this purpose, the illuminator 110 is configured to provide different illumination patterns with which a target area 160 may be illuminated. The illumination patterns may differ from each other in that the target area 160 is irradiated spatially different with light radiation 170. The difference may relate to intensity and / or wavelength or color of the light radiation 170.

[0084] Figure 5 shows, by way of lateral and perspective views, a configuration of the illuminator 110 comprising a lightgenerating device 111 with a plurality of light-emitting pixels 112. The pixels 112 are arranged next to each other in the form of a matrix. Deviating from the schematic representation of figure 5, the illuminator 110 may comprise another or a much larger number of light-emitting pixels 112. Each of the pixels 112 may be separately controlled and operated to produce and emit light radiation 170. This is done by corresponding control by the controller 130. As an example, the plurality of pixels 112 may be constituted by an emitter array such as an array of LEDs or laser diodes, and may e.g. constitute a structured light flash. Via each of the pixels 112, a respective subarea 161 of the target area 160 may be selectively illuminated. For this purpose, the illuminator 110 may additionally comprise a beam shaping or directing device such as an appropriate optics downstream of the lightgenerating device 111 (not depicted) . In this way, by changing the operation of at least one pixel 112, the spatial distribution of subareas 161 of the target area 160 illuminated by the illuminator 110, and thus the illumination pattern provided by the illuminator 110, may be changed. Figure 5 shows an example of a change between an illumination pattern (left side) and another illumination pattern (right side) by operating and not operating several pixels 112 differently. In figure 5, the operated pixels 112 emitting light radiation 170 are additionally indicated by hatching. With reference to figure 5, a modification in relation to an illumination pattern may also be achieved by operating the pixels 112 in different ways to generate light radiation 170 with different intensities (not depicted) .

[0085] Figure 6 shows, by way of lateral views, another configuration of the illuminator 110 for providing different illumination patterns. Here, the illuminator 110 is configured to change an angle of emission of light radiation 170 that may be emitted from the illuminator 110 or a light-generating device 111 of the same. This is accomplished by respective control by the controller 130. Figure 6 illustrates an example of a change between an illumination pattern (left side) and another illumination pattern (right side) by emitting the light radiation 170 with a modified emission angle, as indicated by an angular change 172. In case the illuminator 110 comprises light-emitting pixels 112 as described above, a change in angle of emission may be provided in relation to individual, a plurality or all of the (operated) pixels 112 or light radiations 170 or beams emitted from them. The change in angle of emission of light radiation 170, via which a target area 160 may be illuminated in a spatially different manner, may be realized by an appropriate configuration of the illuminator 110 or of a respective beam shaping or directing device of the same. Possible examples are described in the following.

[0086] Providing a change in angle of emission of light radiation 170 and thus changing an angular pattern may be realized by the illuminator 110 comprising a plurality of emitters configured to emit coherent light radiation 170, i.e. monochromatic light radiation 170. For this purpose, the emitters may e.g. be laser diodes. Figure 7 illustrates such a configuration via two pixels 112 of a light-generating device 111 of the illuminator 110 emitting monochromatic light radiation 170. Here, the light radiation 170 is illustrated in the form of a sinusoidal wave propagating along a spatial direction (to the right, indicated by an arrow) , wherein the depicted change or oscillation in amplitude may visuali ze a changing electric field strength . In this configuration, changing the angle of emission may be provided by changing a phase relationship between coherent light radiations 170 emitted from pixels 112 of the illuminator 110 . Figure 7 shows an example of a change between a condition in which the two depicted pixels 112 emit light radiations 170 with matching phases ( left side ) and a condition in which the pixels 112 emit light radiations 170 phase-shi fted from each other, as indicated by a phase di f ference 171 . The change in phase relationship may be accomplished by di f ferent on and of f switching of the pixels 112 , which is accomplished by corresponding control by the controller 130 . As a consequence , the angle of emission of a total or superimposed light radiation 170 may be changed (not depicted) . This ef fect may be based on the fact that light radiation 170 may be emitted not only in one direction, but in a spatial or angular area, and by dif ferent phase relationships constructive and destructive interference may appear in di f ferent directions .

[0087] With regard to figure 7 , it is pointed out that the illuminator 110 may comprise a larger number of light-emitting pixels 112 . By changing the phase relationship of light radiations 170 emitted from a plurality of or all of these pixels 112 , it may be caused to emit light radiations 170 or respective light beams with di f ferent angles of emission and therefore into di f ferent spatial areas and thus subareas 161 of a target area 160 (not depicted) .

[0088] Figure 8 shows , by way of lateral views , a further configuration of the illuminator 110 via which a change in angle of emission of light radiation 170 may be provoked, thus making it possible to illuminate a target area 160 with di f ferent illumination patterns . The illuminator 110 comprises a lightgenerating device 111 with a plurality of light-emitting pixels 112 , and a beam directing device in the form of a micromirror device 115 comprising a plurality of micromirrors 116 whose rotational position may be set and changed . The micro- mirror device 115 may be a digital micromirror device ( DMD) . Deviating from the schematic illustration of figure 8 , the illuminator 110 may comprise another or a much larger number of pixels 112 and micromirrors 116 . Via the micromirrors 116 , light radiation 170 emitted from the pixels 112 may be reflected . The direction in which the reflected radiation 170 is radiated depends on the respective rotational position of the micromirrors 116 . Setting or changing the rotational position of one or a plurality of or all micromirrors 116 , which is performed by respective control by the controller 130 , therefore makes it possible to provide a change in angle of emission in relation to individual or a plurality of or all light radiations 170 generated by the pixels 112 . Figure 8 shows an example of a change between a condition in which the depicted micromirrors 116 comprise a corresponding rotational position ( left side ) and another condition in which one of the micromirrors 116 is brought into a di f ferent rotational position ( right side , upper micromirror 116 ) .

[0089] I lluminating a target area 160 with di f ferent spatial illumination patterns may also be brought about by changing a color or color distribution of light radiation 170 emitted from the illuminator 110 . A possible configuration is illustrated in figure 9 by way of lateral views . Here , the illuminator 110 comprises a light-generating device 111 with a plurality of light-emitting pixels 112 , each pixel 112 comprising three subpixels 113 configured to emit a di f ferent-colored light radiation 170 . This is indicated in figure 9 by reference numerals 113- 1 , 113-2 , 113-3 for the subpixels 113 , and by reference numerals 170- 1 , 170-2 , 170-3 for the light radiations 170 generated by them . In this regard, a RGB subpixel arrangement may be present for each of the pixels 112 in which the subpixel 113- 1 may emit a red light radiation 170-1 , the subpixel 113-2 may emit a green light radiation 170-2 , and the subpixel 113-3 may emit a blue light radiation 170-3 . Deviating from the schematic illustration of figure 9 , the illuminator 110 may comprise another or a much larger number of pixels 112 and subpixels 113 . The light-generating device 111 with the pixels 112 and subpixels 113 may be constituted e . g . by a LED array or LED display, an OLED array or OLED display, or a LCD display .

[0090] The emission of light radiation 170 from the pixels 112 and subpixels 113 is carried out on the basis of a PWM (pulsewidth modulation) control scheme , in which the subpixels 113 may be periodically energi zed for certain times and remain off in between such times (not depicted) . The ratio of durations of the on and of f times and thus the duty cycle may define the perceivable average intensity and brightness of the respective light radiation 170 generated by a subpixel 113 . In this way, the intensities of light radiations 170 emitted by the subpixels 113 , and therefore the color of a total or superimposed light radiation 170 emitted from a pixel 112 , may be set and changed by respectively setting and changing the duty cycles . Therefore , by changing the PWM control scheme , the color distribution of light radiation 170 emitted from the illuminator 110 may be changed . The application and change of the PWM control scheme , and accordingly the control of the pixels 112 and subpixels 113 , is carried out by the controller 113 .

[0091] The provision of the PWM control scheme and the change thereof is indicated in figure 9 by the arrows illustrating the light radiation 170 . In this regard, figure 9 shows an example of a change between a condition in which the depicted subpixels 113 of the pixels 112 emit their di f ferent-colored light radiations 170 with the same average intensity ( left side ) and another condition in which a modi fication exists in relation to the subpixels 113 of two pixels 112 ( right side , middle and lower pixel 112 ) . For the middle pixel 112 , the subpixel 113-2 is not operated such that the light radiation 170-2 is not emitted and therefore its intensity is zero . For the lower pixel 112 , the operation of the subpixels 113-2 , 113-3 is modi fied such that a smaller average intensity of the light radiation 170-2 and a greater average intensity of the light radiation 170-3 , indicated by a smaller or bigger width of the associated arrows , is present .

[0092] The modi fication of the image capture setting, which is performed by corresponding control by the controller 130 , may also comprise modi fying an operating mode of the image sensor unit 120 so that an image recording setting of the image sensor unit 120 is changed . For this purpose , the image sensor unit 120 is configured to be operated with di f ferent image recording settings .

[0093] Figure 10 shows , by way of lateral views , a configuration of the image sensor unit 120 comprising an image sensor 121 with a plurality of light-sensing pixels 122 via which l ight radiation 170 may be detected . The image sensor 121 may be an active-pixel image sensor such as a CMOS image sensor or a CCD image sensor . The pixels 122 may be arranged next to each other in the form of a matrix . Deviating from the schematic representation of figure 10 , the image sensor 121 may comprise another or a much larger number of light-sens ing pixels 122 . Moreover, the image sensor unit 120 may compri se , upstream of the image sensor 121 , a beam shaping component such as receiving optics (not depicted) .

[0094] With regard to the configuration illustrated in figure 10 , changing the image recording setting of the image sensor unit

[0095] 120 is ef fected by changing a coding of exposure of the image sensor unit 120 or its image sensor 121 . This is controlled by the controller 130 . For this purpose , an exposure time 125 of the image sensor 121 in which the image sensor 121 senses light radiation 170 is modi fied . This may e . g . be applied to accomplish defocusing . Figure 10 shows an example o f a change between an image recording setting in which the image sensor

[0096] 121 is operated with a respective exposure time 125 ( left side ) and another image recording setting in which the image sensor 121 is operated with a shorter exposure time 125 ( right side ) . It is pointed out that the configurations of the il luminator 110 and image sensor unit 120 described above with reference to figures 5 to 10 only represent an exemplary, non- exhaustive selection . Apart from that , other configurations may be considered . Moreover, changing an image capture setting controlled by the controller 130 may comprise changing an operating mode of the illuminator 110 and thus an illumination pattern provided by the illuminator 110 and / or changing an operating mode of the image sensor unit 120 and thus an image recording setting of the same . In other words , a respective change may be provided for either the illuminator 110 or the image sensor unit 120 , so that there is no change in the operating mode for one of the illuminator 110 and the image sensor unit 120 , or changes may be provided for both the illuminator 110 and image sensor unit 120 together .

[0097] Similar to the illuminator 110 and image sensor unit 120 , multiple di f ferent variants may be applied for the predefined criterion used in the system 100 and method when analyzing a captured image ( step 202 in figure 2 ) . This may depend on the respective use case . Possible examples are described in the following .

[0098] In a possible configuration, the predefined criterion is a predefined balance of spatial frequency content of the analyzed image , i . e . the analysis is carried out to determine whether a spatial frequency content of the image in question complies with a predefined balance requirement . To check this , step 202 of the method shown in figure 2 may comprise performing the Fourier trans form of the respective image using its image data 220 in order to provide its spatial frequency content , and performing an evaluation based on the presence of a proportion or percentage of preselected spatial frequencies . In doing so , the existing proportion o f these spatial frequencies may be compared with a predefined threshold level , and ful filment of the predefined criterion may be determined by whether the threshold level is undercut or exceeded (not depicted) . The analyzed image may e.g. be very blurred, which may be due to a (too) large proportion of low spatial frequencies. In this case, after performing the Fourier transform to provide the spatial frequency content, a certain portion of low or lowest spatial frequencies of the image may be averaged, and then it may be checked if the averaged portion exceeds a predefined threshold. If this is the case, this may mean that the image is too blurry, and consequently the predefined criterion is not met. In this respect, providing control information 222 for a modified image capture setting (step 203) allows to capture a subsequent image (step 204) which may be less blurred and comprises a better resolution.

[0099] In a similar variant, the predefined criterion is a predefined blur level, i.e. the analysis (step 202) is carried out to determine whether a blur level of the respective image fulfills a predefined blur level requirement. Here, to check if the analyzed image complies with the predefined criterion, a procedure as described above may be applied, i.e. the Fourier transform of the respective image using its image data 220 may be performed in order to provide its spatial frequency content, and an evaluation based on the presence of a proportion or percentage of preselected spatial frequencies may be carried out. In this regard, the existing proportion of these spatial frequencies may be compared with a predefined threshold level, and compliance with the predefined criterion may be determined as to whether the threshold level is undercut or exceeded (not depicted) .

[0100] If, according to the example given above, the analyzed image is again very blurred, this may be due to a (too) small proportion of high spatial frequencies. In this case, after performing the Fourier transform to provide the spatial frequency content, a certain portion of high or highest spatial frequencies of the image may be averaged, and following this, it may be evaluated if the averaged portion exceeds a predefined threshold. If this is not the case, this may mean that the image is too blurry, and therefore the predefined criterion is not met. In this regard, providing control information 222 for a modified image capture setting (step 203) makes it possible to capture a subsequent image (step 204) which may be sharper .

[0101] According to a further configuration, the predefined criterion is a predefined color balance, i.e. the analysis is carried out to determine whether a color content of the respective image complies with a predefined color balance requirement. In order to check this, step 202 of the method illustrated in figure 2 may comprise generating a color histogram of the analyzed image using its image data 220, and then evaluating the color histogram. The color histogram may be generated by classifying colors or color ranges of the respective image into bins in accordance with their frequency of occurrence in the image. In the evaluation of the color histogram, it may be checked if certain histogram bins of the color histogram have enough frequency. For this purpose, the color histogram of the image may be compared with a reference color histogram (not depicted) .

[0102] The reference color histogram may e.g. be provided through a dataset used in the training of the applied artificial neural network 151. If the comparison reveals that the considered histogram bins of the color histogram have a too low frequency, this may mean that there is an inadequate color balance for the analyzed image. In this respect, providing control information 222 for a modified image capture setting (step 203) allows to capture a subsequent image (step 204) comprising an enhanced color balance.

[0103] The system 100 and method may also be applied in the context of classifying, tracking or recognizing an object, the object being reflected by a captured image. In this regard, a further variant may be employed in which the predefined criterion used in the image analysis (step 202) is a predefined recognition level of an object. Here, the analysis is carried out to determine whether a degree or percentage of recognition of the object reproduced by the respective image complies with a predefined score or likelihood. Examples for the object may include a finger, a fingerprint of a finger, a face or an iris of an eye. In this respect, it is noted that the considered object is not an individual or unique object such as a unique fingerprint of a person, but a general object of an object class or group, and therefore e.g. any finger, any fingerprint, any face or any iris of an eye.

[0104] In this configuration, step 202 of the method shown in figure 2 comprises processing the image or its image data 220 to determine the degree of recognition of the object in question, and subsequently checking if the determined degree of recognition is below a predefined threshold level (not depicted) . If this is the case, this may mean that the recognition level is too low. In this regard, providing control information 222 for a modified image capture setting (step 203) makes it possible to capture a subsequent image (step 204) via which an increased level of likelihood may be provided.

[0105] With respect to the aforementioned predefined criterion, i.e. the predefined recognition level of an object, the training of the artificial neural network 151 depicted in figure 4 may accordingly be performed with reference images reflecting objects of the associated object class. When operating the system 100 and carrying out the method (using the trained artificial neural network 151) , the aim is to provide an improvement in relation to the image capture setting to increase the recognition level of the object of the respective object class. If the analysis of a captured image (step 202) reveals that the image complies with the predefined criterion, i.e. the recognition level is fulfilled, the image may subsequently be subjected to further processing or analysis using its image data 220. With reference to the applications mentioned above, only then may the actual classifying of an object or recognizing of an individual object be performed, or may, in the case of tracking, e.g. object coordinates of the object reflected by the image be produced . I f applicable , this may be done in step 205 and performed by the processing module 140 or another component (not depicted) .

[0106] With regard to tracking an obj ect , the system 100 and method may be applied to successively capture images of the obj ect during movement of the same , wherein each of these images may comply with the predefined criterion . Here , the obj ect movement may require modi fying an image capture setting such as an illumination pattern provided by the illuminator 110 on the fly or in real-time , which may be accomplished in the manner described above by the system 100 and method . This variant or application of the method may also be re ferred to as "tracking-mode" .

[0107] The method and therefore the operation of the system 100 may be carried out with a plurality of di f ferent predef ined criteria . A possible variant is described in the following with reference to the flowchart depicted in figure 11 . The method according to figure 11 substantially corresponds to the method of figure 2 , but comprises an additional step 200 that is carried out before step 201 ( capturing a first image of a target area 160 ) . The step 200 may comprise selecting the predefined criterion from a plurality of di f ferent predefined criteria, and selecting an arti ficial neural network 151 from a plurality of arti ficial neural networks 151 depending on the selected predefined criterion (not depicted) . In this case , each of the arti ficial neural networks 151 may be trained in the above-described manner to generate control information 222 for a modi fied image capture setting on the basis of respective input information 221 , the training being performed with regard to the associated predefined criterion . Moreover, a cloud-based implementation may be considered for each of the arti ficial neural networks 151 , i f appl icable .

[0108] In this configuration, the system 100 and method may be used in relation to multiple di f ferent tasks , wherein each of the tasks is associated with one of the predefined criteria . The predefined criteria may comprise a plurality are all of the predefined criteria described above. The selecting of the predefined criterion and associated artificial neural network 151 may e.g. be performed by the controller 130 of the system 100 or another component, and may e.g. be based on input or user action by a user of a device (for example a mobile phone) on which the system 100 may be implemented. After the selection (step 200) , the method is proceeded as described above, i.e. a first image is captured (step 201) , and then the first image is analyzed (step 202) , this being performed in relation to the selected predefined criterion. Depending on the result of the analysis, the method is continued as specified above. When performing step 203 in order to provide control information 222, the selected artificial neural network 151 is used.

[0109] The initial or first image capture setting applied in the method of figures 2 and 11 according to which the illuminator 110 and image sensor unit 120 are controlled to capture a first image (step 201) , may be a default image capture setting. Such an image capture setting may ensure acceptable image capture on average under most conditions. The default image capture setting may comprise a default illumination pattern provided by the illuminator 110 and / or a default image recording setting of the image sensor unit 120. In this regard, only one default image capture setting may be applied, and this variant of the method may also be referred to as "fast-mode" .

[0110] In a further variant, a base of a plurality of different default image capture settings is applied, and the first image capture setting is selected from these different default image capture settings. This non-shown procedure may be related to boundary conditions such as hardware setup of the system 100, typical scene or use case (e.g. car cabin, fingertracking etc.) , and may also be carried out within step 200 of the flowchart illustrated in figure 11. Selecting the first image capture setting from a plurality of default image capture settings makes it possible to implement a preoptimization . The plurality of default image capture settings may enable highlighting di f ferent aspects or characteri stics of an image captured of a target area 160 such as high spatial frequencies or low spatial frequencies . For the selection, images ( or pre-images ) of the target area 160 may be sequentially captured using the illuminator 110 and image sensor unit 120 , hereby being sequentially controlled by the controller 130 according to each of the di f ferent default image capture settings . Subsequently, these images may be analyzed to determine whether they meet a predefined pre-cri terion . This may be carried out e . g . by the processing module 140 . Thereafter, one of the default image capture settings may be set as the first image capture setting depending on which of the captured images best meets the predefined pre-criterion or comes closest to meeting the predefined pre-criterion .

[0111] This may also be performed e . g . by the processing module 140 or another component . The predefined pre-criterion applied here may correspond to the predefined criterion used in the analysis of the first image and any subsequent image in step 202 , but its ful filling may be subj ect to less demanding requirements ( e . g . lower threshold) . This variant of the method, in which more pre-processing is involved, may also be referred to as " slow-mode" .

[0112] With reference to the following figures , an exemplary, non- exhaustive selection of use cases is described that may be considered for the system 100 and method .

[0113] A possible use case is fingerprint recognition on a display . In this regard, figure 12 illustrates a bidirectional display 180 which may e . g . be part of a mobile phone (not depicted) , the bidirectional display 180 comprising a display 181 for visual image display and an integrated image sensor unit 120 . The display 181 also constitutes an illuminator 110 of the system 100 . The display 181 may be a LED, OLED or LCD display . Via the illuminator 110 , a large variety of di f ferent illumination patterns may be produced which may vary in terms of intensity and wavelength content or light color . Figure 12 further depicts a finger 300 whose fingerprint is to be recogni zed . With the aid of the system 100 and by carrying out the method described above , an image reproducing the finger 300 or its fingerprint may be captured that complies with a predefined criterion . In this case , the predefined criterion may be a predefined recognition level of an ( arbitrary) fingerprint . After capturing the image , further proces sing may be performed using the image or its image data 220 to recognize or identi fy the individual fingerprint of the finger 300 .

[0114] Figure 13 shows a further use case in the form of f inger tracking on a display . Here again, a bidirectional display 180 which may e . g . be part of a mobile phone (not depicted) is applied, the bidirectional display 180 comprising a display 181 for visual image display and an integrated image sensor unit 120 . An illuminator 110 of the system 100 is integrated in the display 181 , as described in detail below . Figure 13 further depicts a finger 300 at an elevated distance to the bidirectional display 180 . The finger 300 or its movement is to be tracked in order to control a respective device ( e . g . the mobile phone ) comprising the bidirectional display 180 . Since the finger 300 may not necessari ly be in touch with the bidirectional display 180 , the bidirectional display 180 additionally comprises optics 182 downstream of the illuminator 110 for radiation shaping .

[0115] In order not to disturb showing information on the display 181 , transparent optics 182 is applied . This may be a transparent optical element such as a phase mask made of an absorbing dye-based polymer . Moreover, it is provided that the illumination from the display 181 ( and not the illuminator 110 ) does not interact with the optics 182 . For thi s purpose , the optics 182 or its absorbing polymer acts in a wavelength region di f ferent from the visible wavelength region in which the display 181 operates for visual image display . This may be the infrared wavelength region . The illuminator 110 acts in the same wavelength region of the optics 182 , i . e . the infrared wavelength region . For this purpose , the display 181 may comprise RGB pixels for emitting red, green and blue light radiation 170 , and the illuminator 110 may be configured using infrared emitters ( e . g . infrared LEDs ) for emitting infrared light radiation 170 integrated in the display 181 or its RGB pixels (not depicted) .

[0116] With reference to figure 13 , the system 100 and the abovedescribed method may be applied to capture an image reproducing the finger 300 that complies with a predefined criterion . In this case , the predefined criterion may be a predefined recognition level of an ( arbitrary) finger . After capturing the image , further processing may be performed using the image or its image data 220 in order to e . g . provide position information of the finger 300 used for finger tracking . I f the finger 300 moves and changes its position, as i llustrated in figure 13 by the dotted finger, this requires the acquisition of further images and corresponding image processing . In this context , the system 100 and method may again be used to capture the images in such a way that they comply with the predefined criterion . The system 100 and method may in this context promote the finger tracking, which may result i . a . from the finger 300 to be able to be better discriminated from background .

[0117] Figure 14 illustrates a further use case , in which the system 100 and method are applied on a mobile phone 185 in connection with providing a structured light source . In this regard, an illuminator 110 in the form of or constituted by a structured and addressable light flash of the mobile phone 185 is applied to emit light radiation 170 , and thus to provide di f ferent illumination patterns . Apart from the illuminator 110 , other components of the system 100 depicted in figure 1 may be installed on or integrated in the mobile phone 185 . Deviating from this , a generating module 150 or an arti ficial neural network 151 thereof may be reali zed as a cloud-based component . With the aid of the system 100 and by carrying out the method as described above , images may be captured that ful fill a predefined criterion . It is also possible to implement the above-mentioned configuration in which a plurality of predefined criteria and associated artificial neural networks 151 are applied . In this respect , the system 100 may be used for multiple tasks , and for each task, a di fferent arti ficial neural network 151 may be used or recalled, e . g . from the cloud . Such tasks may e . g . relate to color balance , face recognition, target depth estimation etc .

[0118] Figure 15 shows a further use case , in which the system 100 and method are applied on AR glasses 187 in connection with iris recognition or eye tracking . Here , an illuminator 110 of the system 100 may be reali zed at an outer edge of a frame of the AR glasses 187 , as depicted in figure 15 , or in a region of the spectacle lenses of the AR glasses 187 , i f the illuminator 110 is made from transparent materials . For the illuminator 110 , a configuration comprising emitters or l ightemitting pixels such as OLEDs or micro-LEDs may be considered . An image sensor unit 120 may be integrated in the frame of the AR glasses 187 , as well . Other components of the system 100 depicted in figure 1 may be installed on an arm of the AR glasses 187 . With the aid of the system 100 and by carrying out the method as described above , images may be captured ful filling a predefined criterion, in this case a predefined recognition level of an ( arbitrary) iris . In this application, the system 100 and method may promote iris recognition by enhancing the scoring accuracy, or promote eye tracking, which may depend on ambient light conditions .

[0119] Besides the embodiments described above and depicted in the figures , further embodiments are conceivable which may comprise further modi fications and / or combinations of features .

[0120] In this regard, deviating from the above-described configurations of an illuminator 110 and image sensor unit 120 , other configurations may be considered . In a similar way, apart from the predefined criteria mentioned above , other prede- fined criteria may be used in an analysis of a captured image ( step 202 ) . In addition, implementations and use cases other than those described above may be considered for the system 100 and method .

[0121] A further variant may be considered with regard to changing an image recording of an image sensor unit 120 . The image sensor unit 120 may comprise not only an image sensor 121 , but also a component for beam shaping upstream of the image sensor 121 such as receiving optics . For the case that the image sensor unit 120 is additionally configured to provide dif ferent settings or positions of the beam shaping component , changing an image recording setting applied in the method may also comprise changing a setting or position of the beam shaping component .

[0122] Moreover, with regard to applying a plurality of di f ferent predefined criteria in the method, reference is made to the possibility of performing the analysis of a captured image ( step 202 ) successively with respect to the plurality of the predefined criteria, and in the event that one of the predefined criteria is not satis fied, the generating of control information 222 for a modi fied image capture setting ( step 203 ) and capturing a subsequent image ( step 204 ) are performed . In this case , similar to the method variant described above , for the generating of the control information 222 , an arti ficial neural network 151 may be selected from a plurality of arti ficial neural networks 151 depending on the respective predefined criterion not being ful filled, each of the arti ficial neural networks 151 being associated with and trained in relation to a respective one of the predefined criteria .

[0123] Another variant with respect to applying a plurality of di fferent predefined criteria consists in carrying out the method in such a way that successively images of a target area 160 are captured in relation to each of the plurality of di fferent predefined criteria . In this regard, a sequence of im- ages is captured, wherein each image fulfills a respective one of the plurality of predefined criteria. Afterwards, a processed or composite image may be generated from the sequence of images using their image data 220, e.g. by stitch- ing or fusing these images. In this way, the composite image may fulfill several or all predefined criteria at the same time .

[0124] Although the invention has been more specifically illustrated and described in detail by preferred exemplary embodiments, nevertheless the invention is not restricted by the examples disclosed and other variations may be derived therefrom by a person skilled in the art, without departing from the scope of protection of the invention.

[0125] REFERENCE SYMBOLS system illuminator light-generating device light-emitting pixel subpixel micromirror device micromirror image sensor unit image sensor light-sensing pixel exposure time controller processing module generating module arti ficial neural network input layer hidden layers output layer target area subarea light radiation phase di f ference angular change bidirectional display display optics mobile phone AR glasses image image image method step method step method step method step method step 205 method step

[0126] 206 method step

[0127] 211 control signal

[0128] 212 control signal 220 image data

[0129] 221 input information

[0130] 222 control information

[0131] 225 output

[0132] 300 finger

Claims

CLAIMS1. A method for adapted image capture comprising: capturing a first image of a target area (160) by illuminating the target area (160) with an illuminator (110) and performing image recording of the target area (160) with an image sensor unit (120) , the illuminator (110) and the image sensor unit (120) being controlled according to a first image capture setting; analyzing the first image to determine whether it meets a predefined criterion; generating control information (222) for a second image capture setting if the predefined criterion is not met on the basis of input information (221) relating to the first image, the second image capture setting comprising a modification compared to the first image capture setting in order to provide an improvement in terms of meeting the predefined criterion; and capturing a second image of the target area (160) by illuminating the target area with the illuminator (110) and performing image recording of the target area (160) with the image sensor unit (120) , the illuminator (110) and the image sensor unit (120) being controlled according to the control information (222) generated on the basis of the input information (221) relating to the first image and thus according to the second image capture setting.

2. The method according to claim 1, wherein the generating of the control information is carried out using an artificial neural network (151) to which the input information (221) is fed, the artificial neural network (151) being trained to generate the control information (222) on the basis of the input infer-mation (221) .

3. The method according to any one of the preceding claims, wherein the modification of the image capture setting comprises at least modifying an operating mode of the illuminator (110) .

4. The method according to any one of the preceding claims, wherein the modification of the image capture setting comprises changing an illumination pattern provided by the illuminator (110) .

5. The method according to any one of the preceding claims, wherein the modification of the image capture setting comprises changing a spatial distribution of subareas (161) of the target area (160) that may be illuminated by the illuminator (110) .

6. The method according to any one of the preceding claims, wherein the modification of the image capture setting comprises changing an angle of emission of light radiation (170) that may be emitted from the illuminator (110) .

7. The method according to any one of the preceding claims, wherein the illuminator (110) comprises a plurality of emitters (112) configured to emit coherent light radiation (170) , and the modification of the image capture setting comprises changing a phase relationship between coherent light radiations (170) emitted from the emitters (112) .

8. The method according to any one of the preceding claims, wherein the illuminator (110) comprises a micromirror device (115) , and the modification of the image capture setting comprises changing a position of at least one micromirror (116) of the micromirror device (115) .

9. The method according to any one of the preceding claims, wherein the modification of the image capture setting comprises changing a color or color distribution of light radiation (170) that may be emitted from the illuminator (110) .

10. The method according to any one of the preceding claims, wherein the illuminator (110) comprises light-emitting pixels (112) each comprising subpixels (113) configured to emit different-colored light radiations (170) , the emission of light radiation (170) from the pixels (112) and subpixels (113) is carried out on the basis of a PWM control scheme, and the modification of the image capture setting comprises changing the PWM control scheme.

11. The method according to any one of the preceding claims, wherein the modification of the image capture setting comprises changing at least one of the following: an image recording setting of the image sensor unit (120) ; and a coding of exposure of the image sensor unit (120) .

12. The method according to any one of the preceding claims, wherein the predefined criterion is a predefined balance of spatial frequency content.

13. The method according to any one of the preceding claims, wherein the predefined criterion is a predefined blur level .

14. The method according to any one of the preceding claims, wherein the predefined criterion is a predefined color balance .

15. The method according to any one of the preceding claims, wherein the predefined criterion is a predefined recog-nition level of an object.

16. The method according to any one of the preceding claims, further comprising: selecting the predefined criterion from a plurality of predefined criteria; selecting an artificial neural network (151) from a plurality of artificial neural networks (151) depending on the selected predefined criterion, each of the artificial neural networks (151) being associated with a respective one of the plurality of predefined criteria; and performing the generating of the control information (222) by using the selected artificial neural network (151) and feeding the input information (221) to the selected artificial neural network (151) .

17. The method according to any one of the preceding claims, wherein the first image capture setting is selected from a plurality of different default image capture settings.

18. The method according to claim 17, wherein the selecting comprises successively capturing images of the target area (160) using the illuminator (110) and the image sensor unit (120) , hereby being controlled in a successive manner according to each of the different default image capture settings, analyzing the captured images in relation to meeting a predefined precriterion, and then setting one of the default image capture settings as the first image capture setting depending on which of the captured images best meets the predefined pre-criterion or comes closest to meeting the predefined pre-criterion.

19. The method according to any one of the preceding claims, wherein the second image is analyzed to determine whether it meets the predefined criterion, and if this is not the case, the method is repeated iteratively until the predefined criterion is met.

20. A system (100) for adapted image capture comprising an illuminator (110) , an image sensor unit (120) , a controller (130) for controlling the illuminator (110) and the image sensor unit (120) , a processing module (140) and a generating module (150) , the system (100) being configured to carry out the following steps: capturing a first image of a target area (160) by illuminating the target area (160) with the illuminator(110) and performing image recording of the target area (160) with the image sensor unit (120) , the illuminator (110) and the image sensor unit (120) being controlled by the controller (130) according to a first image capture setting; analyzing the first image to determine whether it meets a predefined criterion using the processing module (140) ; generating control information for a second image capture setting if the predefined criterion is not met using the generating module (150) to which input information (221) relating to the first image is fed, the second image capture setting comprising a modification compared to the first image capture setting in order to provide an improvement in terms of meeting the predefined criterion, and the generating module (150) being configured to generate the control information (222) on the basis of the input information (221) ; and capturing a second image of the target area (160) by illuminating the target area (160) with the illuminator(110) and performing image recording of the target area (160) with the image sensor unit (120) , the illuminator (110) and the image sensor unit (120) being controlled by the controller (130) according to the control infor- mation (222) generated by the generating module (150) on the basis of the input information (221) relating to the first image and thus according to the second image capture setting.

Images