Method and device for acquiring a stack of images of a scene with improved luminosity

Abstract

The invention relates to a method (100) for acquiring a stack of images of a scene at different focusing distances, the method (100) furthermore comprising, for at least one of the focusing distances, an acquisition phase (120) comprising the following steps: - comparing (126) a luminosity value of at least part of the scene located at a depth of field associated with the focusing distance with at least one predetermined luminosity threshold value, - when the luminosity value does not satisfy the at least one luminosity threshold value, adjusting (128) the luminosity for acquiring an image of the scene at the focusing distance, by modifying at least one parameter of the device, and - acquiring (130) an image of the scene at the focusing distance. The invention also relates to a computer program, a device, a system and a vehicle implementing such a method.

Description

DESCRIPTION Title: Method and device for acquiring a stack of images of a scene with enhanced brightness

[0001] The present invention relates to a method for obtaining a stack of images of a scene with enhanced, or balanced, brightness. It also relates to a computer program and a device implementing such a method. It further relates to an apparatus and a vehicle implementing such a method.

[0002] The field of the invention is the field of obtaining a stack of images of a scene comprising images acquired at different focusing distances. State of the art

[0003] We know the technique, called focus bracketing, or focus stacking, which allows capturing a stack of images of a scene, each image being captured at a different focusing distance, or focus. Then, the scene can be represented on a display medium, with a greater depth of field, by exploiting the stack of images, for example by combining them or displaying them in turn.

[0004] This technique, although improving the depth of field, has drawbacks and can still be improved. Indeed, in a scene, objects at different depths are not necessarily in the same imaging conditions, and in particular in the same lighting conditions, which produces a lack of details. However, the focus bracketing technique does not take these differences into account and provides a stack of images of the scene including images whose qualities vary depending on the focuses used, in particular in terms of brightness. This can generate a significant difference in brightness between the images in the stack. of images. Such a difference in brightness results in a rendering that is degraded when representing the scene from the image stack.

[0005] An aim of the present invention is to remedy at least one of the aforementioned drawbacks.

[0006] Another aim of the invention is to propose a solution for imaging a scene and providing a stack of images of the scene of better quality, in particular in terms of brightness or lighting. Statement of the invention

[0007] The invention proposes to achieve at least one of the aforementioned aims by a method of acquiring a stack of images of a scene comprising images of said scene at different focusing distances, with an apparatus comprising a camera module formed by an optical lens associated with an image sensor; said method further comprising, for at least one of said focusing distances, an acquisition phase comprising the following steps: - comparison of a brightness value of at least one part of said scene located at a depth of field associated with said focusing distance, with at least one predefined brightness threshold value, - when said brightness value does not satisfy said at least one brightness threshold value, adjusting the brightness for the acquisition of an image of said scene at said focusing distance, by modifying at least one parameter of said device for, and - acquisition of an image of said scene at said focusing distance.

[0008] Thus, like known techniques, the invention makes it possible to image a scene to obtain a stack of several images captured at different focusing distances, or at different focuses. Such a stack of images makes it possible to represent the scene with a greater depth of field.

[0009] In addition to what is proposed by current techniques, the invention proposes to adjust the brightness with which the scene is imaged for at least one focusing distance, as a function of the brightness received by the apparatus for the parts of the scene located at the depth of field associated with said focusing distance. Thus, the invention makes it possible to image a scene more efficiently and provide a stack of images of the scene whose quality, and in particular the brightness, is better, or at least better harmonized and balanced for the entire stack of images, compared to current techniques. With the present invention, it is possible to acquire a stack of images of the scene while avoiding certain parts of the scene being saturated and other parts of the scene being under-illuminated. The invention makes it possible to better control and adjust the brightness when acquiring a stack of images of the scene.

[0010] By image, we mean a digital image, and in particular a matrix image, and more particularly an RGB matrix image for example.

[0011] Depth of field refers to the extent of the area of ​​sharpness that appears in an image, that is, the area between the sharp foreground and the sharpest plane of the image.

[0012] The term "sharpness range" refers to the distance over which the sharp part of the image extends, that is, the distance from the sharp foreground to the sharpest last plane of the image.

[0013] Focusing distance refers to the distance at which an optical lens is focused, with reference to the position of the lens. The focusing distance for capturing an image is generally adjusted by changing the distance between the image sensor and the optical lens. Thus, a first image of a scene acquired at a first focusing distance will clearly represent a first part of the scene, and a second image of a scene acquired at a second focusing distance will clearly represent a second part of the scene.

[0014] According to a non-limiting example of embodiment, given as an illustration of the definitions indicated above, the focusing distance can be adjusted to 15 meters. The sharpness range can be 1.50 meters. and the depth of field can be 14.50 meters - 16 meters. In this case, the image will clearly represent all objects, or parts of the scene, located at a distance of between 14.5m and 16m from the optical lens.

[0015] In the following, "brightness value associated with a focusing distance" or "brightness value for a focusing distance" means the brightness value of at least a part of the scene located in the depth of field associated with said focusing distance.

[0016] The at least one predetermined threshold value may be, or may include, a minimum brightness value to be achieved.

[0017] The at least one predetermined threshold value may be, or may include, a maximum brightness value not to be exceeded.

[0018] Preferably, the at least one predetermined threshold value may be, or may comprise, a predetermined interval of brightness values ​​to be satisfied, said interval being bordered by a minimum brightness threshold value and by a maximum brightness threshold value. Thus, the scene is imaged and several images of the scene are obtained, each for a different focusing distance, and each representing the scene with a brightness value included in an interval. This makes it possible to obtain a stack of images of the scene, said images being acquired at different focusing distances, the brightness of which is more uniform and harmonized, regardless of the focusing distance.

[0019] The at least one threshold value may be determined independently of the scene. In embodiments, the at least one threshold value may be determined as a brightness value, or a range of brightness values, desired for imaging the scene.

[0020] The at least one threshold value may be determined as a function of the brightness of the scene, or as a function of the scene. For example, an average brightness value of the scene may be determined / measured, and an interval of brightness values, of a given width, may be centered on said average value. For example, if the average brightness value of the scene is VLM, then a brightness interval can be determined as being (Vmin-Vmax), with Vmin= VLM-X%VLM and Vmax=VLM+Y%VLM, with X=Y, or X*Y. According to a non-limiting example X=Y=10. In this case, the scene will be imaged and a stack of images of the scene will be acquired comprising images at different focusing distances, and whose brightness is controlled and varies little from one image to another.

[0021] The average brightness of the scene can be calculated from an exploratory image of the scene captured prior to the acquisition of the image stack. Alternatively, or in addition, the average brightness of the scene can be measured with a sensor provided for this purpose and equipped in the device used to image the scene.

[0022] According to embodiments, for at least one focusing distance, the brightness adjustment can achieve an increase in brightness during image acquisition at said focusing distance.

[0023] Indeed, the scene may include certain areas that are under-illuminated, compared to the rest of the scene. When, for a focusing distance, the depth of field corresponds to these areas, or includes these areas, in this case the brightness can be increased for the acquisition of the image at said focusing distance. Thus, the image acquired for said focusing distance will represent the scene with increased, or improved, lighting for said areas, so as to reduce the difference in lighting between these areas and the rest of the scene.

[0024] According to embodiments, for at least one focusing distance, the brightness adjustment can achieve a decrease in brightness during image acquisition at said focusing distance.

[0025] Indeed, the scene may include certain areas that are overlit, compared to the rest of the scene. When the depth of field associated with a focusing distance corresponds to these areas, or includes these areas, the brightness may be reduced for the acquisition of the image at said focusing distance. Thus, the image acquired for said focusing distance focus will represent the scene with diminished or reduced lighting for said areas, so as to reduce the difference in lighting between these areas and the rest of the scene.

[0026] According to embodiments, the brightness adjustment can achieve: - a trigger, and / or - an adjustment of the average intensity: of a means of lighting the scene, and in particular of a flash, equipping the imaging device.

[0027] For example, for at least one focusing distance, when acquiring the image of the scene at said focusing distance, the flash may be triggered to increase the brightness of the scene.

[0028] For example, for at least one focusing distance, when acquiring the image of the scene at said focusing distance, the flash may be triggered and the average intensity of said flash may be adjusted, up or down, to illuminate the scene, and thus adjust the brightness of the scene for acquiring an image at said focusing distance.

[0029] According to an exemplary embodiment, the average intensity of the flash can be adjusted by adjusting the instantaneous intensity of said flash. In this case, the flash emits a more or less strong light to illuminate the scene.

[0030] According to an exemplary embodiment, the average intensity of the flash may be adjusted by time modulation of said flash. In this case, the flash emits light whose intensity is constant, but the total emission time of the flash is adjusted upwards or downwards to increase or decrease the light emitted towards the scene. Such modulation of the flash may be achieved with a flash that performs sequential light emission, for example in the form of a series of light pulses, and the time modulation may be achieved by adjusting either the emission frequency or the width of each pulse.

[0031] According to embodiments, adjusting the brightness may effect a modification of an exposure time of the image sensor.

[0032] Indeed, the longer the exposure time of the image sensor, the more light the image sensor collects from the scene, and therefore the more the scene is imaged with increased brightness, and vice versa. From this, for at least one focusing distance, the brightness with which the scene is imaged at said focusing distance can be adjusted upwards or downwards, by adjusting the exposure time of the image sensor upwards or downwards.

[0033] According to embodiments, the adjustment of the brightness can result in a modification of the opening of a diaphragm equipping the device.

[0034] Indeed, the larger the aperture of the diaphragm, the more light the image sensor collects from the scene, and therefore the more the scene is imaged with increased brightness, vice versa. From this, for at least one focusing distance, the brightness with which the scene is imaged at said focusing distance can be adjusted upwards or downwards, by adjusting the aperture of the diaphragm upwards or downwards.

[0035] According to embodiments, the method according to the invention may comprise a step of determining a depth map of the scene, the focusing distances for which the images are acquired being selected as a function of said depth map.

[0036] The depth map of the scene can be determined using any known technique.

[0037] According to embodiments, the depth map can be determined by analyzing an image of said scene, called an exploratory image, taken prior to the acquisition of the stack of images, and in particular prior to the first acquisition phase.

[0038] The exploratory image can be analyzed by any known technique to determine the depths of objects in the scene. Thus, a depth map of the scene can be obtained, indicating the depths of different objects in the scene.

[0039] Following a first example, it is possible to use the data captured by the dual pixels of the image sensor used to capture said exploratory image. A dual pixel is a pixel split into two half pixels, noted "pa" and "pb" in the following, whose micro lens placed just above, therefore illuminates for one half the half-pixel pa and for the other half the half-pixel pb. When the image is in focus, the intensity captured by the half-pixel, Ipa, is equal to the intensity, noted Ipb, captured by the half-pixel pb. When the image is out of focus, we have a difference whose direction is linked to the direction of difference in focus: for example Ipa>Ppb when the observed points are further than the focusing distance. And when Ipa<Ipb, the observed points are closer.These half-pixels are therefore used to focus on an area of ​​the image: the focusing motor / mechanism is actuated in the direction indicated by (Ipa-Ipb) in the area considered, and is stopped when (Ipa-Ipb) reaches zero or is below a certain threshold (absolute value (Ipa-Ipb) < threshold). These dual pixels can also be used to find the focus difference of the area considered. It is then necessary to analyze a set of pixels in the area, to find a certain (blurred) image on the pa pixels of the area, to find the corresponding image on the pb pixels of the area. Comparing the 2 images provides an idea of ​​the defocus, and knowing the focus distance, we can deduce the depth(s) of the area, and therefore of each object in the area. Following a second example, it is possible to use the optical transfer functions, in particular the PSF (for "Point Spread Function") of the optical lens used to capture the image.Indeed, the PSF of the optical lens varies with the distance between the optical lens and the object in the scene. By knowing the different PSFs of the optical lens, it is possible to determine the depth of each object in the scene in the exploratory image by convolution / deconvolution of said exploratory image with the different PSFs. Of course, other techniques are possible and the invention is not limited to the examples that have just been described.

[0040] According to embodiments, the depth map of the scene can be determined with at least one sensor to measure the depths of objects in the scene.

[0041] Such a sensor can be a lidar, a time-of-flight camera, etc.

[0042] Such a sensor can equip the device used to image the scene.

[0043] The method according to the invention comprises, for at least one, and in particular each, focusing distance, a step of determining the brightness of the scene for said focusing distance, or for at least one area / part of the scene located in the depth of field associated with said focusing distance.

[0044] According to embodiments, for at least one focusing distance, the brightness value of the scene at said focusing distance can be determined by analyzing an exploratory image of the scene taken prior to the acquisition of the stack of images, and in particular prior to the first acquisition phase.

[0045] In this case, the areas of the scene falling within the depth of field associated with said focusing distance are identified on said exploratory image. The brightness of said areas can then be calculated from the brightness values ​​of each pixel of the exploratory image corresponding to said areas.

[0046] According to embodiments, for at least one focusing distance, the value of the brightness of the scene at said focusing distance can be determined by analyzing an image, called a preliminary image, of the scene taken at said focusing distance.

[0047] In this case, the preliminary image is taken at the said focusing distance. The sharp areas of the scene are determined on the said image. The brightness value of the scene can then be determined from the brightness values ​​of the pixels belonging to the sharp areas of the said preliminary image.

[0048] In the preliminary image, and generally in an image of the scene, the sharp areas of the scene can be determined by any known technique. According to a non-limiting exemplary embodiment, the sharp areas of the scene can be determined for example by analysis of local variance of the image: the areas correspond to the areas for which this local variance is greater than a certain predefined threshold.

[0049] The brightness value of a pixel in an image can, for example, correspond to an average of the values ​​of said pixel.

[0050] For example, for each pixel in an RGB color image, there is a value for each color: a value for red, a value for green, and a value for blue. In this case, the brightness value for that pixel can be calculated as an average of the red, green, and blue values ​​for that pixel. Of course, this example is for illustration purposes only, and other brightness calculation techniques can be used.

[0051] According to embodiments, for at least one focusing distance, the method according to the invention may comprise: - a flash-free acquisition of a first image of the scene at said focusing distance, and - a flash acquisition of a second image of the scene at the said focusing distance.

[0052] In this case, the image stored in the image stack for said focusing distance may be either the first image taken without flash, or the second image taken with flash, or an image obtained from said first and second images, for example an image corresponding to the average of said images.

[0053] According to embodiments, at least one image of the scene may be a 2D image.

[0054] According to embodiments, at least one image of the scene may be a 3D image.

[0055] According to another aspect of the same invention, there is provided a computer program comprising executable instructions which, when executed by a computer device, implement all the steps of the method according to the invention.

[0056] The computer program can be in any computer language, such as machine language, C, C++, JAVA, Python, etc.

[0057] Such a computer program may be presented as a standalone application. Alternatively, such a computer program may be integrated into a photo or video application, or into an image or video playback application.

[0058] According to another aspect of the invention, a device is proposed comprising means configured to implement all the steps of the method according to the invention.

[0059] The device according to the invention can be, or be integrated into, any type of device such as a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0060] According to another aspect of the invention, there is provided an apparatus comprising: - at least one camera module, - at least one computing unit; configured to implement all the steps of the method according to the invention.

[0061] According to embodiments, the apparatus may further comprise a flash, and optionally a mechanism for adjusting the average intensity of the flash.

[0062] According to embodiments, the apparatus may further include a mechanism for adjusting the exposure time of the image sensor.

[0063] According to embodiments, the apparatus may further include a diaphragm and a mechanism for adjusting the aperture of said diaphragm.

[0064] In particular, the device may be a user device such as a smartphone, tablet, etc. comprising a display screen.

[0065] In this case, the user device may further comprise a display screen, a capacitive sensing surface, etc.

[0066] In particular, the device may be a computer-type user device.

[0067] In this case, the computer-type user device may comprise a display screen, a touch surface, in particular integrated into, or associated with, the display screen of said computer, etc.

[0068] In particular, the device may be a television.

[0069] In particular, the device may be a virtual reality headset or an augmented reality headset.

[0070] In this case, the headset may include a screen for displaying at least one image, one or more sensors, particularly optical sensors, etc.

[0071] In particular, the device may be a medical imaging device.

[0072] In particular, the medical imaging device may be an endoscope, an ultrasound device, etc.

[0073] Of course, the apparatus according to the invention is not limited to the examples which have just been given.

[0074] According to another aspect of the present invention, there is provided a vehicle comprising: - at least one camera module, and - at least one computing unit; configured to implement all the steps of the method according to the invention.

[0075] According to embodiments, the vehicle may further include a flash, and optionally a mechanism for adjusting the average intensity of the flash.

[0076] According to embodiments, the vehicle may further include a mechanism for adjusting the exposure time of the image sensor.

[0077] According to embodiments, the vehicle may further have a diaphragm and a mechanism for adjusting the opening of said diaphragm.

[0078] According to embodiments, the vehicle may be a ground vehicle, such as a car, which is autonomous, semi-autonomous or non-autonomous.

[0079] According to embodiments, the vehicle may be a flying vehicle, such as a drone, an airplane, a helicopter, autonomous, semi-autonomous or non-autonomous.

[0080] According to embodiments, the vehicle may be a maritime vehicle, such as a boat or a submarine, which is autonomous, semi-autonomous or non-autonomous. Description of figures and embodiments

[0081] Other advantages and characteristics will appear on examining the detailed description of non-limiting embodiments, and the attached drawings in which: - FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the invention; - FIGURE 2 is a schematic representation of another non-limiting example of embodiment of a method according to the invention; - FIGURE 3 is a schematic representation of a non-limiting exemplary embodiment of a device according to the invention; - FIGURES 4-6 are schematic representations of non-limiting exemplary embodiments of an apparatus according to the invention; and - FIGURE 7 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the invention.

[0082] It is understood that the embodiments which will be described below are in no way limiting. In particular, it is possible to imagine variants of the invention comprising only a selection of characteristics described below isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection includes at least one preferably functional characteristic without structural details, or with only part of the structural details if it is this part which is only sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.

[0083] In particular, all the variants and embodiments described can be combined with each other if there is no technical obstacle to this combination.

[0084] In the figures and in the rest of the description, the elements common to several figures retain the same reference.

[0085] FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the present invention.

[0086] The method 100 of FIGURE 1 can be used to obtain, with a device, a stack of images of the scene, comprising images acquired at different focusing distances, the brightnesses of which are improved and in particular harmonized.

[0087] The apparatus used to image the scene comprises a camera module comprising an image sensor associated with an optical lens. Optionally, but preferably, the apparatus comprises a flash and possibly a first mechanism for adjusting the average intensity of the flash. Optionally, but preferably, the apparatus comprises a second mechanism for adjusting the exposure time of the image sensor. Optionally, but preferably, the apparatus comprises a diaphragm, and possibly a third mechanism for adjusting the opening of said diaphragm. Of course, the apparatus may include means other than those indicated here, instead of or in combination with at least one of the means listed above.

[0088] The method 100 comprises an optional step 102 for establishing a depth map of the scene. Such a depth map can be established in different ways.

[0089] In the example shown in FIGURE 1, the depth map is established from an exploratory image of the scene. In this case, an image of the scene is acquired by the device during a step 104: this image is called the first exploratory image. The first exploratory image is analyzed during a step 106 to determine the depth of the objects in the scene. For example, a depth value is determined for each object in the scene, indicating the distance at which said object is located relative to the device used to image the scene.

[0090] Determining the depth of an object in an image can be done using any known technique. As a first example, it is possible to use the data captured by the dual pixels of the image sensor used to capture the image. A dual pixel is a pixel split into two half pixels, denoted "pa" and "pb" in the following, whose micro lens placed just above, therefore illuminates half the half-pixel pa and half the half-pixel pb. When the image is in focus, the intensity captured by the half-pixel, Ipa, is equal to the intensity, denoted Ipb, captured by the half-pixel pb. When the image is out of focus, there is a difference whose direction is linked to the direction of the difference in focus: for example Ipa>Ppb when the observed points are further away than the focusing distance. And when Ipa<Ipb, the observed points are closer.These half-pixels are therefore used to focus on an area of ​​the image: the focusing motor / mechanism is actuated in the direction indicated by (Ipa-Ipb) in the area considered, and is stopped when (Ipa-Ipb) reaches zero or is below a certain threshold (absolute value (Ipa-Ipb) < threshold). These dual pixels can also be used to find the focus difference of the area considered. It is then necessary to analyze a set of pixels in the area, to find a certain (blurred) image on the half-pixels pa of the area, to find the corresponding image. on the half-pixels pb of the area. Comparing the 2 images provides an idea of ​​the defocus, and knowing the focus distance, we can deduce the depth(s) of the area, and therefore of each object in the area. According to a second example, it is possible to use the optical transfer functions, in particular the PSF (for "Point Spread Function") of the optical lens used to capture the image. Indeed, the PSF of the optical lens varies with the distance between the optical lens and the object in the scene. By knowing the different PSFs of the optical lens, it is possible to determine the depth of each object in the scene by convolution / deconvolution with the different PSFs. Of course, other techniques are possible and the invention is not limited to the examples just described.

[0091] The depth map can alternatively be determined by other techniques, for example using a time-of-flight sensor, or a lidar sensor, equipping the device used to image the scene.

[0092] Thus, at the end of step 106 a map indicating the depths of the objects in the scene is determined. This depth map is used to choose the focusing distances for which the scene is imaged in order to obtain the image stack.

[0093] This step 102 is optional because the depth map can be provided by another device. Alternatively, the method 100 may not use a depth map for acquiring the image stack: in this case, the focusing distances for which the scene is imaged in order to obtain the image stack can be determined randomly, or according to a predetermined rule or predetermined focusing distance values.

[0094] The method 100 then comprises an optional step 110 for determining at least one brightness threshold value which will be used in the remainder of the method for imaging the scene.

[0095] In the example shown in FIGURE 1, the at least one brightness threshold value is a function of the scene, and in particular of the brightness of the different parts of the scene.

[0096] In this case, an image of the scene is acquired by the device during a step 112: this image is called the second exploratory image. This second exploratory image is acquired at an intermediate focusing distance from the scene, for example determined from a depth map of the scene. This depth map may for example be the one determined in step 102, or provided by another device. Alternatively, the second exploratory image may be acquired at a randomly chosen focusing distance.

[0097] This second exploratory image is analyzed, during a step 114, to determine an average brightness of the scene. According to a non-limiting exemplary embodiment, the average brightness can be determined as being the average of the brightness values ​​for each pixel of the second exploratory image. According to a non-limiting exemplary embodiment, the brightness value of a pixel can be the average of the values ​​representing said pixel.

[0098] In a step 116, at least one brightness threshold value may be calculated based on the average brightness value obtained in step 114. For example, a minimum brightness threshold value may be determined. For example, a maximum brightness threshold value may be determined. According to a non-limiting example, and without loss of generality: - the minimum brightness threshold value may correspond to a certain percentage X%, in particular 90%, of said average brightness value; and / or - the maximum brightness threshold value may correspond to a certain percentage Y%, in particular 110%, of said average brightness value. In the following, it is considered, without loss of generality, that step 116 provides an interval of brightness values ​​defined by a minimum threshold value and a maximum brightness threshold value.

[0099] According to alternative embodiments, the at least one brightness threshold value may be determined differently from what is described above.

[0100] According to alternative embodiments, step 110 is not performed and the at least one brightness threshold value may be predetermined and previously provided.

[0101] The method 100 then comprises a phase 120 of acquiring images of the scene at different focusing distances. This acquisition phase 120 is carried out individually, for example in turn, for each focusing distance at which the scene is to be imaged, for example in turn.

[0102] The acquisition phase 120 comprises a step 122 during which the imaging module is set to the focusing distance for which the scene is to be imaged. For example, the distance between the optical lens and the image sensor can be adjusted to obtain said focusing distance.

[0103] When the imaging module is adjusted for the correct focusing distance, the acquisition phase 120 comprises a step 124 of acquiring an image, called a preliminary image, of the scene at said focusing distance.

[0104] In a step 125, the preliminary image is used to determine the brightness of the part of the scene located in the depth of field associated with the focusing distance, or of at least one area of ​​said part of the scene. To do this, the sharp areas of the preliminary image are identified: these sharp areas of the image corresponding to the part of the scene located at the depth of field associated with the focusing distance. The brightness value of the part of the scene, respectively of an area of ​​the scene, located at the depth of field associated with said focusing distance, can then be determined from the brightness values ​​of the pixels corresponding to said part of the scene, respectively to said area of ​​the scene.

[0105] The brightness calculated in step 125 is compared, in a step 126, to at least one brightness threshold value. The at least one brightness threshold value may have been determined in the optional step 110. Alternatively, the at least one brightness threshold value may have been previously provided, for example by the user.

[0106] If the brightness value calculated in step 125 satisfies the at least one brightness threshold value, for example if it is within an interval of brightness threshold values, then the preliminary image is retained as the final image, at said focusing distance, or another image can be taken under the same conditions at the focusing distance, and the acquisition phase for this focusing distance is terminated. A new acquisition phase for another focusing distance can then be carried out.

[0107] If the brightness value calculated in step 125 does not satisfy the at least one brightness threshold value, for example if it is not within a threshold value interval, then the brightness is adjusted in a step 128 for the acquisition of a new image of the scene at the focusing distance.

[0108] In step 128, brightness adjustment can be achieved in a number of ways.

[0109] According to an exemplary embodiment, the brightness of the scene can be adjusted by triggering a flash to illuminate the scene, during a step 128i. In addition, optionally, when the flash is triggered, it is possible to adjust the average intensity of the flash, during a step 1282: - either by adjusting the instantaneous intensity of the flash, - either by performing a temporal modulation of the flash.

[0110] Alternatively, or in addition, the brightness at which the scene will be imaged can be adjusted by changing the total exposure time of the image sensor in a 128s step. The total exposure time of the image sensor can be adjusted by any known means, for example by controlling the opening time of a shutter located in front of the sensor or in front of the camera module. [YES] Alternatively, or in addition, the brightness with which the scene will be imaged may be adjusted by changing the aperture of a diaphragm of the camera module, or of the image sensor, in a step 1284. The aperture of the diaphragm may be adjusted by any known means, for example by controlling a mechanism controlling the size of the aperture of the diaphragm.

[0112] Of course, the brightness with which the scene is to be imaged can be adjusted by performing all of steps 1281-1284, or any combination of at least two of these steps, or only one of these steps 1281-1284. Steps 1281-1284 can be performed in the order in which they have just been described or in another order.

[0113] The brightness adjustment can be carried out according to a predetermined relationship taking as input: - the scene brightness value calculated in step 125; and - at least one brightness threshold value, and providing at least one of the following values ​​as output: - flash triggering or not, - an average flash intensity value, - an exposure time of the image sensor, - an opening of the diaphragm. The predetermined relationship can be any type of relationship. For example, the predetermined relationship can be a mathematical relationship. The predetermined relationship can be a lookup table. The predetermined relationship can be an artificial intelligence model, such as a neural network, previously trained for this purpose.

[0114] According to a non-limiting example embodiment, the value provided may be the intensity of the flash, noted I_FLASH, varying between 0 and I_FLASH_MAX. We observe, for example by means of switching on the flash before taking an image, the variation in brightness linked to the action on I_FLASH. Thus, the relationship below can be known: LUM_ECL = LUM_AMB + L_SUR_F * I_FLASH. Or : - LUM_ECL is the brightness obtained under artificial lighting - the flash - and ambient lighting, on the considered part of the scene; and - LUM_AMB is the same brightness obtained when the flash is off. Assuming a linear relationship between brightness and the flash control parameter, the L_SUR_F parameter provides the proportionality coefficient between the brightness obtained and the flash control. Then the predetermined relationship FLASH_ACTION (CURRENT_LUM, L_OBJ) is written: FLASH_ACTION(CURRENT_LUM,L_OBJ) = (L_OBJ-CURRENT_LUM) / L_SUR_F Or : - L_OBJ is the brightness to obtain, and - LUM_ACTUELLE is the brightness before using the process. This gives an I_FLASH value to apply to the flash for the next image capture. Of course, this example is given for informational purposes only and is in no way limiting.

[0115] The acquisition phase 120 then comprises a step 130 of acquiring an image of the scene, at the focusing distance, and with at least one brightness adjustment parameter determined in step 128, for example: - by triggering the flash, possibly with an average intensity corresponding to the value calculated during step 1282, and / or - with a sensor exposure time calculated in step 128s, and / or - with a diaphragm opening calculated in step 1284.

[0116] In a step 132, the image acquired in step 130 with brightness adjustment is stored in an image stack. Optionally, the preliminary image acquired in step 124 may also be stored.

[0117] The acquisition phase 120 is repeated for each focusing distance for which an image of the scene is to be acquired.

[0118] When the acquisition phase 120 is performed for several, and in particular all, focusing distances, an image stack of the scene is obtained, with harmonized brightness. The brightness of the scene on all the images of the image stack satisfies the at least one threshold value. Thus, the scene is imaged in the image stack with improved brightness and the image stack does not include an image on which the scene is underlit or the scene is overlit.

[0119] FIGURE 2 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0120] The method 200 of FIGURE 2 may be used to obtain, with an apparatus, a stack of images of the scene, including acquired images at different focusing distances, the brightness of which is improved and particularly harmonized.

[0121] The method 200 of FIGURE 2 includes all of the steps of the method 100 of FIGURE 1, except for the differences noted below.

[0122] The method 200 does not include the step 124 of acquiring a preliminary image at the focusing distance, nor the step 125 of calculating the brightness value of the scene at said focusing distance.

[0123] Instead, the method 200 comprises a step 202 calculating the brightness of the scene for all the focusing distances at which it is desired to take an image of the scene. This step 202 takes as input an exploratory image of the scene, for example the first exploratory image or the second exploratory image, or another exploratory image acquired during said step 202. In the following, without loss of generality, it is considered that step 202 takes as input the second exploratory image.

[0124] In the second exploratory image, for each focusing distance, the areas of the image corresponding to the depth of field associated with said focusing distance are identified. The brightness value is calculated for the focusing distance, from the brightness of each of the pixels belonging to said areas, for example as an average of the brightnesses of the pixels belonging to said areas.

[0125] An area of ​​the exploratory image located at a depth of field associated with a given focusing distance can be identified using any known technique. According to a non-limiting exemplary embodiment, it is possible to use the optical transfer functions, PSF, (for "Point Spread Function") of the optical lens of the camera module. The exploratory image is first deconvolved with the PSF corresponding to the focusing distance with which said exploratory image is acquired, then convolved with the PSF specific to the given focusing distance to identify the area(s) of the image located at the depth of field associated with the desired focusing distance: this or these areas are sharp in the image obtained after convolution with the PSF specific to the given focusing distance.

[0126] Thus, step 202 provides a brightness value for each focusing distance for which the scene is to be imaged. These calculated brightness values ​​are stored, each with the corresponding focusing distance.

[0127] In method 200, step 126 compares the brightness value calculated in step 202 for the focusing distance to the at least one threshold value. If the brightness value is not satisfactory, steps 128-132 are performed. If the brightness value is satisfactory, then an image of the scene is acquired at said focusing distance, without brightness adjustment, during a step 204. This image is stored during step 132 in the image stack.

[0128] The acquisition phase 120 is repeated for all the desired focusing distances in order to constitute a stack of images of the scene comprising images acquired at different focusing distances.

[0129] In the method 200, when the brightness value is not satisfactory in step 126, optionally, an image of the scene may be acquired with the unsatisfactory brightness, i.e., without brightness adjustment, in addition to the image acquired in step 130 with adjusted brightness. This image may be stored, in addition to the brightness-adjusted image. This image may, for example, be used to process the image acquired in step 130 with adjusted brightness. Alternatively, or in addition, this image may be retained for other reasons.

[0130] In methods 100 and 200, optionally, the depth map of the scene is determined from the first exploratory image. Alternatively, the depth map of the scene can be determined with at least one sensor measuring the depths of the objects in the scene, for example a LIDAR type sensor, time-of-flight camera, etc.

[0131] In methods 100 and 200, optionally, the brightness of the scene, and in particular the average brightness of the scene or the brightness of the scene at a focusing distance, is determined from image(s) of the scene. Alternatively, the average brightness of the scene, respectively the brightness of the scene for a given focusing distance can be determined with at least one sensor measuring said brightness, for example a light sensor equipping the device, or the image sensor of the camera module without acquiring an image of the scene, etc.

[0132] FIGURE 3 is a schematic representation of a non-limiting exemplary embodiment of a device according to the present invention.

[0001] The device 300 of FIGURE 3 can be used to obtain a stack of images of the scene, comprising images acquired at different focusing distances, the brightnesses of which are improved and in particular harmonized.

[0002] The device 300 of FIGURE 3 can be used to implement a method according to the invention, and in particular any one of the methods 100 or 200 of FIGURES 1 and 2.

[0003] The device 300 comprises a calculation unit 302 implementing the steps of the method according to the invention.

[0004] The calculation unit 302 comprises an optional module 304 for determining a depth map of the scene by analyzing an exploratory image of the scene. This module 304 is, for example, configured / programmed to carry out step 102 of the methods 100 or 200. Alternatively, the module 304 can be replaced by a sensor, such as a lidar or a time-of-flight camera, possibly associated with a processing module, for determining the depth map.

[0005] The calculation unit 302 further comprises an optional module 306 for determining at least one brightness threshold value, by analyzing a second exploratory image of the scene. This module 306 is for example configured / programmed to carry out step 110 of the methods 100 and 200. Alternatively, the module 306 can be replaced by a sensor, such as an optical sensor, possibly associated with a processing module, for determining the at least one brightness threshold value. According to yet another alternative, the module 306 can be coupled to the image sensor used for acquiring images of the scene, in order to determine the at least a brightness threshold value from the brightness captured by said image sensor without acquiring an image.

[0006] The calculation unit 302 further comprises an optional module 308 for determining the brightness of the parts of the scene located at a depth of field associated with a given focusing distance, in other words a brightness value associated with a focusing distance. This module 308 is in particular configured / programmed to carry out step 126 of the method 100, or step 202 of the method 200 of FIGURE 2.

[0007] The calculation unit 302 comprises a module 310 for adjusting the brightness of the scene for acquiring an image of the scene at a given focusing distance. This module 310 may for example be configured to implement step 128 of the methods 100 and 200.

[0008] The computing unit 302 further comprises a module 312 performing the adjustment of the imaging module to a focusing distance, for example by adjusting the distance between the image sensor and the lens. This module 312 may for example be configured to implement step 122 of the methods 100 and 200. This module 312 may be part of the device 300. Alternatively, this module 312 may exist in the apparatus used to image the scene. In this case, the device 300 may use such a module existing in the apparatus.

[0009] The calculation unit 302 further comprises a module 314 controlling the triggering of the acquisition of an image of the scene, this image being able to be an exploratory image, a preliminary image, a definitive image at a given focusing distance, etc. This module 314 can for example be configured to trigger the acquisition of image(s) during at least one of the steps 104, 112, 124, 130, 204 of the methods 100 and 200. This module 314 can be part of the device 300. Alternatively, this module 314 can exist in the apparatus used to image the scene. In this case, the device 300 can use such a module existing in the apparatus.

[0010] At least one of the modules 304-314 may be a module independent of the others.

[0011] At least two of the modules 304-314 can be integrated within the same module.

[0012] At least one of modules 304-314 may be a hardware module, such as a processor, a microchip, etc.

[0013] At least one of the modules 304-314 may be a software module, such as a computer program.

[0014] At least one of the modules 304-314 may be a combination of at least one software module and at least one hardware module.

[0015] In particular, at least one of the modules 304-314 can be integrated into an electronic chip, or even into an application installed in a user device.

[0016] In particular, the calculation unit 302 may be, or may be integrated, in an electronic chip, or even in an application installed in a user device.

[0017] The device 300 may further comprise, optionally, an image storage means 316, for storing the acquired images. This module 314 may for example be configured to store an image, or images, during step 132 of the methods 100 and 200. This storage means 316 is optional because such a storage means may already exist in the apparatus used to image the scene. In this case, the device according to the invention may use such a module existing in the apparatus.

[0018] The device 300 further comprises at least one image acquisition means 318, such as a camera module, comprising an optical lens and an image sensor, for acquiring an image or a stack of images of the scene. Such an image acquisition means 318 may be part of the device 300. Alternatively, such an image acquisition means 318 may exist in the apparatus used to image the scene. In this case, the device 300 may use such an image acquisition means 318 equipping the apparatus or cooperate with said image acquisition means equipping the apparatus. According to yet another alternative, the device 300 may be connected to an external image acquisition means, or to an external apparatus having an image acquisition means 318 or itself connected to an image acquisition means 318.

[0019] Optionally, the device 300 may further comprise at least one means for illuminating the scene, and in particular a flash 320. Such a means 320 for illuminating the scene is optional because the device 300 may not comprise such an image acquisition means 318. For example, the device 300 may be integrated into an apparatus which already has such a means.

[0020] The device 300 may further comprise, optionally, - a mechanism (not shown) for adjusting the intensity of the flash, and / or - a mechanism (not shown) for adjusting the exposure time of the image sensor, and / or - a mechanism (not shown) for adjusting the aperture of a diaphragm (not shown) positioned between the image sensor and the scene. This or these mechanisms may be part of the device 300, or may be integrated into an apparatus comprising the device 300.

[0021] FIGURE 4 is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0022] The apparatus 400 of FIGURE 4 comprises means configured to implement the invention, and in particular any one of the methods 100 and 200.

[0023] The apparatus 400 of FIGURE 4 may comprise a device according to the invention, and in particular the device 300 of FIGURE 3.

[0024] In the example shown in FIGURE 4, the device 400 is a smartphone, or a tablet, comprising the device 300 of FIGURE 3. In particular, the device 400 of FIGURE 4 comprises a camera module 318 and at least one flash 320 for illuminating the scene.

[0025] Optionally, the apparatus 400 may further comprise: - a mechanism (not shown) for adjusting the intensity of the flash 320, and / or - a mechanism (not shown) for adjusting the exposure time of the image sensor of the camera module 318, and / or - a mechanism (not shown) for adjusting the aperture of a diaphragm (not shown) of the camera module 318, positioned between the image sensor and the scene.

[0026] In addition, the apparatus 400 comprises a display screen 402 equipped with a detection surface 404, for example capacitive.

[0027] Of course, the apparatus 400 may comprise other members than those indicated above.

[0028] FIGURE 5 is a schematic representation of another non-limiting exemplary embodiment of an apparatus according to the present invention.

[0029] The apparatus 500 of FIGURE 5 comprises means configured to implement the invention, and in particular any one of the methods 100 and 200.

[0030] The apparatus 500 of FIGURE 5 may comprise a device according to the invention, and in particular the device 300 of FIGURE 3.

[0031] In the example shown in FIGURE 5, the device 500 is a virtual reality, VR, headset, or an augmented reality headset, comprising the device 300 of FIGURE 3. The headset 500 of FIGURE 5 comprises in particular a camera module 318 and at least one flash 320 for illuminating the scene.

[0032] Optionally, the helmet 500 may further comprise: - a mechanism (not shown) for adjusting the intensity of the flash 320, and / or - a mechanism (not shown) for adjusting the exposure time of the image sensor of the camera module 318, and / or - a mechanism (not shown) for adjusting the aperture of a diaphragm (not shown) of the camera module 318, positioned between the image sensor and the scene.

[0033] In addition, the helmet 500 comprises a display screen 502 in / on a visor of said helmet 500.

[0034] Of course, the helmet 500 may include other members than those indicated above.

[0035] FIGURE 6 is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0036] The apparatus 600 of FIGURE 6 comprises means configured to implement the invention, and in particular any one of the methods 100 or 200.

[0037] The apparatus 600 of FIGURE 6 may comprise a device according to the invention, and in particular the device 300 of FIGURE 3.

[0038] In the example shown in FIGURE 6, the apparatus is a medical imaging apparatus, such as an endoscope, an ultrasound apparatus, etc. In particular, the medical imaging apparatus 600 comprises a camera module 318 and at least one flash 320 for illuminating the scene.

[0039] Optionally, the medical imaging device 600 may further comprise: - a mechanism (not shown) for adjusting the intensity of the flash 320, and / or - a mechanism (not shown) for adjusting the exposure time of the image sensor of the camera module 318, and / or - a mechanism (not shown) for adjusting the aperture of a diaphragm (not shown) of the camera module 318, positioned between the image sensor and the scene.

[0040] In addition, the medical imaging device 600 comprises a display screen 602 equipped with a detection surface 604, for example capacitive.

[0041] Of course, the medical imaging device 600 may include other organs than those indicated above.

[0042] FIGURE 7 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the present invention.

[0043] The vehicle 700 of FIGURE 7 comprises means configured to implement the invention, and in particular any one of the methods 100 and 200 of FIGURES 1 and 2.

[0044] The vehicle 700 of FIGURE 7 may comprise a device according to the invention, and in particular the device 300 of FIGURE 3.

[0045] In the example shown in FIGURE 7, the vehicle 700 is a land vehicle, in particular a car, comprising the device 300 of FIGURE 3. In particular, the vehicle 700 comprises a camera module 318 and at least one flash 320 for illuminating the scene.

[0046] Optionally, the vehicle 700 may further comprise: - a mechanism (not shown) for adjusting the intensity of the flash 320, and / or - a mechanism (not shown) for adjusting the exposure time of the image sensor of the camera module 318, and / or - a mechanism (not shown) for adjusting the aperture of a diaphragm (not shown) of the camera module 318, positioned between the image sensor and the scene.

[0047] In addition, the vehicle 700 comprises a display screen 702 equipped with a detection surface 704, for example capacitive, arranged in the passenger compartment of the vehicle 700.

[0048] Of course, the vehicle 700 may include other components than those indicated above.

[0133] Of course, the invention is not limited to the examples which have just been described.

Claims

CLAIMS 1. Method (100;200) for acquiring a stack of images of a scene comprising images of said scene at different focusing distances, with an apparatus (400;500;600;700) comprising a camera module (318) formed by an optical lens associated with an image sensor; said method (100;200) further comprising, for at least one of said focusing distances, an acquisition phase (120) comprising the following steps: - comparison (126) of a brightness value of at least one part of said scene located at a depth of field associated with the focusing distance, to at least one predetermined brightness threshold value, - when said brightness value does not satisfy said at least one brightness threshold value, adjustment (128) of the brightness for the acquisition of an image of said scene at said focusing distance, by modification of at least one parameter of said apparatus for, and - acquisition (130) of an image of said scene at said focusing distance.

2. Method (100;200) according to the preceding claim, characterized in that, for at least one focusing distance, the adjustment (128) of the brightness achieves: - an increase in brightness when acquiring the image at said focusing distance; or - a decrease in brightness when acquiring the image at the said focusing distance.

3. Method (100;200) according to any one of the preceding claims, characterized in that the adjustment (128) of the brightness carries out: - a trigger (128i), and / or - an adjustment (128z) of the average intensity: of a means of lighting (320) the scene, and in particular of a flash, equipping the imaging device.

4. Method (100;200) according to any one of the preceding claims, characterized in that the adjustment (128) of the brightness effects a modification (128s) of an exposure time of the image sensor.

5. Method (100;200) according to any one of the preceding claims, characterized in that the adjustment (128) of the brightness effects a modification (1284) of the opening of a diaphragm equipping the device.

6. Method (100; 200) according to any one of the preceding claims, characterized in that it comprises a step (102) of determining a depth map of the scene, the focusing distances for which the images are acquired being selected as a function of said depth map.

7. Method (100;200) according to the preceding claim, characterized in that the depth map is determined by analysis (106) of an image of said scene, called exploratory image, taken prior to the acquisition of the stack of images.

8. Method (100;200) according to any one of claims 1 to 4, characterized in that the depth map is determined with at least one sensor for measuring the depths of the objects located in the scene.

9. Method (200) according to any one of the preceding claims, characterized in that, for at least one focusing distance, the value of the brightness of the scene associated with said focusing distance is determined by analysis (202) of an exploratory image of the scene taken prior to the acquisition of the stack of images.

10. Method (100) according to any one of the preceding claims, characterized in that, for at least one focusing distance, the brightness value of the scene at said focusing distance is determined by analysis (125) of an image, called a preliminary image, of the scene taken at said focusing distance.

11. Method (100;200) according to any one of the preceding claims, characterized in that it comprises, for at least one focusing distance, an acquisition of several images at different brightness values, for example with and without flash.

12. Computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method (100;200) according to any one of the preceding claims.

13. Device (300) comprising means configured to implement all the steps of the method according to any one of claims 1 to 11.

14. Apparatus (400; 500; 600) comprising: - at least one camera module (318), - at least one calculation unit (302); configured to implement all the steps of the method (100;200) according to any one of claims 1 to 11.

15. Apparatus (700) comprising: - at least one camera module (318), - at least one calculation unit (302); configured to implement all the steps of the method (100;200) according to any one of claims 1 to 11.

Images