Imaging method and associated imaging device

Abstract

The invention relates to an imaging method (20), which comprises the steps of: - acquiring (22) a first image by means of a camera module (10), wherein a focus of the camera module (10) is adjusted such that a sharpness of the first image acquired is maximum at an area of interest (30) of a scene (15); - acquiring (24) a second image by means of the camera module (10), wherein a focus of the camera module is adjusted such that a sharpness of the second image acquired is less than a predetermined low sharpness threshold in at least a portion of the second image that is not representative of the area of interest (30); - detecting (26), in the first image, pixels of interest that are representative of the area of interest (30); and - generating (28) a hybrid image using the pixels of interest of the first image and pixels of the second image.

Description

Imaging method and associated imaging device DESCRIPTION technical field

[0001] The present invention relates to an imaging method.

[0002] The invention also relates to a computer program and a device implementing such a method.

[0003] The invention applies to the field of image acquisition. State of the art

[0004] Today, smartphones have become the primary tools for capturing images, especially portraits.

[0005] However, due to the small size of their sensors and lenses, such smartphones generally produce images with a large depth of field.

[0006] However, this is detrimental in the case where a desired result is a portrait in which the subject visually stands out from the background, which is often sought for artistic or aesthetic effects.

[0007] The known solutions to this problem rely mainly on post-processing techniques, such as digital clipping and the application of "bokeh" type filters.

[0008] However, such methods do not provide complete satisfaction.

[0009] Indeed, such methods are likely to produce artificial results and lack precision, particularly with regard to the transition between the subject and the background.

[0010] One object of the present invention is to remedy at least one of the drawbacks of the prior art.

[0011] Another aim of the invention is to provide an imaging process that results in a better quality of the portrait obtained. Description of the invention

[0012] To this end, the invention relates to a method of the aforementioned type, comprising the following steps: - acquisition of a first image using a camera module, the focus of the camera module being set so that the sharpness of the first image acquired is maximum at the level of an area of ​​interest of a scene; - acquisition of a second image using the camera module, with the focus of the camera module set so that the sharpness of the second acquired image is less than a predetermined low sharpness threshold in at least a part of the second image not representative of the area of ​​interest; - detection, in the first image, of pixels of interest representative of the area of ​​interest; and - generation of a hybrid image from the pixels of interest of the first image and pixels of the second image.

[0013] Indeed, thanks to the first image acquisition stage, fine details in the area of ​​interest (for example, fine details of a subject's face) are acquired with high precision.

[0014] In addition, thanks to the second image acquisition step, a blurred background is generated optically rather than synthetically, producing a more natural effect to the eye, and without tedious or computationally intensive image processing.

[0015] Furthermore, such a method is likely to be easily integrated into existing electronic devices with software adjustments, without requiring significant hardware modifications.

[0016] Advantageously, the process according to the invention has one or more of the following characteristics, taken individually or in any technically feasible combination:

[0017] The area of ​​interest includes: - an area of ​​the scene selected by a user via a human-machine interface; or - an area of ​​the scene located at a predetermined distance from the camera module; or - an element of the scene automatically detected as belonging to a predetermined target category;

[0018] in the first image, the pixels of interest correspond to the adjacent pixels for which the sharpness is greater than a predetermined high sharpness threshold;

[0019] the generation of the hybrid image includes, for each pixel of the first image distinct from a pixel of interest, an assignment to said pixel of the value taken by said pixel in the second image;

[0020] prior to the generation stage, at least one of the first or second image is recalibrated according to characteristics of said first and second image;

[0021] a time gap between the acquisition of the first image and the acquisition of the second image is less than or equal to a predetermined time gap;

[0022] for each pixel, a corresponding sharpness is equal to the value taken, at said pixel, by a predetermined variance estimator, a predetermined entropy estimator or a predetermined energy estimator.

[0023] According to another aspect of the invention, an imaging device is proposed comprising a camera module, the imaging device being configured to: - acquire a first image using the camera module, with the camera module's focus set so that the sharpness of the first image acquired is maximum at the level of an area of ​​interest in a scene; - acquire a second image using the camera module, with the camera module's focus set so that the sharpness of the second acquired image is less than a predetermined low sharpness threshold in at least a part of the second image that is not representative of the area of ​​interest; - detect, in the first image, pixels of interest representative of the area of ​​interest; and - generate a hybrid image from the pixels of interest of the first image and pixels of the second image.

[0024] The device according to the invention can be any type of device such as a server, a computer, a tablet, a calculator, a processor, a computer chip, programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0025] According to another aspect of the invention, an electronic device comprising an imaging device as defined above is proposed.

[0026] Advantageously, the electronic device also includes a display support for displaying the generated hybrid image.

[0027] Preferably, the electronic device is a smart mobile phone (or "smartphone" in English), also called a smartphone or multifunction mobile phone.

[0028] In this case, the invention also relates to the use of an imaging method according to the invention, within the smart mobile phone. to generate an image of a scene from images acquired using a camera module included in the smartphone.

[0029] Alternatively, the electronic device is a touchscreen tablet.

[0030] In this case, the invention also relates to the use of an imaging method according to the invention, within a touch tablet, to generate an image of a scene from images acquired by means of a camera module included in the touch tablet.

[0031] Alternatively, the electronic device is a computer.

[0032] In this case, the invention also relates to the use of an imaging method according to the invention, within the computer, to generate an image of a scene from images acquired by means of a camera module included in the computer.

[0033] Alternatively, the electronic device is a television or a display terminal.

[0034] In this case, the invention also relates to the use of an imaging method according to the invention, within the television or display terminal, to generate an image of a scene from images acquired by means of a camera module included in the television or display terminal.

[0035] Alternatively, the electronic device is a virtual reality headset or an augmented reality headset.

[0036] In this case, the invention also relates to the use of an imaging method according to the invention, within the virtual reality headset or the augmented reality headset, to generate an image of a scene from images acquired by means of a camera module included in the virtual reality headset or the augmented reality headset.

[0037] Alternatively, the electronic device is included in a medical imaging system, for example an endoscope.

[0038] In this case, the invention also relates to the use of an imaging method according to the invention, within the medical imaging system, to generate an image of a scene from images acquired by means of a camera module included in the medical imaging system.

[0039] Alternatively, the electronic device is included in a vehicle, autonomous or not, such as a land vehicle (e.g., a car), an aircraft (e.g., a drone, a plane, a helicopter, etc.) or a maritime vehicle (e.g., a boat, a submarine, etc.).

[0040] In this case, the invention also relates to the use of an imaging method according to the invention, within the vehicle, to generate an image of a scene from images acquired by means of a camera module included in the vehicle. Brief description of the figures

[0041] The invention will be better understood upon reading the following description, given solely by way of non-limiting example and made with reference to the accompanying drawings in which:

[0042] Figure 1 is a schematic representation of an electronic device according to the invention; and

[0043] Figure 2 is a flowchart of an imaging process implemented by the electronic device in Figure 1.

[0044] It is understood that the embodiments described below are by no means exhaustive. In particular, variants of the invention may be conceived comprising only a selection of the features described below, isolated from the other features described, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from the prior art. prior. This selection includes at least one functional preference feature without structural details, or with only part of the structural details if that part is sufficient to confer a technical advantage or to differentiate the invention from the prior art.

[0045] In particular, all the variants and embodiments described can be combined with each other if there are no technical obstacles to this combination.

[0046] In the figures and in the rest of the description, elements common to several figures retain the same reference. Detailed description

[0047] An electronic device 2 according to the invention is illustrated by figure 1.

[0048] The electronic device 2 includes, in particular, an imaging device 4.

[0049] Preferably, the electronic device 2 also includes a display support 6.

[0050] Display support 6 can be any type of display support, for example a display screen or a projector projecting images onto a display surface.

[0051] Preferably, the electronic device 2 includes a human / machine interface (not shown), for example a touch surface superimposed on the display support 6, a button, a selection wheel, etc.

[0052] Electronic device 2 is, for example, a digital camera, a camcorder, a smartphone, a tablet, a computer, etc.

[0053] The imaging device 4 comprises a control module 8 and a camera module 10.

[0054] More specifically, the camera module 10 includes a lens 12 associated with a sensor 14, the sensor 14 being arranged to acquire an image of a scene 15 viewed through the lens 12. The lens 12 and the sensor 14 are, for example, mounted in a housing 16 to form said camera module 10.

[0055] Each camera module 10 is capable of being controlled to modify a corresponding focusing distance. For example, such a focusing distance can be modified by adjusting the optical elements of the lens 12 and / or the relative position of the sensor 14 with respect to the lens 12.

[0056] The control module 8 is connected to the camera module 10 to control the operation of the camera module 10, and to receive the images acquired by the camera module 10.

[0057] Preferably, the control module 8 is also connected to the display support 6 for the transmission of generated images to be displayed.

[0058] Preferably, the control module 8 is also connected to the human / machine interface to receive instructions from a user.

[0059] More specifically, the control module 8 is configured to generate a hybrid image of an area of ​​interest in the scene 15, by implementing an imaging process 20 (Figure 2). For example, if the area of ​​interest in the scene 15 is a subject 18 (such as a person, an animal, etc.), such a hybrid image is preferably a portrait of said subject 18.

[0060] Preferably, the control module 8 is configured to implement process 20 in response to a specific instruction, such as a user instruction. Such an instruction corresponds, in particular, to the activation of a particular operating mode of the imaging device 4, usually called "portrait mode".

[0061] The process 20 includes a first acquisition step 22, a second acquisition step 24, a detection step 26 and a hybrid image generation step 28 (called the "generation step").

[0062] First acquisition 22

[0063] The control module 8 is configured to control the acquisition, during the first acquisition step 22, of a first image by means of a camera module 10.

[0064] More specifically, during the acquisition of the first image, the focus of the camera module is set so that the sharpness of the first image acquired is maximum at the level of an area of ​​interest in the scene 15.

[0065] In particular, such an area of ​​interest is a face 30 of the subject 18, or a specific part of the subject 18 or of an object in the scene of interest.

[0066] For example, such sharpness is directly set by a user, for example by manually acting on the focus of the camera module 10, in particular by acting on the lens 12.

[0067] Alternatively, or as a complement, the control module 8 is configured to modify a setting of the camera module 10 to maximize sharpness in an area selected by the user, for example selected on a preview of the scene 15 displayed on the display support 6. In this case, such a selection is implemented via the human-machine interface mentioned above.

[0068] In other words, in this case, the area of ​​interest includes the area of ​​the scene selected by the user via the human-machine interface.

[0069] Alternatively, or in addition, the control module 8 is configured to modify a setting of the camera module 10 to maximize sharpness in an area of ​​the scene located at a first predetermined distance from the camera module 10.

[0070] In other words, in this case, the area of ​​interest includes the area of ​​the scene located at said first predetermined distance from the camera module 10.

[0071] Alternatively, or as a complement, the control module 8 is configured to automatically detect the area of ​​interest in the scene 15, based on an image or image stream delivered by the camera module 10. In this case, the control module 8 is configured to modify a setting of the camera module 10 to maximize sharpness at the detected area of ​​interest.

[0072] For example, to perform such detection, the control module 8 is configured to implement an object detection algorithm trained to detect elements belonging to a predetermined target category. For example, the object detection algorithm is trained to recognize human faces, or a specific type of object.

[0073] In other words, in this case, the area of ​​interest includes the scene element automatically detected as belonging to the predetermined target category.

[0074] Preferably, for each pixel, a corresponding sharpness is equal to the value taken, at said pixel, by a predetermined variance estimator, a predetermined entropy estimator, or a predetermined energy estimator.

[0075] Second acquisition 24

[0076] The control module 8 is also configured to control the acquisition, during the second acquisition step 24, of a second image by means of the camera module 10.

[0077] More specifically, during such an acquisition, the focus of the camera module 10 is set so that the sharpness of the second acquired image is less than a predetermined low sharpness threshold in at least a part of the second image, advantageously in a part of the second image not representative of the area of ​​interest, preferably on the major part of the second image which is not representative of the area of ​​interest.

[0078] For example, the focus of camera module 10 is set so that the sharpness in every pixel of the second acquired image is below the predetermined low sharpness threshold.

[0079] According to another example, the focus of the camera module 10 is set so that, in the second image acquired, the sharpness of the pixels representing elements of the scene located at a second predetermined distance from the camera module 10 is less than the predetermined low sharpness threshold.

[0080] According to another example, the focus of camera module 10 is set so that, in the second image acquired, the sharpness of the pixels imaging predetermined areas of the field of camera module 10 is less than the predetermined low sharpness threshold.

[0081] Preferably, the control module 8 is configured to control the focus of the camera module 10 so that the condition on the sharpness of the pixels of the second image is met.

[0082] Alternatively, such focus is set by the user, for example by manually acting on the camera module 10, in particular on the lens 12. In this case, the control module 8 is advantageously configured to deliver a signal to the user when the desired sharpness condition is met, i.e. when the sharpness in at least part of the second image is below the predetermined lower sharpness threshold.

[0083] Advantageously, the time difference between the acquisition of the first image and the acquisition of the second image is less than or equal to a predetermined time difference. For example, such a time difference is less than 500 ms (milliseconds).

[0084] Such a feature is advantageous, as it reduces the probability of movement of the elements of scene 15 between the first acquisition 22 and the second acquisition 24. This contributes to the generation of a better quality portrait.

[0085] Detection 26

[0086] The control module 8 is further configured to detect, during the detection step 26, pixels of interest in the first image that are representative of the area of ​​interest. For example, the pixels of interest are representative of the face 30 of subject 18.

[0087] Preferably, the command module 8 is configured to detect pixels of interest as the pixels in the first image for which the sharpness is greater than a predetermined high sharpness threshold, and more specifically the adjacent pixels in the first image for which the sharpness is greater than a predetermined high sharpness threshold.

[0088] Alternatively, or as a complement, to perform such detection, control module 8 is configured to implement an object detection algorithm trained to detect elements in an image belonging to the predetermined target category mentioned earlier. For example, the object detection algorithm is trained to recognize human faces, or a specific type of object. In this case, the pixels of interest are, for example, the pixels delimited by a bounding box generated by the object detection algorithm.

[0089] The control module is likely to be configured to implement the detection step 26 before or after the second acquisition step 24.

[0090] Implementing the second acquisition step 24 after the detection step 26 is likely to offer an advantage. Specifically, in this case, detecting the pixels of interest in the first image is likely to result in better focus adjustment during the acquisition of the second image.

[0091] Implementing the second acquisition step 24 before the detection step 26 is also likely to offer an advantage, particularly by reducing the time elapsed between the acquisition of the first and second image.

[0092] Generation 28

[0093] The control module 8 is also configured to generate, during generation step 28, a hybrid image from the pixels of interest of the first image and pixels of the second image.

[0094] This results in a portrait of subject 18, in the case where the area of ​​interest is the face 30 of subject 18.

[0095] Preferably, to generate such a hybrid image, the command module 8 is configured to assign, to each pixel of the first image that is distinct from a pixel of interest, the value taken by said pixel in the second image.

[0096] Advantageously, the control module 8 is configured to implement image registration prior to the generation of the hybrid image.

[0097] In this case, control module 8 is preferably configured to realign at least one of the first or second images based on characteristics of said first and second images. In particular, control module 8 is configured to realign the second image.

[0098] To perform such registration, the command module 8 is preferably configured to implement an analysis based on features of the first and second images, such as feature recognition between pixels of the first and second images, correlation analysis, or a Lucas-Kanade method. This results in at least one displacement vector representative of the registration to be performed, specifically indicative of the registration of the second image.

[0099] In this case, the command module 8 is configured to realign at least one of the first or second images, especially the second image, based on an analysis result.

[0100] Such a feature is advantageous, insofar as it allows compensation for any movement of the camera module 10 and / or the scene 15 in the interval between the acquisition of the first image and the acquisition of the second image.

[0101] Preferably, the generated hybrid image is displayed on display support 6. Functioning

[0102] The operation of electronic device 2 will now be described with reference to the figures.

[0103] During the first acquisition step 22, the control module 8 commands the acquisition of a first image by means of a camera module 10.

[0104] More specifically, during the acquisition of the first image, the focus of the camera module is set so that the sharpness of the first image acquired is maximum at the level of an area of ​​interest of a scene 15.

[0105] Then, during the second acquisition step 24, the control module 8 commands the acquisition of a second image using the camera module 10.

[0106] More specifically, during such an acquisition, the focus of the camera module 10 is set so that, in at least part of the second image acquired, the sharpness is below the predetermined low sharpness threshold.

[0107] As mentioned previously, the time gap between the acquisition of the first image and the acquisition of the second image is advantageously less than or equal to the predetermined time gap, for example less than or equal to 500 ms.

[0108] In addition, during detection step 26, the control module 8 detects, in the first image, the pixels of interest representative of the area of ​​interest.

[0109] The detection step 26 is likely to be implemented before or after the second acquisition step 24.

[0110] Then, during generation step 28, the control module 8 generates, a hybrid image from the pixels of interest of the first image and pixels from the second image.

[0111] Preferably, the generated hybrid image is displayed on display support 6.

[0112] Of course, the invention is not limited to the examples that have just been described.

Claims

DEMANDS 1. Imaging procedure (20) comprising the steps: - acquisition (22) of a first image by means of a camera module (10), the focus of the camera module (10) being set so that the sharpness of the first image acquired is maximum at the level of an area of ​​interest (30) of a scene (15); - acquisition (24) of a second image by means of the camera module (10), a focus of the camera module being set so that a sharpness of the second image acquired is less than a predetermined low sharpness threshold in at least a part of the second image not representative of the area of ​​interest (30); - detection (26), in the first image, of pixels of interest representative of the area of ​​interest (30); and - generation (28) of a hybrid image from the pixels of interest of the first image and pixels of the second image.

2. A method according to claim 1, wherein the area of ​​interest (30) comprises: - an area of ​​the scene (15) selected by a user via a human-machine interface; or - an area of ​​the scene (15) located at a first predetermined distance from the camera module (10); or - an element of the scene (15) automatically detected as belonging to a predetermined target category.

3. Method according to claim 1 or 2, wherein, in the first image, the pixels of interest correspond to the adjacent pixels for which the sharpness is greater than a predetermined high sharpness threshold.

4. A method according to any one of claims 1 to 3, wherein the hybrid image generation comprises, for each pixel of the first distinct image of a pixel of interest, an assignment to said pixel of the value taken by said pixel in the second image.

5. A method according to any one of claims 1 to 4, wherein, prior to the generation step, at least one of the first or second image is recalibrated according to characteristics of said first and second image.

6. A method according to any one of claims 1 to 5, wherein a time gap between the acquisition (22) of the first image and the acquisition (24) of the second image is less than or equal to a predetermined time gap.

7. A method according to any one of claims 1 to 6, wherein, for each pixel, a corresponding sharpness is equal to the value taken, at said pixel, by a predetermined variance estimator, a predetermined entropy estimator, or a predetermined energy estimator.

8. Imaging device (4) comprising a camera module (10), the imaging device (4) being configured to: - acquire (22) a first image using the camera module (10), the focus of the camera module (10) being set so that the sharpness of the first image acquired is maximum at the level of an area of ​​interest (30) of a scene (15); - acquire (24) a second image using the camera module, the focus of the camera module being set so that the sharpness of the second image acquired is less than a predetermined low sharpness threshold in at least a part of the second image not representative of the area of ​​interest; - detect (26), in the first image, pixels of interest representative of the area of ​​interest; and - generate (28) a hybrid image from the pixels of interest of the first image and pixels of the second image.

9. Electronic device (2) comprising an imaging device (4) according to claim 8.

10. Electronic device (2) according to claim 9, further comprising a display support (6) for displaying the generated hybrid image.

Images