Method and apparatus for acquiring an image stack of a scene with adjusted sharpness magnitude

Abstract

The invention relates to a method (200) of imaging a scene using at least one camera module, said method comprising acquiring a plurality of images (IM1-IMn) of said scene at different discrete focusing distances DFi, each image having a respective depth of field PCi = {PCi,min; PCi,max}, said depth of field comprising a respective focusing distance and extending over a respective range of sharpness NETi = PCi,max - PCi,min, said method further comprising a step (202) of determining: - a minimum focusing distance (DF1) such that the lower bound (PC1,min) of the depth of field (PC1) associated therewith is less than or equal to a predetermined lower bound distance (Dmin); and / or - a maximum focusing distance (DF1) such that the upper bound (PCn,max) of the depth of field (PCn) associated therewith is greater than a predetermined upper bound distance (Dmax). The invention also relates to a computer program, a device and a vehicle implementing such a method.

Description

[0001] This invention relates to a method for acquiring an image stack of a scene with adjusted sharpness amplitude. The invention also relates to computer programs and apparatus for implementing this method. Furthermore, the invention relates to devices and vehicles for implementing this method.

[0002] The present invention is in the field of acquiring an image stack of a scene, the image stack of the scene comprising images acquired at different focus distances. Existing technology

[0003] The camera module acquires an image of the scene at a focus distance DF, with a depth of field PC = {PC}. min PC max The sharpness range NET=PC near the focus distance DF is defined. max PC min Therefore, the image clearly and in focus depicts a limited portion of the scene, particularly those areas within the depth of field where the camera module is located (i.e., the PC). min To PC max All objects in PC. min It is the distance from the camera module to the nearest plane in the scene where the image is sharpest, PC max It is the distance from the camera module to the farthest plane in the scene where the image is clearest.

[0004] For example, a technique known as focus bracketing or focus stacking is made possible by capturing a stack of images of a scene, each captured at a different focus distance. This image stack can be used to obtain a composite image that sharply focuses and depicts a larger portion of the scene, and therefore has a greater depth of field.

[0005] In some focus bracketing techniques, images of the scene are acquired at randomly selected focus distances. When the scene is depicted based on these images, random results are produced. Some objects in the scene may never be depicted in sharp focus, or the image may be taken of parts of the scene that do not contain any objects, which is practically meaningless.

[0006] Other techniques acquire images at a focus distance corresponding to previously measured distances to objects in the scene. The aim of these techniques is to match sharp areas of the scene with objects located within them. However, these techniques are complex because they require detecting all objects in the scene and measuring their distance / depth relative to the camera module. Furthermore, these techniques are inefficient because they rely on efficient object detection in the scene, which is not always feasible. Additionally, certain areas in the scene that do not contain objects are not sharp in the acquired image, which is a major limitation.

[0007] One object of the present invention is to solve at least one of the above-mentioned disadvantages.

[0008] Another object of the present invention is to provide a solution for imaging a scene with improved clarity.

[0009] Another object of the present invention is to provide a solution for imaging a scene in a simpler and more efficient manner with improved sharpness. Summary of the Invention

[0010] The present invention achieves at least one of the aforementioned objectives by means of a method for imaging a scene using at least one camera module, each camera module including an optical objective lens associated with an image sensor; the method includes focusing at different discrete focusing distances (hereinafter referred to as DF). i Acquire multiple images of the scene (hereinafter referred to as IM1-IM) n Each image has a corresponding depth of field (denoted as PC). i ={PC i,min PC i,max The depth of field includes the corresponding focus distance DF. i And within the corresponding resolution range NET i =PC i,max PC i,min Extend upwards;

[0011] With focus distance DF i The acquired image clearly and in focus depicts the scene located at the focus distance DF. i The associated depth PC i The portion within the distance;

[0012] The method is characterized by further comprising the step of determining:

[0013] - Minimum focusing distance such that the lower limit of the associated depth of field is less than or equal to a predetermined lower bound distance; and / or

[0014] - Determine the maximum focusing distance such that the upper limit of the depth of field associated with it is greater than the predetermined upper limit distance.

[0015] Therefore, as with known techniques, this invention enables the imaging of a scene to obtain a stack of multiple images captured at different focus distances. Such an image stack allows for the depiction of the scene with a greater depth of field because each image sharply focuses on a different part of the scene.

[0016] Furthermore, contrary to known solutions, this invention provides methods for selecting the minimum or maximum focusing distance based on predetermined lower or upper bound distances, respectively. Therefore, this invention enables imagery of a scene while ensuring sharpness from the lower to the upper bound distance, which are selected rather than randomly determined or based on a depth map of the scene, a determination that is not only complex but can also be a source of error. Thus, this invention enables imagery of a scene with improved sharpness and in a simpler and faster manner because it eliminates the need to determine a depth map of the scene.

[0017] As used in this article, “image” refers to a digital image, particularly a raster image, and more particularly, for example, an RGB raster image.

[0018] As used herein, "focusing distance" refers to the distance at which an optical objective lens is positioned relative to the lens for focusing. The focusing distance used to capture an image is typically adjusted by modifying the distance between the image sensor and the optical objective lens. Thus, a first image of a scene acquired at a first focusing distance depicts a first portion of the scene in sharp focus, and a second image of a scene acquired at a second focusing distance depicts a second portion of the scene in sharp focus.

[0019] As used in this article, "depth of field" refers to the extent of the area of ​​sharpness in an image, that is, the distance between the first and last sharp focus planes in the image.

[0020] As used in this article, "sharpness range" refers to the distance that the sharp, in-focus portion of an image extends, or the depth of field, which is the distance from the first plane of sharpness to the last plane of sharpness in the image.

[0021] In a non-limiting example, to illustrate the definitions given above, the focusing distance can be 15 meters. The sharpness range can be 1.50 meters, and the depth of field can be 14.50 meters to 16 meters. In this case, the image sharply focuses on all objects or parts of the scene located at distances from the optical objective lens between 14.5 meters and 16 meters.

[0022] According to some embodiments, the predetermined lower bound distance can be selected by the user.

[0023] This provides users with the option to set a lower limit for the sharpness of the image used to image the scene, regardless of the imaging device used to image the scene or the at least one camera module. Therefore, the present invention enables the imagery of a scene in a customizable manner according to each user's wishes and / or intentions.

[0024] Alternatively, the predetermined lower bound distance can correspond to the distance to the nearest object in the scene, specifically the distance between the imaging module and the nearest object in the scene.

[0025] In this embodiment, the minimum focusing distance for imaging the scene can be adapted to the composition of the scene, particularly to objects located within it. Therefore, imaging the scene ensures that the closest object / part of the scene to the camera module is clearly focused in at least one image in the image stack. Furthermore, this embodiment allows for the avoidance of acquiring images whose depth of field does not correspond to any object in the scene, particularly images at distances smaller than the distance to the closest object in the scene.

[0026] Alternatively, the predetermined lower bound distance may correspond to the minimum sharpness distance allowed by the at least one camera module, and in particular to the lower limit of the minimum focus distance allowed by the at least one camera module.

[0027] Such embodiments provide the possibility of fully utilizing the capabilities of the at least one camera module used for imaging a scene, particularly for imaging the nearest object / area of ​​the scene.

[0028] When the predetermined lower bound distance corresponds to the distance to the nearest object in the scene, the method according to the invention may further include the step of determining the distance to the nearest object in the scene.

[0029] It can measure / estimate the distance to the nearest object.

[0030] There are several ways to measure / estimate the distance to the nearest object:

[0031] - Use sensors, such as LiDAR sensors or time-of-flight cameras;

[0032] - Based on the signal received by the at least one camera module, the at least one camera module gradually increases the focus starting from the minimum focus distance allowed by the at least one camera module;

[0033] - Based on one or more images captured by the at least one camera module. In practice, distances to objects in the scene can be measured using techniques such as triangulation.

[0034] - Based on the image captured by the camera module and the optical transfer function of the camera module, particularly the PSF (point spread function); or

[0035] - Based on the input captured by the camera module, an image is generated and previously trained to provide the distance to an artificial intelligence model.

[0036] Of course, other embodiments are also possible, and the invention is not limited to these examples, which are given for illustrative purposes only.

[0037] According to some embodiments, the predetermined upper bound distance can be selected by the user.

[0038] This provides users with the option to set an upper limit on the sharpness used to image a scene, regardless of the imaging device used to image the scene or the at least one camera module. Therefore, this invention enables scene imaging in a customizable manner according to each user's wishes and / or intentions.

[0039] Alternatively, the predetermined upper bound distance can correspond to the distance to the farthest object in the scene.

[0040] In this embodiment, the maximum focusing distance for imaging the scene can be adapted to the composition of the scene, particularly to objects located within it. Therefore, scene imaging ensures that the object / part of the scene furthest from the camera module is clearly focused and imaged. Furthermore, this embodiment avoids acquiring images whose depth of field does not correspond to any object in the scene, especially images at distances greater than the distance to the furthest object in the scene.

[0041] Alternatively, the predetermined upper limit distance may correspond to the maximum sharpness distance allowed by the at least one camera module, and in particular to the upper limit of the maximum focus distance allowed by the at least one camera module.

[0042] Such an embodiment provides the possibility of fully utilizing the capabilities of the at least one camera module used for imaging a scene, particularly for imaging the farthest objects / areas of the scene.

[0043] When the predetermined upper bound distance corresponds to the distance to the farthest object in the scene, the method according to the invention may further include the step of determining the distance to the farthest object in the scene.

[0044] It can measure / estimate the distance to the farthest object.

[0045] There are several ways to measure / estimate the distance to the farthest object:

[0046] - Use sensors, such as LiDAR sensors or time-of-flight cameras;

[0047] - Based on the signal received by the at least one camera module, the at least one camera module gradually reduces the focus starting from the maximum focus distance allowed by the at least one camera module;

[0048] - Based on one or more images captured by the at least one camera module. In practice, distances to objects in the scene can be measured using techniques such as triangulation.

[0049] - Based on the image captured by the camera module and the optical transfer function of the camera module, particularly the PSF (point spread function); or

[0050] - Based on the input captured by the camera module, an image is generated and previously trained to provide the distance to an artificial intelligence model.

[0051] Of course, other embodiments are also possible, and the invention is not limited to these examples, which are given for illustrative purposes only.

[0052] According to some embodiments, for at least one focusing distance, the depth of field associated with said focusing distance may be adjacent to or overlap with the depth of field associated with the next focusing distance in ascending or descending order of focusing distance.

[0053] In this configuration, sharpness continuity is achieved between at least two images of a scene acquired at two consecutive focusing distances in ascending or descending order. In other words, the sharpness of the scene is continuous across the two images acquired at different focusing distances, ensuring that there are no blurred or out-of-focus areas of the scene between the two focusing distances.

[0054] According to some embodiments, for each focus distance, the depth of field associated with said focus distance may be adjacent to or overlap with the depth of field associated with the next focus distance in ascending or descending order of focus distance.

[0055] In this configuration, sharpness continuity is achieved across all images of the scene acquired at different focus distances. In other words, the scene sharpness is continuous across all images acquired at various focus distances. Therefore, there are no blurred or out-of-focus areas of the scene between the focus distances used to acquire the images, particularly between the lower limit of the minimum focus distance and the upper limit of the maximum focus distance.

[0056] This embodiment maximizes the sharpness range of the scene depicted in the image because it uses all acquired images IM1-IM1. n It can be accessed from a PC. 1,min To PC n,max Depicting the scene in sharp focus at all distances, with a total sharpness range equal to NET. total =PC n,max PC 1,min .

[0057] According to some embodiments, for at least one (especially each) focusing distance, the depth of field associated with said focusing distance may be adjacent to the depth of field associated with the next focusing distance.

[0058] In this scenario, there is no overlap between the corresponding depths of field. This allows for the selection of a focus distance to minimize the total number of images to be acquired, while still allowing for continuity of sharpness between images of the scene.

[0059] For example, you can select the focus distance DF i and DF i+1 (where DF) i <DF i+1 ) makes PC i,max =PC i+1,min Therefore, based on the focusing distance DF... i and DF i+1 Acquired image IM i and IM i+1 From PC i,min To PC i+1,max It depicts the scene clearly and continuously across the entire depth.

[0060] According to some embodiments, for at least one (especially each) focusing distance, the depth of field associated with said focusing distance overlaps with the depth of field associated with the next focusing distance.

[0061] In this situation, the corresponding depth of field overlaps. This allows for the selection of a focus distance to ensure continuity of sharpness between scene images.

[0062] For example, you can select the focus distance DF i and DF i+1 (where DF) i <DF i+1 ) makes PC i,max PC i+1,min Therefore, based on the focusing distance DF... i and DF i+1 Acquired Image IM i and IM i+1 From PC i,min To PC i+1,max The scene is clearly focused and continuously depicted across the entire depth, and further details are shown in both images IM1 and IM2. i+1 There is a clear overlap between them. In fact, the scene is located within the range [PC]. i+1,min PC i,max The depth portion within the image IM i and image IM 1+1 The middle part is depicted with clear focus.

[0063] In one embodiment, at least two (in particular all) images can be acquired using the same camera module.

[0064] In this case, even if the imaging device used to image the scene includes multiple camera modules, only one of these modules is used to acquire the image.

[0065] In this case, the images are acquired sequentially.

[0066] In one embodiment, at least two (in particular all) images can be acquired using different camera modules.

[0067] In this scenario, at least two camera modules are used to acquire the images. In this case, images acquired by multiple camera modules can be acquired simultaneously or sequentially.

[0068] For a given number of images, using multiple camera modules makes it possible to image a scene much faster compared to using a single camera module.

[0069] Furthermore, using multiple camera modules allows for imaging of the scene at a greater total depth of field. In fact, camera modules in devices (such as smartphones) enable imaging of the scene at different focusing distances. For example, a smartphone can be equipped with a wide-angle camera module and a telephoto camera module, allowing for imaging of the scene at different focusing distances.

[0070] According to some embodiments, for at least one focus distance DF i The associated depth PC i ={PC i,min PC i,max The depth of field is known and provided by the imaging module. In reality, the depth of field is determined by the hardware that constitutes the camera module and depends on the physical characteristics of the camera module.

[0071] According to some embodiments, for at least one focus distance DF i Related depth PC i ={PC i,min PC i,max This can be measured and stored in association with the focusing distance.

[0072] Such measurements can be performed in different ways.

[0073] For example, with focus distance DF i Related depth PC i ={PC i,min PC i,max The following measurements can be taken:

[0074] - The camera module is configured to acquire an image of the reference scene at the said focus distance.

[0075] - Take an image of the reference scene at the stated focusing distance.

[0076] - Identify the first clear object in the image and measure its distance: this distance corresponds to the PC. i,min ;

[0077] - Identify the last sharp object in the image and measure its distance: this distance corresponds to the PC. i,max ;

[0078] Therefore, the depth of field PC associated with the focusing distance was determined. i It is also possible to determine the sharpness range NET associated with the focal distance. i =PC i,max PC i,min .

[0079] In some embodiments, at least one image of the scene may be a 2D image.

[0080] In some embodiments, at least one image of the scene may be a 3D image.

[0081] According to another aspect of the invention, a computer program is proposed that includes executable instructions that, when executed by a computing device, implement all the steps of the method according to the invention.

[0082] Computer programs can be written in any computer language, such as machine language, C, C++, JAVA, Python, etc.

[0083] Such a computer program can be a standalone application. Alternatively, it can be integrated into an application used to acquire photos or videos.

[0084] According to another aspect of the present invention, an apparatus is provided comprising:

[0085] - At least one camera module,

[0086] - At least one computing unit;

[0087] It is configured to implement all the steps of the method according to the invention.

[0088] Specifically, the device may be a user device, such as a smartphone, tablet, etc.

[0089] In this context, the user equipment may also include a display screen, a capacitive sensing surface, etc.

[0090] In particular, the device may be a computer-type user equipment.

[0091] In this context, a computer-type user equipment may include a display screen, a touch surface (particularly a touch surface integrated into or associated with the display screen of the computer), etc.

[0092] In particular, the device may be a television set.

[0093] Specifically, the device may be a virtual reality headset or an augmented reality headset.

[0094] In this context, the head-mounted device may include a display screen, one or more sensors (especially optical sensors), etc.

[0095] In particular, the device may be a medical imaging device.

[0096] Specifically, medical imaging devices can be endoscopes, ultrasound machines, etc.

[0097] Of course, the devices according to the present invention are not limited to the examples disclosed above.

[0098] According to another aspect of the present invention, a vehicle is provided, comprising:

[0099] - At least one camera module, and

[0100] - At least one computing unit;

[0101] It is configured to implement all the steps of the method according to the invention.

[0102] According to some embodiments, the vehicle may be a land vehicle, such as an automobile, and may be autonomous, semi-autonomous, or non-autonomous.

[0103] According to some embodiments, the vehicle may be an aircraft, such as a drone, airplane, or helicopter, and may be autonomous, semi-autonomous, or non-autonomous.

[0104] According to some embodiments, the vehicle may be a marine vehicle, such as a ship or submarine, and may be autonomous, semi-autonomous, or non-autonomous.

[0105] Description of the Drawings and Detailed Description of the Embodiments

[0106] Other advantages and features will become apparent upon examination of the detailed description of the entirely non-limiting embodiments and the accompanying drawings, wherein:

[0107] - Figure 1 This is a schematic diagram illustrating non-limiting examples of different variables used in this invention;

[0108] - Figure 2-5 This is a schematic diagram of a non-limiting exemplary embodiment of the method according to the present invention;

[0109] - Figure 6 This is a schematic diagram of a non-limiting exemplary embodiment of the device according to the present invention;

[0110] - Figure 7-9 These are schematic diagrams of non-limiting exemplary embodiments of the device according to the present invention; and

[0111] - Figure 10 This is a schematic diagram of a non-limiting exemplary embodiment of a vehicle according to the present invention.

[0112] It should be clearly understood that the embodiments described below are by no means limiting. In particular, variations of the invention are conceivable that include only selections of features isolated from the other disclosed features, if such selection is sufficient to impart a beneficial technical effect or distinguish the invention from the prior art. Such selections include at least one preferred functional feature that has no structural details, or only a portion thereof, if such portion alone is sufficient to impart a beneficial technical effect or distinguish the invention from the prior art.

[0113] In particular, all the described variations and embodiments can be combined with each other if there are no technical obstacles.

[0114] In the remainder of the drawings and description, the same reference numerals are used for features common to multiple drawings.

[0115] Figure 1 This is a schematic diagram illustrating non-limiting examples of different variables used in this invention.

[0116] Figure 1 A camera module 102 is depicted. The camera module 102 includes an optical objective 104, also referred to as an optical system, which includes one or more optical elements, such as lenses. The camera module 102 also includes an image sensor 106, such as a CCD or CMOS sensor, which is arranged opposite to the optical objective 104. The function of the optical objective 104 is to guide light from different parts of the scene to be imaged onto the image sensor 106.

[0117] The distance (denoted as DOC) between the optical objective lens 104 and the image sensor 106 can be adjusted to change the area of ​​the scene that will be perceived as being in sharp focus on the image sensor 106. This distance DOC can be adjusted in various ways, for example by a mechanism (not shown) that changes the distance DOC by moving the image sensor 106, or the optical objective lens 104, or both.

[0118] Specifically, changing the distance DOC modifies the focus 110 of the camera module 102. The distance between the camera module 102 and the focus 110 is referred to below as the focus distance DF.

[0119] When the camera module 102 is adjusted to a given focus distance DF, the focus point 110 at focus distance DF is in sharp focus on the image sensor 106. Furthermore, the image sensor 106 also receives some points located behind the focus point 110 and some other points located before the focus point 110, and these points are also in sharp focus. Therefore, the image sensor receives the region including the focus point 110 in sharp focus. The distance range including the focus point 110 and received in sharp focus on the image sensor 102 is called the depth of field, denoted by PC, where PC = {PC...} min PC max},in

[0120] PC min The distance between camera module 102 and the nearest plane of the scene that is clearly focused on and received on image sensor 106; and

[0121] PC max The distance between the camera module 102 and the farthest plane of the scene that is clearly focused and received on the image sensor 106.

[0122] In addition, depth PC max PC min The width is called the resolution range and is denoted as NET=PC. max PC min .

[0123] When DF is used at different focusing distances i -DF n Acquire multiple images I1-IM n At that time, with the focus distance DF i Each image captured IM i The scene is clearly and accurately depicted at the focus distance DF. i Related depth PC i ={PC i,min PC i,max Objects / regions at a distance within (relative to) the image sensor. Image IM i With resolution range NET i =PC i,max PC i,min .

[0124] Figure 2 This is a schematic diagram of a first non-limiting exemplary embodiment of the method according to the present invention.

[0125] Figure 2 Method 200 can be used to obtain an image stack of a scene using at least one camera module, the image stack including images acquired at different focus distances (and therefore different depths of field).

[0126] At least one camera module may include an image sensor associated with an optical objective, such as... Figure 2 The camera module 202. An imaging device for imaging a scene may include a single camera module. Alternatively, an imaging device may include multiple camera modules.

[0127] Method 200 includes step 202, based on a predetermined minimum distance D. min and / or the predetermined maximum distance D max This determines one or more focus distances used to acquire an image of the scene.

[0128] According to an exemplary embodiment, the focus distance can be selected as follows:

[0129] - Select the first focus distance DF1, which has a first depth of field PC1={PC 1,min PC 1,max}, making the lower limit PC of depth of field PC1 1,min Equal to or slightly less than the minimum distance D min ;

[0130] - Select the final focus distance (DF) n It has the final depth of field PC n ={PC n,min PC n,max}, making the depth PC n PC limit n,max Equal to or slightly greater than the maximum distance D max ;

[0131] - And optionally, at focus distances DF1 and DF n Select one or more focus distances DF i .

[0132] This embodiment does not necessarily guarantee the continuity of sharpness between the acquired images.

[0133] According to another exemplary embodiment, the focusing distance can be from the minimum distance D min Start selecting:

[0134] - Select the first focus distance DF1, which has a first depth of field PC1={PC 1,min PC 1,max}, making the lower limit PC of depth of field PC1 1,min Equal to or slightly less than the minimum distance Dmin ;

[0135] - Then, select the next second focus distance DF2, which has a second depth of field PC2={PC 2,min PC 2,max}, making the lower limit PC of depth of field PC2 2,min Equal to or slightly below the upper limit of the depth of field PC1 associated with the focus distance DF1. 1,max ;

[0136] - And so on, until the distance to DF n It has a depth-of-field PC n ={PC n,min PC n,max}, where the upper limit is PC n,max Equal to or slightly greater than the maximum distance D max .

[0137] As an alternative to the aforementioned method, the focusing distance can be adjusted from the maximum distance D. max Start by selecting, instead of starting with the minimum distance D. min Begin the selection. Based on another alternative, we can start with the minimum distance D. min and maximum distance D max Equidistant center distance D moy Let's begin. Of course, these examples are not restrictive.

[0138] This embodiment enables the assurance of sharpness continuity between images to be acquired: specifically, two images acquired at two consecutive focus distances in ascending or descending order enable the depiction of a scene with two adjacent or overlapping depths of field. In the given example, and without loss of generality, step 202 determines the focus distance used to achieve sharpness continuity across all acquired images.

[0139] In each case, step 202 provides the focus distances DF1-DF1 for acquiring the image of the scene. n .

[0140] Next, method 200 includes focusing at the focal distance DF1-DF n Acquire image IM1-IM n Step 204. Image IM1-IM n Some of these images can be acquired sequentially by the same camera module. Alternatively or additionally, images IM1-IM2... n Some of the data can be captured simultaneously or sequentially by at least two camera modules.

[0141] The focus distance DF of a camera module can be modified in several ways. In one example, the focus distance DF of a camera module can be modified by changing the distance DOC between the image sensor and the optical objective of the camera module.

[0142] Depth of field associated with focusing distance is an optical variable that depends on the architecture of the camera module, particularly the optical objectives. Typically, depth of field associated with focusing distance is provided by the manufacturer or pre-measured and stored in association with the camera module, such as in the imaging device containing the camera module.

[0143] Figure 3 This is a schematic diagram of a method according to another non-limiting exemplary embodiment of the present invention.

[0144] Figure 3 Method 300 can be used to obtain an image stack of a scene using at least one camera module, the image stack including images acquired at different focus distances (and therefore different depths of field).

[0145] Method 300 includes Figure 2 Method 200 includes all steps.

[0146] Furthermore, prior to steps 202-204, method 300 includes step 302, which targets at least one of the camera modules, particularly each focus distance DF. i Determine the focusing distance DF i Related depth PC i .

[0147] Such measurements can be performed in different ways.

[0148] Based on a non-limiting exemplary example, and without loss of generality, the depth of field measurement of the focusing distance is performed as follows:

[0149] - Adjust the camera module to obtain the focus distance DF i For example, by changing the distance between the image sensor and the optical lens of the camera module;

[0150] - Use the camera module to image a reference scene (where the distance is known) at the focus distance to obtain a calibration image;

[0151] - Identify the first sharp focus plane in the calibration image, i.e., the plane in the scene that is closest to the camera module in sharp focus: the known distance to this first sharp focus plane corresponds to the focus distance PF. i The lower limit of PF i,min ;

[0152] - Identify the last sharp focus plane in the calibration image, i.e., the plane in the scene that is farthest from the camera module in sharp focus: the known distance to this second sharp focus plane corresponds to the focus distance PF. i upper limit PF i,max .

[0153] This operation can be performed in step 302 for all focus distances of the camera module. If the imaging device includes multiple camera modules, step 302 can be repeated for each camera module sequentially or simultaneously. Therefore, at the end of step 302, for each focus distance DF of each camera module... i The focusing distance DF was obtained. i Related depth PC i .

[0154] Step 302 can be performed exactly before steps 202-204.

[0155] Alternatively, step 302 may be performed long before steps 202-204, for example, during the manufacture of the camera module, or during the integration of the camera module into the imaging device, or during the calibration operation of the camera module and / or the imaging device, or during the calibration operation of the imaging application installed in the imaging device.

[0156] Figure 4 This is a schematic diagram of a method according to another non-limiting exemplary embodiment of the present invention.

[0157] Figure 4 Method 400 can be used to obtain an image stack of a scene using at least one camera module, the image stack including images acquired at different focus distances (and therefore different depths of field).

[0158] Method 400 includes Figure 2 Method 200 includes all steps. Alternatively, method 400 may include... Figure 3 Method 300 includes all steps.

[0159] Method 400 further includes step 402, which measures the distance between the camera module and the nearest object / region in the scene to the camera module, denoted as DS. min Such measurements can be performed in different ways. For example, a lidar sensor can be used to perform the measurement. The distance DS measured in this way... min Used as the minimum distance D in step 202 min This makes D min =DS min Therefore, a minimum focusing distance DF1 can be selected to achieve sharp focusing imaging of the nearest object (area) in the scene in the corresponding image acquired at the stated focusing distance.

[0160] Method 400 further includes step 404, measuring the distance between the camera module and the object / region in the scene farthest from the camera module, denoted as DS. max Such measurements can be performed in different ways. For example, a lidar sensor can be used to perform the measurement. The distance DS measured in this way... max The maximum distance D is used in step 202. max This makes D max =DS max Therefore, the maximum focusing distance DF is selected. n So that at the said focus distance DF n The acquired image clearly focuses on the farthest object (area) in the scene.

[0161] Of course, method 400 may include only one of steps 402 and 404.

[0162] In addition, method 400 may optionally include step 302 for determining the depth of field at at least one focus distance.

[0163] Figure 5 This is a schematic diagram of a method according to another non-limiting exemplary embodiment of the present invention.

[0164] Figure 5 Method 500 can be used to obtain an image stack of a scene using at least one camera module, the image stack including images acquired at different focus distances (and therefore different depths of field).

[0165] Method 500 includes Figure 2 Method 200 includes all steps. Alternatively, method 400 may include... Figure 3 Method 300 includes all steps.

[0166] Method 500 further includes step 502, for selecting the minimum distance D. min This minimum distance can be selected by the user or manufacturer, or even by the imaging application used. This minimum distance may or may not depend on the scene. This minimum distance can depend on the imaging mode selected from multiple imaging modes, such as landscape mode, portrait mode, etc. In the given example, and without loss of generality, the minimum distance D... min It was selected manually. Minimum distance D min You can use a numeric keypad, cursor, or any other user interface to indicate.

[0167] Method 500 further includes step 504, for selecting the maximum distance D. maxThis maximum distance can be selected by the user or manufacturer, or even by the imaging application used. This maximum distance may or may not depend on the scene. This maximum distance can depend on the imaging mode selected from multiple imaging modes, such as landscape mode, portrait mode, etc. In the given example, and without loss of generality, the maximum distance D... max It was selected manually. Maximum distance D max You can use a numeric keypad, cursor, or any other user interface to indicate.

[0168] Of course, method 500 may include only one of steps 502 and 504.

[0169] In addition, method 500 may optionally include Figure 3 Step 302, to determine at least one depth of field at at least one focus distance.

[0170] In all the examples described, step 202 considers the predetermined minimum distance D. min To determine the focusing distance. Alternatively, step 202 may not be by considering the minimum distance D. min Instead of determining the focusing distance directly, the focusing distance is determined by considering the minimum technically achievable focusing distance of the at least one camera module. In this case, the selection of the focusing distance can begin or end with the minimum focusing distance and its associated depth of field. In this case, the minimum distance D... min It is selected as the lower limit of the minimum focusing distance that is technically achievable for the at least one camera module.

[0171] In all the examples described, step 202 considers the predetermined maximum distance D. max This is to determine the focusing distance. (For reference) Figure 2 An alternative to the described scheme, step 202, can determine the focus distance by considering the maximum technically achievable focus distance of the at least one camera module, rather than the maximum distance D. max To determine the focusing distance. In this case, the selection of the focusing distance can start or end from the maximum focusing distance and its associated depth of field. In this case, the maximum distance D max It is selected as the upper limit of the maximum focusing distance that is technically achievable by the at least one camera module.

[0172] Figure 6 This is a schematic diagram of a non-limiting exemplary embodiment of the device according to the present invention.

[0173] Figure 6 The device 600 can be used to acquire an image stack of a scene, the image stack including images acquired at different focus distances (and therefore different depths of field).

[0174] Figure 6 The apparatus 600 can be used to implement the method according to the invention, in particular Figure 2-5 Any one of methods 200, 300, 400, or 500.

[0175] Device 600 includes at least one camera module. In the described example, and by no means limiting, device 600 includes K camera modules 6021-602. K Where K ≥ 1. In the following text, reference numeral 602 refers to the camera module.

[0176] Each camera module 602 includes an optical objective lens associated with an image sensor, and optionally a mechanism for changing the focus distance of the camera module.

[0177] At least two camera modules can have the same focusing distance.

[0178] Alternatively or additionally, at least two camera modules can have different focus distances: in other words, one camera module can image the scene at a focus distance that the other camera module cannot reach.

[0179] The device 600 also includes a computing unit 604, which is configured to perform various steps of the method according to the invention.

[0180] The calculation unit 604 includes an optional module 606 for determining the depth of field at at least one focus distance, for example, determining the depth of field at at least one focus distance based on a reference scene. This module 606 can, for example, be configured / programmed to perform step 302 of methods 300, 400, and 500.

[0181] The calculation unit 604 also includes an optional module 608 for determining the distance to at least one object in the scene, particularly the distance to the nearest object and / or the distance to the farthest object. This optional module 608 may, for example, be configured / programmed to perform at least one of steps 402 and 404 of method 400.

[0182] The calculation unit 604 also includes an optional module 610 for selecting at least one distance, particularly a minimum distance and / or a maximum distance, to determine a focusing distance for imaging the scene. This optional module 610 may, for example, be configured / programmed to perform at least one of steps 502 and 504 of method 500.

[0183] The calculation unit 604 also includes a module 612 for selecting / determining a focus distance for imaging the scene. This module 612 may, for example, be configured / programmed to perform at least step 202 of any of methods 200, 300, 400, and 500.

[0184] The computing unit 604 also includes a module 614 for controlling at least one camera module to acquire images of the scene at different focusing distances. This module 614 may, for example, be configured / programmed to perform at least step 204 of any one of methods 200, 300, 400, and 500.

[0185] At least one of modules 606-614 can be a module that is independent of the other modules.

[0186] At least two of modules 606-614 can be merged into the same module.

[0187] At least one of the computing unit 604 and / or modules 606-614 may be a hardware module, such as a processor, electronic chip, etc.

[0188] At least one of the computing unit 604 and / or modules 606-614 may be a software module, such as a computer program.

[0189] At least one of the computing unit 604 and / or modules 606-614 may be a combination of at least one software module and at least one hardware module.

[0190] In particular, at least one of the computing unit 604 or modules 606-614 may be integrated into an electronic chip or into an application installed in a user device.

[0191] The apparatus 600 may also optionally include an image storage device 616 for storing the acquired images. The module 616 may be configured, for example, to store at least one of the images acquired during step 204 of methods 200, 300, 400, and 500.

[0192] Figure 7 This is a schematic diagram of a non-limiting exemplary embodiment of the device according to the present invention.

[0193] Figure 7 The device 700 includes devices configured to implement the present invention, particularly any one of methods 200, 300, 400, and 500.

[0194] Figure 7 The device 700 may include Figure 6 All components of the 600 device.

[0195] exist Figure 7 In the example shown, device 700 is a smartphone or tablet, which includes Figure 6 All components of device 600. (Non-exhaustive list) Figure 7 The device 700 includes a camera module 602 and a computing unit 604.

[0196] In addition, the device 700 includes a display screen 702 equipped with a sensing surface 704, such as a capacitive sensing surface.

[0197] Of course, device 700 may include other components besides those mentioned above.

[0198] Figure 8 This is a schematic diagram of a device according to another non-limiting exemplary embodiment of the present invention.

[0199] Figure 8 The device 800 includes devices configured to implement the present invention, particularly devices implementing any one of the methods 200, 300, 400, and 500 of the present invention.

[0200] Figure 8 The device 800 may include Figure 6 All components of the 600 device.

[0201] exist Figure 8 In the example shown, device 800 is a virtual reality (VR) or augmented reality (AR) headset, which includes... Figure 6 All components of the device 600. In particular, Figure 8 The head-mounted device 800 shown includes a camera module 602 and a computing unit 604.

[0202] In addition, the head-mounted device 800 includes a display screen 802 in / on the goggles of the head-mounted device 800.

[0203] Of course, the head-mounted device 800 may include components other than those mentioned above.

[0204] Figure 9 This is a schematic diagram of a non-limiting exemplary embodiment of the device according to the present invention.

[0205] Figure 9 The device 900 includes devices configured to implement the present invention, particularly devices implementing any one of the methods 200, 300, 400, and 500 of the present invention.

[0206] Figure 9 Device 900 may include Figure 6 All components of the device 600 shown.

[0207] exist Figure 9 In the example shown, device 900 is a medical imaging device, such as an endoscope, ultrasound equipment, etc. Specifically, medical imaging device 900 includes a camera module 602 and a computing unit 604.

[0208] In addition, the medical imaging device 900 includes a display screen 902 equipped with a sensing surface 904, such as a capacitive sensing surface.

[0209] Of course, the medical imaging device 900 may include other components besides those indicated above.

[0210] Figure 10 This is a schematic diagram of a non-limiting exemplary embodiment of a vehicle according to the present invention.

[0211] Figure 10 The vehicle 1000 includes equipment configured to implement the present invention, particularly equipment for implementing any one of the methods 200, 300, 400 and 500 of the present invention.

[0212] Figure 10 The vehicle 1000 may include Figure 6 All components of the 600 device.

[0213] exist Figure 10 In the example shown, vehicle 1000 is a land vehicle, specifically an automobile, which includes Figure 6 All components of the device 600. In particular, the vehicle 1000 includes a camera module 602 and a computing unit 604.

[0214] In addition, the vehicle 1000 includes a display screen 1002 arranged in the passenger compartment of the vehicle 1000, the display screen 1002 being equipped with a sensing surface 1004, such as a capacitive sensing surface.

[0215] Of course, vehicle 1000 may include other components besides those mentioned above.

[0216] Of course, the present invention is not limited to the examples disclosed above.

Claims

1. A method for imaging a scene using at least one camera module (200; 300; 400; 500), each camera module including an optical objective associated with an image sensor; The method includes using different discrete focus distances DF i Acquire multiple images of the scene (IM1-IM2) n Each image has a corresponding depth of field PC. i ={PC i,min PC i,max The depth of field includes the corresponding focus distance DF. i And within the corresponding resolution range NET i =PC i,max PC i,min Extend upwards; With focus distance DF i The acquired image clearly and in focus depicts the scene located at the focus distance DF. i The associated depth PC i The portion within the distance; Its features are, The method further includes step (202), which determines: - The minimum focusing distance (DF1) makes the lower limit (PC1) of the associated depth of field (PC) 1,min ) less than or equal to the predetermined lower bound distance (D) min ); and / or - The maximum focusing distance (DF1) results in a depth of field (PC) associated with it. n The upper limit of PC n,max ) greater than the predetermined upper bound distance (D) max ).

2. The method according to any one of the preceding claims (200; 300; 400; 500), characterized in that, The predetermined lower bound distance (D) min ): - Selected by the user; or - Corresponds to the distance to the nearest object in the scene; or - With at least one camera module (6021-602) K The minimum allowable resolution distance corresponds to this.

3. The method (400) according to the preceding claim, characterized in that, It also includes the step (402) of determining the distance to the nearest object in the scene.

4. The method according to any one of the preceding claims (200; 300; 400; 500), characterized in that, The predetermined upper bound distance (D) max ): - Selected by the user; or - Corresponds to the distance to the farthest object in the scene; or - With at least one camera module (6021-602) K The maximum allowable resolution distance corresponds to this.

5. The method according to the preceding claim (200; 300; 400; 500), characterized in that, It also includes the step (404) of determining the distance to the farthest object in the scene.

6. The method (200; 300; 400; 500) according to any one of the preceding claims, characterized in that, For at least one focusing distance, the depth of field associated with said focusing distance is adjacent to or overlaps with the depth of field associated with the next focusing distance.

7. The method according to the preceding claim (200; 300; 400; 500), characterized in that, For each focusing distance, the depth of field associated with that focusing distance is adjacent to or overlaps with the depth of field associated with the next focusing distance.

8. The method (200; 300; 400; 500) according to any one of claims 6 or 7, characterized in that, For at least one focusing distance, the depth of field associated with said focusing distance is adjacent to the depth of field associated with the next focusing distance.

9. The method (200; 300; 400; 500) according to any one of claims 6 to 8, characterized in that, For at least one focusing distance, the depth of field associated with said focusing distance overlaps with the depth of field associated with the next focusing distance.

10. The method (200; 300; 400; 500) according to any one of the preceding claims, characterized in that, At least two images were acquired using the same camera module (602), and in particular all images were acquired using the same camera module (602).

11. The method according to any one of the preceding claims, characterized in that, At least two images were taken using different camera modules (6021-602). K (Originated) 12. A computer program comprising executable instructions that, when executed by a computing device, implement all the steps of the method (200; 300; 400; 500) according to any one of the preceding claims.

13. A device (600; 700; 800; 900), comprising: - At least one camera module (602), - At least one computing unit (604); It is configured to implement all the steps of the method (200; 300; 400; 500) according to any one of claims 1 to 11.

14. A vehicle (1000), comprising: - At least one camera module (602), - At least one computing unit (604); It is configured to implement all the steps of the method (200; 300; 400; 500) according to any one of claims 1 to 11.

Images