Method and device for determining a depth map of a scene from images of said scene

Abstract

The invention relates to a method (100) for determining a depth map of a scene, comprising the following steps: - estimating (104) a plurality of first depth maps (PCP1-PCPn) of said scene, each one obtained with a previously trained artificial intelligence, AI, model, taking as input a different respective source image (IM1-IMn) of said scene, - and obtaining (108) a depth map referred to as a consolidated depth map (CPC) from at least two of said first depth maps (PCP1-PCPn). The invention also relates to a computer program, a device, an apparatus and a vehicle implementing such a method.

Description

DESCRIPTION Title: Method and device for determining a depth map of a scene from images of said scene

[0001] The present invention relates to a method for determining a depth map of a scene from several two-dimensional (2D) image(s) of said scene. It also relates to a computer program and a device implementing such a method.

[0002] The domain of the invention is the domain of obtaining a depth map of a scene from image(s) of said scene. State of the art

[0003] The depth map of a scene, when only two-dimensional (2D) image(s) are available, is important data. It can be used to perform various functions, for example, to image the scene using a focus bracketing technique, or to provide a representation of the scene with a 3D effect, etc., particularly in a virtual reality (VR) or augmented reality (VA) headset.

[0004] Solutions based on artificial intelligence (AI) exist for estimating the depth map of a scene from a 2D image of that scene. In short, a 2D image of the scene is fed into an AI model, such as a convolutional neural network (CNN), which has been previously trained on a training dataset. This AI model then outputs an estimated depth map of the scene. Examples of known models available to professionals include the Marigold Depth Estimation Neural Network and the Depth Anything Neural Network, both of which can generate a depth map of a scene from a 2D image.

[0005] These solutions represent a significant advance, but still have drawbacks. In particular, the accuracy of these solutions depends of the training dataset used because the images processed when using the AI ​​model generally have differences with those of the training dataset, which leads to a decrease in the accuracy and performance of the AI ​​model when used.

[0006] Added to this is another, more significant difficulty related to the use of a 2D image of the scene. A 2D image provides limited information about the scene, entirely dependent on the viewpoint used to image it, and more generally on the image acquisition conditions. These conditions can cause optical effects on the 2D image, leading to errors in depth map estimation, not to mention optical effects deliberately introduced into the imaged scene that are then reflected in the 2D image provided as input to the AI ​​model.

[0007] One objective of the present invention is to remedy at least one of the aforementioned drawbacks.

[0008] Another objective of the invention is to provide a more precise and efficient solution for estimating the depth map of a scene from 2D image(s) of said scene. Description of the invention

[0009] The invention proposes to achieve at least one of the aforementioned goals by a method for determining a depth map of a scene comprising the following steps: - estimation of several initial depth maps of said scene, each obtained with a previously trained artificial intelligence (AI) model, taking as input a different respective source image of said scene, and - obtaining a consolidated depth map from at least two of the aforementioned first depth maps.

[0010] Thus, like current solutions, the invention proposes to determine the depth map of a scene using an AI model that takes as input a source image, in particular a 2D source image, of The scene is used to provide a depth map. Such models are familiar to those in the trade.

[0011] Unlike current techniques, and in an inventive manner, the invention proposes: - to use several different source images of the scene, - estimate several initial depth maps of said scene, and - obtain a consolidated depth map from said initial depth maps. Thus, the invention enables a more efficient and accurate estimation of a scene's depth map. Indeed, using multiple images of the scene provides more information about it, ultimately leading to a more precise estimation of the depth map. For example, using several different images of the scene makes the depth map estimation less susceptible to optical effects, whether intentionally introduced into the scene or not, and present in one of the source images.

[0012] By image, or source image, we mean a digital image, and in particular a raster image, and more specifically an RGB raster image for example.

[0013] By "depth of field" we mean the extent of the area of ​​sharpness that appears on an image, that is to say the area between the first sharp plane and the last sharp plane of the image.

[0014] By "extent of sharpness" we mean the distance over which the sharp part of the image extends, that is to say the distance from the first sharp plane to the last sharp plane of the image.

[0015] Focal length refers to the distance at which an optical lens is focused, relative to the lens's position. The focal length for capturing an image is typically adjusted by changing the distance between the image sensor and the optical lens. Thus, a first image of a scene acquired at a first focal length will clearly depict one part of the scene, and a second image of a scene acquired at a second focal length will clearly depict another part of the scene.

[0016] Following a non-limiting example, given to illustrate the definitions indicated above, the focusing distance can be adjusted to 15 meters. The depth of field can be 1.50 meters and the depth of field can be 14.50 meters to 16 meters. In this case, the image will sharply depict all objects, or parts of the scene, located between 14.5m and 16m from the optical lens.

[0017] By "angle of view", we mean the angular opening of the module allowing the image of the scene to be captured.

[0018] By "viewing angle", we mean two angles which allow us to characterize the direction in which the image is taken in relation to the scene.

[0019] The consolidated depth map can be obtained using any type of tool that takes as input at least two of the first depth maps.

[0020] Depending on the embodiment, the consolidated depth map can be obtained with a predefined mathematical relationship taking as input at least two of the first depth maps.

[0021] For example, the consolidated depth map can be a mathematical average of the first depth maps, for example at each point.

[0022] Optionally, it is possible to exclude one or more extreme depth maps to eliminate results that are furthest from other depth maps.

[0023] Depending on the embodiment, the consolidated depth map can be obtained using a second, pre-trained AI model. This second AI model can: - take as input, at least two initial maps of depth, and - provide the consolidated depth map as output.

[0024] Such a second AI model could, for example, be a neural network, in particular a deep learning neural network, in in particular a convolutional neural network or a regression neural network.

[0025] In this case, depending on the embodiment, the second AI model may include: - as many neurons as there are points in each depth map, or in the largest depth map one wishes to obtain; and - each neuron has as many inputs as there are depth maps. Of course, the number of neurons input to the second AI model may differ from what is indicated above, for example depending on the definition of the input depth maps, the spatial extent with which each respective source image represents the scene, etc.

[0026] Alternatively, the second AI model can be a machine vector support, a decision tree, etc., or any other AI model.

[0027] The second AI model can take as input only the first depth maps and generate the consolidated depth map based on said first depth maps.

[0028] According to embodiments, the second AI model can take as input, for at least one, in particular each, first depth map, at least one image parameter, or a set of several image parameters, relative to the source image from which said first depth map was generated.

[0029] In this case, the second AI model takes into account not only the first depth maps, but also the parameters of the source images used to generate said depth maps.

[0030] For example, an image parameter relative to an image may be relative to how said source image represents the scene.

[0031] Such an image parameter can be of any type.

[0032] Based on implementation examples, such an image parameter can be any one, or a combination of any one, of the following parameters: - focal distance with which the source image represents the scene; - field of view angle with which the source image represents the scene; - viewing angle with which the source image represents the scene; - position from which the source image represents the scene; - brightness with which the source image represents the scene; - etc. In this case, for example, the AI ​​model takes the following data as input: {(PCPi ; PI^-PIi" 1 ), ..., (PCP n ; Pl^-PIn" 1 )} with : - PCPi, ..., PCPn: the first depth maps, with n>l; and - Plj^PIj 171 the image parameters relative to the source image IMj with which the first depth map PCPj was obtained.

[0033] According to embodiments, the process according to the invention may further include a step of training the second AI model.

[0034] Such training can be carried out with a training database, called a second training database, comprising a multitude of training games, each training game including: - several initial depth maps, known as training maps, and - a consolidated depth map, known as a training map.

[0035] Each training game can be obtained in different ways.

[0036] Following an example implementation, a training set can be obtained from a scene, called a training scene. In this case, each initial training depth map can be obtained by inputting a different source image from the training scene into the first AI model: thus, it is possible to obtain several initial training depth maps by inputting several source images from the scene into the first AI model, for example, in turn. Furthermore, the consolidated training depth map can be determined: - either by designing / constructing said training scene, where - either by measurement with at least one measurement sensor, such as a LIDAR or a time-of-flight camera: By using a multitude of different training scenes, it is possible to obtain a multitude of training games comprising the second training database used to train the second AI model.

[0037] The second AI model can be trained with any known and suitable training algorithm. For example, the second AI model can be trained using a backpropagation algorithm. Of course, this example is only illustrative and is by no means exhaustive.

[0038] Of course, when image parameters are taken into account by the second AI model for the generation of the consolidated depth map, each training set may further include said image parameter(s), for each first training depth map.

[0039] Depending on the embodiment, the consolidated depth map can have the same definition as the source images given as input to the first models.

[0040] In this case, for each pixel of the source image(s) of the scene, the consolidated depth map indicates a depth value of the point in the scene to which each pixel corresponds.

[0041] If the source images do not all have the same resolution, it is possible to: - to keep the resolution of the image with the highest resolution, - to keep the resolution of the image with the lowest resolution high, or - choose another resolution.

[0042] In this case, for each pixel of the source image(s) of the scene, the consolidated depth map indicates a depth value for the point in the scene represented by said pixel.

[0043] Depending on the embodiment, the consolidated depth map may have a lower resolution than the source image(s). Indeed, a depth map is generally used to determine the distance between the image sensor and each object in the scene. Therefore, in some cases, it may be sufficient for the consolidated depth map to indicate a depth for the different areas of the scene, each corresponding to an object in the scene: in this case, the consolidated depth map will have a lower resolution than the source images.

[0044] In all cases, the depth map can be indexed to points in the scene, to indicate a depth at each of these points.

[0045] Depending on the implementation methods, the source images may have the same definition.

[0046] Alternatively, at least two source images of the scene may have different resolutions: in this case, the consolidated depth map may have the same resolution as the source image with the highest resolution, or the lowest resolution.

[0047] Each initial AI model can be any type of AI model.

[0048] Following an example implementation, at least one, in particular each, first model can be a convolutional neural network.

[0049] Such neural networks are well known to those in the field, such as, for example: - the Marigold Depth Estimation model, see https: / / github.com / prs-eth / Marigold for more information; - the Depth Anything model, see https: / / depth-anything.github.io / for more information, - etc.

[0050] As mentioned above, the respective source images input into the first AI model are all different. In other words, each first depth map is obtained with a different source image than the other first depth maps.

[0051] At least two respective source images of the scene, used to generate two initial depth maps, may differ from each other in various ways.

[0052] Following examples of implementation, at least two source images of the scene may differ from each other in that they are acquired at two different focal distances.

[0053] Thus, the source images represent the scene with different areas of sharpness. Using source images with different focal lengths increases and varies the scene information taken into account when creating the depth map of said scene, thereby reducing errors due to optical effects that may be present in a given source image and that would be related to the use of a particular localization distance.

[0054] Alternatively, or in addition, at least two source images of the scene may differ from each other in that they are acquired with different field angles.

[0055] Thus, one of the source images is sharper than the other, but the other provides a wider view of the scene. Using source images with different field of view increases and diversifies the scene information considered when creating the depth map of that scene. This reduces errors due to optical effects that may be present in a particular source image and related to the use of a specific field of view.

[0056] Alternatively, or in addition, at least two source images of the scene may differ from each other in that they are acquired from different viewing angles.

[0057] Thus, the two images represent the scene from different viewpoints. Using source images with different viewpoints increases and varies the scene information considered when creating the scene's depth map, thereby reducing errors due to optical effects that may be present in a given source image and related to the use of a particular viewpoint.

[0058] Following examples of implementation, at least two source images of the scene may differ from each other in that they are acquired from two different positions. Thus, the source images represent the scene from different positions. Using source images acquired from different positions increases and varies the information about the scene taken into account when creating the depth map of said scene, which reduces errors due to optical effects that may be present in a given source image and that would be related to the use of a particular acquisition position.

[0059] Following realization examples, at least two source images of the scene may differ from each other in that they are acquired with different brightness levels for the scene.

[0060] Thus, the source images represent the scene with varying brightness levels for part or all of the scene. Using source images with different brightness levels increases and diversifies the scene information considered when creating the scene's depth map, particularly regarding areas that might be overexposed or underexposed in one of the source images. This helps reduce errors due to brightness deficiencies in one of the source images or optical effects related to such brightness deficiencies.

[0061] According to embodiments, the process according to the invention may further include a spatial registration step of at least one source image.

[0062] This registration process eliminates any potential discrepancies between source images of the same scene. For example, the registration step eliminates differences in viewpoint, viewpoint, slight movements, etc., that may exist between the different source images.

[0063] The registration of a source image can be performed relative to another source image. In particular, one of the source images can be chosen as the reference image, and the other source image, and indeed each of the other source images, can be spatially registered relative to said reference image. Following a non-limiting example, the source image chosen as the reference image could be the one representing the scene with the widest field of view.

[0064] Alternatively, the registration of a source image can be performed against a reference format that is independent of the source images. In this case, the source image, and in particular each of the source images, can be registered against said reference format.

[0065] The registration of a source image can be performed using any known technique, for example, by using points of interest or objects located within the source image. Alternatively, or in addition, the registration of a source image can be performed using velocity or acceleration data associated with the pixels of the source image.

[0066] The registration step can be carried out before the generation of the first depth maps.

[0067] All source images can have the same size.

[0068] Alternatively, at least two source images can have different sizes, and in particular represent the scene from different angles. In this case, the smaller source image can be supplemented with predetermined values, especially constant ones, such as zero. Thus, there is no data loss.

[0069] Of course, other implementations are possible. For example, the larger source image can be reduced, for instance, by removing the data located at the periphery of the source image (this data is generally the least defined), to adjust its size to that of the smaller source image. Other implementation examples are possible.

[0070] All source images can have the same resolution.

[0071] Alternatively, at least two source images may have different definitions.

[0072] In this case, the least defined source image can be supplemented with predetermined values, specifically identical values ​​such as zero or undefined, to adjust the aspect ratio of said less defined source image to the aspect ratio of the most defined source image. Specifically, all source images can be adjusted to the aspect ratio of the most defined source image.

[0073] Alternatively, the higher-resolution source image can be adjusted to the aspect ratio of the lower-resolution source image, for example, by removing some pixels or merging pixels within the higher-resolution source image. In particular, all source images can be adjusted to the aspect ratio of the lower-resolution source image.

[0074] Alternatively, all source images can be adjusted to another resolution, for example to a resolution equal to the average resolution of the source images.

[0075] According to another aspect of the same invention, a computer program is proposed comprising executable instructions which, when executed by a computer device, implement all the steps of the process according to the invention.

[0076] The computer program can be in any computer language, such as for example machine language, C, C++, JAVA, Python, etc.

[0077] Such a computer program can be presented as a standalone application. Alternatively, such a computer program can be integrated into a photo or video application, or even into an image or video playback application.

[0078] The computer program can be stored in a non-transient, or non-volatile, manner in a storage medium.

[0079] According to another aspect of the invention, a device is proposed comprising means configured to implement all the steps of the process according to the invention.

[0080] The device according to the invention can be a computer, a processor, a computer chip, etc. programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0081] The device according to the invention can be integrated into any type of device such as a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, etc.

[0082] The device according to the invention may include, in terms of technical means, at least one, or any combination of at least two, of the characteristics described above with reference to the method according to the invention, and which will not be repeated here exhaustively for the sake of brevity.

[0083] In particular, the system may include: - a first module, running a first AI model, to generate several initial depth maps of a scene, each with a different respective source image of the scene, and - a second module to obtain a consolidated depth map of the scene from said first depth maps, for example with a mathematical relationship or with a second AI model executed by said second module.

[0084] According to another aspect of the invention, an apparatus is proposed, and in particular a user apparatus, comprising a device according to the invention.

[0085] In particular, the device can be a user device such as a smartphone, tablet, etc.

[0086] The user device may also include a display screen.

[0087] Alternatively, or in addition, the user device may further include at least one camera module to image a scene and obtain source images of said scene.

[0088] In particular, the device can be a user device such as a computer.

[0089] The computer-type user device may also include a display screen.

[0090] Alternatively, or in addition, the computer-type user device may further include at least one camera module to image a scene and obtain source images of said scene.

[0091] In particular, the device could be a television.

[0092] Television may also include a display screen.

[0093] Alternatively, or in addition, the television may further include at least one camera module to image a scene and obtain source images of said scene.

[0094] In particular, the device can be a virtual reality headset or glasses, or an augmented reality headset or glasses.

[0095] The helmet, or glasses respectively, according to the invention may further comprise at least one display screen.

[0096] Alternatively, or in addition, the helmet, or glasses respectively, according to the invention may further comprise at least one camera module for imaging a scene and obtaining source images of said scene.

[0097] In particular, the device may be a medical imaging device.

[0098] In particular, the medical imaging device can be an endoscope, an ultrasound machine, etc.

[0099] The medical imaging device may also include at least one display screen.

[0100] Alternatively, or in addition, the medical imaging device may further include at least one camera module to image a scene and obtain source images of said scene.

[0101] Of course, the device according to the invention is not limited to the examples of devices that have just been given.

[0102] According to another aspect of the present invention, a vehicle comprising a device according to the invention is proposed.

[0103] The vehicle may further include a display screen, for example arranged in a passenger compartment of the vehicle, or a projector to project at least one image onto a display surface generally known as a "head-up display".

[0104] Alternatively, or in addition, the vehicle may further include at least one camera module to image a scene and obtain source images of said scene.

[0105] Depending on the embodiment, the vehicle can be a land vehicle, such as a car, autonomous, semi-autonomous or non-autonomous.

[0106] Depending on the embodiment, the vehicle can be a flying vehicle, such as a drone, an airplane, a helicopter, autonomous, semi-autonomous or non-autonomous.

[0107] Depending on the embodiment, the vehicle can be a maritime vehicle, such as a boat or a submarine, autonomous, semi-autonomous or non-autonomous. Description of the figures and methods of implementation

[0108] Other advantages and features will become apparent upon examination of the detailed description of non-limiting embodiments and the accompanying drawings, in which: - FIGURE 1 is a schematic representation of a non-limiting example of an embodiment of a process according to the invention; - FIGURE 2 is a schematic representation of a non-limiting example of an embodiment of a device according to the invention; - Figures 3-5 are schematic representations of non-limiting examples of embodiments of a device according to the invention; and - FIGURE 6 is a schematic representation of a non-limiting example embodiment of a vehicle according to the invention.

[0109] It is understood that the embodiments described below are by no means exhaustive. In particular, variants of the invention may be conceived comprising only a selection of the features described below, isolated from the other features described, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from the prior art. This selection includes at least one preferably functional feature without structural details, or with only a portion of the structural details if this portion alone is sufficient to confer a technical advantage or to differentiate the invention from the prior art.

[0110] In particular, all the variants and embodiments described can be combined with each other if there are no technical obstacles to this combination. [YES] In the figures and in the rest of the description, elements common to several figures retain the same reference.

[0112] FIGURE 1 is a schematic representation of a non-limiting example of an embodiment of a method according to the present invention.

[0113] Method 100 of FIGURE 1 can be used to determine, by estimation, the depth map of a scene from source images of the scene.

[0114] The process 100 includes a phase 102 of estimating the depth map of the scene from at least two source images of the scene which are different.

[0115] Estimation phase 102 includes a step 104 for determining a first depth map from a source image of the scene, using a first artificial intelligence (AI) model, and in particular a first neural network previously trained for this purpose. Such a first AI model can be any type of model already known for estimating the depth map of a scene from a source image of said scene. such as, for example, the Marigold Depth Estimation model, the Depth Anything model, etc.

[0116] This step 104 provides for an IMi image as the source of the scene a first depth map, PCPi, which is stored during a step 106.

[0117] Steps 104 and 106 are repeated as many times as desired to generate several first depth maps, each with a respective source image of the scene, different from the other source images used for generating the other first depth maps.

[0118] Steps 104 and 106 are repeated at least twice. Following a general formulation, step 102 is repeated N times, with N > 2, using N source images denoted IMi-IM n , preferably all different from each other, to generate N first depth maps, noted PCPi-PCPn, for the same scene.

[0119] Then, in step 108, a consolidated depth map, denoted CPC, is generated from at least two of the said, in particular from all the first depth maps PCPi-PCPn.

[0120] The consolidated CPC depth map can be generated in different ways.

[0121] In some embodiments, the consolidated CPC depth map can be generated using a predetermined mathematical relationship. For example, the consolidated CPC depth map can be obtained by averaging the initial PCPi-PCPn depth maps, specifically point by point. In another example, the consolidated CPC depth map can be obtained by averaging, for each point, the extreme depth values, {min; max}, obtained for all the initial PCPi-PCPn depth maps. In yet another embodiment, the consolidated CPC depth map can be obtained by averaging the initial PCPi-PCPn depth maps, specifically point by point, after excluding the extreme values, {min; max}, for each point. These embodiments are by no means exhaustive.

[0122] Depending on embodiments, the consolidated CPC depth map can be generated using a second artificial intelligence model, AI, previously trained.

[0123] This second AI model can be any type of suitable AI model.

[0124] Following one embodiment example, the second AI model can be a convolutional neural network pre-trained to provide the consolidated CPC depth map as a function of at least two of, and in particular all of, the first PCPi-PCPn depth maps.

[0125] Optionally, the second AI model can also take as input, for at least one and in particular each, first depth map, at least one image parameter relating to the source image used to generate said first depth map. This image parameter can relate to how the scene is represented in said source image, for example, the focal length of the source image, the field of view of the source image, the viewing angle of the source image, the position in which the source image represents the scene, the direction in which the source image represents the scene, the brightness of the image, etc.

[0126] As mentioned above, at least two of, and in particular all of, the source images used to generate the first depth maps are different from each other.

[0127] For example, at least two source images differ from each other in that they are acquired: - at two different focusing distances, - with different field angles, - with different viewing angles, - from two different positions, and / or - with different lighting levels for the scene. Thus, each of these source images provides different information about the scene, which allows for the generation of initial depth maps that are potentially different as well.

[0128] When the consolidated CPC depth map is generated by an AI model, particularly a convolutional neural network, the process 100 may optionally include a phase 110 of training said neural network.

[0129] Training phase 110 can be carried out just before scene depth map estimation phase 102, so that estimation phase 102 is carried out immediately after training phase 110.

[0130] Alternatively, training phase 110 can be carried out well before scene depth map estimation phase 102, so that estimation phase 102 is not carried out immediately after training phase 110 and a non-negligible time lapse occurs between training phase 110 and estimation phase 102.

[0131] The training phase 110 is common to several iterations of the estimation phase 102, so the second model trained during training phase 110 is used in several iterations of estimation phase 102, for the same scene or for different scenes. Alternatively, training phase 110 can be specific to an estimation phase and be repeated before each estimation phase 102, for example, to update the second AI model based on a particular type of scene.

[0132] When the second model is a convolutional neural network, the training phase 110 can be carried out in all possible ways.

[0133] In particular, training can be carried out using a training database, called a second training database, comprising a multitude of training games. Each training game may include: - several initial depth charts, known as training charts, designated PCPEi-PCPEk, - a consolidated depth chart, known as a training chart, denoted CPCE, and - optionally, for each first PCPi depth map, one or more image parameters relating to the source image with which said first depth map was generated.

[0134] Each training game can be obtained in different ways.

[0135] Following an example implementation, a training set can be obtained from a known training scene. In this case, each first training depth map (PCPEj) can be obtained by inputting a different source image of the training scene into the first AI model: thus, it is possible to obtain several first training depth maps (PCPEj) by inputting several source images of the scene into the first AI model. Furthermore, the consolidated training depth map (CPCE) can be known. - either by design / construction of said training scene, which, we recall, is known; - either by measurement with at least one measurement sensor, such as a LIDAR or a time-of-flight camera. By using a multitude of different training scenes, it is possible to obtain a multitude of training games for the second AI model, each training game including: - several initial training depth maps, PCPEi-PCPEk, each obtained by inputting a source image of the training scene into the first AI model - a consolidated training depth map, CPCE, obtained either by design or by measurement; and - optionally, for each first PCPi depth map, one or more image parameters relating to the source image with which said first depth map was generated.

[0136] The second AI model can be trained with any known and suitable training algorithm. For example, the second AI model can be trained using a backpropagation algorithm. Of course, this example is only illustrative and is by no means exhaustive.

[0137] The process according to the invention may optionally include a phase 120 of training the first AI model, carried out according to any known method.

[0138] In particular, the training of the first AI model can be carried out with a training dataset, called the first dataset training, including a multitude of training games, each training game including: - a source image for training, of a scene, and - a training depth map, of said scene.

[0139] Each training game can be obtained in different ways.

[0140] Following an example implementation, a training set can be obtained from a known training scene. In this case, a source image of the training scene is acquired: this source image constitutes the training source image that makes up a training set. Furthermore, the depth map of the training scene can be determined. - either by design / construction of said training scene, which, we recall, is known; - either by measurement with at least one measurement sensor, such as a LIDAR or a time-of-flight camera. This depth map can serve as the training depth map for the first model's training set. By using a multitude of different training scenes, it is possible to obtain a multitude of training games for the first AI model, each training game including: - a source image of a training scene; and - a depth map of said training scene.

[0141] The first AI model can be trained with any known and suitable training algorithm. For example, the first AI model can be trained using a backpropagation algorithm. Of course, this example is only illustrative and is by no means exhaustive.

[0142] Optionally, process 100 may include an optional step 132 of spatial registration of at least one source image.

[0143] In particular, the registration step 132 allows for the spatial registration of all source images.

[0144] This registration process eliminates discrepancies between source images depicting the same scene. For example, the registration step allows you to eliminate differences in view position, view direction, view angle, slight movements, etc. that may exist between the different source images of the same scene.

[0145] In the example of realization of FIGURE 1, in no way limitingly, at least one, in particular each, source image is spatially recalibrated with respect to the source image representing the scene with the largest field of view.

[0146] Registration is performed by detecting the same objects of interest in the source images and using their position to register the source images to each other.

[0147] The process 100 may further include an optional step 134 of adjusting at least one source image.

[0148] In particular, adjustment step 104 can adjust the size of at least one source image, especially when the source images are of different sizes. In the example shown in Figure 1, the size of the source images is adjusted, without limitation, to match the size of the largest source image, for example, the source image representing the scene with the widest field of view. In this case, each smaller source image (IMi) is padded with predetermined values, specifically constant values ​​such as zero or undefined values, to make it the same size as the largest source image.

[0149] Alternatively, or in addition, step 104 can adjust the resolution of the source images, particularly when the source images have different resolutions. In the example shown in Figure 1, the resolution of the source images is adjusted, without limitation, to match the resolution of the highest-resolution source image, for example, the image representing the scene at the greatest zoom level. In this case, each lower-resolution source image (IMj) is supplemented with predetermined, or constant, values, such as zero or undefined values, so that it has the same resolution as the highest-resolution source image.

[0150] FIGURE 2 is a schematic representation of a non-limiting example embodiment of a device according to the present invention.

[0151] The device 200 in FIGURE 2 can be used to obtain, by estimation, a depth map of a scene from several source images of said scene, different from each other.

[0152] The device 200 of FIGURE 2 can be used to implement a method according to the invention, and in particular the method 100 of FIGURE 1.

[0153] Device 200 may include an optional module 202 for training the first AI model. This module 202 is, for example, configured / programmed to perform training phase 120 in FIGURE 1.

[0154] Device 200 may include an optional module 204 for training the second AI model. This module 204 is, for example, configured / programmed to perform training phase 110 of FIGURE 1.

[0155] Device 200 includes a module 206, running a first AI model 208, to provide an estimated depth map from a given source image as input to said first AI model. This module 206 is, for example, configured / programmed to perform step 104 of process 100 in FIGURE 1.

[0156] Device 200 includes a module 210 to provide a consolidated depth map from several estimated depth maps given as input to said module. For example, module 210 can run a second AI model 212. This module 210 is, for example, configured / programmed to perform step 108 of process 100 in Figure 1.

[0157] Device 200 may also optionally include storage means 214.

[0158] At least one of the modules 202-206 and 210 can be a module independent of the others.

[0159] At least two of the modules 202-206 and 210 can be integrated within the same module. In particular, modules 202-206 and 210 can be integrated within a computing unit 220.

[0160] At least one of modules 202-206 and 210 can be a hardware module, such as a processor, an electronic chip, etc.

[0161] At least one of modules 202-206 and 210 may be a software module, such as a computer program.

[0162] At least one of the modules 202-206 and 210 may be a combination of at least one software module and at least one hardware module.

[0163] In particular, at least one of the modules 202-206 and 210 can be integrated into an electronic chip, or into an application installed in a user device.

[0164] In particular, the 220 computing unit can be, or can be integrated, into an electronic chip, or into an application installed in a user device.

[0165] FIGURE 3 is a schematic representation of a non-limiting example embodiment of a device according to the present invention.

[0166] The apparatus 300 of FIGURE 3 includes means configured to implement the invention, and in particular the method 100 of FIGURE 1.

[0167] The device 300 of FIGURE 3 may include a device according to the invention, and in particular the device 200 of FIGURE 2.

[0168] In the example shown in FIGURE 3, device 300 is a smartphone, or a tablet, comprising device 200 from FIGURE 2.

[0169] Optionally, the 300 unit may also include: - a 302 camera module, in particular for taking source images of the scene which can be used to estimate the depth map of said scene, according to the invention; and - a display screen 304, optionally equipped with a detection surface 306, for example capacitive.

[0170] Of course, the 300 device may include other components than those indicated above.

[0171] FIGURE 4 is a schematic representation of another non-limiting embodiment of a device according to the present invention.

[0172] The apparatus 400 of FIGURE 4 includes means configured to implement the invention, and in particular the method 100 of FIGURE 1.

[0173] The device 400 of FIGURE 4 may include a device according to the invention, and in particular the device 200 of FIGURE 2.

[0174] In the example shown in FIGURE 4, device 400 is: - a virtual reality (VR) headset or glasses, or - an augmented reality headset, or glasses; including device 200 of FIGURE 2.

[0175] Optionally, the 400 unit may also include: - a 402 camera module, in particular for taking source images of the scene which can be used to estimate the depth map of said scene, according to the invention; and - a 404 display screen, for example in / on a visor of said 400 helmet.

[0176] Of course, the 400 helmet may include other components than those listed above.

[0177] FIGURE 5 is a schematic representation of a non-limiting example embodiment of a device according to the present invention.

[0178] The apparatus 500 of FIGURE 5 includes means configured to implement the invention, and in particular the method 100 of FIGURE 1.

[0179] The device 500 of FIGURE 5 may include a device according to the invention, and in particular the device 200 of FIGURE 2.

[0180] In the example shown in FIGURE 5, device 500 is a medical imaging device, such as an endoscope, an ultrasound machine, etc.

[0181] Optionally, the 500 unit may also include: - a 502 camera module, in particular for taking source images of the scene which can be used to estimate the depth map of said scene, according to the invention; - a display screen 504, optionally equipped with a sensing surface 506, for example capacitive.

[0182] Of course, the 500 medical imaging device may include other organs than those indicated above.

[0183] FIGURE 6 is a schematic representation of a non-limiting example embodiment of a vehicle according to the present invention.

[0184] The vehicle 600 of FIGURE 6 includes means configured to implement the invention, and in particular the method 100 of FIGURE 1.

[0185] The vehicle 600 of FIGURE 6 may include a device according to the invention, and in particular the device 200 of FIGURE 2.

[0186] In the example shown in FIGURE 6, vehicle 600 is a land vehicle, in particular a car, comprising device 200 of FIGURE 3.

[0187] Optionally, the 600 vehicle can also include: - a 602 camera module, in particular for taking source images of the scene which can be used to estimate the depth map of said scene, according to the invention; - a display screen 604, optionally equipped with a sensing surface 606, for example capacitive, arranged in the passenger compartment of the vehicle 600.

[0188] Of course, the 600 vehicle may include other components than those indicated above.

[0189] Of course, the invention is not limited to the examples that have just been described.

Claims

DEMANDS 1. Method (100) for determining a depth map of a scene comprising the following steps: - estimation (104) of several initial depth maps (PCPi-PCPn) of said scene, each obtained with an artificial intelligence model (208), AI, previously trained, taking as input a respective different source image (IMi-IM n ) of said scene, - obtaining (108) a consolidated depth map (CPC) from at least two of the aforementioned first depth maps (PCPl-PCPn).

2. Method (100) according to the preceding claim, characterized in that the consolidated depth map (CPC) is obtained with a second AI model (212) previously trained: - taking as input at least two initial depth maps (PCPi-PCPn), and - providing the consolidated depth map (CPC) as output.

3. Method (100) according to the preceding claim, characterized in that the second AI model (212) further takes as input, for at least one, in particular each, first depth map, at least one image parameter, or a set of several image parameters, relating to the source image from which said first depth map was generated.

4. Method (100) according to any one of the preceding claims, characterized in that it further comprises a step (110) of training the second AI model (212).

5. A method (100) according to any one of the preceding claims, characterized in that the consolidated depth map (CPC) has the same definition as the source images (IMi-IM n ).

6. Method (100) according to any one of the preceding claims, characterized in that at least a first AI model (208) is a convolutional neural network.

7. A method (100) according to any one of the preceding claims, characterized in that at least two source images (IMi-IM n ) of the scene differ from one another in that they are acquired: - at two different focusing distances, - with different field angles, - with different viewing angles, - from two different positions, and - with different lighting levels for the scene.

8. Computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the process (100) according to any one of the preceding claims.

9. Device (200) comprising means (202-214,220) configured to carry out all the steps of the process (100) according to any one of claims 1 to 7.

10. Apparatus (300;400;500) comprising a device (200) according to the preceding claim.

11. Vehicle (600) comprising a device according to claim 10.

Images