Method and device for estimating a depth map associated with a digital hologram representing a scene, and associated computer program
The use of a convolutional neural network to process disjoint thumbnails of reconstructed images from digital holograms addresses the computational inefficiencies of existing methods, enhancing speed and accuracy in depth estimation.
Patent Information
- Application Number
- FR2022011433
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-11-03
- Publication Date
- 2024-12-13
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing methods for determining depth from digital holograms are computationally expensive and time-consuming due to the need for extensive processing of multiple holographic reconstruction planes.
A method utilizing a convolutional neural network to process disjoint thumbnails of reconstructed images from a digital hologram, where each thumbnail is associated with a depth value, allowing for faster and more precise determination of depth values through an artificial neural network.
The method significantly reduces computational resources and processing time while improving the accuracy of depth estimation in digital holograms by leveraging the independence of thumbnails and the efficiency of neural networks.
Smart Images
Figure 00000016_0000 
Figure 00000016_0001 
Figure 00000016_0002
Abstract
Description
Title of the invention: Method and device for estimating a depth map associated with a digital hologram representing a scene and associated computer program Technical field of the invention
[0001] The present invention relates to the technical field of digital holography.
[0002] It relates in particular to a method and device for estimating a depth map associated with a digital hologram representing a scene. It also relates to an associated computer program. State of the art
[0003] Digital holography is an immersive technology that records the characteristics of a wave diffracted by an object present in a three-dimensional scene in order to reproduce a three-dimensional image of this object. The digital hologram obtained then contains all the information allowing this three-dimensional scene to be described.
[0004] However, extracting this information from the digital hologram itself in order to reconstruct the three-dimensional scene is not simple because the plane of the digital hologram does not include the spatial location of the objects in the three-dimensional scene.
[0005] It is known in particular to use methods for determining depth (or "Depth from focus" according to the original Anglo-Saxon term) relative to the plane of the hologram to determine, from the hologram, information concerning the geometry of the scene. An example of such a method is for example described in the article "Depth from focus", by Grossmann, Pavel, Pattern Recognit. Lett. 5, 63-69, 1987.
[0006] In such methods, a reconstruction volume is obtained from several holographic reconstruction planes calculated at different focusing distances chosen in a predefined interval. From these reconstruction planes, the associated depth is estimated by applying, on each of these planes, focusing operators in order to select, for each pixel, the depth of the reconstruction plane for which the focusing is optimal.
[0007] However, such methods are long to implement and very expensive in terms of computing resources. Presentation of the invention
[0008] In this context, the present invention proposes to improve the determination of depth values relative to the plane of a digital hologram associated with a three-dimensional scene.
[0009] More particularly, according to the invention, a method is proposed for estimating a depth map associated with a digital hologram representing a scene, the method comprising steps of:
[0010] - reconstruction of a plurality of images of the scene from the hologram digital, each reconstructed image being associated with a depth of the scene,
[0011] - decomposition of each reconstructed image into a plurality of thumbnails, said thumbnails of said plurality being adjacent to each other, each thumbnail being associated with the depth of the scene corresponding to the reconstructed image concerned, each thumbnail comprising a plurality of pixels,
[0012] - determination, for each image, of a focusing map by providing, in input of an artificial neural network, values associated with said pixels of the image concerned, so as to obtain, at the output of the artificial neural network, said focusing map comprising a focusing level associated with the pixel concerned, and
[0013] - determination of a depth value, for each point of a map of depth, depending on the focus levels obtained respectively for the pixels associated with said point of the depth map in the focus maps respectively determined for the thumbnails containing a pixel corresponding to said point of the depth map.
[0014] Thus, the different processed images are independent of each other because they are formed from sets of disjoint pixels. This independence then allows for faster implementation of the method. In addition, the necessary computing resources are less expensive thanks to the limited number of areas to be analyzed (namely the different images).
[0015] Furthermore, the use of the artificial neural network allows faster processing of all the thumbnails and more precise determination of the focusing levels associated with the pixels of the thumbnails.
[0016] Other non-limiting and advantageous characteristics of the method according to the invention, taken individually or in all technically possible combinations, are the following:
[0017] - for each point in the depth map, the associated depth value is obtained by determining the depth corresponding to the highest focus level among all the pixels associated with said point of the depth map in the thumbnails containing said pixels;
[0018] - the artificial neural network is a convolutional neural network; and
[0019] - the reconstruction step is implemented in such a way that the images re constructed are respectively associated with uniformly distributed depths between a minimum depth and a maximum depth of the scene.
[0020] The present invention also relates to a device for estimating a depth map associated with a digital hologram representing a scene, the device comprising:
[0021] - a module for reconstructing a plurality of images of the scene from the digital hologram, each reconstructed image being associated with a depth of the scene,
[0022] - a module for decomposing each reconstructed image into a plurality of thumbnails, said thumbnails of said plurality being adjacent to each other, each thumbnail being associated with the depth of the scene corresponding to the reconstructed image concerned, each thumbnail comprising a plurality of pixels,
[0023] - a module for determining, for each image, a focusing map by providing, as input to an artificial neural network, the values associated with said pixels of the image concerned, so as to obtain, as output from the artificial neural network, said focusing map comprising a focusing level associated with the pixel concerned, and
[0024] - a module for determining a depth value, for each point of a depth map as a function of the focus levels obtained respectively for the pixels associated with said point of the depth map in the thumbnails containing said pixels.
[0025] The present invention finally relates to a computer program comprising instructions executable by a processor and designed to implement a method as introduced previously when these instructions are executed by the processor.
[0026] Of course, the various features, variants and embodiments of the invention may be combined with each other in various combinations to the extent that they are not incompatible or mutually exclusive. Detailed description of the invention
[0027] Furthermore, various other characteristics of the invention emerge from the appended description given with reference to the drawings which illustrate non-limiting forms of embodiment of the invention and where:
[0028] [Fig-1] represents, in functional form, a device for estimating a map of depth designed to implement a method for estimating a depth map in accordance with the invention,
[0029] [Fig.2] represents an example of a digital hologram associated with the estimated depth map according to the invention,
[0030] [Fig.3] is a schematic representation of an example of the architecture of an artificial neural network implemented during the process of estimating a map of depth in accordance with the invention, and
[0031] [Fig.4] represents, in the form of a flowchart, an example of a method for estimating a depth map in accordance with the invention.
[0032] It should be noted that, in these figures, the structural and / or functional elements common to the different variants may have the same references.
[0033] [Fig.l] represents, in functional form, a device 1 for estimating (also referred to as device 1 in the following) a depth map C from a digital hologram H.
[0034] The digital hologram H represents a given three-dimensional scene. This three-dimensional scene comprises, for example, one or more objects. The three-dimensional scene is defined in a frame of reference (O, x, y, z).
[0035] As shown in [Fig.2], the digital hologram H is defined by a matrix of pixels in the (x, y) plane. The z axis, called the z depth axis, is orthogonal to the (x, y) plane of the digital hologram H. As can be seen in [Fig.2], the digital hologram H is here defined in the plane of equation z=0.
[0036] The digital hologram H has, for example, a size of 1024x1024 pixels.
[0037] The device 1 for estimating a depth map C is designed to estimate the depth map C associated with the digital hologram H. For this, the device 1 comprises a processor 2 and a storage device 4. The storage device 4 is for example a hard disk or a memory.
[0038] The device 1 also comprises a set of functional modules. It comprises for example a reconstruction module 5, a decomposition module 6, a module 8 for determining a focusing map Ci;jjk and a module 9 for determining a depth value djs+q>ks+r.
[0039] Each of the different modules described is for example implemented by means of computer program instructions designed to implement the module concerned when these instructions are executed by the processor 2 of the device 1 for estimating the depth map C.
[0040] However, as a variant, at least one of the aforementioned modules can be produced by means of a dedicated electronic circuit, for example an integrated circuit with a specific application.
[0041] The processor 2 is also designed to implement an artificial neural network NN, involved in the method of estimating the depth map C associated with the digital hologram H.
[0042] An example of the architecture of this artificial neural network NN is shown in [Fig.3]. In this example, the artificial neural network NN is a convolutional neural network, for example of the U-Net type.
[0043] Generally, such an artificial neural network NN comprises a plurality of convolution layers distributed according to different levels, such as is explained below and shown in [Fig.3]. More details on a U-Net type neural network can also be found in the article “U-Net: Convolutional Networks for Biomedical Image Segmentation” by Ronneberger, O., Fischer, P. & Brox, T., CoRR, abs / 1505.04597, 2015.
[0044] In order to describe the architecture of the artificial neural network NN, it is considered here that an image Ie is provided as input to this artificial neural network NN. In practice, this image Ie is an image derived from the digital hologram H, as will be explained later.
[0045] As shown in [Fig.3], the artificial neural network NN here comprises a first part 10, a connecting bridge 20 and a second part 30.
[0046] The first part 10 is a so-called contraction part. Generally speaking, this first part 10 has the encoder function and makes it possible to reduce the size of the image provided as input while retaining its characteristics. For this purpose, it here comprises four levels 12, 14, 16, 18. Each level 12, 14, 16, 18 comprises a convolution block Conv and a subsampling block D.
[0047] The convolution block Conv comprises at least one convolution layer whose kernel is a matrix of size nxn. Preferably here, each convolution block has two successive convolution layers. Here, each convolution layer has a kernel with a matrix of size 3x3.
[0048] Then, the convolution layer (or the convolution layers if there are several) is followed by an activation function of the rectified linear unit type (or ReLu for "Rectified Linear Unit" according to the commonly used English term). Finally, the convolution block Conv comprises, for the result obtained after application of the activation function, a so-called batch normalization. This batch is here formed by the number of images provided as input to the artificial neural network NN. During the step of training the artificial neural network as described below, the size of the batch is greater than or equal to 1 (that is to say that at least two images are provided as input to allow the training of the artificial neural network). In the case of the method of estimating the depth map as described later, the size of the batch is, for example here, given by the number of reconstructed images h (see below).
[0049] As shown in [Fig.3], at the output of the convolution block Conv, each level 12, 14, 16, 18 comprises the sub-sampling block D. This sub-sampling block D makes it possible to reduce the dimensions of the result obtained at the output of the convolution block Conv. This involves, for example, a reduction by 2 of these dimensions, for example by selecting the maximum pixel value among the four pixels of a pixel window of size 2x2 (this is then referred to as “max-pooling 2x2” according to the commonly used English expression).
[0050] Thus, taking the example shown in [Fig.3], the input image Ie is provided as input to the first level 12 of the first part 10. The convolution block Conv and the sub-sampling block D of this first level 12 then make it possible to obtain, as output, a first data item X0 0. This first data item X0 0 has, for example, dimensions reduced by half compared to the input image Ie.
[0051] Then, this first data item X0'0 is provided as input to the second level 14 of the first part 10 so as to obtain, at the output thereof, a second data item X1'°. This second data item X10 has, for example, dimensions reduced by half compared to the first data item X0'°.
[0052] The second data X1'0 is then provided as input to the third level 16 of the first part 10 so as to obtain, at the output thereof, a third data item X2'°. This third data item X2'0 has, for example, dimensions reduced by half compared to the second data item X10.
[0053] Then, this third data X2'0 is provided as input to the fourth level 18 of the first part 10 so as to obtain, as output, a fourth data X3 0. This fourth data X3- 0 has, for example, dimensions reduced by half compared to the third data X2- °.
[0054] Thus, the operations of processing the input image L by the first part 10 of the artificial neural network NN can be expressed in the following form:
[0055] xiJ = D ( Conv ( j ) )
[0056] with j=0 and i between 1 and 4, the operator Conv corresponding to the processing implemented by the convolution block Conv and the operator D being associated with the sub-sampling block D.
[0057] As can be seen in [Fig.3], the artificial neural network NN comprises, at the output of the first part 10, the connecting bridge 20. This connecting bridge 20 makes it possible to make the link between the first part 10 and the second part 30 of the artificial neural network NN. It comprises a convolution block Conv as described previously. Here, it thus receives as input the fourth data item X3'0 and provides, as output, a fifth data item X4- °.
[0058] The second part 30 of the artificial neural network NN is called expansion. Generally, this second part 30 presents the decoder function and makes it possible to form an image having the size of the image provided as input and which only contains the characteristics essential to the processing.
[0059] For this, the second part 30 here comprises four levels 32, 34, 36, 38. By analogy with the first part 10, the first level 38 of the second part 30 is defined as that positioned at the same level as the first level 12 of the first part 10. The second level 36 of the second part 30 is positioned at the same level as the second level 14 of the first part 10 of the artificial neural network NN. The third level 34 of the second part 30 is positioned at the same level as the third level 16 of the first part 10 of the artificial neural network NN. Finally, the fourth level 32 of the second part 30 is positioned at the same level as the fourth level 18 of the first part 10 of the artificial neural network NN. This definition is used to match the levels of the artificial neural network processing data of the same dimensions.
[0060] Each level 32, 34, 36, 38 comprises an oversampling block U, a concatenation block Conc and a convolution block Conv (such as the one introduced previously in the first part).
[0061] Each oversampling block U aims to increase the dimensions of the data received as input. This is an “upscaling” operation according to the commonly used English expression. For example, here, the dimensions are multiplied by 2.
[0062] Following the oversampling block U, each level 32, 34, 36, 38 comprises the concatenation block Conc. The latter aims to concatenate the data obtained at the output of the oversampling block U of the level concerned with the data of the same size obtained at the output of one of the levels 12, 14, 16, 18 of the first part 10 of the artificial neural network NN. The involvement of data from the first part of the artificial neural network NN in the concatenation operation is represented by broken lines in [Fig.3].
[0063] This concatenation block then allows the transmission of the extracted high frequency information obtained in the first part 10 of the artificial neural network NN also in the second part 30. Without this concatenation block Conc, this information could be lost following the multiple undersampling and oversampling operations present in the artificial neural network NN.
[0064] Then, at the output of the concatenation block Conc, each level 32, 34, 36, 38 of the second part 30 comprises a convolution block Conv such as that described previously in the first part 10 of the artificial neural network NN. Each convolution block Conv comprises in particular here at least one convolution layer followed by an activation function of the rectified linear unit type and a batch normalization operation.
[0065] Thus, on the basis of the example shown in [Fig.3], the fifth data item X4'0 is provided as input to the fourth level 32 of the second part 30. The oversampling block U then makes it possible to obtain, as output, a first intermediate data item Xintl, which has the same dimensions as the fourth data item X3 0 obtained as output from the fourth level 18 of the first part 10. This first intermediate data item mediator Xintl and the fourth data X3'0 are then concatenated by the concatenation block Conc. The result obtained at the output of the concatenation block Conc is then provided as input to the convolution block Conv so as to obtain, at the output, a sixth data X3' *.
[0066] This sixth data item X3'1 is then provided as input to the third level 34 of the second part 30, and more particularly, as input to the oversampling block U. At the output of this oversampling block U, a second intermediate data item X int2, of the same dimensions as the third data item X2' °, is obtained. The second intermediate data item Xint2 and the third data item X2'0 are concatenated by the concatenation block Conc. The result obtained at the output of the concatenation block Conc is provided as input to the convolution block Conv so as to obtain, at the output, a seventh data item X2'2.
[0067] Then, as shown in [Fig.3], the seventh data item X2'2 is provided as input to the second level 36 of the second part 30 (and therefore as input to the oversampling block U of this second level 36). A third intermediate data item Xint3 is obtained as output from this oversampling block U. This third intermediate data item Xint3 has the same dimensions as the second data item X1'°. The third intermediate data item Xint3 and the second data item X10 are then concatenated by the concatenation block Conc. The result obtained as output from the concatenation block Conc is provided as input to the convolution block Conv so as to obtain, as output, an eighth data item X13.
[0068] Then, this eighth data item X1'3 is provided as input to the first level 38 of the second part 30. The oversampling block U then makes it possible to obtain a fourth data item Xint4. The latter has the same dimensions as the first data item X0'°. The fourth intermediate data item Xint4 and the first data item X0'0 are then concatenated by the concatenation block Conc. The result obtained at the output of the concatenation block Conc is provided as input to the convolution block Conv so as to obtain, at the output, a final data item X0'4. This final data item X0'4 has the same dimensions and the same resolution as the input image Ie. In practice here, this final data item X0 4 is for example associated with a focusing map as described below.
[0069] Thus, the operations of processing the fifth data X4'0 by the second part 30 of the artificial neural network NN can be expressed in the following form:
[0070] X ] )
[0071] with j>l and i between 0 and 3, the operator Conv corresponding to the processing implemented by the convolution block Conv, the operator Conc corresponding to the processing implemented by the concatenation block Conc and the operator U being associated with the upsampling block U.
[0072] [Fig.4] is a flowchart representing an example of a method for estimating the depth map C associated with the digital hologram H, implemented in the context described above. This method is for example implemented by the processor 2. Generally, this method is implemented by computer.
[0073] As shown in [Fig. 4], the method begins at step E2 during which the processor 2 determines a minimum depth zmin and a maximum depth zmax of the z coordinate in the three-dimensional scene of the digital hologram H. These minimum and maximum depths are for example previously recorded in the storage device 4.
[0074] The method then continues with a step E4 of reconstructing a plurality of two-dimensional images of the three-dimensional scene represented by the digital hologram H.
[0075] For this, the reconstruction module 5 is configured to reconstruct n images h of the scene by means of the digital hologram H, with i an integer ranging from 1 to n.
[0076] Each reconstructed image f is defined in a reconstruction plane which is perpendicular to the depth axis of the digital hologram H. In other words, each reconstruction plane is perpendicular to the depth axis z. Each reconstruction plane is associated with a depth value, making it possible to associate with each reconstructed image h a depth zi5, the index i referring to the index of the reconstructed image I;. Each depth value defines a distance between the plane of the digital hologram and the reconstruction plane concerned.
[0077] Preferably here, the reconstruction step E4 is implemented such that the depths z; associated with the reconstructed images h are uniformly distributed between the minimum depth zmin and the maximum depth zmax. In other words, the reconstructed images f are uniformly distributed along the depth axis, between the minimum depth zmin and the maximum depth zmax. Thus, the first reconstructed image h is spaced from the plane of the digital hologram H by the minimum depth zmin while the last reconstructed image In is spaced from the plane of the digital hologram H by the maximum depth zmax.
[0078] The reconstruction planes associated with the reconstructed images h are, for example, spaced two by two by a distance ze. The distance ze between each reconstruction plane is, for example, of the order of 50 micrometers (pm).
[0079] Preferably, the n images obtained in reconstruction step E4 are calculated using a propagation of the angular spectrum defined by the following formula: / / vy) = yWec F and F 1 corresponding to direct and inverse Fourier transforms, respectively, and fx and fy are the frequency coordinates of the digital hologram H in the Fourier domain along a first spatial direction x and a second spatial direction y of the digital hologram, X the acquisition wavelength of the digital hologram H, i the index of the reconstructed image I with i ranging from 1 to n and z; the depth given in the reconstruction plane of the image h.
[0080] Each reconstructed image f is defined by a plurality of pixels. Preferably, the reconstructed images are formed of as many pixels as the digital hologram H. Thus, the reconstructed images f and the digital hologram H are of the same size. For example, in the case of a digital hologram H of size 1024x1024, each reconstructed image h also has a size of 1024x1024.
[0081] As shown in [Fig.4], the method continues in step E6. During this step, the decomposition module 6 is configured to decompose each reconstructed image h obtained in step E4 into a plurality of thumbnails Ji;jjk. In other words, during this decomposition step E6, each reconstructed image h is divided into a plurality of thumbnails Ji;jjk. In other words, each thumbnail Ji;jjk corresponds to a sub-part of the reconstructed image h concerned.
[0082] Each image F ;jjk is defined by the following formula:
[0083] :(j+l).s,ks :(Æ+1)^)}
[0084] with j — [ [ ~r ] and k = 1 [ “F ] ' with Sw ct Sh 'cs dimensions (respectively height and width) of the reconstructed image F, s the size of the thumbnail Ji;jjk, IxJ the notation corresponding to the modulus of the data Xi and Lx2J, the notation corresponding to the lower integer part of the number x2. The notation yi : y2 means that, for the variable concerned, the thumbnail Ji;jjk is defined between the pixel yi and the pixel y2. In other words here, the previous formula defines the thumbnail Ji;jjk, according to the dimension x, between the pixels js and (j+l)s of the reconstructed image F and, according to the dimension y, between the pixels ks and (k+l)s of the reconstructed image F-
[0085] Each image F ;jjk comprises a plurality of pixels. This plurality of pixels corresponds to a portion of the pixels of the associated reconstructed image F.
[0086] Here, the thumbnails Ji;j>k are adjacent to each other. In practice, each thumbnail F ;jjk is formed from a set of contiguous pixels of the reconstructed image F-Here, the sets of pixels of the reconstructed image F (forming respectively each of the thumbnails F;j>k) are disjoint. In other words, this means that the thumbnails Ji;j >k associated with a reconstructed image F do not overlap with each other. Each thumbnail F ;jjk therefore comprises pixels which do not belong to the other thumbnails associated with the same reconstructed image F- In other words, the thumbnails F;j,k associated with a reconstructed image F are independent of each other.
[0087] This property of independence between the thumbnails is particularly advantageous for the method according to the invention because it allows for faster implementation. In addition, the necessary computing resources are less expensive thanks to the limited number of areas to be analyzed (namely the different thumbnails).
[0088] As each image J; ;jjk comes from a reconstructed image h associated with a depth zi5, each image Ji;jjk is also associated with this same depth z; (of the three-dimensional scene).
[0089] In the case where the digital hologram H has a size of 1024x1024, each image J; ;jjk may for example have a size of 32x32.
[0090] In the case where the digital hologram H has a size of lHxlw, each image Ji;jjk has a size of (32xsH)x(32xsw) with sH=lH / 1024 and sw=lw / 1024.
[0091] This definition of the size of the thumbnails makes it possible to ensure a size of these thumbnails adapted to the size of the digital hologram H so as to improve the speed of implementation of the method for estimating the depth map associated with the digital hologram H.
[0092] As shown in [Fig.4], the method then continues in step E8. During this step, the processor 2 determines, for each image Ji;jjk, a focus map Ci;jjk. This focus map Ci;jjk comprises a plurality of elements (each identified by the indices js+q, ks+r). Each element of the focus map C i;jjk is associated with a pixel of the image Ji;jjk concerned.
[0093] Here, each element of the focusing map Ci;jjk corresponds to a focusing level (corresponding to a focusing level associated with the pixel concerned in the image Ji;j4t). In other words, the focusing map Ci;jjk associates with each pixel of the image Ji;jjk concerned a focusing level.
[0094] In practice, this step E8 is implemented via the artificial neural network NN. The latter receives, as input, each of the imagettes Ji;jjk and provides as output the focusing levels associated with each of the pixels of the imagette J i;jjk concerned.
[0095] More particularly, the artificial neural network NN receives, as input, each of the pixels of the image Ji;jjk and provides, as output, the associated focus level. This focus level is for example between 0 and 1 and is similar to a sharpness level associated with the pixel concerned. For example, in the case of a blurred pixel, the focus level is close to 0 while in the case of a substantially sharp pixel, the focus level is close to 1.
[0096] Advantageously, the use of the artificial neural network allows faster processing of all the thumbnails and more precise determination of the focusing levels associated with the pixels of the thumbnails.
[0097] Prior to the implementation of the estimation method, a learning step (not shown in the figures) allows the training of the artificial neural network NN. For this, computer-calculated holograms are used, for example. For these calculated holograms, the exact geometry of the scene (and therefore the associated depth map) is known. A set of basic images is derived from these calculated holograms.
[0098] For each base image in this set, each pixel is associated with a focus level. Indeed, for each pixel in each base image, the focus level is equal to 1 if the corresponding pixel, in the depth map, is equal to the associated depth. If this is not the case, the focus level is equal to 0.
[0099] The learning step then consists of adjusting the node weights of the different convolution layers included in the different convolution blocks Conv described previously so as to minimize the error between the focus levels obtained at the output of the artificial neural network NN (when the base images are provided as input to this network) and those determined from the known depth map. For example, a cross-entropy loss method can be used here so as to minimize the distance between the focus levels obtained at the output of the artificial neural network NN (when the base images are provided as input to this network) and those determined from the known depth map.
[0100] In other words, the weights of the nodes of the different convolution layers are adjusted so as to make the focus levels obtained at the output of the artificial neural network NN converge towards the focus levels determined from the known depth map.
[0101] In practice, the artificial neural network NN receives as input all the imagettes Ji;jjk associated with each reconstructed image f and carries out parallel processing of each of the imagettes
[0102] Alternatively, the imagettes Ji;jjk could be processed successively, one after the other.
[0103] At the end of step E8, the processor 2 therefore knows, for each image Ji;jjk, the associated focusing map Ci;jjk which lists the focusing levels obtained at the output of the artificial neural network NN associated with each pixel of the image Ji;jjk concerned. Each focusing map Ci;jjk is associated with the corresponding image J i;jjk, therefore with the depth z; (of the three-dimensional scene).
[0104] As shown in [Fig.4], the method then comprises a step E10 of estimating the depth map C associated with the digital hologram H. This depth map C comprises a plurality of depth values djs+qjks+r. Each depth value djs+qjks+r is associated with a pixel among the different pixels of the thumbnails Ji;jjk. The depth value djs+qjks+r is determined on the basis of the levels of focusing determined in step E8.
[0105] Indeed, during step E8, given the plurality of reconstructed images I; each corresponding to a different depth z;, a pixel of a thumbnail is associated with different focusing levels (depending on the depth of the reconstructed image h from which the thumbnail concerned is derived). In other words, for each pixel (associated with a depth value djs+q ks+r of the depth map C), several focusing levels are known.
[0106] Thus, here, the processor 2 determines, for each pixel associated with the depth value djs+q ks+r concerned, the depth for which the focusing level is the highest:
[0107] = argniax.=LN(Ci.hk(q, r))
[0108] with (js+q,ks+r) the pixel to which the determined depth value is assigned, NN the operator corresponding to the implementation of the artificial neural network NN and argmax, the operator translating the determination of the maximum value of the focus level (obtained at the output of the artificial neural network).
[0109] In other words, for each pixel with indices (js+q, ks+r), the processor 2 determines the depth at which the focus level is highest. This depth then corresponds to the depth value djs+qjks+r (element of the depth map C).
[0110] Alternatively, the depth value could be determined using a method other than determining the maximum value of the focus level. For example, an area formed by a plurality of adjacent pixels may be defined and the depth value may be determined by considering the depth for which a maximum deviation is observed from the average of the focus levels over the defined area of pixels.
[0111] At the end of step E10, the depth map C is therefore, for example, here estimated from these determined depth values. Thus, each element of the depth map C comprises a depth value djs+q>ks+r associated with each pixel of indices (js+q, ks+r).
[0112] This estimated depth map C finally makes it possible to have spatial information in the form of a matrix of depth values representing the three-dimensional scene associated with the digital hologram H.
[0113] Of course, the method described above for a digital hologram applies in the same way for a plurality of holograms. For a plurality of digital holograms, the implementation of the method can be successive for each hologram or in parallel for the plurality of digital holograms.
Claims
Claims
1. Method for estimating a depth map (C) associated with a digital hologram (H) representing a scene, the method comprising steps of: - reconstruction (E4) of a plurality of images (f) of the scene from the digital hologram (H), each reconstructed image (h) being associated with a depth (¾) of the scene, - decomposition (E6) of each reconstructed image (f) into a plurality of thumbnails (Ji;jjk), said thumbnails (Ji^k) of said plurality being adjacent to each other, each thumbnail (Ji;j4t) being associated with the depth (¾) of the scene corresponding to the reconstructed image (f) concerned, each thumbnail (Ji^k) comprising a plurality of pixels, - determination (E8), for each thumbnail (Ji;j,k), of a focus map (Ci;jjk) by supplying, as input to an artificial neural network (NN), values associated with said pixels of the image (Ji;j>k) concerned, so as to obtain, at the output of the artificial neural network (NN), said focus map (Ci;j,k) comprising a focus level associated with the pixel concerned, and - determination (E10) of a depth value (djS+qks+r), for each point of a depth map (C), as a function of the focus levels obtained respectively for the pixels associated with said point of the depth map (C) in the focus maps (Ci;j,k) respectively determined for the thumbnails (Ji^k) containing a pixel corresponding to said point of the depth map (C).;
2. Method according to claim 1, in which, for each point of the depth map (C), the associated depth value (djS+qjks+r) is obtained by determining the depth corresponding to the highest focus level among all the pixels associated with said point of the depth map (C) in the thumbnails (Ji;j>k) containing said pixels.
3. The method of claim 1 or 2, wherein the artificial neural network (NN) is a convolutional neural network.
4. Method according to any one of claims 1 to 3, in which the reconstruction step (E4) is implemented in such a way that the reconstructed images (h) are respectively associated with depths uniformly distributed between a minimum depth (zmin) and a maximum depth (zmax) of the scene.
5. Device (1) for estimating a depth map (C) associated with a
6. digital hologram (H) representing a scene, the device (1) comprising: - a reconstruction module (5) of a plurality of images (f) of the scene from the digital hologram (H), each reconstructed image (h) being associated with a depth (z;) of the scene, - a decomposition module (6) of each reconstructed image (f) into a plurality of thumbnails (Ji;j>k), said thumbnails (Ji;j>k) of said plurality being adjacent to each other, each thumbnail (Ji^k) being associated with the depth (¾) of the scene corresponding to the reconstructed image (f) concerned, each thumbnail (Ji;j>k) comprising a plurality of pixels, - a determination module (8), for each image (Ji;j,k), of a focusing map (Ci;jjk) by supplying, as input to an artificial neural network (NN), the values associated with said pixels of the image (Ji^k) concerned, so as to obtain, as output from the artificial neural network (NN), said focusing map (Ji^k) comprising a focusing level associated with the pixel concerned, and - a module (9) for determining a depth value (djS+qjks+r), for each point of a depth map (C) as a function of the focusing levels obtained respectively for the pixels associated with said point of the depth map (C) in the thumbnails (Ji^k) containing said pixels. Computer program comprising instructions executable by a processor (2) and designed to implement a method according to one of claims 1 to 4 when these instructions are executed by the processor (2).