Method and apparatus for depth map estimation

Description

1. Technical field

[0001] A method and an apparatus for depth-map estimation are disclosed.2. Background art

[0002] Depth-map estimation is made with a pair of images obtained by two cameras for which precise location in space according to the so-called world coordinate system is known.

[0003] A common method for estimating depth associated with a pixel of an image of a first camera is to find a corresponding pixel in an image of a second camera that best matches the pixel of the first image. Such matching is determined using similarity estimators such as a L1 or L2 norm computed between the two pixels.

[0004] However, when the similiarity between two pixels is computed using only the color components of the pixels, such as the R,G, B components (for Red Green Blue), the depth estimation is very sensitive to noise.

[0005] To overcome this limitation, the similarity between two pixels may be computed using a patch of pixels surrounding the pixel being considered. This technique requires requires much more computation since it requires T 2< more computation for a patch of TxT pixels compared to computing similarity between 2 pixels.

[0006] Anders Olofsson addresses the problem of stereo correspondence between stereoscopic images in a report "Modern Stereo Correspondence Algorithms: Investigations and evaluation". This report provides a survey of stereo algorithms and a comparison between an algorithm relying on local winner-take-all optimization and another algorithm minimizing a global energy function formulated at the pixel level. This report acknowledges that computational complexity is an issue within stereo matching.

[0007] This is a critical point for real-time estimation and especially when embedded into mobile devices. Thus, there is a need for a new method for depth-map estimation that allows fast and accurate depth estimation.3. Summary

[0008] The invention is defined by the appended claims.

[0009] A method for depth-map estimation between at least two images respectively captured by at least two cameras is disclosed. The method for depth-map estimation comprises: obtaining extrinsic and intrinsic parameters of the at least two cameras, obtaining local spatial structure descriptors for each pixel of the at least two images, by transforming a patch of pixels surrounding each pixel, obtaining depth for at least one pixel of a first image among the at least two images, among at least two depth candidates, each depth candidate being associated with a corresponding pixel in a second image of the at least two images, said corresponding pixel in the second image being determined according to the obtained extrinsic and intrinsic parameters, wherein obtaining depth comprises: determining a similarity measure between the at least one pixel of the first image and each corresponding pixel in the second image, the similarity measure taking into account color components and local spatial structure descriptors of the corresponding pixel and the at least one pixel of the first image, selecting a pixel among the corresponding pixels in the second image that minimizes the similarity measure.

[0010] According to the present principle, depth for a pixel in an image is estimated by taking into account local spatial structure surrounding the pixel. Advantageously, local spatial structure descriptors can be determined ahead of depth estimation, for instance at the time images are captured by cameras.

[0011] According to an embodiment of the present principle, the number of local spatial structure descriptors obtained is lower than the number of pixels in the patch of pixels surrounding the pixel. According to this embodiment, comparing patches between two pixels necessitates less computation than in the prior art as fewer coefficients are used for describing a patch surrounding a pixel.

[0012] According to an embodiment of the present principle, transforming a patch of pixels surrounding each pixel further comprises: obtaining an auto-correlation matrice between the patch of pixels and surrounding patches, determining the local spatial structure descriptors for the pixel from the auto-correlation matrice.

[0013] According to this embodiment, the patch of pixels surrounding each pixel can be described by few components characterizing its correlation with surrounding patches.

[0014] According to an embodiment of the present principle, the local spatial structure descriptors are determined from a transformation of the auto-correlation matrice.

[0015] According to an embodiment of the present principle, the auto-correlation matrice is 3 by 3 array of scalars with a null coefficient at the center.

[0016] According to an embodiment of the present principle, the local spatial structure descriptors are obtained from the auto-correlation matrice using the following transformation: I = a b c h 0 d g f e 1 1 1 1 0 1 1 1 1 , d x = a b c h 0 d g f e − 1 − 1 − 1 1 0 − 1 1 1 1 , d y = a b c h 0 d g f e 1 1 − 1 1 0 − 1 1 − 1 − 1 , Δ x = a b c h 0 d g f e − 1 − 1 1 1 0 1 1 − 1 − 1 , Δ y = a b c h 0 d g f e 1 − 1 − 1 1 0 1 − 1 − 1 1 , with a, b, c, d,e, f, g, h being coefficients of the auto-correlation matrice.

[0017] According to another embodiment of the present principle, the patch of pixels is transformed using a Discrete Cosine Transform. Such an embodiment allows to take advantage of the decorrelation feature of the DCT transform which allows to extract the frequencies of the patch surrounding the considered pixel.

[0018] According to an embodiment of the present principle, the patch of pixels is a patch of 8x8 pixels.

[0019] According to an embodiment of the present principle, the local spatial structure descriptors are selected among the Discrete Cosine Transform coefficients.

[0020] It should be noted that a step of obtaining an element / value in the present document can be viewed either as a step of reading such element / value in a memory unit of an electronic device or a step of receiving such element / value from another electronic device via communication means, or also a step of computing or determining such element / value via a processor of an electronic device.

[0021] According to another aspect of the disclosure, an apparatus for depth-map estimation between at least two images respectively captured by at least two cameras is disclosed. Such an apparatus comprises: means for obtaining extrinsic and intrinsic parameters of the at least two cameras, means for obtaining local spatial structure descriptors for each pixel of the at least two images, comprising means for transforming a patch of pixels surrounding each pixel, means for obtaining depth for at least one pixel of a first image among the at least two images, among at least two depth candidates, each depth candidate being associated with a corresponding pixel in a second image of the at least two images, said corresponding pixel in the second image being determined according to the obtained extrinsic and intrinsic parameters, wherein said means for obtaining depth comprises: means for determining a similarity measure between the at least one pixel of the first image and each corresponding pixel in the second image, the similarity measure taking into account color components and local spatial structure descriptors of the corresponding pixel and the at least one pixel of the first image, means for selecting a pixel among the corresponding pixels in the second image that minimizes the similarity measure.

[0022] According to an embodiment of the present principle, the means for transforming a patch of pixels surrounding each pixel comprises: means for obtaining an auto-correlation matrice between the patch of pixels and surrounding patches, means for determning the local spatial structure descriptors for the pixel from the auto-correlation matrice.

[0023] According to another embodiment of the present principle, the patch of pixels is transformed using a Discrete Cosine Transform.

[0024] One or more of the present embodiments also provide a computer readable storage medium having stored thereon instructions for depth-map estimation, according to any one of the methods described above. The present embodiments also provide a computer program product including instructions for performing any one of the methods described.4. Brief description of the drawings

[0025] Figure 1 illustrates projection of a 3D point X into a reference image of a first camera and into a second image of a second camera for depth-map estimation. Figure 2 illustrates an exemplary method for depth-map estimation according to an embodiment of the present principle, Figure 3 illustrates an exemplary method for local spatial structure estimation according to an embodiment of the present principle, Figure 4 illustrates exemplary patches used for local spatial structure estimation according to an embodiment of the present principle, Figure 5 illustrates an exemplary method for local spatial structure estimation according to another embodiment of the present principle, Figure 6 illustrates an exemplary patch used for local spatial structure estimation according to another embodiment of the present principle, Figure 7 illustrates an exemplary apparatus for depth-map estimation according to an embodiment of the present disclosure. 5. Description of embodiments

[0026] An exemplary method for depth-map estimation is disclosed herein in reference to figure 2.

[0027] Depth estimation is made with a pair of images. The pair of images can be obtained from a stereo camera having a left view and a right view or from an array of cameras, considering at least 2 cameras. Depth estimation is made possible if the 2 cameras are precisely located in space according to the so-called world coordinate system.

[0028] In step 20, the 2 cameras are located in space using a camera calibration procedure which for instance uses a chessboard observed at different positions from the 2 cameras. Several pictures are taken with the chessboard positioned at different positions from the cameras. On each picture, the 2D coordinate of the corners delimited by 2 black and 2 white squares of the chessboard are extracted. The 2D coordinates of the corners are associated with the 2D coordinates of the same corners observed by the second camera.

[0029] Having all the couple of 2D coordinates of corners observed by the 2 cameras and also for the different exposures, it is possible to estimate the position of the cameras according to a world coordinate system. In this system, the center of the entry pupil of the camera i is positioned in space by a translation vector T i = (X, Y, Z) t< , and the optical axis is defined by a 3D rotation matrix R i . The pose matrix of the camera i is defined by P i = R i T i ∈ ℝ 3 × 4 . The extrinsic matrix of the camera i is defined by Q i = R i − 1 − R i − 1 . T i ∈ ℝ 3 × 4 . From the camera calibration at step 20, the so-called extrinsic camera parameters (a translation vector T and a rotation matrix R) are deduced. Such extrinsic parameters allow to convert a 3D world coordinate into a 2D camera coordinate.

[0030] Now, we describe in reference with figure 1, a method for estimating depth of a 3D point in space. Let x L (x, y) being the pixel from the left camera which observes the object in space at X (X, Y, Z). O L is the optical center of the left camera. Any object X i which lies on the line (O L , X ) is observed by the left camera at the same pixel x L . On the right camera, object X i are observed at coordinates x r which all lie on one line on the sensor, the so-called epipolar line.

[0031] A common method for estimating the depth associated with a pixel is made using the epipolar line.

[0032] One considers one pixel p ref (x, y) of the reference camera at pixel coordinate (x, y). The pixel p ref is de-projected into the world coordinate system at various distance candidates Z c with the rotation and translation matrix associated with the reference camera. The minimum and maximum values for Z c are selected depending on the nearest and farthest objects lying in the scene. The interval between these two values is cut linearly into N slices which defines the N distance candidate Z c .

[0033] The number of slices depends on the desired speed and accuracy of the depth-estimation process. For instance, between 30 and 150 slices may be used. Several 3D points in the world coordinate system are thus obtained at coordinates P ref (X c , Y c , Z c ). All these 3D points P ref (X c , Y c , Z c ) are all observed by the pixel p ref (x, y) in the image of the reference camera.

[0034] For instance, a number N of candidates are evaluated. An exemplary method for depth estimation is given in B.D. Lucas, T.Kanade, et al. "An iterative image registration technique with an application to stereo vision", in DARPA Image Understanding Workshop, Vancouver, BC, Canada, 1981.. For instance, N is equal to 100 for a good depth estimation. The number N is also named the number of slices, as the candidates Z c define planes which cut the 3D space in parallel slices where depth-map is estimated.

[0035] The obtained candidates P ref are then projected into the second camera according to the extrinsic and intrasic camera parameters. One deduces N coordinates p sec (x zc , y zc ) on the second camera which all depends on the distance candidate Z c .

[0036] The distance Z pref of the real physical object P ref observed at pixel p ref on the reference camera is equal to the distance candidate Z c for which p ref (x, y) is the most similar to p sec (x zc , y zc ). Such similarity may be computed using various estimators. For instance, an L1 norm computed between 2 pixels may be used.

[0037] If the pixel p being observed is a color pixel defined by 3 scalars corresponding to the 3 color components Red, Green and Blue (p R , p G , p B ). The L1 norm between 2 pixels p ref (x, y) and p sec (x zc, y zc ) is defined by s L1 (p ref (x, y), p sec (x zc ,y zc )) = |p ref,R (x,y) - p sec,R (x zc , y zc )| + |p ref,G (x, y) - p sec,G (x zc , y zc )| + |p ref,B (x, y) - p secB (x zc , y zc )|. From the N candidates p sec (x zc ,y zc ), the pixel having the smallest L1 norm with p ref (x, y) is said to observe to same object in space. The estimated depth of the pixel p ref is thus the candidate Z c corresponding to the candidate p sec (x zc , y zc ) having the smallest L1 norm with p ref (x, y).

[0038] According to another example, an L2 norm computed between 2 pixels may be used. The L2 norm is similar to the L1 norm except that the similarity measure is defined by s L2 (p ref, p sec ) = |p ref,R - p sec,R | 2< + |p ref,G - p sec,G | 2< + |p ref,B - p sec,B | 2< . In practice if the similarity is estimated only with the color component of one pixel, the depth estimation is very sensitive to noise. To overcome this limitation the similarity between 2 pixels may be computed using a patch of few surrounding pixels. This technique refers to cross-patch depth-estimation.

[0039] Obviously, such a method requires much more computation since for each candidate, similarity measure has to be computed between patches of T × T pixels. Therefore, the method requires T 2< more computation for a patch of T × T pixels compared to similarity measured between only 2 pixels. This is a critical point for real time estimation and especially when embedded into mobile devices.

[0040] The similarity operator described above (L1 and L2 norms) may be used for patches surrounding a pixel. The L1 norm computed between 2 patches may be computed as follows. Let P ref,T (x, y) being a T by T pixels patch surrounding the pixel p ref (x, y) and respectively let P sec,T (x zc , y zc ) being a T by T pixels patch surrounding the pixel p sec (x zc , y zc ). The L1 norm between the 2 patches is defined by S L 1 P ref , T P sec , T = ∑ k = − T / 2 k = T / 2 ∑ l = − T / 2 l = T / 2 S L 1 p ref x − k , y − l , p sec x z c − k , y Z c − l . From the N candidates p sec (x zc , y zc ), the one having the smallest L1 norm with p ref (x, y) is said to observe to same object in space. The corresponding Z c is the depth estimation associated with pixel p ref (x, y). Similarity estimators which use patches allow more accurate depth estimation but with a higher computational cost. The use of patches for similarity estimation between the 2 pixels (p ref and p sec ), allows to take into account the local spatial structure of the pixels being compared, by computing the similarity measure on the color components of the surrounding pixels. The use of patches prevents for instance to estimate that 2 pixels are similar if they have the same color (p ref (x, y) ≈ p sec (x, y) but having different spatial structure (one pixel is lying on an edge and the other one on a flat area for instance). Only patches permit to discriminate pixels considering the underlying spatial structures around the pixels.

[0041] According to the principle disclosed herein, depth-map is estimated taking into account of local spatial structure. According to an embodiment of the present principle, local spatial structure of a pixel is determined ahead of the depth-map estimation by determining characteristics of a patch surrounding the pixel, also called local spatial structure descriptors. Back to figure 2, in step 21, local spatial structure of each pixel of the 2 images is determined. Embodiments for such determination are further discussed in reference with figures 3-6. According to the present principle, local spatial structure descriptors are determined for each pixel of the two images, by transforming a patch of T by T pixels surrounding each pixel. According to the present principle, the number L of local spatial structure descriptors determined at step 21 from the patch surrounding each pixel is lower than the number of pixels in the patch. In this way, the similarity measure computation necessitates less computation than in the prior art because fewer coefficients are used for describing a patch surrounding a pixel.

[0042] For each pixel of the 2 images to be compared, the L local structure descriptors determined are stick to the 3 scalars corresponding to the 3 color components of each pixel. One derives 2 new special images where each pixel is made of L+3 components. In step 22, depth is estimated for each pixel p ref (x, y) of the reference image in a way similar as the one disclosed above.

[0043] More particularly, in step 220, the 2D pixel coordinate p ref (x, y) is de-projected into 3D coordinates P(X c , Y c , Z c ) in the world coordinate system for N candidates Z c according to the extrinsic parameters of the reference camera.

[0044] In step 221, the N 3D coordinates candidates P(X c , Y c , Z c ) are projected into the 2D coordinates p sec (x Zc , y Zc ) of the second camera according to the extrinsic parameter of the second camera.

[0045] In step 222, the similarity measure is computed between the reference pixel p ref (x, y) and each of the pixel candidate p sec (x Zc , y Zc ). The similarity is computed using the L local spatial structure descriptors determined at step 21 and the 3 color components of the pixels p ref (x, y) and p sec (x Zc , y Zc ). For instance, if the L1 norm is used, the similarity measure is computed as s L 1 p ref x y , p sec x Z c y Z c = ∑ k = 1 k = L + 3 p ref x y k − p sec x y k where k ∈ [1, L + 3] is the is k th< component of the pixels, the components of a pixel being given by the 3 color components of the pixel and the L local spatial structure descriptors determined at step 21.

[0046] In step 223, the similarity measures computed at step 222 are compared and the pixel p sec (x Zc ,y Zc ) that minimizes the similiarity measure is selected as the best candidate. That is the best candidates c is selected if p ref (x, y) is the most similar to p sec (x Zc , y Zc ). In step 224, the depth associated with the pixel coordinate p ref (x, y) is equal to the distance Z c of the best candidate c.

[0047] According to the present principle, the method for depth-map estimation allows to describe the patches surrounding the pixels to be compared with few coefficients such that comparing 2 patches requires less computation. Furthermore, the local spatial structure of the patches can be estimated per pixel prior to the depth-estimation.

[0048] A method for local spatial structure estimation of a pixel is disclosed below in reference with figures 3 and 4 according to an embodiment of the present disclosure. According to the embodiment disclosed herein, local spatial structure descriptors are computed for describing a patch of 3x3 pixels P 3 (x, y) surrounding the pixel P(x, y) such as disclosed in figure 4.

[0049] The patch P 3 (x, y) is characterized by estimating the L1-norm L(i, j) between P 3 (x, y) and the 3 × 3 surrounding patches P 3 (x - i - 1,y - j - 1) with (i, j) ∈ [0,2] 2< . Figure 4 illustrates in bolded lines the central patch P 3 (x, y) surrounding pixel P(x,y) (in grey on figure 4), and 2 patches P 3 (x - 1,y - 1) and P 3 (x + 1,y) in dotted lines. In step 30, an auto-correlation matrice L(i, j) is determined between the central patch P 3 (x, y) and the surrounding 3x3 patches as follows: L i j = ∑ k = − 1 k = 1 ∑ l = − 1 l = 1 s L 1 p x − k , y − l , p x − k − i + 1 , y − l − j + 1 L(i, j) is a 3 by 3 array of scalars with a null coefficient in the middle. The 8 other coefficients depend on the local spatial structures of the 3 × 3 pixels around p(x, y). In step 31, 5 local spatial structure descriptors are then computed from the auto-correlation matrice. According to the embodiment disclosed herein, the 8 coefficients of the auto-correlation matrice are compressed into 5 coefficients following the transformation given below: I = a b c h 0 d g f e 1 1 1 1 0 1 1 1 1 = a + b + c + d + e + f + g + h d x = a b c h 0 d g f e − 1 − 1 − 1 1 0 − 1 1 1 1 = − a − b − c − d + e + f + g + h d y = a b c h 0 d g f e 1 1 − 1 1 0 − 1 1 − 1 − 1 = a + b − c − d − e − f + g + h Δ x = a b c h 0 d g f e − 1 − 1 1 1 0 1 1 − 1 − 1 = − a − b + c + d − e − f + g + h Δ y = a b c h 0 d g f e 1 − 1 − 1 1 0 1 − 1 − 1 1 = a − b − c + d + e − f − g + h where a, b, c, d, e, f, g, h are the 8 coefficients of the array L(i, j).

[0050] The scalar I represents how the 8 pixels are similar: I is close to 0 for flat areas, and I has high values for textured areas.

[0051] The 2 scalars (d x , d y ) represent the asymmetry of the array L. The values are close to 0 for pixels belonging to flat area. The values become positives and negatives on both side of an edge with an intensity which depends on the contrast of the edge.

[0052] The 2 scalars (Δ x , Δ y ) represent the orientation of an edge if any.

[0053] The 5 scalars (I, d x , d y , Δ x , Δ y ) permit to describe the local spatial structure for the pixel p(x, y).

[0054] The original images are made of 3 scalars per pixel corresponding to the 3 color components. With the local spatial structure estimation disclosed above, one adds 5 components to describe a pixel. For each pixel of the input image, one forms a new image where each pixels is made of 8 components [R; G; B; s I I; s d d x ; s d d y ; s Δ Δ x ; s Δ Δ y ], where s I , s d , s Δ are 3 scale-factors which controls the amplitude of the 5 scalars versus the 3 color components. A method for local spatial structure estimation of a pixel is disclosed below in reference with figures 5 and 6 according to another embodiment of the present disclosure.

[0055] According to the embodiment disclosed herein, local spatial structure descriptors are computed for describing a patch of 8x8 pixels P 8 (x, y) surrounding the pixel P(x, y) such as disclosed in figure 6.

[0056] In the embodiment disclosed herein, the local spatial descriptor are based on the Discrete Cosine Transform (DCT) computed on a patch of 8 × 8 pixels P 8 (x, y) surrounding a pixel P(x, y) as illustrated in figure 6. The DCT transform allows to extract the frequencies which lies within a patch of 8 by 8 pixels. Selecting some coefficients provided by the DCT transform allows to characterize the shape of the patch surrounding the pixel P(x, y). In step 50, DCT transform is applied on the patch P 8 (x, y) surrounding the pixel P(x, y). The 8 × 8 DCT(u, v) coefficients with (u, v) ∈ [0,8[ 2< are computed as follows: DCT u v = ∑ i = 0 7 ∑ j = 0 7 p x + i − 3 , y + j − 3 cos 2 x + 1 uπ 16 cos 2 y + 1 vπ 16 For each pixel p(x, y), 64 values of the DCT(u, v) are obtained.

[0057] In step 51, L characteristics associated with the pixel P(x, y) are selected from these 64 values. Several selections can be used.

[0058] For example, with L = 5, the selected DCT values are: DCT(0,0), DCT(0,1), DCT(0,2), DCT(1,0), and DCT(2,0).

[0059] According to another example, with L = 6, the selected DCT values are the coefficients (u, v) such that u + v < 3: DCT(0,0), DCT(0,1), DCT(0,2), DCT(1,0), DCT(1,1)and DCT(2,0). According to yet another example, with L = 10, the selected DCT values are the coefficients (u, v) such that u + v < 4: DCT(0,0), DCT(0,1), DCT(0,2), DCT(0,3), DCT(1,0), DCT(1,1), DCT(1,2), DCT(2,0), DCT(2,1) and DCT(3,0). Using L = 10 provides an accurate spatial descriptor with only 10 coefficients per pixel compared to the 64 values of the DCT.

[0060] The original images are made of 3 scalars per pixel corresponding to the 3 color components. With the local spatial structure estimation, one adds L components to describe a pixel. For each pixel of the input image, one forms a new image where each pixels is made of (3 + L) components [R; G; B; and the L selected DCT coefficients] . The selected DCT coefficients may be scaled by a scale factor s DCT that controls the amplitude of the L scalars versus the 3 color components. Figure 7 illustrates an examplary apparatus for depth-map estimation according to an embodiment of the present disclosure. FIG. 7 illustrates a block diagram of an exemplary system 700 in which various aspects of the exemplary embodiments may be implemented. The system 700 may be embodied as a device including the various components described below and is configured to perform the processes described above. Examples of such devices, include, but are not limited to, mobile devices, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The system 700 may be communicatively coupled to other similar systems, and to a display via a communication channel. Various embodiments of the system 700 include at least one processor 710 configured to execute instructions loaded therein for implementing the various processes as discussed above. The processor 710 may include embedded memory, input output interface, and various other circuitries as known in the art. The system 700 may also include at least one memory 720 (e.g., a volatile memory device, a non-volatile memory device). The system 700 may additionally include a storage device 740, which may include non-volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drive, and / or optical disk drive. The storage device 740 may comprise an internal storage device, an attached storage device, and / or a network accessible storage device, as non-limiting examples. Program code to be loaded onto one or more processors 710 to perform the various processes described hereinabove may be stored in the storage device 740 and subsequently loaded onto the memory 720 for execution by the processors 710. In accordance with the exemplary embodiments, one or more of the processor(s) 710, the memory 720, and the storage device 740, may store one or more of the various items during the performance of the processes discussed herein above, including, but not limited to the input images, the output depth-maps, equations, formulas, matrices, variables, operations, and operational logic. The system 700 may also include a communication interface 750 that enables communication with other devices via a communication channel. The communication interface 750 may include, but is not limited to a transceiver configured to transmit and receive data from the communication channel. The communication interface 750 may include, but is not limited to, a modem or network card and the communication channel 750 may be implemented within a wired and / or wireless medium. The various components of the system 700 may be connected or communicatively coupled together (not shown) using various suitable connections, including, but not limited to internal buses, wires, and printed circuit boards. The exemplary embodiments may be carried out by computer software implemented by the processor 710 or by hardware, or by a combination of hardware and software. As a non-limiting example, the exemplary embodiments may be implemented by one or more integrated circuits. The memory 720 may be of any type appropriate to the technical environment and may be implemented using any appropriate data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 710 may be of any type appropriate to the technical environment, and may encompass one or more of microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non-limiting examples. The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed may also be implemented in other forms (for example, an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users.

Claims

1. A computer-implemented method for depth-map estimation between at least two images respectively captured by at least two cameras, comprising: - obtaining (20) extrinsic and intrinsic parameters of the at least two cameras, - for each pixel of the at least two images, applying a first transform to a patch of pixels surrounding the pixel to obtain (21) a first set of coefficients representative of a local spatial structure of the patch of pixels, wherein a second transform is applied to the first set of coefficients to compress the first set of coefficients in a second set of coefficients representative of the local spatial structure of the patch of pixels, wherein the patch of pixels surrounding the pixel comprises a first number of pixels, and the second set of coefficients comprises a second number of coefficients that is lower than the first number of pixels of the patch of pixels, - obtaining (22) depth for at least one pixel of a first image among the at least two images, among at least two depth candidates, each depth candidate being associated with a corresponding pixel in a second image of the at least two images, said corresponding pixel in the second image being obtained according to the extrinsic and intrinsic parameters, wherein obtaining depth comprises: - determining (222) a similarity measure between the at least one pixel of the first image and each corresponding pixel in the second image, the similarity measure taking into account color components and the second set of coefficients of the corresponding pixel and the at least one pixel of the first image, - selecting (223) a pixel among the corresponding pixels in the second image that minimizes the similarity measure.

2. The method according to claim 1, wherein the first set of coefficients is obtained by determining an autocorrelation matrix between the patch of pixels and surrounding patches, the first set of coefficients corresponding to autocorrelation matrix coefficients.

3. The method for depth-map estimation according to claim 2, wherein the autocorrelation matrix is a 3 by 3 array of scalars with a null coefficient at the center, the other coefficients depending on the local spatial structure of patches of pixels surrounding the pixel.

4. The method for depth-map estimation according to claim 2, wherein the second set of coefficients is obtained from the autocorrelation matrix using the following transformation: I = a b c h 0 d g f e 1 1 1 1 0 1 1 1 1 , d x = a b c h 0 d g f e − 1 − 1 − 1 1 0 − 1 1 1 1 , d y = a b c h 0 d g f e 1 1 − 1 1 0 − 1 1 − 1 − 1 , Δ x = a b c h 0 d g f e − 1 − 1 1 1 0 1 1 − 1 − 1 , Δ y = a b c h 0 d g f e 1 − 1 − 1 1 0 1 − 1 − 1 1 , with a, b, c, d, e, f, g, h being the coefficients of the autocorrelation matrix, I representing how the pixels of the patch surrounding the pixel are similar, dx,dy representing an asymmetry of the autocorrelation matrix, Δx, Δy representing a potential orientation of an edge, I, dx, dy, Δx, Δy being the second set of coefficients for the pixel p(x, y).

5. The method according to claim 1, wherein the first set of coefficients is obtained using a discrete cosine transform, the first set of coefficients corresponding to the discrete cosine transform coefficients.

6. The method for depth-map estimation according to claim 5, wherein the patch of pixels is a patch of 8x8 pixels.

7. The method for depth-map estimation according to claim 5, wherein the second set of coefficients is selected among the discrete cosine transform coefficients.

8. A data processing apparatus for depth-map estimation between at least two images respectively captured by at least two cameras, comprising: means for obtaining extrinsic and intrinsic parameters of the at least two cameras; means for applying, for each pixel of the at least two images, a first transform to a patch of pixels surrounding the pixel, to obtain a first set of coefficients representative of a local spatial structure of the patch of pixels; means for applying a second transform to the first set of coefficients to compress the first set of coefficients in a second set of coefficients representative of the local spatial structure of the patch of pixels, wherein the patch of pixels surrounding the pixel comprises a first number of pixels, and the second set of coefficients comprises a second number of coefficients that is lower than the first number of pixels of the patch of pixels; means for obtaining depth for at least one pixel of a first image among the at least two images, among at least two depth candidates, each depth candidate being associated with a corresponding pixel in a second image of the at least two images, said corresponding pixel in the second image being obtained according to the extrinsic and intrinsic parameters, wherein said means for obtaining depth comprises: means for determining a similarity measure between the at least one pixel of the first image and each corresponding pixel in the second image, the similarity measure taking into account color components and the second set of coefficients of the corresponding pixel and the at least one pixel of the first image; and means for selecting a pixel among the corresponding pixels in the second image that minimizes the similarity measure.

9. The apparatus according to claim 8, wherein the first set of coefficients is obtained by determining an autocorrelation matrix between the patch of pixels and surrounding patches, the first set of coefficients corresponding to autocorrelation matrix coefficients.

10. The apparatus according to claim 9, wherein the autocorrelation matrix is a 3 by 3 array of scalars with a null coefficient at the center, the other coefficients depending on the local spatial structure of patches of pixels surrounding the pixel.

11. The apparatus according to claim 9, wherein the second set of coefficients is obtained from the autocorrelation matrix using the following transformation: I = a b c h 0 d g f e 1 1 1 1 0 1 1 1 1 , d x = a b c h 0 d g f e − 1 − 1 − 1 1 0 − 1 1 1 1 , d y = a b c h 0 d g f e 1 1 − 1 1 0 − 1 1 − 1 − 1 , Δ x = a b c h 0 d g f e − 1 − 1 1 1 0 1 1 − 1 − 1 , Δ y = a b c h 0 d g f e 1 − 1 − 1 1 0 1 − 1 − 1 1 , with a, b, c, d, e, f, g, h being the coefficients of the autocorrelation matrix, l representing how the pixels of the patch surrounding the pixel are similar, dx,dy representing an asymmetry of the autocorrelation matrix, Δx, Δy representing a potential orientation of an edge, I, dx, dy, Δx, Δy being the second set of coefficients for the pixel p(x, y).

12. The apparatus according to claim 8, wherein the first set of coefficients is obtained using a discrete cosine transform, the first set of coefficients corresponding to the discrete cosine transform coefficients.

13. The apparatus according to claim 12, wherein the patch of pixels is a patch of 8x8 pixels.

14. The apparatus according to claim 12, wherein the second set of coefficients is selected among the discrete cosine transform coefficients.

15. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 1 to 7.