Method for encoding and decoding multiview video

Abstract

In general, the goal is to improve upon the shortcomings of conventional technologies, particularly in the field of immersive video, such as multi-view video. [Solution] The present invention relates to a method for encoding views that simultaneously represent 3D scenes from different positions or different viewing angles, which is implemented by an encoding device, and comprises: dividing the depth component of at least one view into at least one block (C2); obtaining depth information for at least one block from the texture data of at least one texture component of the view (C4); obtaining at least one depth estimation parameter from this information (C5); and encoding at least one depth estimation parameter because the depth information for at least one block is not encoded (C6).

Description

[Technical Field]

[0001] The present invention generally relates to the field of immersive video, particularly multiview video. More specifically, the present invention relates to encoding and decoding multiple captured views to generate such immersive video and to synthesizing uncaptured intermediate views. The present invention may be applied in particular, but not exclusively, to video encoding and corresponding video decoding implemented in current AVC and HEVC video coders and their extensions (MVC, 3D-AVC, MV-HEVC, 3D-HEVC, etc.). [Background technology]

[0002] It is commonplace to use 360° cameras to generate omnidirectional images, such as 360° video. Such 360° cameras consist of several 2D cameras mounted on a spherical platform. Each 2D camera captures a specific angle of the 3D scene from the same viewpoint, and the set of views captured by the cameras makes it possible to generate video representing the 3D scene according to a 360° x 180° field of view from a single viewpoint. It is also possible to capture a 3D scene according to a 360° x 180° field of view using a single 360° camera. Of course, such a field of view can be small, for example, 270° x 135°.

[0003] Subsequently, such 360° video allows users to view the scene as if they were at its center, looking around in 360°, thus providing a new way of seeing images. Such images are generally played back on virtual reality headsets, also known as HMDs, an abbreviation for "head-mounted device." However, these images can also be displayed on a 2D screen equipped with appropriate user interaction means. The number of 2D cameras used to capture the 360° scene varies depending on the platform used.

[0004] However, the aforementioned 360° technique is limited because the viewer can only see the scene from a single viewpoint.

[0005] To allow viewers to change their viewpoint, a multi-view capture system is used, in which a 3D scene is captured by a set of 2D type cameras, each capturing the scene at a specific angle. To enhance the sense of immersion in the scene, one or more missing views, i.e., views not captured by the cameras, are composited from the existing views.

[0006] The processing of these missing or intermediate views is performed by a so-called "view compositing" algorithm. For example, VSRS ("View Compositing Reference") software can be used as a view compositing algorithm.

[0007] Traditionally, such synthesis algorithms rely not only on the texture components of views captured by each camera at different points in time, but also on the depth components of these views, known as "depth maps." A depth map represents the distance between each pixel in a view and the camera that captured that view. Thus, each camera captures a view of the 3D scene in the form of texture components from its respective viewpoint, but these texture components are associated with a depth map of the 3D scene as seen from each camera's viewpoint. Several methods exist for constructing depth maps, including radar, laser, and computational methods using pixels from the current view and adjacent views.

[0008] There are known techniques for estimating depth from a set of texture components. Thus, the DERS method described in the document “(Non-Patent Document 1)” first estimates the displacement of one pixel between two views, i.e., the disparity between the two views measured in terms of the number of pixels, using at least two texture components. To do this, for a block surrounding the current pixel in the current view, DERS continuously applies a “block matching” step to identify, in another view, the block that minimizes the error with respect to the block in the current view. This search is performed horizontally because the views are considered calibrated. The search is performed within a predetermined disparity interval. That is, block matching is performed for all blocks between the minimum disparity D min and the maximum disparity D max . There is a direct link between the disparity “d” of the scene and the depth “Z”, and the depth “Z” of a pixel with disparity “d” is Z=(b*f) / d , where “b” is the distance between the optical centers of the cameras that captured the views and “f” is the focal length of the camera. Thus, the minimum disparity D min corresponds to the maximum depth Z max assumed in the scene, and the maximum disparity D max corresponds to the minimum depth Z min assumed in the scene. In a specific case, the capture of the scene is performed by specifying a predetermined value, for example, Z of 0.3 m min and, for example, Z of 5 m max . Thereby, the values D min and D max that determine the number of disparity hypotheses to be evaluated are directly generated. For example, D min = 2 pixels and D maxIf it is determined that the number of pixels is 192, this means that the block matching technique evaluates the block matching cost for all disparity hypotheses D=2,3,4,...,191,192 pixels, i.e., 191 calculations are stored and compared with one another. Thus, the DERS algorithm determines the disparity to be selected from the 191 possible disparities as a function of the cost associated with each disparity, and one or more regularization parameters, including the parameter Sc (abbreviation for "smoothing coefficient"). This coefficient determines the regularity of the generated depth map. Therefore, a lower coefficient results in a more accurate depth map, but carries the risk of containing noise, while a higher coefficient results in a very regular depth map with uniform depth zones, but carries the risk of only poorly representing small local variations.

[0009] For example, other depth estimation methods, such as the IVDE (Immersive Video Depth Estimation) method, are presented in "(Non-Patent Document 2)," which involves the parallel and collaborative determination of depth maps for a series of views.

[0010] In conventional immersive video coders, texture components and their respective depth maps are encoded and sent to a decoder, for example, installed in the user's display device. On the decoder side, each view is decoded, including the decoding of the texture components of that view, and the depth map associated with the texture components of that view is decoded. A synthesis algorithm then constructs an intermediate view corresponding to the viewpoint requested by the user from one or more decoded depth maps and one or more decoded texture components.

[0011] "(Non-Patent Document 3)" describes an immersive video coder in which the depth map is not encoded. Only the texture component of the view is encoded and sent to the decoder. On the decoder side, the texture component is decoded, and then a depth estimation method such as DERS is applied to the decoded texture component to generate an estimated depth map. The VSRS (abbreviation for "View Synthesis Reference Software") synthesis algorithm then performs view synthesis using these estimated depth maps.

[0012] This technique reduces the encoding bitrate for omnidirectional video because it eliminates the need to encode and transmit depth maps. Furthermore, the number of pixels decoded to obtain the composite view is less than that used in conventional immersive video decoders.

[0013] However, the decoding method used in this technique is computationally intensive because it requires the decoder to implement a depth estimation step. Furthermore, the depth estimated by the decoder is based on the decoded texture component, which is of lower quality than the original texture component, so the estimated depth itself is not of very good quality. Therefore, view compositing implemented with this technique is not optimal in terms of both the quality of the image rendered to the user and the consumption of computational resources. [Prior art documents] [Non-patent literature]

[0014] [Non-Patent Document 1] O.Stankiewicz, K.Wegner, M.Tanimoto and M.Domanski, “Enhanced Depth Estimation Reference Software(DERS) for Free-viewpoint Television”, ISO / IEC JTC1 / SC29 / WG11 Doc.MPEG M31518, Geneva, 2013 [Non-Patent Document 2] Dawid Mieloch,Adrian Dziembowski,Jakub Stankowski,Olgierd Stankiewicz,Marek Domanski,Gwangsoon Lee,Yun Young Jeong[MPEG-I Visual]Immersive video depth estimation,ISO / IEC JTC1 / SC29 / WG11 MPEG2020 m53407 [Non-Patent Document 3] Patrick Garus,Jung Joel,Thomas Maugey,Christine Guillemot.Bypassing Depth Maps Transmission For Immersive Video Coding.PCS 2019-Picture Coding Symposium,Nov 2019,Ningbo,China.pp.1-5.hal-02397800 [Overview of the Initiative] [Problems that the invention aims to solve]

[0015] One of the objectives of the present invention is to improve upon the shortcomings of the prior art described above. [Means for solving the problem]

[0016] For this purpose, one object of the present invention is a method for encoding views that simultaneously represent a 3D scene from different positions or different viewing angles, which is implemented by an encoding device, wherein for the depth component of at least one view, - Dividing the aforementioned depth component into at least one block, - Obtaining depth information of at least one block from the texture data of at least one texture component of the view, - Obtain at least one depth estimation parameter from this depth information. - Encoding the at least one depth estimation parameter, wherein the depth information of the at least one block is either not encoded or encoded. Regarding methods including

[0017] Such an encoding method according to the present invention avoids encoding the components or depth blocks of the depth map associated with the view when encoding the view, thereby reducing the computations implemented by the coder and allowing them to be stored on memory resources, eliminating the need to store encoded data of the depth blocks. Since the depth blocks are not encoded, the encoded data concerning these depth blocks is not sent to the decoder, thereby reducing the cost of signaling information transmitted between the coder and the decoder. To compensate for the lack of encoded data concerning the depth blocks and to enable decoding of the depth blocks, the encoding method according to the present invention implements encoding of at least one depth estimation parameter associated with the depth blocks, so that the depth estimation parameter can be used by the decoder to reconstruct the depth blocks without requiring the depth blocks to be decoded first.

[0018] According to a particular embodiment, the at least one depth estimation parameter is either a depth value of the at least one block that is greater than each of the depth values ​​of the at least one block, or a depth value of the at least one block that is less than each of the depth values ​​of the at least one block.

[0019] According to this particular embodiment, the decoder's depth estimator no longer needs to evaluate the likelihood of each possible corresponding depth of the depth block using each pixel of the reconstructed texture block from one or more views in order to reconstruct the depth block. In fact, according to this embodiment, the depth estimator simply estimates the depth of the depth block being reconstructed only within the interval between the minimum and maximum depth values ​​of the block. Such depth estimation significantly speeds up the depth estimation step, which is a complex aspect of the prior art.

[0020] According to another specific embodiment, the at least one depth estimation parameter is a parameter used by the depth estimation method.

[0021] According to this particular embodiment, the depth estimation parameters used by a known depth estimation method are favorably optimized to yield an estimated depth block that is as close as possible to the original depth block. Such parameters are, for example, regularization parameters that allow the depth estimation algorithm to find a depth map with little noise, or confidence parameters that allow the depth estimation algorithm to continue improving the depth value if its confidence is otherwise too low.

[0022] According to another specific embodiment, information representing the depth estimation method is encoded.

[0023] According to this particular embodiment, the coder can test different available depth estimation methods, each potentially yielding more or less favorable results for a given content or block, select the depth estimation method that yields the best depth estimation for the current block, and encode this selection for transmission to the decoder to apply the selected depth estimation to the current block.

[0024] Various embodiments or features described above can be added individually or in combination to the encoding method defined above.

[0025] The present invention is a device for encoding views that simultaneously represent 3D scenes from different positions or different viewing angles, wherein for the depth component of at least one view, - Divide the depth component into at least one block. - Obtaining depth information of at least one block from the texture data of at least one texture component of the view, - Obtain at least one depth estimation parameter from this depth information. - Encoding the at least one depth estimation parameter, wherein the depth information of the at least one block is either not encoded or encoded. This also relates to devices that include a processor configured to implement this.

[0026] Such encoding devices can implement the encoding methods described above, in particular.

[0027] The present invention relates to a method for decoding views that simultaneously represent a 3D scene from different positions or different viewing angles, implemented by a decoding device, wherein the depth component of at least one view is divided into at least one block, - Reading at least one depth estimation parameter associated with the at least one block in the data signal, - Decoding at least one depth estimation parameter, - Obtain depth information for the at least one block from the at least one depth estimation parameter and the texture data of the at least one reconstructed texture component of the view. This also relates to methods that include this.

[0028] Such a decoding method according to the present invention has low computational complexity and, advantageously, can be stored on memory resources. In fact, since the block depth information is not encoded and therefore not transmitted to the decoder, the decoder does not need to decode or store this information. During decoding, the decoder only needs to decode at least one depth estimation parameter transmitted in the received data signal to reconstruct the block depth information, and the at least one depth estimation parameter is less costly to transmit than the depth information.

[0029] According to a particular embodiment, the at least one depth estimation parameter is either a depth value of the at least one block that is greater than each of the depth values ​​of the at least one block, or a depth value of the at least one block that is less than each of the depth values ​​of the at least one block.

[0030] According to a particular embodiment, the at least one depth estimation parameter is a parameter used by the depth estimation method.

[0031] According to another specific embodiment, information representing the depth estimation method is decoded.

[0032] Various embodiments or features described above can be added individually or in combination to the decoding method defined above.

[0033] The present invention is a device for decoding views that simultaneously represent a 3D scene from different positions or different viewing angles, wherein the depth component of at least one view is divided into at least one block, - Reading at least one depth estimation parameter associated with the at least one block in the data signal, - Decoding at least one depth estimation parameter, - Obtain depth information for the at least one block from the at least one depth estimation parameter and the texture data of the at least one reconstructed texture component of the view. This also relates to devices that include a processor configured to implement this.

[0034] Such a decryption device can implement the decryption method described above.

[0035] The present invention relates to a view synthesis method implemented by a decoding device or a view synthesis device, - Reconstructing the view according to the decoding method according to the present invention as mentioned above. - Synthesizing at least a portion of the view from the reconstructed view and from the obtained information representing the depth of at least one block. This also relates to view compositing methods, including those that include this.

[0036] The present invention also relates to a computer program which, when executed by a processor, includes instructions for implementing an encoding method, decoding method, or synthesis method according to the present invention, in accordance with any one of the specific embodiments described above.

[0037] Such instructions can be permanently stored in a non-temporary memory medium of an encoding device implementing the above-described encoding method, a decoding device implementing the above-described decoding method, or a synthesis device implementing the above-described synthesis method.

[0038] This program can use any programming language and may be in the form of source code, object code, or intermediate code between source code and object code, a partially compiled form, or any other desirable form.

[0039] The present invention also covers computer-readable storage media or information media containing computer program instructions as described above.

[0040] The storage medium can be any entity or device capable of storing a program. For example, the medium may include storage means such as ROM, e.g., CD-ROM or miniature electronic circuit ROM, or further magnetic storage means such as a USB key or hard disk.

[0041] Similarly, the storage medium may be a medium that can be transmitted wirelessly or by other means, such as electrical signals or optical signals that can be transmitted via electrical cables or optical cables. The program according to the present invention can be downloaded, in particular, onto an internet-type network.

[0042] Alternatively, the storage medium may be an integrated circuit into which a program is incorporated, which is adapted to perform the encoding method, decoding method, or synthesis method described above, or an integrated circuit used when performing such methods.

[0043] Other features and advantages will become apparent by reading the exemplary and non-exclusive examples and specific embodiments of the invention provided as accompanying drawings. [Brief explanation of the drawing]

[0044] [Figure 1] This represents the progression of a method for encoding a view in a particular embodiment of the present invention. [Figure 2A] This figure shows a first embodiment of the step of obtaining depth estimation parameters implemented by the encoding method shown in Figure 1. [Figure 2B] This figure shows a second embodiment of the step of obtaining depth estimation parameters implemented using the encoding method shown in Figure 1. [Figure 3A] Figure 1 shows a first embodiment of signal transmission of information encoded by the encoding method. [Figure 3B] This figure shows a second embodiment of signal transmission of information encoded by the encoding method shown in Figure 1. [Figure 4] This figure shows a video encoding device that implements the encoding method shown in Figure 1. [Figure 5] This represents the progression of a method for decoding a view in a particular embodiment of the present invention. [Figure 6] Figure 5 shows a video decoding device that implements the decoding method. [Figure 7] This illustrates the progress of a missing view synthesis method in a specific embodiment of the present invention. [Figure 8A] This figure shows a synthesis device that implements the synthesis method shown in Figure 7 in a specific embodiment of the present invention. [Figure 8B] This figure shows a synthesis device that implements the synthesis method shown in Figure 7, in another specific embodiment of the present invention. [Modes for carrying out the invention]

[0045] Examples of implementation forms of multi-view video encoding methods The method for encoding multiview video is described below, and any type of multiview video coder, such as one conforming to the 3D-HEVC or MV-HEVC standard or other standards, can be used.

[0046] Referring to Figure 1, this encoding method is used for multiple views V1, ..., V N This applies to the current view, which forms part of the overall scene. These multiple views each represent a 3D scene from multiple field of view or multiple positions / orientations of the camera capturing the scene.

[0047] The encoding method according to the present invention, as of now, - View V1, - View V2, - ..., - View V i , - ..., - View V N This encodes [something].

[0048] One view considered from among N options could equally be a texture component, a depth component, or a map associated with this view. For example, the current view V i (1≦i≦N) conventionally represents the texture components T of Q (Q≧1) pixels. i and, for example, texture component T i A depth component P having Q depth values ​​associated with Q pixels of at least one texture component, such as i It is associated with, or among N, View V i It is associated with the texture components of the view other than the depth component P. i This is a texture image T i It can be generated either from or directly by capturing volumetric data of a 3D scene using a device such as LIDAR (Light Detection and Ranging).

[0049] In C1, the current view V i At this point in time, is selected, and each of the N views is selected one after another in a predetermined order.

[0050] In C2, the at least one view V i Depth component P i This is a set of multiple blocks B1, B2, ..., B j ,...,B M It is divided into (1 ≤ j ≤ M). In one possible embodiment, a single depth block is an undivided depth component P. i This is considered to correspond to: The depth component block is of a predefined size (e.g., 64x64 pixels) and can be parameterizable (and the size used is then transmitted in an encoded form) or adaptable using signal transmission of the size used, similar to that implemented in the HEVC standard. In the latter case, the depth component P iFirst, the image is divided into blocks of the largest size (e.g., 64x64 pixels), and then binary information indicating whether the block needs to be subdivided into smaller blocks is repeatedly transmitted for each block until a predefined minimum block size (e.g., 4x4 pixels) is reached, at which point no further information is transmitted. This results in the depth component P i This makes it possible to define the subdivision into blocks.

[0051] In C3, the depth component P i Current Block B j Selected, depth component P i Each of the blocks is selected one after another in a predetermined order.

[0052] In C4, the at least one current block B j In contrast, depth information IP j This is obtained. As is known by itself, the texture component T i Of the pixels in block BT of the pixels, all or part and / or N pixels of view V i For all or part of the texture components of the other views, the depth values ​​Z1~Z corresponding to these pixels R This yields the depth block BP corresponding to the pixel block BT. j It forms.

[0053] In C5, at least one depth estimation parameter PE is the depth information IP i It is obtained from.

[0054] According to the first acquisition embodiment shown in Figure 2A, in C51a, R depth values ​​Z1~Z R From among them, the maximum depth value Z max and / or minimum depth value Z min This will be decided.

[0055] In C52a, Z min Or Z max or interval [Z min ,Z max] is assigned to the depth estimation parameter PE.

[0056] According to the second acquisition embodiment shown in Figure 2B, at C51b, at least one depth estimation parameter PME of the depth estimation method is selected. This is, for example, a parameter of the depth estimation algorithm DERS mentioned above. In another example, it could be a parameter of another depth estimation algorithm, such as the IVDE algorithm. With respect to the DERS algorithm, the selected parameter is, for example, a regularization parameter or "smoothing coefficient" Sc that allows the depth estimation algorithm DERS to find a noise-free depth block corresponding to a block BT of pixels. Other examples of parameters for depth estimation algorithms include, for example, - If the confidence level is too low, confidence parameters such as the DERS algorithm parameter λ allow the DERS algorithm to continue improving the depth value. - For example, smoothing parameters such as the initial smoothing parameter β0 used in the IVDE algorithm and described in the above-mentioned document "Dawid Mieloch, Adrian Dziembowski, Jakub Stankowski, Olgierd Stankiewicz, Marek Domanski, Gwangsoon Lee, Yun Young Jeong [MPEG-I Visual] Immersive video depth estimation, ISO / IEC JTC1 / SC29 / WG11 MPEG2020 m53407" It would also be possible to use things like the above.

[0057] In C52b, the selected depth estimation method has X (X≧1) possible values ​​for the depth estimation parameter PME, Val1,...,Val k ,...,Val X For a finite set consisting of (1≦k≦X), there are X estimated depth blocks BPE1,...,BPE k ,...,BPE XThese are obtained respectively. For the parameter Sc, the finite set of possible values ​​is, for example, {0.01, 0.02, 0.04, 0.08, 0.16}. Needless to say, other values ​​are also possible depending on the current video context.

[0058] In C53b, the depth of X blocks BPE1, ..., BPE k ,...,BPE X From among them, the estimated depth obtained from the pixel block BT is the original depth block BP j The block closest to the specified value is selected. Such selection is implemented using distortion measurements such as PSNR (an abbreviation for "peak signal-to-noise ratio"), mean squared error, sum of absolute differences, or any other similar measurement. In the example shown, it is, for example, the selected estimated depth block BPE. k That is the case.

[0059] In C54b, the estimated depth block BPE selected in C53b k The value of the depth estimation parameter PME of the selected depth estimation method, Val k This is then selected as the value of the depth estimation parameter PE.

[0060] Referring again to Figure 1, at C6, the depth estimation parameter PE is encoded using a lossless coding method such as CABAC (an abbreviation for "Context-Adaptive Binary Arithmetic Coding"), or further by Huffman coding or Lempel-Ziv coding. In the example in Figure 2A, it is the value Z min Or Z max Or further interval [Z min ,Z max ] are also, and they are encoded in C6. In the example in Figure 2B, it is the value of the regularization parameter Sc, Val, that is encoded in C6. k This value Val is either the confidence parameter or, in yet another embodiment, the confidence parameter or smoothing parameter described above.k In addition, the values ​​of other parameters, such as the confidence parameter or smoothing parameter mentioned above which could be obtained at the end of step C54b, can be encoded. At the end of step C6, the encoded depth estimation parameter PE C This can be obtained.

[0061] Optionally, and for this reason represented by the dotted line in Figure 1, information IMEP, e.g., DERS or IVDE, representing the depth estimation method used in C5, is encoded in C7 using a lossless encoding method such as CABAC. Such IMEP information is shown in View V i or View V located at a different point in time i The sequence can be encoded. At the end of step C7, the encoded information IMEP C This can be obtained.

[0062] In C8, texture component T i This is encoded using a conventional video coder such as HEVC. At the end of step C8, the encoded texture component T i C This can be obtained.

[0063] According to the first embodiment shown in Figure 3A, the encoded depth estimation parameter PE C , encoded information IMEP C and encoded texture component T i C The data is contained in the same data signal F that is intended to be transmitted to the decoder described later in this specification.

[0064] According to the second embodiment shown in Figure 3B, the encoded depth estimation parameter PE C and encoded information IMEP C While the same data signal F is included in the encoded texture component T, i CThe data is contained in another data signal F', and signals F and F' are intended to be transmitted to the decoder described above.

[0065] Advantageously, according to the present invention, the encoding method is an encoded depth block BP j C It does not generate. As a result, in the examples in Figures 3A and 3B, signal F is encoded depth block BP j C It does not include.

[0066] The encoding method just described above then involves the depth component P. i Each of the blocks B1~B M Each, then, views V1~V N It can be implemented for each of these.

[0067] Examples of implementation forms for video encoding devices Figure 4 shows a simplified structure of a coding device (COD) suitable for implementing an coding method according to any one of the specific embodiments of the present invention.

[0068] According to a particular embodiment of the present invention, the actions performed by the encoding method are implemented by computer program instructions. For this purpose, the encoding device COD has a conventional computer architecture and, in particular, includes a memory MEM_C and a processing unit UT_C equipped with, for example, a processor PROC_C, and is driven by a computer program PG_C stored in the memory MEM_C. The computer program PG_C, when the program is executed by the processor PROC_C, includes instructions for implementing the actions of the encoding method as described above.

[0069] Upon initialization, the code instructions of the computer program PG_C are loaded, for example, into the RAM memory (not shown), before being executed by the processor PROC_C. The processor PROC_C of the processing unit UT_C, in particular, implements the actions of the encoding method described above, in accordance with the instructions of the computer program PG_C.

[0070] Example of an implementation form of a multi-view video decoding method A method for decoding a multi-view video that can use any type of multi-view video decoder compliant with, for example, the 3D-HEVC or MV-HEVC standard or other standards will be described below.

[0071] Referring to FIG. 5, such a decoding method is applied to a data signal representing the current view encoded according to the above-described encoding method, and the current view forms part of a plurality of views V1,...,V N and.

[0072] The decoding method according to the present invention - A data signal representing the encoded view V1, - A data signal representing the encoded view V2, -..., - A data signal representing the encoded view V i and. -..., - A data signal representing the encoded view V N and. is to decode.

[0073] The decoding method includes the following for the data signal F (FIG. 3A) or the data signals F and F' (FIG. 3B) representing the encoded current view V i to be reconstructed.

[0074] In D1, the encoded current view V i is selected at the current time, and each of the N views is selected one after another in a predetermined order.

[0075] In D2, the at least one view V i of the reconstructed depth component P i is divided into a plurality of blocks B1, B2, ..., B j , ..., B M (1 ≦ j ≦ M). In a possible embodiment, it is considered to correspond to a depth component P of a depth block that is not divided i . The depth block has a predefined size (e.g., 64×64 pixels), is parameterizable (and then the size used is decoded after being transmitted in an encoded form), or is adaptable using signaling of a size similar to that read in signal F and implemented according to the HEVC standard. In the latter case, the depth component P i is first divided into blocks of a maximum size (e.g., 64×64 pixels), and then, for example, in signal F or another signal, binary information indicating whether the blocks have to be subdivided into smaller blocks is repeatedly read for each block until a predefined minimum block size (e.g., 4×4 pixels) is reached, and no information is read for this minimum block size. This makes it possible to define the subdivision of the depth component P i into blocks.

[0076] In D3, the current block B i of the depth component P j is selected, and the blocks of the depth component P i are successively selected one after another in a predetermined order.

[0077] In D4, optionally and for this reason as represented by the dotted line in FIG. 5, the encoded information IMEP C representing the depth estimation method used in C5 (FIG. 1) is read in the data signal F (FIG. 3A or FIG. 3B).

[0078] In D5, the encoded information IMEP CThis information is decoded, for example, using a lossless decoding method such as CABAC, or further by Huffman decoding or Ramper-Ziff decoding. C This is the current view V i or View V located at a different point in time i The sequence can be decoded. At the end of step D5, the information IMEP is obtained.

[0079] Instead of the optional steps D4 and D5, the depth estimation method used in C5 can be predefined in the decoder. In this case, the IMEP information representing the depth estimation method is directly available.

[0080] In D6, the depth component P i The current Block B that will be reconstructed j and at least one encoded depth estimation parameter PE associated with C This is read in data signal F in Figure 3A or Figure 3B.

[0081] In D7, the at least one encoded depth estimation parameter PE C This can be decoded, for example, using a lossless decoding method such as CABAC, or otherwise by Huffman decoding or Ramper-Ziff decoding. The depth value Z is encoded in C6. min or Z max If so, or otherwise, interval [Z min ,Z max If it was ], - Depth value Z min This is decoded in D7 and assigned to the depth estimation parameter PE, - Or depth value Z max This is decoded in D7 and assigned to the depth estimation parameter PE, or - Depth interval [Z min ,Z max Is ] decoded in D7 and assigned to the depth estimation parameter PE? It is one of the following:

[0082] The value of the regularization parameter Sc, Val, is what is encoded in C6. k In the case of and / or the value of the confidence parameter or smoothing parameter Val according to other embodiments k Even in this case, the value that is decoded in D7 and assigned to the depth estimation parameter PE is Val k This value is Val k In addition, the values ​​of other parameters, such as the confidence parameter or smoothing parameter obtained at the end of step C54b, can be decoded.

[0083] In D8, the at least one view V i Texture component T i This is reconstructed using, for example, a conventional video decoder, such as HEVC. The reconstructed texture component T i R This is obtained at the end of step D8. Needless to say, step D8 can be implemented before steps D1-D7 or at any point in time upon receiving the data signal F (Figure 3A) or F' (Figure 3B). For this purpose, the encoded texture component T i C The data is read at signal F (Figure 3A) or F' (Figure 3B), then reconstructed, and the reconstructed texture component T i R Generates.

[0084] In D9, the current block B j Depth information IP j This includes the at least one depth estimation parameter PE decoded in D7 and the reconstructed texture component T. i R Or, out of N, view V i It is obtained from the texture data (pixels) of the reconstructed texture components of the view other than the one shown.

[0085] According to the first embodiment of step D9, by using a predefined depth estimation method or one corresponding to the IMEP parameters obtained in D5, the depth search is performed on the reconstructed depth component P i Current Block B j Depth value Z min Alternatively, the depth value Z max Or further intervals of depth values ​​[Z min ,Z max Using ], the reconstructed texture component T i R This is implemented for each pixel in the block. The advantage of this behavior is that it is not between the theoretically possible maximum depth and minimum depth (generally from 0 meters to infinity), but at most [Z min ,Z max ] and the current Block B j This is where the depth is determined. This makes it possible to reduce the complexity of the depth estimation algorithm by significantly reducing the number of depths to be tested. At the end of the first embodiment in step D9, the current block B j Reconstructed depth block BP corresponding to this j R This can be obtained.

[0086] According to the second embodiment of step D9, the value of the regularization parameter Sc (or, for example, the value of the confidence parameter or smoothing parameter) Val, which is decoded in D7, corresponds to a predefined depth estimation method or the IMEP parameter obtained in D5. k By using the current block B j Apply to the reconstructed texture component T i R A depth search is performed for each pixel in the block. At the end of the second embodiment in step D9, the current block B j Reconstructed depth block BP corresponding to this j R This is obtained. In this second embodiment, the reconstructed depth block BP j RThis is the depth block BP obtained at C4 during the encoding method in Figure 1. j It approaches as close as possible, and advantageously, according to the present invention, this depth block BP j This is neither encoded nor transmitted in signal F or F'.

[0087] In D10, the reconstructed depth block BP j R Next, the reconstructed texture component T i R The corresponding, now reconstructed P i R It is included in the depth component.

[0088] The decoding method just described above then proceeds to reconstruct each pixel block B1-B M Each of the following views V1-V is then reconstructed. N It is implemented for each instance.

[0089] Examples of implementation forms for video decoding devices Figure 6 shows a simplified structure of a decoding device (DEC) suitable for implementing a decoding method according to any one of the specific embodiments of the present invention.

[0090] According to a particular embodiment of the present invention, the actions performed by the decoding method described above are implemented by computer program instructions. For this purpose, the decoding device DEC has a conventional computer architecture and includes, in particular, memory MEM_D and a processing unit UT_D, the processing unit UT_D being equipped, for example, with a processor PROC_D and driven by a computer program PG_D stored in memory MEM_D. The computer program PG_D includes instructions for implementing the actions of the decoding method as described above, once the program is executed by the processor PROC_D.

[0091] During initialization, the code instructions of the computer program PG_D are loaded, for example, into RAM memory (not displayed) before being executed by the processor PROC_D. The processor PROC_D of the processing unit UT_D implements the actions of the decoding method described above, in particular, according to the instructions of the computer program PG_D.

[0092] Examples of view composition method implementations The view compositing method, which uses the view reconstructed according to the decoding method in Figure 5, will now be explained with reference to Figure 7.

[0093] As illustrated in Figure 7, the synthesis method according to the present invention obtains N reconstructed views V1 at the end of the decoding method in Figure 5. R ,...,V N R Use at least one reconstructed view from among them.

[0094] In S1, at least one reconstructed view V q R (1≦q≦N) is selected from N reconstructed views. Reconstructed view V q R This is the reconstructed texture component T q R and the associated reconstructed depth component P q R This includes.

[0095] In S2, at least one composite partial PV of a missing view or intermediate view, such as a view that the user has requested to be displayed on the screen of their computer or phone. sy This is the reconstructed texture component T q R From, and this reconstructed texture component T q R Reconstructed block B of pixels y R and at least one reconstructed depth block BP associated with ity R It is calculated from the following. Note that 1 ≤ y ≤ M. The composite partial PV of missing or intermediate views. sy This is computed using conventional synthesis algorithms such as the VSRS algorithm, the RVS ("Reference View Synthesizer") algorithm, and the VVS ("General-Purpose View Synthesizer") algorithm.

[0096] Examples of synthetic device implementations Figure 8A or Figure 8B presents a simplified structure of a synthetic device SYNT suitable for implementing the synthesis method of Figure 7, according to one of the specific embodiments of the present invention.

[0097] According to a particular embodiment of the present invention, the actions performed by the synthesis method shown in Figure 7 are implemented by computer program instructions. For this purpose, the synthesis device SYNT has a conventional computer architecture and includes, in particular, memory MEM_S and a processing unit UT_S, the processing unit UT_S being equipped, for example, with a processor PROC_S and driven by a computer program PG_S stored in memory MEM_S. The computer program PG_S includes instructions for implementing the actions of the synthesis method as described above, once the program is executed by the processor PROC_S.

[0098] During initialization, the code instructions of the computer program PG_S are loaded, for example, into RAM memory (not displayed) before being executed by the processor PROC_S. The processor PROC_S of the processing unit UT_S implements the actions of the synthesis method described above, in particular, according to the instructions of the computer program PG_S.

[0099] According to the embodiment shown in Figure 8A, the synthesis device SYNT is located at the output of the decoder DEC, as illustrated in Figure 8A.

[0100] According to the embodiment shown in Figure 8B, the synthesis device SYNT forms an integral part of the decoder DEC, as illustrated in Figure 8B. [Explanation of Symbols]

[0101] C1 Step C2 Step C3 Step C4 Step C5 Step C6 Step C7 Step C8 Step

Claims

1. A method for encoding views that simultaneously represent a 3D scene from different positions or different viewing angles, implemented by an encoding device, wherein for the depth component of at least one view, - Dividing the aforementioned depth component into at least one block (C2), - Obtain depth information of the at least one block from the texture data of at least one texture component of the view (C4), - Obtain at least one depth estimation parameter from the above information (C5), - Encoding the at least one depth estimation parameter (C6), wherein the depth information of the at least one block is not encoded (C6). A method that includes this.

2. The at least one depth estimation parameter is greater than each of the depth values ​​of the at least one block (Z ｍａｘ ) or the depth value of the at least one block (Z) which is smaller than each of the depth values ​​of the at least one block ｍｉｎ The encoding method according to claim 1, wherein it is one of the following:

3. The at least one depth estimation parameter is a parameter (Sc; λ; β) used by the depth estimation method. ０ The encoding method according to claim 1, wherein the encoding method is as follows:

4. The encoding method according to any one of claims 1 to 3, wherein information representing the depth estimation method is encoded.

5. A device for encoding views that simultaneously represent a 3D scene according to different positions or different viewing angles, wherein for the depth component of at least one view, - Dividing the aforementioned depth component into at least one block, - Obtaining depth information of at least one block from the texture data of at least one texture component of the view, - Obtain at least one depth estimation parameter from the aforementioned information, - Encoding the at least one depth estimation parameter, wherein the depth information of the at least one block is not encoded, or is encoded. A device containing a processor (UT_C) configured to implement [the specified feature].

6. A computer program which, when executed on a computer, includes program code instructions for implementing the encoding method described in any one of claims 1 to 4.

7. A computer-readable information medium containing instructions for the computer program described in claim 6.

8. A method for decoding views that simultaneously represent a 3D scene from different positions or different viewing angles, implemented by a decoding device, wherein for the depth component of at least one view, which is divided into at least one block, - Reading at least one depth estimation parameter associated with the at least one block in the data signal (D6), - Decoding the at least one depth estimation parameter (D7), - Obtain depth information of the at least one block from the at least one depth estimation parameter and the texture data of the at least one reconstructed texture component of the view (D9) A method that includes this.

9. The at least one depth estimation parameter is greater than each of the depth values ​​of the at least one block (Z ｍａｘ ) or the depth value of the at least one block (Z) which is smaller than each of the depth values ​​of the at least one block ｍｉｎ The decoding method according to claim 8, wherein it is one of the following:

10. The decoding method according to claim 8, wherein the at least one depth estimation parameter is a parameter (Sc) used by the depth estimation method.

11. The decoding method according to any one of claims 8 to 10, wherein information representing the depth estimation method is decoded.

12. A device for decoding views that simultaneously represent 3D scenes from different positions or different viewing angles, wherein the depth component of at least one view is divided into at least one block, - Reading at least one depth estimation parameter associated with at least one block in the data signal, - Decoding the aforementioned at least one depth estimation parameter, - Obtain depth information of the at least one block from the at least one depth estimation parameter and the texture data of the at least one reconstructed texture component of the view. A device containing a processor (UT_D) configured to implement [the specified feature].

13. A view compositing method implemented by a decoding device or a view compositing device, - Reconstructing the view according to the decoding method described in any one of claims 8 to 11. - To synthesize at least a portion of the view from the reconstructed view and from the obtained information representing the depth of at least one block. A view compositing method that includes this.

14. A computer program that, when executed on a computer, includes program code instructions for implementing the decoding method described in any one of claims 8 to 11 or the synthesis method described in claim 13.

15. A computer-readable information medium containing instructions for the computer program described in claim 14.

Images