Video coding method with syntax element signaling of an assignment function, used by a cube-based projection, and associated video decoding method

Abstract

Video coding methods, comprehensive: Encoding a projection-based frame (IMG) to generate a portion of a bitstream (BS), wherein at least a portion of a 360-degree content of a sphere (200) is mapped via a cube-based projection onto projection surfaces (0, 1, 2, 3, 4, 5), and the projection-based frame (IMG) comprises the projection surfaces (0, 1, 2, 3, 4, 5) packed in a cube-based projection arrangement (202); and Signaling at least one syntax element (SE) via the bitstream (BS), wherein the at least one syntax element (SE) is linked to a mapping function which is used by the cube-based projection to determine sample locations for each of the projection surfaces (0, 1, 2, 3, 4, 5); wherein the mapping function is a parameterized mapping function used by the cube-based projection to adjust sample locations for each of the projection surfaces (0, 1, 2, 3, 4, 5), and wherein the at least one syntax element (SE) signaled via the bitstream (BS) has a plurality of syntax elements which specify at least coefficients of the parameterized mapping function; and where each of the projection surfaces (0, 1, 2, 3, 4, 5) has the majority of syntax elements: a first syntax element designed to specify a coefficient used in the parameterized mapping function of a first axis (u, v); characterized in that for each of the projection surfaces (0, 1, 2, 3, 4, 5) the majority of syntax elements further exhibit: a second syntax element designed to indicate whether the parameterized mapping function of the first axis (u, v) refers to a position value of a second axis (v, u); where if the second syntax element is equal to 1, it indicates that the parameterized mapping function of the first axis (u, v) refers to a position value of the first axis (u, v) and to the position value of the second axis (v, u), and where if the second syntax element is equal to 0, it indicates that the parameterized mapping function of the first axis (u, v) refers to the position value of the first axis (u, v) and does not refer to the position value of the second axis (v, u).

Description

Cross-reference to related patent application(s)

[0001] This application claims the benefits of preliminary US application No. 62 / 869,627, filed on July 2, 2019, preliminary US application No. 62 / 871,840, filed on July 9, 2019, preliminary US application No. 62 / 903,056, filed on September 20, 2019, and US application No. 16 / 899,575, filed on June 11, 2020. Technical field

[0002] The present invention relates to video encoding and video decoding, and more precisely to a video encoding method with syntax element signaling of an imaging function used by a cube-based projection, and an associated video decoding method. In particular, the present invention relates to a video encoding method according to the preamble of independent claim 1 and to a video decoding method according to the preamble of independent claim 8. Such methods are known from WO 2018 / 218028 A1. Such methods are also known from WO 2019 / 083943 A1, WO 2017 / 205648 A1, and Ye Yan; Boyce Jill: Algorithm description of projection format conversion and video quality metrics in 360Lib Version 9 (JVET-M1004). In: Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 13th Meeting: Marrakech, MA, January 9-18, 2019. As of February 17, 2019. URL: https: / / jvet-experts.org / doc_end_user / current_document.php?id=5757 [accessed on 25.08.2022] known. background

[0003] Virtual reality (VR) with head-mounted displays (HMDs) is associated with a variety of applications. The ability to present a user with a wide field of view can be used to deliver immersive visual experiences. A real-world environment must be captured in all directions, resulting in a 360-degree video that corresponds to a sphere of view. With advancements in camera setups and HMDs, delivering VR content may soon become a bottleneck due to the high bitrate required to represent such 360-degree content. If the resolution of the 360-degree video is 4K or higher, data compression / encoding is crucial for reducing the bitrate.

[0004] Generally, the 360-degree video corresponding to a sphere is transformed into a frame containing 360-degree image content represented by projection surfaces arranged in a 360-degree virtual reality (360-VR) projection setup. The resulting frame is then encoded into a bitstream for transmission. Assume that conventional cube mapping projection (CMP) is used to project the 360-degree content onto six faces of the cube. Due to inherent properties of conventional CMP, the sampling rate near the center of the projection surfaces is much lower than at the projection surface boundary.Therefore, there is a need for a modified cube mapping projection design that can adapt the sampling distribution of the conventional CMP to achieve a more uniform sampling rate, and can signal (a) syntax element(s) associated with the modified cube mapping projection to a video decoder. Summary

[0005] One of the objects of the claimed invention is to provide a video coding method with syntax element signaling of an imaging function, which is implemented by a cube-based projection, and an associated video decoding method. Such a video coding method is the video coding method according to independent claim 1. Such a video decoding method is the video decoding method according to independent claim 8. The dependent claims contain advantageous embodiments of the invention.

[0006] According to a first aspect of the present invention, an exemplary video coding method is disclosed. The exemplary video coding method comprises: encoding a projection-based frame to generate a portion of a bitstream, wherein at least a portion of a 360-degree content of a sphere is mapped onto projection surfaces via a cube-based projection, and the projection-based frame comprises the projection surfaces packed in a cube-based projection arrangement; and signaling at least one syntax element via the bitstream, wherein the at least one syntax element is associated with a mapping function used by the cube-based projection to determine sample locations for each of the projection surfaces.

[0007] According to a second aspect of the present invention, an exemplary video decoding method is disclosed. The exemplary video decoding method comprises: decoding a portion of a bitstream to generate a decoded frame, wherein the decoded frame is a projection-based frame having projection surfaces packed in a cube-based projection arrangement, and at least a portion of a 360-degree content of a sphere is mapped onto the projection surfaces via a cube-based projection; and analyzing at least one syntax element from the bitstream, wherein the at least one syntax element is associated with a mapping function used by the cube-based projection to determine sample locations for each of the projection surfaces.

[0008] These and other problems of the present invention will undoubtedly become apparent to those with ordinary knowledge in the field after reading the following detailed description of the preferred embodiment, which is illustrated in the various figures and drawings. Brief description of the drawings Fig. Figure 1 is a diagram illustrating a 360-degree virtual reality (360-VR) system according to an embodiment of the present invention. Fig. Figure 2 is a diagram representing a cube-based projection according to an embodiment of the present invention. Fig. Figure 3 is a diagram that represents a mapping between a square projection surface of a cube in a 3D space and a square projection surface on a 2D plane according to an embodiment of the present invention. Detailed description of preferred embodiments

[0009] Certain terms relating to specific components are used within the following description and claims. As anyone with knowledge of the field will recognize, electronic equipment manufacturers may refer to a component by different names. This document does not intend to differentiate between components that differ in name but not in function. In the following description and claims, the terms "include" and "feature" are used in an open manner and should therefore be interpreted as meaning "include, but not be limited to...". Furthermore, the term "connect" is intended to mean either an indirect or a direct electrical connection.Accordingly, if one device is connected to another device, this connection can be made by a direct electrical connection or by an indirect electrical connection via other devices and connections.

[0010] Fig. Figure 1 is a diagram illustrating a 360-degree virtual reality (360-VR) system according to an embodiment of the present invention. The 360-VR system 100 comprises a source electronics device 102 and a target electronics device 104. The source electronics device 102 includes a video recording device 112, a conversion circuit 114, and a video encoder 116. For example, the video recording device 112 can be a 360-degree camera. The conversion circuit 114 generates a projection-based frame IMG with a 360-degree virtual reality (360-VR) projection arrangement L_VR according to a 360-degree video frame S_IN corresponding to a sphere, wherein the 360-degree video frame S_IN encompasses the sphere's 360-degree content.The video encoder 116 is a coding circuit which encodes the projection-based frame IMG (which has projection surfaces packed in the 360-VR projection arrangement L_VR) to generate a portion of a bitstream BS and outputs the bitstream BS to the destination electronic device 104 via a transmission device 103, such as a wired / wireless communication device or a storage medium.

[0011] The targeting electronics device 104 can be a head-mounted display (HMD) device. As in Fig. As shown in Figure 1, the target electronic device 104 comprises a video decoder 122, a graphics playback circuit 124, and a display device 126. The video decoder 122 is a decoding circuit that receives the bitstream BS from the transmission device 103 (e.g., a wired / wireless communication link or a storage medium) and decodes a portion of the received bitstream BS to generate a decoded frame IMG'. In this embodiment, the frame IMG to be encoded by the video decoder 116 has a 360-degree VR projection arrangement L_VR. Therefore, after a portion of the bitstream BS has been decoded by the video decoder 122, the decoded frame (i.e., the reconstructed frame) IMG' has the same 360-degree VR projection arrangement L_VR. In other words, the decoded frame IMG' is also a projection-based frame, which has projection surfaces packed in the 360 ​​VR projection arrangement L_VR.The graphics playback circuit 124 is configured to control / drive the display device 126 to display image content from a viewport area selected by a user. The graphics playback circuit 124 may include a conversion circuit 125, which is configured to process a portion of the image content transmitted by the decoded frame IMG' to obtain pixel data associated with the image content of the selected viewport area.

[0012] In this embodiment, the 360-VR projection arrangement L_VR is a cube-based projection arrangement. Therefore, projection surfaces derived from different faces of a cube are packed into the cube-based projection arrangement, which is deployed by the projection-based frame IMG / the decoded frame IMG'. In one embodiment, a cube-based projection with six square projection surfaces representing a complete 360°×180° panoramic video (i.e., all of 360 degrees of a sphere) can be used. In another embodiment, a cube-based projection with five projection surfaces (comprising one full face and four half faces) representing a 180°×180° panoramic video (i.e., a portion of 360 degrees of content on a sphere) can be used.To put it simply, a cube-based projection can be a regular cube-mapping projection or a hemispherical cube-mapping projection, depending on actual design considerations. For the sake of brevity and simplicity, it is assumed that six square projection surfaces are obtained by the cube-based projection and packed into the cube-based projection array.

[0013] Regarding the conversion circuit 114 of the source electronics device 102, a cube-based projection is used to generate square projection surfaces of a cube in a three-dimensional (3D) space. Fig. Figure 2 is a diagram representing a cube-based projection according to an embodiment of the present invention. The 360-degree content on the sphere 200 is projected onto six faces of a cube 201, comprising a top face (labeled "Top"), a bottom face (labeled "Bottom"), a left face (labeled "Left"), a front face (labeled "Front"), a right face (labeled "Right"), and a back face (labeled "Back"). As shown in Figure 2, the 360-degree content on the sphere 200 is projected onto six faces of a cube 201, comprising a top face (labeled "Top"), a bottom face (labeled "Bottom"), a left face (labeled "Left"), a front face (labeled "Front"), a right face (labeled "Right"), and a back face (labeled "Back"). Fig. As shown in Figure 2, an image content of a northern polar region of sphere 200 is projected onto the upper surface “Top”, an image content of a southern polar region of sphere 200 is projected onto the lower surface “Bottom”, and an image content of an equatorial region of sphere 200 is projected onto the left surface “Left”, the front surface “Front”, the right surface “Right” and the rear surface “Back”.

[0014] In the 3D space defined by an x-axis, a y-axis, and a z-axis, every point lies on six surfaces at (x, y, z), where x, y, z ∈ [-1,1]. Therefore, the front surface "Front" lies on an x-plane with x=1, the back surface "Back" lies on an x-plane with x=-1, the top surface "Top" lies on a z-plane with z=1, the bottom surface "Bottom" lies on a z-plane with z=-1, the left surface "Left" lies on a y-plane with y=1, and the right surface "Right" lies on a y-plane with y=-1.

[0015] A forward transformation is used to transform the 3D space (x, y, z) to the 2D plane (u, v). Therefore, the top face "Top", the bottom face "Bottom", the left face "Left", the front face "Front", the right face "Right", and the back face "Back" of cube 201 in 3D space are transformed into a top face (labeled "2"), a bottom face (labeled "3"), a left face (labeled "5"), a front face (labeled "0"), a right face (labeled "4"), and a back face (labeled "1") on the 2D plane. Each face is defined on a 2D plane by a u-axis and a v-axis, with each point located at (u, v).

[0016] An inverse transformation is used to transform from the 2D plane (u, v) to 3D space (x, y, z). Therefore, the top face (labeled "2"), the bottom face (labeled "3"), the left face (labeled "5"), the front face (labeled "0"), the right face (labeled "4"), and the back face (labeled "1") on the 2D plane are transformed into the top face "Top", the bottom face "Bottom", the left face "Left", the front face "Front", the right face "Right", and the back face "Back" of cube 201 in 3D space.

[0017] Fig. Figure 3 is a diagram representing a mapping between a square projection surface of a cube in 3D space and a square projection surface on a 2D plane according to an embodiment of the present invention. For an integer pixel q with a 2D coordinate (u, v) on the front surface “0”, the 2D coordinate (u, v) can be mapped to a 3D coordinate (x, y, 1) of a point p on the front surface “Front”. Therefore, the integer pixel q is defined by the point p. In particular, for an integer pixel contained in the front surface “0”, its corresponding position in the front surface “Front” can be found by a 2D-to-3D mapping function used by the inverse transformation.

[0018] The inverse transformation can be used by the conversion circuit 114 of the source electronics device 102 to generate the top surface "2", bottom surface "3", left surface "5", front surface "0", right surface "4", and back surface "1". The top surface "2", bottom surface "3", left surface "5", front surface "0", right surface "4", and back surface "1" on the 2D plane are packed to form the projection-based frame IMG, which is to be encoded by the video encoder 116. The projection-based frame IMG to be encoded must be rectangular. For example, the projection-based frame IMG can contain projected image data arranged in the compact cube-based projection array 202 to avoid the use of dummy areas (e.g., black areas, gray areas, or white areas).Since the present invention does not focus on the cube-based projection arrangement design, a further description of the compact cube-based projection arrangement 202 is omitted here for the sake of brevity.

[0019] The video decoder 122 receives the bitstream BS from the transmission device 103 and decodes a portion of the received bitstream BS to generate the decoded frame IMG', which has the same projection arrangement L_VR (e.g., the compact cube-based projection arrangement 202) used on the encoder side. With respect to the conversion circuit 125 of the target electronics device 104, a forward transformation is used to transform from 3D space (x, y, z) to the 2D plane (u, v) in order to determine pixel values ​​of pixels in each of the top face "Top", bottom face "Bottom", left face "Left", front face "Front", right face "Right", and rear face "Back". Or the inverse transformation is used to transform from the 2D space (u, v) to the 3D plane (x, y, z) in order to map the sample locations of a projection-based frame back to the sphere.

[0020] It's back on Fig. Reference is made to Figure 3. For an integer pixel P with a 3D coordinate (x, y, 1) in the front face "Front", the 3D coordinate (x, y, 1) can be mapped to a 2D coordinate (u, v) of a point Q on the front face "0". Therefore, the integer pixel P is defined by the point Q. In particular, for every integer pixel contained in the front face "Front", its corresponding position in the front face "0" can be found by a 3D-to-2D mapping function used by the forward transform.

[0021] It is assumed that the conventional cube mapping projection (CMP) is used to project the 360-degree content on the sphere 200 onto six faces, "Top", "Bottom", "Right", "Front", "Left", and "Back", of the cube 201. The sampling rate near the center of the cube faces is much lower than at the cube face boundary. In other words, the sampling distribution of the conventional CMP is not uniform. To address this problem, the present invention proposes a modified cube mapping projection design which improves the sampling distribution of the conventional CMP by applying a mapping function to (u, v) (e.g., f). x (u, v) and f y(u, v)) can be adjusted to achieve a more uniform sampling rate. In some embodiments of the present invention, the 2D-to-3D mapping function used by the inverse transformation and the 3D-to-2D mapping function used by the forward transformation can be a parameterized mapping function in the conversion circuits 114 and 125. {fx(u,v)=u1+au(1−buv2)(1−u2)fy(u,v)=v1+av(1−bvu2)(1−v2) insert a location of a point on a square projection surface on a 2D plane (e.g., "0", "1", "2", "3", "4", or "5") that is in Fig. 2 are shown) is represented by (u, v), and a location of a point on a square projection surface in a 3D space (e.g., "front", "back", "top", "bottom", "left" or "right", which are in Fig. 2 are shown) by (f x (u, v), f y (u, v)) are represented. Furthermore, coefficients a u , b u , av and b v The parameterized mapping function is specified by syntax elements signaled via the bitstream BS. The parameterized mapping function serves only illustrative purposes and is not intended to limit the present invention. For example, coefficients of F x (u, v) one or both of a u and b u include, and coefficients of F y (u, v) can select one or both of a v and b v include.

[0022] For a better understanding of the technical features of the present invention, several exemplary syntax signaling methods are described below.The video encoder 116 can employ one of the proposed syntax signaling methods to signal (a) syntax element(s) SE that indicates (the) configuration information of a mapping function employed by the conversion circuit 114, and the video decoder 122 can analyze (a) syntax element(s) SE' signaled (by a proposed syntax signaling method employed by the video encoder 116, and can provide the graphics playback circuit 124 (in particular the conversion circuit 125) with the analyzed syntax element(s) SE', so that the graphics playback circuit 124 (in particular the conversion circuit 125) is informed of the configuration information of the mapping function employed by the conversion circuit 114.In this way, when an image content of a viewport area selected by a user is determined, the conversion circuit 125 can refer to the configuration information of the mapping function to perform a transformation correctly. Ideally, the syntax element(s) SE that is encoded into the bitstream BS by the video encoder 116 is the same as the syntax element(s) SE' that is parsed from the bitstream BS by the video decoder 122.

[0023] It should be noted that the descriptors in the following example syntax tables specify the analysis process for each syntax element. For example, the syntax elements could be encoded using a fixed-length encoding (e.g., u(n)). Taking the descriptor u(n) as an example, this describes an unsigned integer using n bits. However, this is for illustrative purposes only and is not intended to be a limitation of the present invention. In practice, the syntax elements can be encoded using a fixed-length encoding (e.g., f(n), i(n), or u(n)) and / or a variable-length encoding (e.g., ce(v), se(v), or ue(v)). The descriptor f(n) denotes a bit sequence of a fixed structure using n bits, written (from left to right) with the leftmost bit first. The descriptor i(n) means a signed integer using n bits.The descriptor ce(v) denotes a context-adaptive, entropy-encoded syntax element of variable length with the leftmost bit first. The descriptor se(v) denotes a signed, integer, exponentially Golomb-encoded syntax element with the leftmost bit first. The syntax element ue(v) denotes an unsigned, integer, exponentially Golomb-encoded syntax element with the leftmost bit first.

[0024] According to a first syntax signaling procedure, the following syntax table can be used.

[0025] The syntax element `gcmp_mapping_function_type` is set to specify the mapping function used to adjust the sample locations of the cube mapping projection. The value of `gcmp_mapping_function_type` should be in the range 0 to 2, inclusive.

[0026] When the syntax element `gcmp_mapping_function_type` is equal to 0, it indicates that the conventional CMP, as specified by the Cube Mapping Projection Additional Extension Information (SEI) message in Rec. ITU-T H.265 | ISO / IEC 23008-2 and Rec. ITU-T H.264 | ISO / IEC 14496-10, is used. For example, in a case where `gcmp_mapping_function_type` is 0, the mapping function used by the conventional cube mapping projection without sampling distribution matching can be expressed as {fx(u)=ufy(v)=v.

[0027] If the syntax element `gcmp_mapping_function_type` is equal to 1, it indicates that the equilateral mapping function is used to fit sample locations of all square projection surfaces. For example, in a case where `gcmp_mapping_function_type` is 1, the mapping function used by the equilateral cube mapping (EAC) projection with a sampling distribution fit can be expressed as {fx(u)=tan(u∗π4)fy(v)=tan(v∗π4).

[0028] If the syntax element `gcmp_mapping_function_type` is equal to 2, it indicates that the aforementioned parameterized mapping function is used to fit sample locations of all square projection surfaces. Additionally, coefficients of the parameterized mapping function used to determine the sample locations of the i-th projection surface (i.e., the nth projection surface) are specified., a projection surface located at surface position index i in the cube-based projection array) is specified by syntax elements gcmp_function_coeff_u[i], gcmp_function_u_affected_by_y_flagl[i], gcmp_function_coeff_y[i] and gcmp_function_y_affected_by_u_flag[i], where the front surface is assigned a surface index gcmp_face_index[i] == 0, the back surface is assigned a surface index gcmp_face_index[i] == 1, the top surface is assigned a surface index gcmp_face_index[i] == 2, the bottom surface is assigned a surface index gcmp_face_index[i] == 3, the right surface is assigned a surface index gcmp_face_index[i] == 4 and the left surface is assigned a surface index gcmp_face_index[i] == 5 is assigned.

[0029] The syntax element gcmp_function_coeff_u[i] specifies the coefficient used in the parameterized mapping function of the u-axis of the i-th projection surface. For example, au = (gcmp_function_coeff_u[i] + 1) / 2 7 If the syntax element gcmp_function_coeff_u[i] is not present, it is inferred to be equal to 0.

[0030] If the syntax element gcmp_function_u_affected_by_y_flag[i] is equal to 1, it indicates that the parameterized mapping function of the u-axis refers to the v-position of the sample location. If the syntax element gcmp_function_u_affected_by_y_flag[i] is equal to 0, it indicates that the parameterized mapping function in the u-axis does not refer to the v-position of the sample location. For example, bu=gcmp_function_u_affected_by_v_flag[i].

[0031] The syntax element gcmp_function_coeff_y[i] is set to specify the coefficient used in the parameterized mapping function of the v-axis of the i-th projection surface. For example, α v = (gcmp_function_coeff_v[i] + 1) / 2 7If the syntax element gcmp_function_coeff_y[i] is not present, it is inferred to be equal to 0.

[0032] If the syntax element gcmp_function_y_affected_by_u_flag[i] is equal to 1, it indicates that the parameterized mapping function of the v-axis refers to the u-position of the sample location. If the syntax element gcmp_function_y_affected_by_u_flag[i] is equal to 0, it indicates that the parameterized mapping function in the v-axis does not refer to the u-position of the sample location. For example, bv=gcmp_function_v_affected_by_u_flag[i].

[0033] The 2D-to-3D mapping function employed by the conversion circuit 114 or 125 can be defined by the conventional CMP mapping function without sampling distribution adjustment, the EAC mapping function with sampling distribution adjustment, or the parameterized mapping function with sampling distribution adjustment, depending on actual application requirements. In some embodiments of the present invention, inputs to the 2D-to-3D mapping process can include a width (faceWidth) and a height (faceHeight) of a projection surface and can further include a sample location (hPos, vPos) of the projection surface on a 2D plane, wherein hPos ∈ [0, faceWidth] and vPos ∈ [0, faceHeight]; and a final output of the 2D-to-3D mapping process can include spherical coordinates (φ, θ) for the sample location in degrees.

[0034] When the 2D-to-3D mapping process begins, the following applies: hPos'=−(2*hPos÷faceWidth)+1, and hPos'=−(2*vPos÷faceHeight)+1, where hPos',vPos'∈[−1,1].

[0035] If gcmp_mapping_function_type is equal to 0, the following applies: hPos''=hPos', and vPos''=vPos'.

[0036] If gcmp_mapping_function_type is equal to 1, the following applies: hPos''=tan(hPos'*π÷4), and vPos''=tan(vPos'*π÷4).

[0037] If gcmp_mapping_function_type is equal to 2, the following applies: coeffU[n]=(gcmp_function_coeff_u[n]+1)÷128, coeffV[n]=(gcmp_function_coeff_v[n]+1)÷128, hPos''=hPos'÷[1+coeffU[n]*(1−gcmp_function_u_affected−by_v_flag[n]*vPos'2)*(1−hPos'2)]), and vPos''=vPos'÷[1+coeffV[n]*(1−gcmp_function_v_affected−by_u_flag[n]*hPos'2)*(1−hPos'2)]), where n is a surface position index in the cube-based where n represents a surface position index in the cube-based projection arrangement. For example, with respect to a sample location (hPos, vPos) of a projection surface located at a surface position index n in the cube-based projection arrangement, a fitted sample position (hPos'', hPos'') can be obtained by the parameterized mapping function.

[0038] After the adjusted sample position (hPos'', hPos'') is obtained by the parameterized mapping function, the following applies: if (gcmp_face_index[n] == 0) { / * positive x front face * / x = 1,0, y = hPos'' z = vPos''} else if (gcmp_face_index[n] == 1) { / * negative x back face * / x = -1,0 y = -vPos'' z = -hPos''} else if (gcmp_face_index[n] == 2) { / * positive z upper face * / x = -hPos'' y = -vPos'' z = 1,0} else if (gcmp_face_index[n] == 3) { / * negative z lower face * / x = hPos'' y = -vPos'' z = -1,0} else if (gcmp_face_index[n] == 5) { / * positive y left face * / x = -hPos'' y = 1.0 z = vPos''} else if (gcmp_face_index[n] == 4) { / * negative y right face * / x = hPos'' y = -1.0 z = vPos'' φ=Atan2(y, x)*180÷π θ=Asin(z÷x2+y2+z2)*180÷π

[0039] Regarding the first mapping function design, the parameterized mapping function is applied to both the u-axis and the v-axis of a projection surface when gcmp_mapping_function_type equals 2. However, this serves only illustrative purposes and is not intended to be a limitation of the present invention. Alternatively, it may be permissible to apply the parameterized mapping function to only one of the u-axis and the v-axis of a projection surface when gcmp_mapping_function_type equals 2.

[0040] According to a second syntax signaling method, the following syntax table can be used.

[0041] If the syntax element `gcmp_u_axis_adjustment_function_enabled_flag[i]` is equal to 0, it specifies that the sampling in the u-axis is not modified. If the syntax element `gcmp_u_axis_adjustment_function_enabled_flag[i]` is equal to 1, it specifies that the parameterized mapping function is used to adjust the sampling with coefficients in the u-axis. As mentioned above, coefficients of the parameterized mapping function applied to the u-axis of the i-th projection surface are specified by the syntax elements `gcmp_function_coeff_u[i]` and `gcmp_function_u_affected_by_v_flagl[i]`.

[0042] If the syntax element `gcmp_v_axis_adjustment_function_enabled_flag[i]` is equal to 0, it specifies that the sampling in the v-axis is not modified. If the syntax element `gcmp_v_axis_adjustment_function_enabled_flag[i]` is equal to 1, it specifies that the parameterized mapping function is used to adjust the sampling with coefficients in the v-axis. As mentioned above, the coefficients of the parameterized mapping function applied to the v-axis of the i-th projection surface are specified by the syntax elements `gcmp_function_coeff_v[i]` and `gcmp_function_v_affected_by_u_flagl[i]`.

[0043] Regarding the first mapping function design and the second mapping function design, a mapping function used by the conversion circuits 114 and 125 can be selected from a group consisting of the conventional CMP mapping function without sampling distribution adaptation, the EAC mapping function with sampling distribution adaptation, and the parameterized mapping function with sampling distribution adaptation. This is for illustrative purposes only and is not intended to be a limitation of the present invention. Alternatively, a mapping function used by the conversion circuits 114 and 125 can be determined solely by the parameterized mapping function. Therefore, the syntax element gcmp_function_mapping_type can be omitted without being signaled via the bitstream BS.

[0044] According to a third syntax signaling method, the following syntax table can be used.

[0045] According to a fourth syntax signaling method, the following syntax table can be used.

[0046] Since a person with knowledge in the field can easily understand the details of the third and fourth mapping function interpretations after reading the preceding paragraphs, which address the first and second mapping function interpretations, further description is omitted here for the sake of brevity.

[0047] In some embodiments of the present invention, the encoder-side conversion circuit 114 and the decoder-side conversion circuit 125 can be configured to support only predefined mapping functions with fixed settings. Therefore, the syntax element gcmp_mapping_function_type can be signaled via the bitstream BS.

[0048] According to a fifth syntax signaling method, the following syntax table can be used.

[0049] The mapping function specified by the syntax element `gcmp_mapping_function_type` can be selected from a group consisting of a conventional cube projection, an equiangular mapping function, and a parameterized mapping function. In one case, where a 2D-to-3D mapping function used by the conversion circuit 114 or 125 is defined by the conventional CMP mapping function without sampling distribution adjustment, the syntax element `gcmp_mapping_function_type` can be set to 0 and then signaled via the bit stream BS. In another case, where a 2D-to-3D mapping function used by the conversion circuit 114 or 125 is defined by the EAC mapping function with sampling distribution adjustment, the syntax element `gcmp_mapping_function_type` can be set to 1 and then signaled via the bit stream BS.

[0050] Those with expertise in the field will readily recognize that numerous modifications and alterations to the apparatus and the method can be made while retaining the teachings of the invention. Accordingly, the foregoing disclosure should be considered as limited only by the dimensions and limits of the appended claims.

Claims

[1] Video coding method comprising: Encoding a projection-based frame (IMG) to generate a portion of a bitstream (BS), wherein at least a portion of a 360-degree content of a sphere (200) is mapped onto projection surfaces (0, 1, 2, 3, 4, 5) via a cube-based projection, and the projection-based frame (IMG) comprises the projection surfaces (0, 1, 2, 3, 4, 5) packed in a cube-based projection array (202); and Signaling at least one syntax element (SE) via the bit stream (BS), wherein the at least one syntax element (SE) is linked to a mapping function used by the cube-based projection to determine sample locations for each of the projection surfaces (0, 1, 2, 3, 4, 5); wherein the mapping function is a parameterized mapping function used by the cube-based projection to adapt sample locations for each of the projection surfaces (0, 1, 2, 3, 4, 5), and the at least one syntax element (SE) signaled via the bit stream (BS) comprises a plurality of syntax elements specifying at least coefficients of the parameterized mapping function; and where for each of the projection surfaces (0, 1, 2, 3, 4, 5) the plurality of syntax elements comprises: a first syntax element configured to specify a coefficient used in the parameterized mapping function of a first axis (u, v); characterized by that for each of the projection surfaces (0, 1, 2, 3, 4, 5) the majority of syntax elements further comprises: a second syntax element configured to indicate whether the parameterized mapping function of the first axis (u, v) refers to a position value of a second axis (v, u); where if the second syntax element is equal to 1, it indicates that the parameterized mapping function of the first axis (u, v) refers to a position value of the first axis (u, v) and to the position value of the second axis (v, u), and where if the second syntax element is equal to 0, it indicates that the parameterized mapping function of the first axis (u, v) refers to the position value of the first axis (u, v) and does not refer to the position value of the second axis (v, u). [2] A video coding method according to claim 1, wherein for each of the projection surfaces (0, 1, 2, 3, 4, 5), the plurality of syntax elements further comprises: a third syntax element configured to specify a coefficient used in the parameterized mapping function of the second axis (v, u). [3] A video coding method according to claim 2, wherein for each of the projection surfaces (0, 1, 2, 3, 4, 5), the plurality of syntax elements further comprises: a fourth syntax element configured to indicate whether the parameterized mapping function of the second axis (v, u) refers to a position value of the first axis (u, v). [4] A video coding method according to claim 2, wherein the at least one syntax element (SE) signaled via the bit stream (BS) further comprises: a fourth syntax element designed to specify that the parameterized mapping function is used by the cube-based projection. [5] A video coding method according to claim 1, wherein an adapted sample location ( f x (u, v), fy (u, v)) is obtained by applying the parameterized mapping function to a target sample location (u, v) on one of the projection surfaces, and the parameterized mapping function is expressed by: {fx(u, v)=u1+au(1−buv2)(1−u2)fy(u, v)=v1+av(1−bvu2)(1−v2), where coefficients a u , b u , a v and b v the parameterized mapping function is specified by the plurality of syntax elements signaled via the bit stream (BS). [6] Video coding method according to claim 1, wherein the at least one syntax element (SE) comprises: a syntax element designed to specify that the mapping function is used by the cube-based projection. [7] A video coding method according to claim 6, wherein the mapping function specified by the syntax element is selected from a group consisting of a conventional cube mapping projection, an equiangular mapping function, and a parameterized mapping function. [8] Video decoding method comprising: Decoding a portion of a bitstream (BS) to generate a decoded frame (IMG'), wherein the decoded frame (IMG') is a projection-based frame (IMG') having projection surfaces (0, 1, 2, 3, 4, 5) packed in a cube-based projection arrangement (202), and at least a portion of a 360-degree content of a sphere (200) is mapped onto the projection surfaces (0, 1, 2, 3, 4, 5) via a cube-based projection; and Analyzing at least one syntax element (SE') from the bit stream (BS), wherein the at least one syntax element (SE') is associated with a mapping function used by the cube-based projection to determine sample locations for each of the projection surfaces (0, 1, 2, 3, 4, 5); wherein the mapping function is a parameterized mapping function used by the cube-based projection to adapt sample locations for each of the projection surfaces (0, 1, 2, 3, 4, 5), and the at least one syntax element (SE') analyzed from the bit stream (BS) comprises a plurality of syntax elements specifying coefficients of the parameterized mapping function; and where for each of the projection surfaces (0, 1, 2, 3, 4, 5) the plurality of syntax elements comprises: a first syntax element configured to specify a coefficient used in the parameterized mapping function of a first axis (u, v); characterized by that for each of the projection surfaces (0, 1, 2, 3, 4, 5) the majority of syntax elements further comprises: a second syntax element configured to indicate whether the parameterized mapping function of the first axis (u, v) refers to a position value of a second axis (v, u); where if the second syntax element is equal to 1, it indicates that the parameterized mapping function of the first axis (u, v) refers to a position value of the first axis (u, v) and to the position value of the second axis (v, u), and where if the second syntax element is equal to 0, it indicates that the parameterized mapping function of the first axis (u, v) refers to the position value of the first axis (u, v) and does not refer to the position value of the second axis (v, u). [9] A video decoding method according to claim 8, wherein for each of the projection surfaces (0, 1, 2, 3, 4, 5), the plurality of syntax elements further comprises: a third syntax element configured to specify a coefficient used in the parameterized mapping function of the second axis (v, u). [10] A video decoding method according to claim 9, wherein for each of the projection surfaces (0, 1, 2, 3, 4, 5), the plurality of syntax elements further comprises: a fourth syntax element configured to indicate whether the parameterized mapping function of the second axis (v, u) refers to a position value of the first axis (u, v). [11] A video decoding method according to claim 9, wherein the at least one syntax element (SE') analyzed from the bit stream (BS) further comprises: a fourth syntax element designed to specify that the parameterized mapping function is used by the cube-based projection. [12] A video decoding method according to claim 8, wherein an adapted sample location (f x (u, v), f y (u, v)) is obtained by applying the parameterized mapping function to a target sample location (u, v) on one of the projection surfaces, and the parameterized mapping function is expressed by: {fx(u, v)=u1+au(1−buv2)(1−u2)fy(u, v)=v1+av(1−bvu2)(1−v2), where coefficients a u , b u , a v and b v the parameterized mapping function is specified by the plurality of syntax elements analyzed from the bit stream (BS). [13] Video decoding method according to claim 8, wherein the at least one syntax element (SE') comprises: a syntax element designed to specify that the mapping function is used by the cube-based projection. [14] A video decoding method according to claim 13, wherein the mapping function specified by the syntax element is selected from a group consisting of a conventional cube mapping projection, an equiangular mapping function, and a parameterized mapping function.

Images