Encoding method, decoding method, encoding device, and decoding device

Abstract

In the present invention, among a plurality of vertices included in a frame to be encoded, the number of one or more second vertices is calculated, the second vertices excluding first vertices set as vertices for which information concerning the vertices is not encoded (step S18801). In encoding the frame to be encoded, the number of groups is determined, the groups serving as units for encoding information concerning the one or more second vertices (step S18802), and for each of the determined number of groups, first information concerning second vertices belonging to the group among the one or more second vertices is encoded (step S18803).

Description

Encoding method, decoding method, encoding device, and decoding device

[0001] The present disclosure relates to encoding methods and the like.

[0002] In US Pat. No. 6,299,549 a method and apparatus for encoding and decoding three-dimensional mesh data is proposed.

[0003] Japanese Patent Application Laid-Open No. 2006-187015

[0004] Further improvements are desired in the process of encoding or decoding motion vectors.The present disclosure aims to improve the process of encoding or decoding motion vectors.

[0005] An encoding method according to one aspect of the present invention is an encoding method that calculates the number of one or more second vertices among a plurality of vertices included in a frame to be encoded, excluding a first vertex that is set as a vertex for which information relating to that vertex is not encoded, determines the number of groups that are units in which information relating to the one or more second vertices is encoded in encoding the frame to be encoded, and encodes first information relating to second vertices that belong to that group among the one or more second vertices for each of the determined number of groups.

[0006] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, and a recording medium.

[0007] The present disclosure may contribute to improving the encoding or decoding process for motion vectors.

[0008] 1 is a conceptual diagram showing a three-dimensional mesh according to an embodiment. FIG. 2 is a conceptual diagram showing basic elements of a three-dimensional mesh according to an embodiment. FIG. 3 is a conceptual diagram showing mapping according to an embodiment. FIG. 4 is a block diagram showing a configuration example of an encoding / decoding system according to an embodiment. FIG. 5 is a block diagram showing a configuration example of an encoding device according to an embodiment. FIG. 6 is a block diagram showing another configuration example of an encoding device according to an embodiment. FIG. 7 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 8 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 9 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 10 is a conceptual diagram showing another configuration example of a bit stream according to an embodiment. FIG. 11 is a conceptual diagram showing yet another configuration example of a bit stream according to an embodiment. FIG. 12 is a block diagram showing a specific example of an encoding / decoding system according to an embodiment. FIG. 13 is a conceptual diagram showing an example configuration of point cloud data according to an embodiment. FIG. 14 is a conceptual diagram showing an example data file of point cloud data according to an embodiment. FIG. 15 is a conceptual diagram showing an example configuration of mesh data according to an embodiment. FIG. 16 is a conceptual diagram showing an example data file of mesh data according to an embodiment. FIG. 17 is a conceptual diagram showing types of three-dimensional data according to an embodiment. FIG. 18 is a block diagram showing an example configuration of a three-dimensional data encoder according to an embodiment. FIG. 19 is a block diagram showing an example configuration of a three-dimensional data decoder according to an embodiment. FIG. 19 is a block diagram showing another configuration example of a three-dimensional data encoder according to an embodiment. FIG. 19 is a block diagram showing another configuration example of a three-dimensional data decoder according to an embodiment. FIG. 1 is a conceptual diagram showing a specific example of encoding processing according to an embodiment. FIG. 2 is a conceptual diagram showing a specific example of decoding processing according to an embodiment. FIG. 3 is a block diagram showing an implementation example of an encoding device according to an embodiment. FIG. 4 is a block diagram showing an implementation example of a decoding device according to an embodiment. FIG. 5 is a block diagram showing a configuration example of an encoding / decoding system according to an embodiment. FIG. 6 is a block diagram showing another configuration example of an encoding device according to an embodiment. FIG. 7 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 8 is a block diagram showing yet another configuration example of an encoding device according to an embodiment. FIG. 9 is a block diagram showing yet another configuration example of a decoding device according to an embodiment. FIG. 10 is a block diagram showing a detailed configuration example of a decoding device according to an embodiment. FIG. 11 is an explanatory diagram showing coordinates of vertices in a three-dimensional mesh according to an embodiment.1 is an explanatory diagram showing prediction information according to an embodiment; an explanatory diagram showing an example of a mesh according to an embodiment; an explanatory diagram showing an example of dividing a mesh into sub-meshes according to an embodiment; an explanatory diagram showing a positional relationship of 3D points according to an embodiment; an explanatory diagram showing a distance between 3D points according to an embodiment; a flow diagram showing a process of selecting adjacent points according to an embodiment; an explanatory diagram showing a process of selecting adjacent points according to an embodiment; a diagram showing a first example of a reference destination of a motion group according to an embodiment; a diagram showing an example of syntax of a base mesh header according to an embodiment; a diagram showing a second example of a reference destination of a motion group according to an embodiment; a diagram showing a third example of a reference destination of a motion group according to an embodiment; a diagram showing a relationship between vertices constituting a mesh and motion groups according to an embodiment; a diagram showing a relationship between vertices constituting a sub-mesh and motion groups according to an embodiment; a flow diagram showing an example of basic encoding process according to an embodiment; a flow diagram showing an example of basic decoding process according to an embodiment; a diagram showing a method of generating LoD according to an embodiment; a diagram showing a method of generating LoD according to an embodiment; a diagram showing a method of generating a predicted value of a motion vector according to an embodiment; a diagram showing an example of syntax according to an embodiment; a diagram showing an example of syntax according to an embodiment. 1 is an explanatory diagram showing an example of calculation of a prediction residual in an embodiment. FIG. 2 is an explanatory diagram showing an example of calculation of a prediction residual in an embodiment. FIG. 3 is a flow diagram showing an example of processing of an encoding device in an embodiment. FIG. 4 is a flow diagram showing an example of processing of a decoding device in an embodiment. FIG. 5 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 6 is an explanatory diagram showing a method of generating a predicted value of a motion vector in an embodiment. FIG. 7 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 8 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 9 is an explanatory diagram showing an example of predicted value information for the X component of a motion vector in an embodiment. FIG. 10 is an explanatory diagram showing an example of predicted value information for the Y component of a motion vector in an embodiment. FIG. 11 is an explanatory diagram showing an example of predicted value information for the Z component of a motion vector in an embodiment.1 is an explanatory diagram showing an example of predicted value information for YZ components of a motion vector in an embodiment. FIG. 2 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 3 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 4 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 5 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 6 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 7 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 8 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 9 is an explanatory diagram showing an example of a prediction mode and binarized data in an embodiment. FIG. 1 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 2 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 3 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 4 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 5 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 6 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 7 is a block diagram showing an example of the configuration of an encoding unit in an embodiment. FIG. 8 is a block diagram showing an example of the configuration of a decoding unit in an embodiment. FIG. 9 is a flow diagram showing an example of processing by an encoding device in an embodiment to determine a prediction mode. FIG. 10 is an explanatory diagram showing an example of processing by an encoding device in an embodiment to calculate a maximum absolute difference value of a motion vector. FIG. 11 is a flow diagram showing an example of processing by a decoding device in an embodiment to determine a prediction mode. FIG. 12 is an explanatory diagram showing an example of syntax in an embodiment.1 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 2 is a flow diagram showing an example of encoding processing of a prediction mode value in an embodiment. FIG. 3 is a flow diagram showing an example of decoding processing of a prediction mode value in an embodiment. FIG. 4 is a flow diagram showing an example of processing of an encoding device in an embodiment. FIG. 5 is a flow diagram showing an example of processing of an encoding device in an embodiment. FIG. 6 is a flow diagram showing an example of processing of an encoding device in an embodiment. FIG. 7 is a flow diagram showing an example of processing of a decoding device in an embodiment. FIG. 8 is a flow diagram showing an example of processing of a encoding device in an embodiment. FIG. 9 is a flow diagram showing an example of processing of a decoding device in an embodiment. FIG. 10 is a flow diagram showing an example of processing of a encoding device in an embodiment. FIG. 11 is a flow diagram showing an example of processing of a decoding device in an embodiment. FIG. 12 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 13 is a flow diagram showing an example of encoding processing of a prediction mode value in an embodiment. FIG. 14 is a flow diagram showing an example of decoding processing of a prediction mode value in an embodiment. FIG. 15 is a flow diagram showing an example of processing of an encoding device in an embodiment. FIG. 16 is a flow diagram showing an example of processing of an encoding device in an embodiment. 1 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 1 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 2 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 3 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 4 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 5 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 6 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 7 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment.1 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 1 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 2 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 3 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 4 is a flow diagram showing an example of processing of a coding device in an embodiment. FIG. 5 is a flow diagram showing an example of processing of a coding device in an embodiment. FIG. 6 is a flow diagram showing an example of processing of a decoding device in an embodiment. FIG. 7 is a flow diagram showing an example of processing of a decoding device in an embodiment. FIG. 8 is a flow diagram showing an example of processing of assigning a predicted value in an embodiment. FIG. 9 is a flow diagram showing an example of processing of assigning a predicted value in an embodiment. FIG. 10 is an explanatory diagram showing a method of calculating a predicted value in an embodiment. FIG. 11 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 12 is an explanatory diagram showing a method of generating a predicted value of a motion vector in an embodiment. FIG. 13 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 14 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 1 is an explanatory diagram showing an example of a target point to be coded in an embodiment. FIG. 2 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 3 is an explanatory diagram showing an example of temporal mv in an embodiment. FIG. 4 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 5 is an explanatory diagram showing an example of a reference destination of a motion group in an embodiment. FIG. 6 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 7 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 8 is a flow diagram showing an example of an encoding process in an embodiment. FIG. 9 is a flow diagram showing an example of a decoding process in an embodiment. FIG. 10 is an explanatory diagram showing an example of predicted value information of a motion vector in an embodiment. FIG. 11 is a flow diagram showing an example of an encoding process in an embodiment. FIG. 12 is a flow diagram showing an example of a decoding process in an embodiment. FIG. 13 is a diagram showing an example of a reference destination of a motion group according to an embodiment.1 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 2 is an explanatory diagram showing an example of a frame to be encoded in an embodiment. FIG. 3 is an explanatory diagram showing an example of a reference frame in an embodiment. FIG. 4 is a flow diagram showing an example of processing by a decoding device in an embodiment. FIG. 5 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 6 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 7 is a diagram showing an example of a reference destination of a motion group in an embodiment. FIG. 8 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 9 is a flow diagram showing an example of processing by a decoding device in an embodiment. FIG. 10 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 11 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 12 is a flow diagram showing an example of encoding processing in an embodiment. FIG. 13 is a flow diagram showing an example of decoding processing in an embodiment. FIG. 14 is an explanatory diagram showing an example of frame reference in an embodiment. FIG. 15 is an explanatory diagram showing examples of a unique point list and a duplicate point list in an embodiment. FIG. 16 is an explanatory diagram showing an example of frame reference in an embodiment. FIG. 17 is a flow diagram showing an example of processing by an encoding device in an embodiment. FIG. 18 is a flow diagram showing an example of processing by a decoding device in an embodiment. FIG. 19 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 1 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 2 is a flow diagram showing an example of encoding processing in an embodiment. FIG. 3 is a flow diagram showing an example of decoding processing in an embodiment. FIG. 4 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 5 is a flow diagram showing an example of calculation processing of MotionCount in an embodiment. FIG. 6 is a flow diagram showing an example of calculation processing of MvFlag in an embodiment. FIG. 7 is a flow diagram showing an example of processing for a motion group in an embodiment. FIG. 8 is an explanatory diagram showing an example of syntax in an embodiment. FIG. 9 is a flow diagram showing an example of encoding processing of a motion vector in an embodiment. FIG. 10 is a flow diagram showing an example of decoding processing of a motion vector in an embodiment. FIG. 11 is a flow diagram showing an example of encoding processing in an embodiment.FIG. 10 is a flowchart illustrating an example of a decoding process according to an embodiment.

[0009] <Summary of the Disclosure> Three-dimensional (3D) meshes are used in computer graphics images, for example, which may be composed of multiple temporally distinct frames, each of which may be represented by a 3D mesh.

[0010] A 3D mesh is composed of vertex information indicating the positions of each of the vertices in 3D space, connectivity information indicating the connections between the vertices, and attribute information indicating the attributes of each vertex or face. Each face is constructed according to the connectivity between the vertices. Various computer graphics images can be expressed using such 3D meshes.

[0011] Furthermore, for transmission and storage of the 3D mesh, efficient encoding and decoding of the 3D mesh is expected. For efficient encoding and decoding of the 3D mesh, arithmetic coding and decoding may be used.

[0012] Further improvements are desired in the encoding or decoding process for three-dimensional data. The present disclosure aims to improve the encoding or decoding process for three-dimensional data.

[0013] Below, examples of inventions that can be obtained from the disclosure of this specification will be given, and the effects and the like that can be obtained from these inventions will be explained.

[0014] (1) An encoding method that calculates the number of one or more second vertices among a plurality of vertices included in a frame to be encoded, excluding a first vertex that is set as a vertex for which information relating to the vertex is not encoded, determines the number of groups that are units for encoding information relating to the one or more second vertices in encoding the frame to be encoded, and encodes first information relating to second vertices that belong to the group among the one or more second vertices for each of the determined number of groups.

[0015] According to the above aspect, an encoding device executing the encoding method can set multiple groups to which vertices (corresponding to second vertices) included in a frame to be encoded belong, excluding a vertex (corresponding to a first vertex) whose information is not coded, and encode information about the second vertex for each group. In other words, when encoding a frame to be encoded, the encoding device can avoid setting multiple groups to which all of the vertices (i.e., all of the vertices including the first vertex) belong. If the encoding device were to set multiple groups to which all of the vertices belong, the number of groups may be greater than if an appropriate number of groups to which the second vertex belongs, which could result in an increase in information processing volume and power consumption. By setting an appropriate number of groups to which the second vertex belongs, the encoding device contributes to suppressing an increase in information processing volume and power consumption. In this way, the above encoding method can improve the encoding process for motion vectors.

[0016] (2) The encoding method described in (1), wherein the first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not encoded, and when encoding the first information, for each of the determined number of groups, information regarding the motion vector of a second vertex belonging to the group among the one or more second vertices is encoded as the first information.

[0017] According to the above aspect, the encoding device sets an appropriate number of groups to which second vertices, excluding vertices set as vertices for which information related to the motion vectors of the second vertices, belong, from among a plurality of vertices included in the encoding target frame. This contributes to suppressing an increase in the amount of information processing and an increase in power consumption in the encoding process of the motion vectors of the second vertices. In this way, the above encoding method can improve the encoding process related to motion vectors.

[0018] (3) The encoding method described in (2), wherein the first vertex is a vertex among the plurality of vertices that has the same position information as a vertex for which information regarding the motion vector of the vertex has already been encoded, and is set as a vertex for which information regarding the motion vector of the vertex is not encoded.

[0019] According to the above aspect, the encoding device sets an appropriate number of groups to which second vertices, excluding vertices that have the same position information as a vertex (in other words, duplicate points) for which information about the motion vector of the vertex has already been encoded, belong, among a plurality of vertices included in a frame to be encoded, and that are set as vertices for which information about the motion vector of the vertex is not to be encoded, thereby contributing to suppressing increases in the amount of information processing and power consumption.In this way, the above encoding method can improve the encoding process for motion vectors.

[0020] (4) The encoding method described in (3), wherein when encoding the first information, information indicating the number of all vertices among the plurality of vertices that have the same position information as a vertex for which information regarding the motion vector of the vertex has already been encoded is further encoded.

[0021] According to the above aspect, the encoding device encodes information indicating the number of vertices included in a frame to be encoded that have the same position information as a vertex for which information related to the motion vector of the vertex has already been encoded, i.e., NumDupPnt. This allows the encoding device to transmit the information, i.e., NumDupPnt, to a decoding device, contributing to the decoding device appropriately decoding the multiple vertices. In this way, the above encoding method can improve the encoding process related to motion vectors.

[0022] (5) The encoding method described in (3), wherein when encoding the first information, for each vertex among the plurality of vertices that has the same position information as a vertex for which information regarding the motion vector of the vertex has already been encoded, information (MvSigFlag) indicating whether the vertex is to be encoded is further encoded.

[0023] According to the above aspect, the encoding device can transmit, to the decoding device, information indicating whether or not a vertex is to be encoded, i.e., MvSigFlag[v] (v is vertex identification information), for each vertex among multiple vertices included in the frame to be encoded that has the same position information as a vertex for which information related to the motion vector of the vertex has already been encoded, thereby contributing to the decoding device appropriately decoding the multiple vertices. In this way, the above encoding method can improve the encoding process related to motion vectors.

[0024] (6) The encoding method described in (1), wherein the first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not encoded, and when encoding the first information, for each of the determined number of groups, information regarding the skip mode of a second vertex belonging to the group among the one or more second vertices is encoded as the first information.

[0025] According to the above aspect, the encoding device sets an appropriate number of groups to which second vertices, excluding vertices set as vertices for which information regarding the motion vectors of the second vertices, belong, from among a plurality of vertices included in the encoding target frame. This contributes to suppressing an increase in the amount of information processing and an increase in power consumption in the encoding process of information regarding the skip mode of the second vertices. In this way, the above encoding method can improve the encoding process of motion vectors.

[0026] (7) A decoding method that calculates the number of one or more second vertices among a plurality of vertices included in a frame to be decoded, excluding a first vertex that is set as a vertex for which information relating to the vertex is not decoded, determines the number of groups that are units for decoding information relating to the one or more second vertices in decoding the frame to be decoded, and decodes, for each of the determined number of groups, first information relating to the second vertices that belong to the group among the one or more second vertices.

[0027] According to the above aspect, a decoding device executing the decoding method can set multiple groups to which vertices (corresponding to second vertices) included in a frame to be decoded belong, excluding a vertex (corresponding to a first vertex) whose information about the vertex is not decoded, and decode information about the second vertex for each group. In other words, when decoding a frame to be decoded, the decoding device can avoid setting multiple groups to which all of the vertices (i.e., all of the vertices including the first vertex) belong. If the decoding device were to set multiple groups to which all of the vertices belong, the number of groups may be greater than if an appropriate number of groups to which the second vertex belongs, which could result in an increase in information processing volume and power consumption. By setting an appropriate number of groups to which the second vertex belongs, the decoding device contributes to suppressing an increase in information processing volume and power consumption. In this way, the above decoding method can improve the decoding process for motion vectors.

[0028] (8) The decoding method described in (7), wherein the first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not decoded, and when decoding the first information, for each of the determined number of groups, information regarding the motion vector of a second vertex belonging to the group among the one or more second vertices is decoded as the first information.

[0029] According to the above aspect, the decoding device sets an appropriate number of groups to which second vertices, excluding vertices set as vertices for which information related to the motion vectors of the second vertices, belong, from among a plurality of vertices included in a frame to be decoded, are not decoded. This contributes to suppressing an increase in the amount of information processing and an increase in power consumption in the decoding process of the motion vectors of the second vertices. In this way, the above decoding method can improve the decoding process related to motion vectors.

[0030] (9) The decoding method described in (8), wherein the first vertex is a vertex among the plurality of vertices that has the same position information as a vertex for which information regarding the motion vector of the vertex has already been decoded, and is set as a vertex for which information regarding the motion vector of the vertex is not decoded.

[0031] According to the above aspect, the decoding device sets an appropriate number of groups to which second vertices, excluding vertices that have the same position information as a vertex (in other words, duplicate points) for which information about the motion vector of the vertex has already been decoded, belong, out of multiple vertices included in the frame to be decoded, but are set as vertices for which information about the motion vector of the vertex is not to be decoded, thereby contributing to suppressing increases in the amount of information processing and power consumption.In this way, the above decoding method can improve the decoding process related to motion vectors.

[0032] (10) A decoding method described in (9), in which, when decoding the first information, information indicating the number of all vertices among the plurality of vertices that have the same position information as a vertex for which information regarding the motion vector of the vertex has already been decoded is further decoded.

[0033] According to the above aspect, the decoding device decodes information indicating the number of all vertices, among the multiple vertices included in the frame to be decoded, that have the same position information as a vertex for which information on the motion vector of the vertex has already been decoded, i.e., NumDupPnt. This allows the decoding device to receive the information, i.e., NumDupPnt, from the encoding device, contributing to appropriate decoding of the multiple vertices. In this way, the above decoding method can improve the decoding process related to motion vectors.

[0034] (11) A decoding method described in (9), in which, when decoding the first information, information indicating whether or not the vertex is to be decoded is further decoded for each vertex among the plurality of vertices that has the same position information as a vertex for which information regarding the motion vector of the vertex has already been decoded.

[0035] According to the above aspect, the decoding device can receive, from the encoding device, information indicating whether or not a vertex is to be decoded, that is, MvSigFlag[v] (v is vertex identification information), for each vertex among multiple vertices included in a frame to be decoded that has the same position information as a vertex for which information regarding the motion vector of the vertex has already been decoded, thereby contributing to appropriate decoding of multiple vertices. In this way, the above decoding method can improve the decoding process regarding motion vectors.

[0036] (12) The first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not decoded, and when decoding the first information, for each of the determined number of groups, information regarding the skip mode of a second vertex belonging to the group among the one or more second vertices is decoded as the first information.

[0037] According to the above aspect, the decoding device sets an appropriate number of groups to which second vertices, excluding vertices set as vertices for which information regarding the motion vectors of the second vertices, belong, from among a plurality of vertices included in a frame to be decoded, are not decoded. This contributes to suppressing an increase in the amount of information processing and an increase in power consumption in the decoding process of information regarding the skip mode of the second vertex. In this way, the above decoding method can improve the decoding process of motion vectors.

[0038] (13) An encoding device comprising a memory and a circuit that can access the memory, wherein the circuit, in operation, calculates the number of one or more second vertices among a plurality of vertices included in a frame to be encoded, excluding a first vertex that is set as a vertex for which information relating to the vertex is not encoded, determines the number of groups that are units for encoding information relating to the one or more second vertices in encoding the frame to be encoded, and encodes, for each of the determined number of groups, first information relating to the second vertices that belong to the group among the one or more second vertices.

[0039] According to the above aspect, the same effects as those of the above encoding method are achieved.

[0040] (14) A decoding device comprising a memory and a circuit that can access the memory, wherein the circuit, in operation, calculates the number of one or more second vertices among a plurality of vertices included in a frame to be decoded, excluding a first vertex that is set as a vertex for which information relating to the vertex is not decoded, determines the number of groups that are units for decoding information relating to the one or more second vertices in decoding the frame to be decoded, and decodes, for each of the determined number of groups, first information relating to the second vertices that belong to the group among the one or more second vertices.

[0041] According to the above aspect, the same effects as those of the above decoding method are achieved.

[0042] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, or a recording medium.

[0043] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0044] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0045] (Embodiment) In this embodiment, an encoding method, a decoding method, etc. will be described.

[0046] <Expressions and Terms> The following expressions and terms are used herein.

[0047] (1) Three-dimensional Mesh A three-dimensional mesh is a collection of multiple faces, and represents, for example, a three-dimensional object. A three-dimensional mesh is mainly composed of vertex information, connectivity information, and attribute information. A three-dimensional mesh may be expressed as a polygon mesh or a mesh. A three-dimensional mesh may also vary over time. A three-dimensional mesh may include metadata related to the vertex information, connectivity information, and attribute information, and may also include other additional information.

[0048] (2) Vertex Information Vertex information is information indicating a vertex. For example, the vertex information indicates the position of a vertex in a three-dimensional space. Furthermore, a vertex corresponds to a vertex of a face that constitutes a three-dimensional mesh. Vertex information may be expressed as "geometry." Furthermore, vertex information may be expressed as position information.

[0049] (3) Connection Information Connection information is information that indicates connections between vertices. For example, connection information indicates connections for forming faces or edges of a three-dimensional mesh. Connection information may be expressed as "Connectivity." Connection information may also be expressed as face information.

[0050] (4) Attribute Information Attribute information is information that indicates attributes of a vertex or a face. For example, attribute information indicates attributes such as a color, an image, and a normal vector associated with a vertex or a face. Attribute information may be expressed as "texture."

[0051] (5) Faces A face is an element that constitutes a three-dimensional mesh. Specifically, a face is a polygon on a plane in three-dimensional space. For example, a face can be defined as a triangle in three-dimensional space.

[0052] (6) Plane A plane is a two-dimensional plane in a three-dimensional space. For example, a polygon is formed on a plane, and multiple polygons are formed on multiple planes.

[0053] (7) Bitstream: A bitstream corresponds to coded information. A bitstream may also be referred to as a stream, a coded bitstream, a compressed bitstream, or a coded signal.

[0054] (8) Encoding and Decoding The term encoding may be substituted with terms such as storing, including, writing, describing, signaling, sending, notifying, saving, or compressing, and these terms may be interchangeable. For example, encoding information may mean including the information in a bitstream. Also, encoding information into a bitstream may mean encoding the information to generate a bitstream that includes the encoded information.

[0055] Additionally, the term "decode" may be replaced with terms such as "read," "decode," "read," "load," "derive," "obtain," "receive," "extract," "reconstruct," "reconstruct," "decompress," or "decompress," and these terms may be interchangeable. For example, decoding information may mean obtaining information from a bitstream. Decoding information from a bitstream may mean decoding the bitstream to obtain information contained in the bitstream.

[0056] (9) Ordinal Numbers In the description, ordinal numbers such as first and second may be assigned to components, etc. These ordinal numbers may be changed as appropriate. Furthermore, new ordinal numbers may be assigned to components, etc., or removed. Furthermore, these ordinal numbers may be assigned to elements in order to identify them, and may not correspond to a meaningful order.

[0057] <Three-dimensional mesh> Fig. 1 is a conceptual diagram showing a three-dimensional mesh according to this embodiment. A three-dimensional mesh is composed of multiple faces. For example, each face is a triangle. The vertices of these triangles are defined in three-dimensional space. The three-dimensional mesh then represents a three-dimensional object. Each face may have a color or an image.

[0058] FIG. 2 is a conceptual diagram showing the basic elements of a three-dimensional mesh according to this embodiment. A three-dimensional mesh is composed of vertex information, connection information, and attribute information. The vertex information indicates the positions of the vertices of a face in three-dimensional space. The connection information indicates the connections between the vertices. A face can be identified by the vertex information and connection information. In other words, a colorless three-dimensional object is formed in three-dimensional space by the vertex information and connection information.

[0059] The attribute information may be associated with a vertex or a face. The attribute information associated with a vertex may be expressed as "Attribute Per Point." The attribute information associated with a vertex may indicate an attribute of the vertex itself, or may indicate an attribute of a face connected to the vertex.

[0060] For example, a color may be associated with a vertex as attribute information. The color associated with a vertex may be the color of the vertex itself, or the color of a face connected to the vertex. The color of a face may be the average of multiple colors associated with multiple vertices of the face. Furthermore, a normal vector may be associated with a vertex or a face as attribute information. Such a normal vector can represent the front and back of a face.

[0061] A two-dimensional image may be associated with a surface as attribute information. The two-dimensional image associated with a surface is also expressed as a texture image or an "Attribute Map." Information indicating mapping between the surface and the two-dimensional image may be associated with the surface as attribute information. Information indicating such mapping may be expressed as mapping information, vertex information of a texture image, texture coordinates, or "Attribute UV Coordinate."

[0062] Furthermore, information such as color, image, and moving image used as attribute information may be expressed as "parametric space."

[0063] The attribute information allows texture to be reflected on the three-dimensional object. That is, a three-dimensional object having color is formed in three-dimensional space based on the vertex information, connection information, and attribute information.

[0064] In the above, the attribute information is associated with the vertices or faces, but it may also be associated with the edges.

[0065] 3 is a conceptual diagram illustrating mapping according to this embodiment. For example, a region of a two-dimensional image on a two-dimensional plane can be mapped onto a surface of a three-dimensional mesh in three-dimensional space. Specifically, coordinate information of the region in the two-dimensional image is associated with the surface of the three-dimensional mesh. As a result, an image of the mapped region in the two-dimensional image is reflected on the surface of the three-dimensional mesh.

[0066] By using the mapping, the 2D image used as attribute information can be separated from the 3D mesh. For example, in encoding the 3D mesh, the 2D image may be encoded by an image encoding method or a video encoding method.

[0067] <System Configuration> Fig. 4 is a block diagram showing an example of the configuration of a coding / decoding system according to this embodiment. In Fig. 4, the coding / decoding system includes a coding device 100 and a decoding device 200.

[0068] For example, the encoding device 100 obtains a three-dimensional mesh and encodes the three-dimensional mesh into a bitstream. Then, the encoding device 100 outputs the bitstream to the network 300. For example, the bitstream includes the encoded three-dimensional mesh and control information for decoding the encoded three-dimensional mesh. By encoding the three-dimensional mesh, information about the three-dimensional mesh is compressed.

[0069] The network 300 transmits a bitstream from the encoding device 100 to the decoding device 200. The network 300 may be the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 300 is not necessarily limited to bidirectional communication, and may be a unidirectional communication network for terrestrial digital broadcasting, satellite broadcasting, or the like.

[0070] Furthermore, the network 300 can be replaced by a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).

[0071] The decoding device 200 obtains a bitstream and decodes a three-dimensional mesh from the bitstream. By decoding the three-dimensional mesh, information about the three-dimensional mesh is expanded. For example, the decoding device 200 decodes the three-dimensional mesh according to a decoding method corresponding to the encoding method used by the encoding device 100 to encode the three-dimensional mesh. That is, the encoding device 100 and the decoding device 200 perform encoding and decoding according to encoding methods and decoding methods that correspond to each other.

[0072] The 3D mesh before encoding may also be referred to as an original 3D mesh, and the 3D mesh after decoding may also be referred to as a reconstructed 3D mesh.

[0073] 5 is a block diagram showing an example of the configuration of a coding device 100 according to this embodiment. For example, the coding device 100 includes a vertex information encoder 101, a connection information encoder 102, and an attribute information encoder 103.

[0074] The vertex information encoder 101 is an electrical circuit that encodes vertex information. For example, the vertex information encoder 101 encodes the vertex information into a bitstream according to a format defined for the vertex information.

[0075] The connection information encoder 102 is an electrical circuit that encodes the connection information, for example, the connection information encoder 102 encodes the connection information into a bitstream according to a format defined for the connection information.

[0076] The attribute information encoder 103 is an electric circuit that encodes the attribute information. For example, the attribute information encoder 103 encodes the attribute information into a bit stream in accordance with a format defined for the attribute information.

[0077] The vertex information, connectivity information, and attribute information may be coded using variable-length coding or fixed-length coding, such as Huffman coding or context-adaptive binary arithmetic coding (CABAC).

[0078] The vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be integrated together, or each of the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be further subdivided into multiple components.

[0079] 6 is a block diagram showing another example of the configuration of the encoding device 100 according to this embodiment. For example, the encoding device 100 includes a pre-processor 104 and a post-processor 105 in addition to the configuration shown in FIG.

[0080] The preprocessor 104 is an electrical circuit that performs processing before encoding the vertex information, connectivity information, and attribute information. For example, the preprocessor 104 may perform a conversion process, a separation process, a multiplexing process, or the like on the 3D mesh before encoding. More specifically, for example, the preprocessor 104 may separate the vertex information, connectivity information, and attribute information from the 3D mesh before encoding.

[0081] The post-processor 105 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are encoded. For example, the post-processor 105 may perform conversion processing, separation processing, multiplexing processing, or the like on the encoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 105 may multiplex the encoded vertex information, connection information, and attribute information into a bitstream. Furthermore, for example, the post-processor 105 may further perform variable-length coding on the encoded vertex information, connection information, and attribute information.

[0082] 7 is a block diagram showing an example of the configuration of a decoding device 200 according to this embodiment. For example, the decoding device 200 includes a vertex information decoder 201, a connection information decoder 202, and an attribute information decoder 203.

[0083] The vertex information decoder 201 is an electrical circuit that decodes vertex information. For example, the vertex information decoder 201 decodes vertex information from a bitstream according to a format defined for the vertex information.

[0084] The connection information decoder 202 is an electrical circuit that decodes the connection information, for example, the connection information decoder 202 decodes the connection information from the bitstream according to a format defined for the connection information.

[0085] The attribute information decoder 203 is an electric circuit that decodes the attribute information. For example, the attribute information decoder 203 decodes the attribute information from the bitstream in accordance with a format defined for the attribute information.

[0086] The vertex information, connection information, and attribute information may be decoded using variable length decoding or fixed length decoding, which may correspond to Huffman coding, context-adaptive binary arithmetic coding (CABAC), or the like.

[0087] The vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be integrated together, or each of the vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be further subdivided into multiple components.

[0088] 8 is a block diagram showing another example of the configuration of the decoding device 200 according to this embodiment. For example, the decoding device 200 includes a pre-processor 204 and a post-processor 205 in addition to the configuration shown in FIG.

[0089] The preprocessor 204 is an electrical circuit that performs processing before decoding the vertex information, connection information, and attribute information. For example, the preprocessor 204 may perform conversion processing, separation processing, multiplexing processing, or the like on the bitstream before decoding the vertex information, connection information, and attribute information.

[0090] More specifically, for example, the preprocessor 204 may separate a sub-bitstream corresponding to vertex information, a sub-bitstream corresponding to connectivity information, and a sub-bitstream corresponding to attribute information from the bitstream. Also, for example, the preprocessor 204 may perform variable-length decoding on the bitstream in advance before decoding the vertex information, connectivity information, and attribute information.

[0091] The post-processor 205 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are decoded. For example, the post-processor 205 may perform conversion processing, separation processing, multiplexing processing, or the like on the decoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 205 may multiplex the decoded vertex information, connection information, and attribute information onto a three-dimensional mesh.

[0092] <Bitstream> Vertex information, connection information, and attribute information are coded and stored in a bitstream. The relationship between this information and the bitstream is shown below.

[0093] 9 is a conceptual diagram showing an example of the configuration of a bitstream according to this embodiment. In this example, connection information, vertex information, and attribute information are integrated in the bitstream. For example, the connection information, vertex information, and attribute information may be included in a single file.

[0094] Furthermore, multiple portions of this information may be stored sequentially, such as a first portion of connection information, a first portion of vertex information, a first portion of attribute information, a second portion of connection information, a second portion of vertex information, a second portion of attribute information, etc. These multiple portions may correspond to multiple portions that are different in time, multiple portions that are different in space, or multiple different faces.

[0095] Furthermore, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0096] 10 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, a plurality of files are included in the bitstream, and connection information, vertex information, and attribute information are stored in different files. Here, a file containing connection information, a file containing vertex information, and a file containing attribute information are shown, but the storage format is not limited to this example. For example, two types of information among the connection information, vertex information, and attribute information may be included in one file, and the remaining type of information may be included in another file.

[0097] Alternatively, the information may be split and stored in more files. For example, multiple pieces of connectivity information may be stored in multiple files, multiple pieces of vertex information may be stored in multiple files, or multiple pieces of attribute information may be stored in multiple files. These multiple pieces may correspond to multiple temporally different pieces, multiple spatially different pieces, or multiple different faces.

[0098] Furthermore, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0099] 11 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, the bitstream is composed of multiple separable sub-bitstreams, and connection information, vertex information, and attribute information are stored in different sub-bitstreams.

[0100] Here, a sub-bitstream containing connection information, a sub-bitstream containing vertex information, and a sub-bitstream containing attribute information are shown, but the storage format is not limited to this example.

[0101] For example, two types of information among the connection information, vertex information, and attribute information may be included in one sub-bitstream, and the remaining type of information may be included in another sub-bitstream. Specifically, attribute information of a two-dimensional image or the like may be stored in a sub-bitstream that complies with an image coding method, separate from the sub-bitstreams of the connection information and vertex information.

[0102] Also, each sub-bitstream may include multiple files, and multiple pieces of connectivity information may be stored in multiple files, multiple pieces of vertex information may be stored in multiple files, or multiple pieces of attribute information may be stored in multiple files.

[0103] 9, 10, and 11, and a storage order different from the above examples may be used. For example, the vertex information, connection information, and attribute information may be stored in the bitstream in this order. Alternatively, the connection information, connection information, and attribute information may be stored in the bitstream in any of the following orders: connection information, attribute information, and vertex information; vertex information, attribute information, and connection information; attribute information, connection information, and vertex information; or attribute information, vertex information, and connection information.

[0104] Furthermore, each of the connection information, vertex information, and attribute information may be divided into a plurality of data, and the plurality of data may be stored in a cyclical or random order within the bitstream.

[0105] 12 is a block diagram showing a specific example of an encoding / decoding system according to this embodiment. In FIG. 12, the encoding / decoding system includes a three-dimensional data encoding system 110, a three-dimensional data decoding system 210, and an external connector 310.

[0106] The three-dimensional data encoding system 110 includes a controller 111, an input / output processor 112, a three-dimensional data encoder 113, a three-dimensional data generator 115, and a system multiplexer 114. The three-dimensional data decoding system 210 includes a controller 211, an input / output processor 212, a three-dimensional data decoder 213, a system demultiplexer 214, a presenter 215, and a user interface 216.

[0107] In the three-dimensional data encoding system 110, sensor data is input from a sensor terminal to a three-dimensional data generator 115. The three-dimensional data generator 115 generates three-dimensional data, such as point cloud data or mesh data, from the sensor data and inputs it to a three-dimensional data encoder 113.

[0108] For example, the three-dimensional data generator 115 generates vertex information, and generates connection information and attribute information corresponding to the vertex information. The three-dimensional data generator 115 may process the vertex information when generating the connection information and attribute information. For example, the three-dimensional data generator 115 may reduce the amount of data by deleting duplicate vertices, or may transform the vertex information (such as by shifting its position, rotating it, or normalizing it). The three-dimensional data generator 115 may also render the attribute information.

[0109] Furthermore, although the three-dimensional data generator 115 is a component of the three-dimensional data encoding system 110 in FIG. 12, it may be arranged externally and independently of the three-dimensional data encoding system 110.

[0110] The sensor terminal that provides the sensor data for generating the three-dimensional data may be, for example, a moving body such as an automobile, a flying object such as an airplane, a mobile terminal, a camera, etc. Furthermore, a distance sensor such as a LIDAR, a millimeter wave radar, an infrared sensor, or a range finder, a stereo camera, or a combination of multiple monocular cameras may also be used as the sensor terminal.

[0111] The sensor data may be the distance (position) of the object, monocular camera images, stereo camera images, color, reflectance, sensor attitude, orientation, gyro, sensing position (GPS information or altitude), speed, acceleration, sensing time, temperature, air pressure, humidity, or magnetism.

[0112] The three-dimensional data encoder 113 corresponds to the encoding device 100 shown in FIG. 5 and other figures. For example, the three-dimensional data encoder 113 encodes three-dimensional data to generate encoded data. The three-dimensional data encoder 113 also generates control information when encoding the three-dimensional data. The three-dimensional data encoder 113 then inputs the encoded data together with the control information to the system multiplexer 114.

[0113] The encoding method for the three-dimensional data may be an encoding method using geometry or an encoding method using a video codec. Here, the encoding method using geometry may also be referred to as a geometry-based encoding method. The encoding method using a video codec may also be referred to as a video-based encoding method.

[0114] The system multiplexer 114 multiplexes the encoded data and control information input from the 3D data encoder 113 to generate multiplexed data using a specified multiplexing method. The system multiplexer 114 may multiplex other media such as video, audio, subtitles, application data, or document files, or reference time information, along with the encoded data and control information of the 3D data. Furthermore, the system multiplexer 114 may multiplex attribute information related to the sensor data or the 3D data.

[0115] For example, the multiplexed data may have a file format for storage or a packet format for transmission. As these formats, ISOBMFF or a format based on ISOBMFF may be used. Also, MPEG-DASH, MMT, MPEG-2 TS Systems, RTP, or the like may be used.

[0116] The multiplexed data is then output as a transmission signal to the external connector 310 by the input / output processor 112. The multiplexed data may be transmitted as a transmission signal by wire or wirelessly. Alternatively, the multiplexed data is stored in an internal memory or a storage device. The multiplexed data may be transmitted to a cloud server via the Internet or may be stored in an external storage device.

[0117] For example, the transmission or storage of the multiplexed data is performed by a method according to the medium for transmission or storage, such as broadcasting or communication. The communication protocol may be http, ftp, TCP, UDP, IP, or a combination thereof. Furthermore, a pull-type communication method or a push-type communication method may be used.

[0118] For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), coaxial cable, etc. may be used. For wireless transmission, 3GPP (registered trademark), 3G / 4G / 5G defined by IEEE, wireless LAN, Wi-Fi, Bluetooth, or millimeter wave may be used. For broadcasting, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 may be used.

[0119] The sensor data may be input to the three-dimensional data generator 115 or the system multiplexer 114. The three-dimensional data or encoded data may be output as a transmission signal directly to the external connector 310 via the input / output processor 112. The transmission signal output from the three-dimensional data encoding system 110 is input to the three-dimensional data decoding system 210 via the external connector 310.

[0120] Furthermore, each operation of the three-dimensional data encoding system 110 may be controlled by a controller 111 that executes an application program.

[0121] In the three-dimensional data decoding system 210, a transmission signal is input to an input / output processor 212. The input / output processor 212 decodes multiplexed data having a file format or a packet format from the transmission signal and inputs the multiplexed data to a system demultiplexer 214. The system demultiplexer 214 obtains coded data and control information from the multiplexed data and inputs them to a three-dimensional data decoder 213. The system demultiplexer 214 may extract other media or reference time information from the multiplexed data.

[0122] The three-dimensional data decoder 213 corresponds to the decoding device 200 shown in Fig. 7 etc. For example, the three-dimensional data decoder 213 decodes three-dimensional data from the encoded data based on a predefined encoding method. The three-dimensional data is then presented to the user by the presenter 215.

[0123] Additionally, additional information such as sensor data may be input to the presenter 215. The presenter 215 may present three-dimensional data based on the additional information. Additionally, a user instruction may be input from a user terminal to the user interface 216. Then, the presenter 215 may present three-dimensional data based on the input instruction.

[0124] The input / output processor 212 may acquire the three-dimensional data and the encoded data from the external connector 310 .

[0125] Furthermore, each operation of the three-dimensional data decoding system 210 may be controlled by a controller 211 that executes an application program.

[0126] 13 is a conceptual diagram showing an example of the configuration of point cloud data according to this embodiment. The point cloud data is data of a group of points representing a three-dimensional object.

[0127] Specifically, a point cloud is made up of a plurality of points, and has position information indicating the three-dimensional coordinate position of each point and attribute information indicating the attribute of each point. The position information is also expressed as geometry.

[0128] The type of attribute information may be, for example, color, reflectance, etc. One point may be associated with attribute information of one type, one point may be associated with attribute information of multiple different types, or one point may be associated with attribute information having multiple values ​​for the same type.

[0129] 14 is a conceptual diagram showing an example of a data file of point cloud data according to this embodiment. This example shows a case where there is a one-to-one correspondence between position information items and attribute information items, and shows position information and attribute information for N points that make up the point cloud data. In this example, the position information is information indicating a three-dimensional coordinate position using three axes, x, y, and z, and the attribute information is information indicating a color using RGB. A PLY file or the like can be used as a representative data file for point cloud data.

[0130] 15 is a conceptual diagram showing an example of the configuration of mesh data according to this embodiment. Mesh data is data used in CG (Computer Graphics) and the like, and is three-dimensional mesh data that shows the three-dimensional shape of an object using multiple surfaces. Each surface is also expressed as a polygon, and has a polygonal shape such as a triangle or a rectangle.

[0131] Specifically, a 3D mesh is composed of a plurality of points constituting a point cloud, as well as a plurality of edges and a plurality of faces. Each point is also expressed as a vertex or a position. Each edge corresponds to a line segment connected by two vertices. Each face corresponds to an area surrounded by three or more edges.

[0132] Furthermore, a three-dimensional mesh has position information indicating the three-dimensional coordinate positions of vertices. The position information is also expressed as vertex information or geometry. A three-dimensional mesh also has connection information indicating the relationship between multiple vertices that make up an edge or a face. The connection information is also expressed as connectivity. A three-dimensional mesh also has attribute information indicating the attributes of the vertices, edges, or faces. The attribute information in a three-dimensional mesh is also expressed as texture.

[0133] For example, the attribute information may indicate the color, reflectance, or normal vector for a vertex, edge, or face. The direction of the normal vector may represent the front and back of the face.

[0134] The mesh data may be stored in a data file format such as an object file.

[0135] 16 is a conceptual diagram showing an example of a data file of mesh data according to this embodiment. In this example, the data file includes position information G(1) to G(N) of N vertices that make up the three-dimensional mesh, and attribute information A1(1) to A1(N) of the N vertices. Also, in this example, M pieces of attribute information A2(1) to A2(M) are included. The attribute information items do not need to correspond one-to-one to vertices or faces. Furthermore, attribute information need not exist.

[0136] The connection information is represented by a combination of vertex indices. n[1, 3, 4] indicates a triangular face formed by three vertices, n=1, n=3, and n=4. Also, m[2, 4, 6] indicates that the attribute information of m=2, m=4, and m=6 corresponds to the three vertices, respectively.

[0137] Furthermore, the actual contents of the attribute information may be written in a separate file. A pointer to that content may be associated with a vertex, a face, or the like. For example, attribute information indicating an image for a face may be stored in a two-dimensional attribute map file. The file name of the attribute map and two-dimensional coordinate values ​​in the attribute map may be written in attribute information A2(1) to A2(M). The method of specifying attribute information for a face is not limited to these methods, and any method may be used.

[0138] 17 is a conceptual diagram showing types of three-dimensional data according to this embodiment. Point cloud data and mesh data may represent static objects or dynamic objects. A static object is an object that does not change over time, and a dynamic object is an object that changes over time. A static object may correspond to three-dimensional data for any point in time.

[0139] For example, point cloud data for a given point in time may be referred to as a PCC frame, mesh data for a given point in time may be referred to as a mesh frame, and PCC frames and mesh frames may be simply referred to as frames.

[0140] The area of ​​the object may be limited to a certain range, as in normal video data, or may not be limited, as in map data. The density of points or surfaces may be determined in various ways. Sparse point cloud data or sparse mesh data may be used, or dense point cloud data or dense mesh data may be used.

[0141] Next, encoding and decoding of a point cloud or a three-dimensional mesh will be described. The device, process, or syntax for encoding and decoding vertex information of a three-dimensional mesh in the present disclosure may be applied to encoding and decoding of a point cloud. The device, process, or syntax for encoding and decoding of a point cloud in the present disclosure may be applied to encoding and decoding vertex information of a three-dimensional mesh.

[0142] Furthermore, a device, process, or syntax for encoding and decoding attribute information of a point cloud in the present disclosure may be applied to encoding and decoding connectivity information or attribute information of a three-dimensional mesh.Furthermore, a device, process, or syntax for encoding and decoding connectivity information or attribute information of a three-dimensional mesh in the present disclosure may be applied to encoding and decoding attribute information of a point cloud.

[0143] Furthermore, at least some of the processing may be shared between the encoding and decoding of point cloud data and the encoding and decoding of mesh data, thereby reducing the scale of the circuit and software program.

[0144] 18 is a block diagram showing an example configuration of a three-dimensional data encoder 113 according to this embodiment. In this example, the three-dimensional data encoder 113 includes a vertex information encoder 121, an attribute information encoder 122, a metadata encoder 123, and a multiplexer 124. The vertex information encoder 121, the attribute information encoder 122, and the multiplexer 124 may correspond to the vertex information encoder 101, the attribute information encoder 103, the post-processor 105, etc. in FIG.

[0145] In this example, the three-dimensional data encoder 113 encodes the three-dimensional data according to a geometry-based encoding method, which takes into account the three-dimensional structure. In addition, in the geometry-based encoding method, attribute information is encoded using configuration information obtained in encoding the vertex information.

[0146] Specifically, first, vertex information, attribute information, and metadata included in three-dimensional data generated from sensor data are input to a vertex information encoder 121, an attribute information encoder 122, and a metadata encoder 123, respectively. Here, connectivity information included in the three-dimensional data may be treated in the same way as attribute information. In addition, in the case of point cloud data, position information may be treated as vertex information.

[0147] The vertex information encoder 121 encodes the vertex information into compressed vertex information and outputs the compressed vertex information as encoded data to the multiplexer 124. The vertex information encoder 121 also generates metadata for the compressed vertex information and outputs it to the multiplexer 124. The vertex information encoder 121 also generates configuration information and outputs it to the attribute information encoder 122.

[0148] The attribute information encoder 122 uses the configuration information generated by the vertex information encoder 121 to encode the attribute information into compressed attribute information and outputs the compressed attribute information as encoded data to the multiplexer 124. The attribute information encoder 122 also generates metadata of the compressed attribute information and outputs it to the multiplexer 124.

[0149] The metadata encoder 123 encodes compressible metadata into compressed metadata and outputs the compressed metadata as encoded data to the multiplexer 124. The metadata encoded by the metadata encoder 123 may be used to encode vertex information and attribute information.

[0150] The multiplexer 124 multiplexes the compressed vertex information, the compressed vertex information metadata, the compressed attribute information, the compressed attribute information metadata, and the compressed metadata into a bitstream, and then inputs the bitstream to the system layer.

[0151] 19 is a block diagram showing an example configuration of a three-dimensional data decoder 213 according to this embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information decoder 221, an attribute information decoder 222, a metadata decoder 223, and a demultiplexer 224. The vertex information decoder 221, the attribute information decoder 222, and the demultiplexer 224 may correspond to the vertex information decoder 201, the attribute information decoder 203, the preprocessor 204, and the like in FIG.

[0152] In this example, the three-dimensional data decoder 213 decodes three-dimensional data according to a geometry-based encoding method. The three-dimensional structure is taken into consideration in the decoding according to the geometry-based encoding method. Furthermore, in the decoding according to the geometry-based encoding method, attribute information is decoded using configuration information obtained in decoding vertex information.

[0153] Specifically, first, a bitstream is input from the system layer to a demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information and compressed vertex information metadata are input to a vertex information decoder 221. The compressed attribute information and compressed attribute information metadata are input to an attribute information decoder 222. The metadata is input to a metadata decoder 223.

[0154] The vertex information decoder 221 decodes vertex information from the compressed vertex information using metadata of the compressed vertex information. The vertex information decoder 221 also generates configuration information and outputs it to the attribute information decoder 222. The attribute information decoder 222 decodes attribute information from the compressed attribute information using the configuration information generated by the vertex information decoder 221 and the metadata of the compressed attribute information. The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 may be used to decode the vertex information and the attribute information.

[0155] Thereafter, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that, for example, this metadata is metadata of the vertex information and attribute information, and can be used in an application program.

[0156] 20 is a block diagram showing another example configuration of the three-dimensional data encoder 113 according to the present embodiment. In this example, the three-dimensional data encoder 113 includes a vertex image generator 131, an attribute image generator 132, a metadata generator 133, a video encoder 134, a metadata encoder 123, and a multiplexer 124. The vertex image generator 131, the attribute image generator 132, and the video encoder 134 may correspond to the vertex information encoder 101 and the attribute information encoder 103 in FIG. 6 , etc.

[0157] In this example, the 3D data encoder 113 encodes the 3D data according to a video-based encoding method. In encoding according to the video-based encoding method, multiple 2D images are generated from the 3D data, and the multiple 2D images are encoded according to a video encoding method. Here, the video encoding method may be High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), or the like.

[0158] Specifically, first, vertex information and attribute information included in three-dimensional data generated from sensor data are input to a metadata generator 133. The vertex information and attribute information are then input to a vertex image generator 131 and an attribute image generator 132, respectively. The metadata included in the three-dimensional data is then input to a metadata encoder 123. Here, connectivity information included in the three-dimensional data may be treated in the same way as attribute information. In the case of point cloud data, position information may be treated as vertex information.

[0159] The metadata generator 133 generates map information of a plurality of two-dimensional images from the vertex information and attribute information, and inputs the map information to the vertex image generator 131, the attribute image generator 132, and the metadata encoder 123.

[0160] The vertex image generator 131 generates a vertex image based on the vertex information and map information, and inputs the generated image to the video encoder 134. The attribute image generator 132 generates an attribute image based on the attribute information and map information, and inputs the generated image to the video encoder 134.

[0161] The video encoder 134 encodes the vertex images and attribute images into compressed vertex information and compressed attribute information, respectively, in accordance with a video encoding method, and outputs the compressed vertex information and compressed attribute information as encoded data to the multiplexer 124. The video encoder 134 also generates metadata for the compressed vertex information and metadata for the compressed attribute information, and outputs them to the multiplexer 124.

[0162] The metadata encoder 123 encodes the compressible metadata into compressed metadata and outputs the compressed metadata as encoded data to the multiplexer 124. The compressible metadata includes map information. The metadata encoded by the metadata encoder 123 may also be used to encode vertex information and attribute information.

[0163] The multiplexer 124 multiplexes the compressed vertex information, the compressed vertex information metadata, the compressed attribute information, the compressed attribute information metadata, and the compressed metadata into a bitstream, and then inputs the bitstream to the system layer.

[0164] 21 is a block diagram showing another example configuration of the 3D data decoder 213 according to this embodiment. In this example, the 3D data decoder 213 includes a vertex information generator 231, an attribute information generator 232, a video decoder 234, a metadata decoder 223, and a demultiplexer 224. The vertex information generator 231, the attribute information generator 232, and the video decoder 234 may correspond to the vertex information decoder 201 and the attribute information decoder 203 in FIG. 8, etc.

[0165] In this example, the 3D data decoder 213 decodes the 3D data according to a video-based coding method. In the decoding according to the video-based coding method, a plurality of 2D images are decoded according to a video coding method, and 3D data is generated from the plurality of 2D images. Here, the video coding method may be High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), or the like.

[0166] Specifically, first, a bitstream is input from the system layer to the demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information, compressed vertex information metadata, compressed attribute information, and compressed attribute information metadata are input to the video decoder 234. The compressed metadata is input to the metadata decoder 223.

[0167] The video decoder 234 decodes the vertex images in accordance with the video encoding method. At this time, the video decoder 234 decodes the vertex images from the compressed vertex information using the metadata of the compressed vertex information. Then, the video decoder 234 inputs the vertex images to the vertex information generator 231. The video decoder 234 also decodes the attribute images in accordance with the video encoding method. At this time, the video decoder 234 decodes the attribute images from the compressed attribute information using the metadata of the compressed attribute information. Then, the video decoder 234 inputs the attribute images to the attribute information generator 232.

[0168] The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 includes map information used to generate vertex information and attribute information. The metadata decoded by the metadata decoder 223 may also be used to decode vertex images and attribute images.

[0169] The vertex information generator 231 reproduces vertex information from the vertex image in accordance with the map information included in the metadata decoded by the metadata decoder 223. The attribute information generator 232 reproduces attribute information from the attribute image in accordance with the map information included in the metadata decoded by the metadata decoder 223.

[0170] Thereafter, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that, for example, this metadata is metadata of the vertex information and attribute information, and can be used in an application program.

[0171] Fig. 22 is a conceptual diagram showing a specific example of encoding processing according to this embodiment. Fig. 22 shows a three-dimensional data encoder 113 and a description encoder 148. In this example, the three-dimensional data encoder 113 includes a two-dimensional data encoder 141 and a mesh data encoder 142. The two-dimensional data encoder 141 includes a texture encoder 143. The mesh data encoder 142 includes a vertex information encoder 144 and a connection information encoder 145.

[0172] The vertex information encoder 144, the connection information encoder 145, and the texture encoder 143 may correspond to the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 in FIG.

[0173] For example, the two-dimensional data encoder 141 operates as a texture encoder 143 and generates a texture file by encoding the texture corresponding to the attribute information as two-dimensional data according to an image encoding method or a video encoding method.

[0174] The mesh data encoder 142 also operates as a vertex information encoder 144 and a connectivity information encoder 145, and generates a mesh file by encoding the vertex information and connectivity information. The mesh data encoder 142 may further encode mapping information for textures. The encoded mapping information may then be included in the mesh file.

[0175] The description encoder 148 also generates a description file by encoding a description corresponding to metadata such as text data. The description encoder 148 may encode the description at the system layer. For example, the description encoder 148 may be included in the system multiplexer 114 of FIG. 12 .

[0176] The above operations generate a bitstream containing texture files, mesh files, and description files, which may be multiplexed into the bitstream in file formats such as glTF (Graphics Language Transmission Format) or USD (Universal Scene Description).

[0177] The three-dimensional data encoder 113 may include two mesh data encoders as the mesh data encoder 142. For example, one mesh data encoder encodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data encoder encodes vertex information and connectivity information of a dynamic three-dimensional mesh.

[0178] Correspondingly, two mesh files may then be included in the bitstream: for example, one mesh file corresponding to a static 3D mesh and another mesh file corresponding to a dynamic 3D mesh.

[0179] Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe coded using interprediction. Furthermore, information on the dynamic three-dimensional mesh may be differential information between vertex information or connectivity information of the three-dimensional mesh of an intraframe and vertex information or connectivity information of the three-dimensional mesh of an interframe.

[0180] Fig. 23 is a conceptual diagram showing a specific example of the decoding process according to this embodiment. Fig. 23 shows a three-dimensional data decoder 213, a description decoder 248, and a renderer 247. In this example, the three-dimensional data decoder 213 includes a two-dimensional data decoder 241, a mesh data decoder 242, and a mesh reconstructor 246. The two-dimensional data decoder 241 includes a texture decoder 243. The mesh data decoder 242 includes a vertex information decoder 244 and a connectivity information decoder 245.

[0181] The vertex information decoder 244, the connection information decoder 245, the texture decoder 243, and the mesh reconstructor 246 may correspond to the vertex information decoder 201, the connection information decoder 202, the attribute information decoder 203, and the post-processor 205 in Fig. 8. The presenter 247 may correspond to the presenter 215 in Fig. 12.

[0182] For example, the two-dimensional data decoder 241 operates as a texture decoder 243, and decodes the texture corresponding to the attribute information from the texture file as two-dimensional data in accordance with an image coding method or a video coding method.

[0183] The mesh data decoder 242 also operates as a vertex information decoder 244 and a connectivity information decoder 245 to decode vertex information and connectivity information from the mesh file. The mesh data decoder 242 may further decode mapping information for textures from the mesh file.

[0184] The description decoder 248 also decodes descriptions corresponding to metadata such as text data from the description file. The description decoder 248 may decode the descriptions at the system layer. For example, the description decoder 248 may be included in the system demultiplexer 214 of FIG. 12 .

[0185] The mesh reconstructor 246 reconstructs a 3D mesh from the vertex information, connectivity information, and textures according to the description. The renderer 247 renders and outputs the 3D mesh according to the description.

[0186] Through the above operations, a 3D mesh is reconstructed and output from a bitstream containing a texture file, a mesh file, and a description file.

[0187] The three-dimensional data decoder 213 may include two mesh data decoders as the mesh data decoder 242. For example, one mesh data decoder decodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data decoder decodes vertex information and connectivity information of a dynamic three-dimensional mesh.

[0188] Correspondingly, two mesh files may then be included in the bitstream: for example, one mesh file corresponding to a static 3D mesh and another mesh file corresponding to a dynamic 3D mesh.

[0189] Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe coded using interprediction. Furthermore, information on the dynamic three-dimensional mesh may be differential information between vertex information or connectivity information of the three-dimensional mesh of an intraframe and vertex information or connectivity information of the three-dimensional mesh of an interframe.

[0190] A dynamic 3D mesh coding method is sometimes called DMC (Dynamic Mesh Coding), and a video-based dynamic 3D mesh coding method is sometimes called V-DMC (Video-based Dynamic Mesh Coding).

[0191] The point cloud encoding method is sometimes called PCC (Point Cloud Compression). The point cloud video-based encoding method is sometimes called V-PCC (Video-based Point Cloud Compression). The point cloud geometry-based encoding method is sometimes called G-PCC (Geometry-based Point Cloud Compression).

[0192] <Implementation Example> Fig. 24 is a block diagram showing an implementation example of the encoding device 100 according to this embodiment. The encoding device 100 includes a circuit 151 and a memory 152. For example, multiple components of the encoding device 100 shown in Fig. 5 etc. are implemented by the circuit 151 and memory 152 shown in Fig. 24.

[0193] The circuit 151 is a circuit that performs information processing and is a circuit that can access the memory 152. For example, the circuit 151 is a dedicated or general-purpose electric circuit that encodes a three-dimensional mesh. The circuit 151 may be a processor such as a CPU. Alternatively, the circuit 151 may be a collection of multiple electric circuits.

[0194] The memory 152 is a dedicated or general-purpose memory that stores information used by the circuit 151 to encode the three-dimensional mesh. The memory 152 may be an electric circuit and may be connected to the circuit 151. The memory 152 may also be included in the circuit 151. The memory 152 may also be a collection of multiple electric circuits. The memory 152 may also be a magnetic disk, an optical disk, or the like, and may also be expressed as a storage, a recording medium, or the like. The memory 152 may also be a non-volatile memory or a volatile memory.

[0195] For example, the memory 152 may store a three-dimensional mesh or a bitstream, or may store a program for the circuit 151 to encode the three-dimensional mesh.

[0196] Note that the encoding device 100 does not necessarily have to implement all of the components shown in Figure 5 and the like, and does not necessarily have to perform all of the processes shown here. Some of the components shown in Figure 5 and the like may be included in another device, and some of the processes shown here may be executed by another device. Furthermore, the encoding device 100 may implement any combination of the components of the present disclosure, and may perform any combination of the processes of the present disclosure.

[0197] Fig. 25 is a block diagram showing an example implementation of a decoding device 200 according to this embodiment. The decoding device 200 includes a circuit 251 and a memory 252. For example, multiple components of the decoding device 200 shown in Fig. 7 and other figures are implemented by the circuit 251 and memory 252 shown in Fig. 25.

[0198] The circuit 251 is a circuit that performs information processing and is a circuit that can access the memory 252. For example, the circuit 251 is a dedicated or general-purpose electric circuit that decodes a three-dimensional mesh. The circuit 251 may be a processor such as a CPU. Alternatively, the circuit 251 may be a collection of multiple electric circuits.

[0199] The memory 252 is a dedicated or general-purpose memory that stores information for the circuit 251 to decode the 3D mesh. The memory 252 may be an electric circuit and may be connected to the circuit 251. The memory 252 may also be included in the circuit 251. The memory 252 may also be a collection of multiple electric circuits. The memory 252 may also be a magnetic disk, an optical disk, or the like, and may also be expressed as a storage, a recording medium, or the like. The memory 252 may also be a non-volatile memory or a volatile memory.

[0200] For example, the memory 252 may store a three-dimensional mesh or a bitstream, or may store a program for the circuit 251 to decode the three-dimensional mesh.

[0201] Note that the decoding device 200 does not necessarily have to implement all of the components shown in Figure 7 and the like, and does not necessarily have to perform all of the processes shown here. Some of the components shown in Figure 7 and the like may be included in another device, and some of the processes shown here may be executed by another device. Furthermore, the decoding device 200 may implement any combination of the components of the present disclosure, and may perform any combination of the processes of the present disclosure.

[0202] The encoding method and the decoding method including the steps performed by each component of the encoding device 100 and the decoding device 200 of the present disclosure may be executed by any device or system. For example, part or all of the encoding method and the decoding method may be executed by a computer including a processor, a memory, an input / output circuit, etc. In this case, the encoding method and the decoding method may be executed by the computer executing a program for causing the computer to execute the encoding method and the decoding method.

[0203] Alternatively, the program or the bitstream may be recorded on a non-transitory computer-readable recording medium such as a CD-ROM.

[0204] An example of a program may be a bitstream. For example, a bitstream including an encoded three-dimensional mesh includes syntax elements for causing the decoding device 200 to decode the three-dimensional mesh. The bitstream then causes the decoding device 200 to decode the three-dimensional mesh according to the syntax elements included in the bitstream. Thus, the bitstream may play a role similar to that of a program.

[0205] The bitstream may be an encoded bitstream containing the encoded 3D mesh, or may be a multiplexed bitstream containing the encoded 3D mesh and other information.

[0206] Furthermore, each component of the encoding device 100 and the decoding device 200 may be configured with dedicated hardware, general-purpose hardware that executes the above-mentioned programs, or a combination of these. The general-purpose hardware may be configured with a memory in which the programs are recorded and a general-purpose processor that reads and executes the programs from the memory. Here, the memory may be a semiconductor memory or a hard disk, and the general-purpose processor may be a CPU.

[0207] Furthermore, the dedicated hardware may be configured with a memory, a dedicated processor, etc. For example, the dedicated processor may execute the encoding method and the decoding method by referring to a memory for recording data.

[0208] Furthermore, as described above, each component of the encoding device 100 and the decoding device 200 may be an electric circuit. These electric circuits may form a single electric circuit as a whole, or each may be a separate electric circuit. Furthermore, these electric circuits may correspond to dedicated hardware, or may correspond to general-purpose hardware that executes the above-mentioned programs, etc. Furthermore, the encoding device 100 and the decoding device 200 may be implemented as an integrated circuit.

[0209] The encoding device 100 may also be a transmitting device that transmits a three-dimensional mesh, and the decoding device 200 may also be a receiving device that receives a three-dimensional mesh.

[0210] Inter and Intra Prediction: A typical 3D model digitally represents an object so that a user can explore the 3D model by zooming, panning, and / or rotating in all three dimensions while rendering it temporally. One way to construct such a representation is to build a 3D mesh using triangles. In a 3D model, the positions of the triangle vertices, their connectivity to each other, and their associated attributes (such as normals and UV patches) are stored. Storing all this information in an uncompressed format requires significant storage space. Therefore, transmitting this information requires significant bandwidth. The triangles that form the 3D mesh often have repeating patterns and similar attributes, especially in temporal and spatial neighborhoods. These repetitions can be used to develop efficient encoding and decoding methods for storage and transmission.

[0211] FIG. 26 is a block diagram showing an example of the configuration of a coding / decoding system according to this embodiment.

[0212] The encoding / decoding system includes an encoding device 100 and a decoding device 200. The encoding / decoding system accepts a 3D mesh frame input in the form of 3D coordinates of vertices, connectivity information, and associated attributes. The encoding device 100 is responsible for encoding all of this associated information into a bitstream (compressed bitstream). The bitstream may consist of multiple bitstreams. The bitstream is transmitted to the decoding device 200 via a transmission channel. The decoding device 200 decodes the bitstream and generates a 3D model (3D mesh frame) using the decoded 3D coordinates of vertices, the decoded connectivity information, and the decoded associated attributes.

[0213] FIG. 27 is a block diagram showing another example configuration of the encoding device 100 according to this embodiment.

[0214] In this example, the encoding device 100 includes a preprocessor 521 and an encoding processor 522 .

[0215] The preprocessor 521 reads the input 3D mesh frame, processes the 3D mesh frame, extracts a base mesh, displacement information, and an attribute map, and outputs them to the encoding processor 522. An example of the displacement information is a displacement vector.

[0216] The encoding processor 522 compresses the base mesh, displacement information, and attribute map separately and combines them to generate a bitstream.

[0217] FIG. 28 is a block diagram showing another example of the configuration of the decoding device 200 according to this embodiment.

[0218] In this example, the decoding device 200 includes a decoding processor 622 and a post-processor 623 .

[0219] The decoding processor 622 reads the bitstream, separates the coded base mesh, the coded displacement information, and the coded attribute map from the read bitstream, decodes them individually, and outputs them to the post-processor 623. An example of the displacement information is a displacement vector.

[0220] The post-processor 623 processes the base mesh using the displacement information and the attribute map to generate a 3D mesh frame. The generated 3D mesh frame is displayed on a display, for example, by being output to the display. By repeating this process, the 3D mesh frame is repeatedly displayed on the display.

[0221] FIG. 29 is a block diagram showing yet another example configuration of the encoding device 100 according to this embodiment.

[0222] In this example, the encoding device 100 comprises a volumetric capturer 511, a projector 512, a base mesh encoder 513, a displacement encoder 514, an attribute encoder 515, and optionally one or more other type encoders 516.

[0223] The volumetric capturer 511 captures content and outputs the captured content to the projector 512 .

[0224] The projector 512 projects the content onto a 3D mesh frame containing vertex geometry coordinates, texture coordinates, and connectivity data. The data is output to a base mesh encoder 513, a displacement encoder 514, an attribute encoder 515, and optionally one or more other type encoders 516. Each encoder compresses the data into a bitstream.

[0225] FIG. 30 is a block diagram showing yet another example configuration of the decoding device 200 according to this embodiment.

[0226] In this example, the decoding device 200 comprises a base mesh decoder 613 , a displacement decoder 614 , an attribute decoder 615 , one or more other type decoders 616 , and a 3D reconstructor 617 .

[0227] The bitstream is sent to a base mesh decoder 613, a displacement decoder 614, an attribute decoder 615, and optionally one or more other type decoders 616. These decoders decode the bitstream to generate decoded data including vertex geometry coordinates, texture coordinates, and connectivity data. The decoded data is then sent to a 3D reconstructor 617, which reconstructs a 3D mesh frame.

[0228] Fig. 31 is a block diagram showing a detailed configuration example of the decoding device 200 according to this embodiment. Specifically, Fig. 31 shows an example of the configuration of a geometry coordinate decoder included in the decoding device 200.

[0229] In this example, the decoding device 200 comprises a frame header decoder 631 , a vertex geometry coordinate predictor 632 , a vertex geometry coordinate difference decoder 633 , and a reconstructor 634 .

[0230] The frame header decoder 631 reads the bitstream and decodes the frame headers in the bitstream to determine whether the frame data is to be intra-decoded (intra-predicted) or inter-decoded (inter-predicted).

[0231] If inter-decoding is selected, the frame data contained in the bitstream is output to a vertex geometry coordinate predictor 632 .

[0232] The vertex geometry coordinate predictor 632 outputs prediction information to the reconstructor 634. An example of prediction information is a motion vector.

[0233] The reconstructor 634 uses prediction information along with vertex coordinates from previously decoded frames to output the three-dimensional coordinates of the vertices (vertex geometry coordinates).

[0234] On the other hand, if intra-decoding is selected, the frame data included in the bitstream is output to the vertex geometry coordinate difference decoder 633 .

[0235] The vertex geometry coordinate differential decoder 633 decodes frame data encoded as differences between the coordinates of the vertices contained in the frame to generate vertex coordinates. Only one of the vertex geometry coordinates from the vertex geometry coordinate differential decoder 633 and the reconstructor 634 is used to generate the decoded 3D mesh frame.

[0236] 32 is an explanatory diagram showing the coordinates of vertices in a 3D mesh according to this embodiment. Specifically, FIG. 32 shows an example in which the entire 3D mesh frame is decoded using the coordinates (positions) of the actual vertices included in the bitstream.

[0237] The coordinates of vertex A included in the 3D mesh frame at time (t) are decoded as (6, 8, 9) using the Cartesian coordinate system (x, y, z) as shown in Figure 32(a). Similarly, the coordinates of vertex B are decoded as (10, 6, 7), and the coordinates of vertex C are decoded as (14, 8, 9). The same is true for vertices D to G.

[0238] Fig. 33 is an explanatory diagram showing prediction information according to this embodiment. Specifically, Fig. 33 shows another example in which the entire 3D mesh frame at time (t) is decoded using a frame (past frame) at time (t-1) and prediction information included in the bitstream.

[0239] The coordinates (6, 8, 9) of vertex A in the frame to be decoded (current frame) are decoded by adding the coordinates (4, 7, 8) of vertex A in the past frame to the value (2, 1, 1) for vertex A indicated by the prediction information. Similarly, the coordinates (10, 6, 7) of vertex B in the current frame are decoded by adding the coordinates (8, 6, 7) of vertex B in the past frame to the value (2, 0, 0) for vertex B indicated by the prediction information.

[0240] One way to encode a 3D mesh frame is to divide the original 3D mesh (original mesh) into several smaller meshes (sub-meshes) so that each sub-mesh can be coded independently. The vertices of the 3D mesh frame are divided so that the coordinates and connectivity information of the vertices within each partition can be coded independently. Each divided mesh is called a sub-mesh.

[0241] Fig. 34 is an explanatory diagram showing an example of a mesh (original mesh) according to this embodiment. Fig. 35 is an explanatory diagram showing an example of division of a mesh into sub-meshes according to this embodiment. Specifically, Fig. 35 is a diagram showing how the mesh shown in Fig. 34 is divided into two sub-meshes.

[0242] Here, vertices A, B, and C of the original mesh are duplicated to vertices A1, B1, and C1, and vertices A2, B2, and C2, and two submeshes (the first submesh and the second submesh) that can be encoded and decoded independently are created (generated). The first submesh and the second submesh are meshes that can be decoded independently.

[0243] In this way, a mesh can be divided into a plurality of smaller parts and coded for each part. When dividing the mesh, the vertices of the mesh are divided in such a way that the coordinates and connectivity information of the vertices in each part can be coded independently.

[0244] The mesh shown in FIG. 34 is an original mesh, which is sometimes called a full mesh in contrast to a sub-mesh.

[0245] <Calculation of Adjacent Points> Next, a method for encoding and decoding the prediction information output from the vertex geometry coordinate predictor 632 shown in FIG. 31 will be described in detail.

[0246] In the following description, an example will be given in which the prediction information is the motion vectors of the vertices (in other words, three-dimensional points) included in the base mesh.

[0247] The prediction information is not necessarily limited to a motion vector, but may be other information possessed by a three-dimensional point, such as position information (geometry) or attribute information (attribute) possessed by a three-dimensional point.

[0248] Here, the position information includes, for example, coordinates (x coordinate, y coordinate, z coordinate) based on a certain point. Also, the attribute information includes, for example, color information (RGB or YUV, etc.), reflectance, normal vector, etc. of each three-dimensional point. Note that the attribute information may be information expressed as a vector, and for example, a motion vector may be an example of the attribute information.

[0249] Furthermore, when the prediction information output by the vertex geometry coordinate predictor 632 is a motion vector, the vertex geometry coordinate predictor 632 may be referred to as a motion decoder.

[0250] In the following description, the motion vector (specifically, the value of the motion vector) is an integer value. For example, if the motion vector has 8-bit precision, the motion vector takes an integer value from 0 to 255, and if the motion vector has 10-bit precision, the motion vector takes an integer value from 0 to 1023.

[0251] If the bit precision of the motion vector is decimal precision, the motion vector may be multiplied by a scale value to obtain an integer value, and then rounded to an integer value.

[0252] This scale value may be added to the bitstream, such as in a header.

[0253] FIG. 36 is an explanatory diagram showing the positional relationship of three-dimensional points according to this embodiment.

[0254] A possible method for encoding a motion vector of a three-dimensional point is to calculate a predicted value of the motion vector of a certain three-dimensional point and encode the difference (prediction residual) between the value of the original motion vector and the predicted value. For example, if the value of the motion vector of a certain three-dimensional point p is Ap and the predicted value is Pp, the encoding device 100 encodes the absolute difference value Diffp = |Ap - Pp|, which indicates the absolute value of the difference. In this case, if the predicted value Pp can be generated with high accuracy, the value of the absolute difference value Diffp will be small. Therefore, for example, the encoding device 100 can reduce the amount of code by performing entropy encoding using a coding table in which the smaller the value, the smaller the number of generated bits.

[0255] One possible method for the encoding device 100 to generate a predicted value of a motion vector is to use the motion vector of another 3D point surrounding the 3D point to be encoded. Here, 3D points surrounding the 3D point refer to another 3D point that is within a predetermined distance (within a predetermined range) from the 3D point. For example, if there are 3D points p = (x1, y1, z1) and 3D points q = (x2, y2, z2) that are the 3D points to be encoded, and if the Euclidean distance d(p, q) = √((x1-y1)2 + (x2-y2)2 + (x3-y3)2) between the 3D points p and q is smaller than a certain threshold THd, the encoding device 100 determines that the position of the 3D point q is close to the position of the 3D point p, and determines to use the value of the motion vector of the 3D point q to generate a predicted value of the motion vector of the 3D point p.

[0256] The distance calculation method may be another method, such as Mahalanobis distance.

[0257] The predetermined distance may be determined arbitrarily and is not particularly limited.

[0258] Furthermore, for example, the encoding device 100 may determine that a 3D point that is farther away from the 3D point to be encoded than a predetermined distance (outside a predetermined range) is not to be used for prediction. For example, if a 3D point r exists and the distance d(p, r) between the 3D point p and the 3D point r is equal to or greater than a threshold THd, the encoding device 100 may determine that the 3D point r is not to be used for prediction.

[0259] The encoding device 100 may add the value of the threshold THd to the header of the bitstream.

[0260] For example, when encoding the motion vector of a three-dimensional point to be encoded using a predicted value, the encoding device 100 uses motion vectors of surrounding three-dimensional points to be used to generate the predicted value, and uses motion vectors that have already been encoded or motion vectors that have already been decoded.

[0261] In addition, when decoding the motion vector of the three-dimensional point to be decoded using a predicted value, the decoding device 200 uses motion vectors that have already been decoded when using the motion vectors of surrounding three-dimensional points to generate the predicted value.

[0262] This allows the same predicted value to be generated during encoding and decoding, thereby enabling the decoding device 200 to correctly decode the bit stream of 3D points generated by the encoding device 100.

[0263] Although it has been described that the points surrounding a 3D point refer to other 3D points within a predetermined range from the 3D point, this is not necessarily limited to this. For example, in the case of 3D point D (i.e., vertex D) shown in FIG. 33, the surrounding 3D points include 3D points A, B, C, E, F, and G. However, the surrounding 3D points (in other words, adjacent points) may be selected according to one or more of the following conditions A and B. In other words, adjacent points are points selected according to certain conditions and are points that are referenced to predict the information of the 3D point to be encoded. An adjacent point may also be referred to as a reference 3D point, a reference point, or a reference vertex.

[0264] Condition A: A 3D point that has connectivity with the target 3D point. Condition B: A 3D point that has been encoded or decoded before the target 3D point.

[0265] For example, when 3D points that satisfy the above conditions A and B are selected as adjacent points, and the 3D points are encoded or decoded in the order of 3D point A, 3D point B, 3D point C, 3D point D, 3D point E, 3D point F, and 3D point G, 3D point A and 3D point C may be selected as adjacent points of 3D point D. Because 3D point A and 3D point C have connectivity with 3D point D, it is highly likely that their motion vectors are also close in value. Furthermore, because 3D point A and 3D point C have been encoded or decoded before 3D point D, the motion vectors of 3D point A and 3D point C can be used to calculate a predicted value of the motion vector of 3D point D.

[0266] This makes it possible to improve the accuracy of the predicted value of the motion vector of the three-dimensional point D, thereby improving the coding efficiency.

[0267] In addition to the above conditions A and B, the number of neighboring points of a three-dimensional point may be limited to a predetermined value (NumNeiCnt) or less. For example, by setting NumNeiCnt = 3, the number of neighboring points of a three-dimensional point may be limited to 3 or less.

[0268] This makes it possible to reduce the memory capacity required to store information about adjacent points of a three-dimensional point, and also to reduce the amount of processing required to predict (calculate) a motion vector.

[0269] The predetermined value may be determined arbitrarily and is not particularly limited.

[0270] Furthermore, for example, the encoding device 100 may add the above-mentioned predetermined value, in other words, NumNeiCnt indicating the maximum number of adjacent points, to the header of the data unit and encode it to add it to the bitstream.

[0271] As a result, the decoding device 200 can properly decode a bitstream in which the maximum number of adjacent points is limited to NumNeiCnt or less by decoding the header of the bitstream.

[0272] In addition, when there are more three-dimensional points satisfying the above conditions A and B than NumNeiCnt as adjacent points, the adjacent points may be selected in order of proximity to the three-dimensional point to be encoded or decoded. For example, when NumNeiCnt = 3, there are five three-dimensional points satisfying the above conditions A and B as adjacent points of three-dimensional point D, namely, three-dimensional point A, three-dimensional point C, three-dimensional point H, three-dimensional point I, and three-dimensional point J, and when the distance to three-dimensional point D is closest in order of three-dimensional point A, three-dimensional point C, three-dimensional point H, three-dimensional point I, and three-dimensional point J, three-dimensional point A, three-dimensional point C, and three-dimensional point H may be selected as adjacent points of three-dimensional point D. Because three-dimensional point A, three-dimensional point C, and three-dimensional point H have connectivity with three-dimensional point D and are close to three-dimensional point D, the values ​​of their respective motion vectors are also likely to be close to the value of the motion vector of three-dimensional point D. Furthermore, 3D point A, 3D point C, and 3D point H have been coded or decoded before 3D point D. Therefore, the motion vectors of 3D point A, 3D point C, and 3D point H can be used to calculate a predicted value of the motion vector of 3D point D.

[0273] This makes it possible to improve the accuracy of the predicted value of the motion vector of the 3D point D. Furthermore, by limiting the number of adjacent points, it is possible to reduce the memory capacity for storing information on adjacent points of the 3D point and also reduce the amount of processing required when calculating (predicting) the motion vector.

[0274] Note that when using connectivity with a target 3D point as condition A when selecting adjacent points of a 3D point to be encoded or decoded (hereinafter also referred to as a target 3D point), it is not limited to using connectivity within the frame to be encoded or decoded (hereinafter also referred to as a target frame). For example, connectivity within a frame that has already been encoded or decoded may be used. For example, in the example shown in FIG. 33, when adjacent points of each 3D point (each target 3D point) in the frame at time (t) (current frame) are selected using the above condition A, the corresponding three-dimensional connectivity within the frame at time (t-1) (past frame) may be used. More specifically, when selecting adjacent points of 3D point D in the current frame using the above condition A, the encoding device 100 and the decoding device 200 may select 3D points A, C, and G with reference to the connectivity of 3D point D in the past frame, and select 3D points A and C that have already been encoded or decoded as adjacent points. In frames that have been coded or decoded before the target frame, such as past frames, the coding device 100 and the decoding device 200 can calculate the connectivity and distance between three-dimensional points, and can therefore appropriately calculate adjacent points of the target three-dimensional point using the above condition A or the distance (distance information) between three-dimensional points.

[0275] In this embodiment, an example has been shown in which a past frame is used as the frame before the target frame, but this is not necessarily limited to this, and any frame that has already been coded or decoded can be used.

[0276] This allows the encoding device 100 and the decoding device 200 to appropriately calculate the neighboring points of the target 3D point using connectivity and / or distance.

[0277] Note that this embodiment may be applied when there is correspondence between 3D points in a target frame and 3D points in frames that have already been encoded or decoded. For example, in the example shown in Fig. 33, there is correspondence between 3D points A, B, C, D, E, F, and G in the current frame and the previous frame, so neighboring points of the 3D points in the current frame may be calculated using the connectivity and / or distance in the previous frame shown in this embodiment.

[0278] In addition, if there is no correspondence between a three-dimensional point in the target frame and a three-dimensional point in a frame that has already been encoded or decoded, the encoding device 100 and the decoding device 200 may calculate (select) adjacent points using the connectivity of the three-dimensional points in the target frame without using distance.

[0279] This allows calculation of adjacent points even when there is no correspondence between each of the three-dimensional points in the coded or decoded frame.

[0280] In addition, the encoding device 100 may add information to the bitstream indicating whether or not there is a correspondence between each of the three-dimensional points in the frame to be encoded and each of the three-dimensional points in frames that have already been encoded or decoded.

[0281] This allows the decoding device 200 to know whether or not there is correspondence between 3D points in a frame to be coded (i.e., a frame coded by the coding device 100 and to be decoded by the decoding device 200) and 3D points in frames that have already been decoded. The decoding device 200 can switch the method of calculating adjacent points so that, for example, if there is correspondence between the 3D points, it calculates adjacent points of the 3D points in the frame to be coded using the connectivity and / or distance in the already decoded frame, and if there is no correspondence between the 3D points, it calculates adjacent points using the connectivity of the 3D points in the frame to be coded without using the distance.

[0282] In addition, when decoding, if the decoding device 200 is unable to calculate the distance between the three-dimensional point to be decoded in the frame to be decoded and an adjacent point before decoding the position information of the three-dimensional point to be decoded, it may calculate the adjacent point of the three-dimensional point to be decoded using the distance between three-dimensional points corresponding to the three-dimensional point to be decoded in a frame that has already been decoded.

[0283] FIG. 37 is an explanatory diagram showing the distance between three-dimensional points according to this embodiment.

[0284] For example, in the example shown in FIG. 37 , the distance between each 3D point in the current frame at time (t) and its adjacent point may be the distance between the corresponding 3D point in the past frame at time (t−1) and its adjacent point. More specifically, the distance between 3D point D in the current frame and adjacent point A, adjacent point C, and adjacent point G may be the distance between 3D point D in the past frame and adjacent point A, adjacent point C, and adjacent point G. Since the decoding device 200 can reliably calculate the distance between 3D points in frames that have been decoded before the frame to be decoded, such as past frames, it can use the distance to appropriately calculate the adjacent points of the 3D point to be decoded.

[0285] In this embodiment, an example has been shown in which a past frame is used as a frame preceding the frame to be decoded, but this is not necessarily limited to this, and any frame that has already been decoded may be used.

[0286] This allows the decoding device 200 to appropriately calculate adjacent points that are close to the 3D point to be decoded, thereby enabling the decoding device 200 to accurately calculate (predict) the 3D motion vector to be decoded, thereby improving coding efficiency.

[0287] In addition, when the decoding device 200 calculates, during decoding, the adjacent points of the three-dimensional point to be decoded using the distance between three-dimensional points corresponding to the three-dimensional point to be decoded within a frame that has already been decoded, the encoding device 100 may also calculate, during encoding, the adjacent points of the three-dimensional point to be encoded using the distance between three-dimensional points corresponding to the three-dimensional point to be encoded within a frame that has already been encoded.

[0288] This allows the method of calculating adjacent points to be the same during encoding and decoding, allowing the decoding device 200 to appropriately decode the bit stream generated by encoding.

[0289] In addition, the encoding device 100 and the decoding device 200 may also calculate the connectivity of a target 3D point using the connectivity between 3D points corresponding to the target 3D point within a frame that has already been encoded or decoded.

[0290] This allows the connectivity and distance to be calculated simultaneously using information on frames that have already been coded or decoded, thereby reducing the amount of processing.

[0291] In addition, the encoding device 100 and the decoding device 200 may select appropriate neighboring points by using connectivity within the target frame and the distance between 3D points corresponding to the target 3D point in an already encoded or decoded frame.

[0292] This allows the encoding device 100 and the decoding device 200 to calculate adjacent points that have connectivity within the target frame and are close to each other using information within frames that have already been encoded or decoded, thereby enabling the motion vector of the target 3D point to be calculated (predicted) with high accuracy and improving encoding efficiency.

[0293] When the adjacent points are calculated without using the distance, the adjacent points may be calculated using the connectivity within the target frame, which can reduce the amount of processing.

[0294] Furthermore, when distance is not used and the number of neighboring points is limited by NumNeiCnt, the encoding device 100 and the decoding device 200 may stop calculating neighboring points when the number of neighboring points reaches NumNeiCnt while adding neighboring points of the target 3D point. This reduces the amount of processing.

[0295] Furthermore, when the number of neighboring points reaches NumNeiCnt while adding neighboring points of the target 3D point, the encoding device 100 and the decoding device 200 may replace at least one of the neighboring points already stored as neighboring points with a newly found neighboring point in subsequent processing. This makes it possible to improve encoding efficiency while limiting the number of neighboring points.

[0296] Fig. 38 is a flow diagram showing the adjacent point selection process according to this embodiment. The flow shown in Fig. 38 is a specific example of a processing procedure executed by the encoding device 100 and the decoding device 200 when calculating adjacent points of a target 3D point.

[0297] First, the encoding device 100 and the decoding device 200 select, from among a plurality of 3D points included in a target frame, 3D points that have connectivity with a target 3D point as first adjacent point candidates (S101).

[0298] Next, the encoding device 100 and the decoding device 200 select, as a second adjacent point candidate, a 3D point that has been coded or decoded before the target 3D point from among the multiple first adjacent point candidates selected in step S101 (S102). For example, the encoding device 100 selects, as a second adjacent point candidate, a 3D point that has been coded before the target 3D point from among the multiple first adjacent point candidates. Also, for example, the decoding device 200 selects, as a second adjacent point candidate, a 3D point that has been decoded before the target 3D point from among the multiple first adjacent point candidates.

[0299] Next, the encoding device 100 and the decoding device 200 calculate the distance between the target 3D point and each of the multiple second adjacent point candidates selected in step S102 (S103).

[0300] Next, the encoding device 100 and the decoding device 200 select the adjacent point with the maximum number of adjacent points by selecting three-dimensional points with a number less than or equal to the maximum number of adjacent points (the above-mentioned NumNeiCnt) from the multiple second adjacent point candidates selected in step S102 in order of closest distance (S104).

[0301] FIG. 39 is an explanatory diagram showing the adjacent point selection process according to this embodiment. In the example shown in FIG. 39 , the target 3D point is 3D point f. Also, in the example shown in FIG. 39 , 3D point a, 3D point b, 3D point c, 3D point d, 3D point e, 3D point f, 3D point g, and 3D point h are encoded or decoded in this order. That is, in the example shown in FIG. 39 , 3D point a, 3D point b, 3D point c, 3D point d, and 3D point e are 3D points that have already been encoded or decoded. Also, in the example shown in FIG. 39 , 3D points that have connectivity are shown connected to each other by solid lines. Also, in the example shown in FIG. 39 , the distance between 3D point f and 3D point x (x is a, b, c, d, e, g, or h) is shown as D(x), and 3D point d, 3D point b, 3D point c, 3D point a, and 3D point e are closest to 3D point f in this order. In the example shown in FIG. 39, the maximum number of adjacent points (the above-mentioned NumNeiCnt) is three.

[0302] For example, in step S101, the encoding device 100 and the decoding device 200 select three-dimensional points a, b, c, d, e, g, and h as first adjacent point candidates, as shown in (a) of Figure 39.

[0303] Also, for example, in step S102, the encoding device 100 and the decoding device 200 select three-dimensional points a, b, c, d, and e as second adjacent point candidates, as shown in (b) of Figure 39.

[0304] Also, for example, in step S104, the encoding device 100 and the decoding device 200 select 3D points b, c, and d as first adjacent points, as shown in (c) of FIG.

[0305] 38 is an example, and the order in which steps S101 to S104 are executed may be arbitrarily interchanged. For example, if step S101 and step S102 are interchanged, the encoding device 100 and the decoding device 200 may select a 3D point that has been encoded or decoded before the target 3D point as a first adjacent point candidate by the processing of step S102, and may select a 3D point that has connectivity with the target 3D point from the first adjacent point candidates as a second adjacent point candidate by the processing of step S101. This improves the flexibility of implementation.

[0306] Furthermore, for example, some of the processes in steps S101 to S104 may be executed in parallel. For example, while the process in step S102 is being executed, the process in step S103 may be executed to calculate the distance between the target 3D point and the first adjacent point candidate in advance in parallel. This reduces the processing time.

[0307] Note that a motion group (MG) may be provided as a prediction unit according to the encoding order or decoding order. When encoding or decoding a motion vector of a three-dimensional point, the encoding device 100 and the decoding device 200 may encode or decode for each MG. For example, the number of three-dimensional points included in an MG (MGSize) may be defined, and the encoding device 100 and the decoding device 200 may divide the three-dimensional points into multiple MGs according to the encoding order or decoding order and encode or decode them.

[0308] Note that the coding order and decoding order of the motion vectors of the 3D points may be any order. For example, the encoding device 100 and the decoding device 200 may generate Level of Detail (hereinafter referred to as LoD) and encode or decode in order for each LoD layer. Alternatively, the encoding device 100 and the decoding device 200 may encode or decode the motion vectors in the coding order or decoding order of the position information of the 3D points (i.e., vertices) without generating LoD. Alternatively, the encoding device 100 and the decoding device 200 may generate Morton codes using the position information of the 3D points and encode or decode in the order of the Morton codes.

[0309] Next, specific examples of motion groups will be described using Figures 40 to 42. Note that in Figures 40, 42, and 43, MG0, MG1, and MGN are each examples of motion groups. Note that N is, for example, an integer equal to or greater than 2, but the number of motion groups may be two, or may be three or more. Furthermore, the multiple three-dimensional points (specifically, information on the three-dimensional points) indicated by circles in each figure are coded or decoded in order from left to right. In other words, the multiple three-dimensional points shown in each figure are coded or decoded in order starting from the three-dimensional point belonging to MG0. Furthermore, the three-dimensional points belonging to the same MG are coded or decoded in order from left to right.

[0310] FIG. 40 is a diagram showing a first example of a reference destination of a motion group according to this embodiment.

[0311] In the first example, 3D points belonging to the same motion group, in other words, 3D points within the same motion group, are defined as not being able to reference each other. That is, in the first example, the motion vectors of 3D points belonging to the same motion group as the target 3D point are not used to calculate the predicted value of the motion vector of the target 3D point. For example, 3D points within the same motion group are not added to adjacent points.

[0312] Furthermore, in the first example, the motion vector of a 3D point that belongs to a different motion group from the target 3D point is used to calculate the predicted value of the motion vector of the target 3D point. Specifically, in the first example, 3D points in a different motion group that have already been coded or decoded are defined as referenceable. In other words, in the first example, the motion vector of a 3D point that has already been coded or decoded, among 3D points that belong to a different motion group from the target 3D point, is used to calculate the predicted value of the motion vector of the target 3D point.

[0313] For example, in the example shown in Figure 40, in calculating the predicted value of the motion vector of the target 3D point belonging to MG1, the motion vector of the 3D point belonging to MG1 is not used, but the motion vector of the 3D point belonging to MG0 is used. Also, in the example shown in Figure 40, in calculating the predicted value of the motion vector of the target 3D point belonging to MG1, the motion vector of the 3D point belonging to MGN (specifically, MGN when N is an integer greater than or equal to 2) is not used.

[0314] It should be noted that, for example, coded or decoded 3D points in different motion groups may be defined as not adding to adjacent points.

[0315] FIG. 41 is a diagram showing an example of the syntax of a base mesh header according to this embodiment.

[0316] As in the syntax shown in Fig. 41, the size (data size) of the motion group may be written in the header of the bitstream, etc. For example, if the size (MGSize) of the motion group is 16, the encoding device 100 may add MGSize = 16 to the header of the bitstream. Alternatively, the encoding device 100 may set MGSize to 2n (n: an integer equal to or greater than 0) and add the value of n to the header of the bitstream.

[0317] The encoding device 100 and the decoding device 200 may encode or decode 3D points in the same motion group in parallel.

[0318] FIG. 42 is a diagram showing a second example of a reference destination of a motion group according to this embodiment.

[0319] In the second example, coded or decoded 3D points within the same motion group are defined as referable. Also in the second example, coded or decoded 3D points within different motion groups are defined as referable. Also in the second example, uncoded or undecoded 3D points are defined as unreferenceable. That is, in the second example, only motion vectors of 3D points that have already been coded or decoded are used to calculate the predicted value of the motion vector of the target 3D point. For example, coded or decoded 3D points within the same motion group may be added as neighboring points. Also, for example, coded or decoded 3D points within different motion groups may be added as neighboring points. On the other hand, for example, uncoded or undecoded 3D points cannot be added as neighboring points, whether within the same motion group or a different motion group.

[0320] For example, in the example shown in Figure 42, in calculating the predicted value of the motion vector of the target 3D point belonging to MG1, the motion vectors of the 3D points that have already been coded or decoded among the 3D points that belong to MG1 may be used, but the motion vectors of the 3D points that have not been coded or decoded are not used. Also, in the example shown in Figure 42, in calculating the predicted value of the motion vector of the target 3D point that belongs to MG1, the motion vectors of the 3D points that belong to MG0 may be used, but the motion vectors of the 3D points that belong to MGN (specifically, MGN when N is an integer greater than or equal to 2) are not used.

[0321] In the second example, the size of the motion group may also be written in the header of the bitstream. For example, if the size of the motion group (MGSize) is 16, the encoding device 100 may add MGSize=16 to the header of the bitstream. Alternatively, the encoding device 100 may set MGSize to 2n and add the value of n to the header of the bitstream.

[0322] In this way, by defining that already coded or decoded three-dimensional points can be referenced even for three-dimensional points within the same motion group, prediction accuracy can be improved, and coding efficiency can be improved.

[0323] FIG. 43 is a diagram showing a third example of a reference destination of a motion group according to this embodiment.

[0324] In the third example, coded or decoded 3D points within the same motion group are defined as referable. On the other hand, in the third example, uncoded or undecoded 3D points are defined as unreferenceable. For example, coded or decoded 3D points within the same motion group may be added as neighboring points. On the other hand, uncoded or undecoded 3D points within the same motion group may not be added as neighboring points.

[0325] Also, in the third example, 3D points in different motion groups are defined as not being able to be referenced, i.e., 3D points in different motion groups cannot be added to adjacent points.

[0326] For example, in the example shown in Figure 43, in calculating the predicted value of the motion vector of the target 3D point belonging to MG1, the motion vectors of the 3D points that have already been coded or decoded among the 3D points that belong to MG1 may be used, but the motion vectors of the 3D points that have not been coded or decoded are not used. Also, in the example shown in Figure 43, in calculating the predicted value of the motion vector of the target 3D point that belongs to MG1, the motion vectors of the 3D points that belong to motion groups other than MG1 are not used.

[0327] In the third example, the size of the motion group may also be written in the header of the bitstream. For example, if the size of the motion group (MGSize) is 16, the encoding device 100 may add MGSize=16 to the header of the bitstream. Alternatively, the encoding device 100 may set MGSize to 2n and add the value of n to the header of the bitstream.

[0328] In this way, by prohibiting references between motion groups and eliminating dependencies between motion groups, the encoding device 100 and the decoding device 200 can encode or decode information on three-dimensional points of multiple motion groups in parallel.

[0329] Furthermore, by defining three-dimensional points that are within the same motion group and that have already been coded or decoded as referenceable in this manner, prediction accuracy can be improved, and coding efficiency can be improved.

[0330] The number of 3D points belonging to each motion group may be determined arbitrarily and is not particularly limited. Furthermore, the number of 3D points belonging to each motion group may be the same for each group or may be different for each group.

[0331] In addition, when a full mesh, which is a mesh before being divided into one or more submeshes, is divided into one or more submeshes and then encoded or decoded, the encoding device 100 and the decoding device 200 may divide the three-dimensional points in each submesh into motion groups according to the encoding order or decoding order, and encode or decode the motion vector of each three-dimensional point for each motion group.

[0332] Fig. 44 is an explanatory diagram showing the relationship between the vertices constituting a mesh (original mesh) and motion groups according to this embodiment. Fig. 45 is an explanatory diagram showing the relationship between the vertices constituting submeshes (first submesh and second submesh) and motion groups according to this embodiment. Note that the first submesh and second submesh shown in Fig. 45 are meshes generated by dividing the original mesh shown in Fig. 44.

[0333] 44 and 45, the 3D points A, B, and C constituting the original mesh (full mesh) are divided into sub-meshes, and are duplicated into A1, B1, and C1 constituting the first sub-mesh and A2, B2, and C2 constituting the second sub-mesh, respectively. For example, the encoding device 100 may assign the motion vectors of the 3D points A1, B1, C1, A2, B2, and C2 to motion groups within each sub-mesh, and encode them using the method shown in the above example.

[0334] This allows the encoding device 100 to assign the motion vectors of the three-dimensional points of each submesh to motion groups within the submesh, while selecting appropriate adjacent points from the three-dimensional points within the submesh to encode the motion vectors.

[0335] Note that 3D points that belong to a submesh different from the 3D point to be encoded may not be included in the adjacent points. This allows each submesh to be encoded or decoded independently without referring to information between the submeshes.

[0336] Alternatively, 3D points belonging to different submeshes may not be included in the same motion group, which prevents reference to information between submeshes and allows the encoding device 100 and the decoding device 200 to encode or decode each submesh independently.

[0337] As described above, for example, in the process of predicting information about vertices (target 3D points) that make up a 3D mesh, the encoding device 100 and the decoding device 200 determine the vertices (adjacent points) to reference using distance information (information indicating the distance between 3D points).

[0338] Furthermore, for example, the vertex information is a motion vector of the vertex coordinates.

[0339] Furthermore, for example, the prediction process is an inter-prediction process.

[0340] Furthermore, for example, the encoding device 100 and the decoding device 200 determine combinations of a plurality of adjacent points.

[0341] Furthermore, for example, the distance information is a difference value between the coordinates of the processing target point (target three-dimensional point) and the coordinates of the adjacent point.

[0342] Furthermore, for example, the encoding device 100 and the decoding device 200 determine, as adjacent points, vertices whose difference values ​​are equal to or less than a predetermined value.

[0343] Furthermore, for example, the encoding device 100 and the decoding device 200 select a predetermined number of vertices in ascending order of difference value as adjacent points.

[0344] Also, for example, the predetermined number is coded into the bitstream.

[0345] Furthermore, for example, the distance information is calculated using information of the reference frame.

[0346] Furthermore, for example, the encoding device 100 and the decoding device 200 derive distance information using a point included in the reference frame and corresponding to the processing target point.

[0347] Also, for example, the reference frame is a frame that precedes the processing target frame in display order.

[0348] Also, for example, the reference frame is a frame that precedes the frame to be processed in coding order or decoding order.

[0349] Furthermore, for example, information other than the distance is derived using information on the processing target frame.

[0350] Furthermore, for example, the encoding device 100 and the decoding device 200 select points having connectivity using processing target points included in the processing target frame.

[0351] Furthermore, for example, the encoding device 100 and the decoding device 200 determine adjacent points using other information in addition to the distance information. Note that one or more pieces of other information used together with the distance information may be used in any combination.

[0352] Furthermore, for example, the encoding device 100 and the decoding device 200 determine, as adjacent points, vertices that have connectivity with the processing target point.

[0353] Furthermore, for example, the encoding device 100 and the decoding device 200 determine, as adjacent points, vertices that have been coded or decoded before the processing target point. For example, the encoding device 100 determines, as adjacent points, vertices that have been coded before the processing target point (the three-dimensional point to be coded). For example, the decoding device 200 determines, as adjacent points, vertices that have been decoded before the processing target point (the three-dimensional point to be decoded).

[0354] Furthermore, for example, the encoding device 100 and the decoding device 200 determine, as adjacent points, vertices that belong to the same submesh as the processing target point.

[0355] Furthermore, for example, the encoding device 100 and the decoding device 200 determine, as adjacent points, vertices that belong to the same motion group as the processing target point.

[0356] Furthermore, for example, the encoding device 100 and the decoding device 200 determine, as adjacent points, vertices that belong to a different motion group from the point to be processed.

[0357] Furthermore, for example, when the number of vertices that are candidates for adjacent points exceeds a predetermined number, the encoding device 100 and the decoding device 200 select a predetermined number of vertices from the candidate vertices using at least one of the above methods.

[0358] <Representative Example> Fig. 46 is a flow diagram showing an example of basic encoding processing according to this embodiment. For example, the circuit 151 of the encoding device 100 shown in Fig. 24 performs the encoding processing shown in Fig. 46 in operation.

[0359] The encoding device 100 executes an encoding method for encoding information on three-dimensional points in a target frame to be encoded.

[0360] First, the encoding device 100 selects one or more reference 3D points from among a plurality of 3D points in the target frame (S201).

[0361] Next, the encoding device 100 calculates a predicted value of the second information of the target 3D point to be encoded in the target frame, using the first information of each of the one or more reference 3D points (S202).

[0362] Here, in the step of selecting one or more reference 3D points (S201), encoding device 100 selects one or more reference 3D points based on the distance between the target 3D point and each of the plurality of 3D points.

[0363] The first information and the second information are, for example, information indicating a motion vector (specifically, prediction information). The first information and the second information may each be any information possessed by a 3D point, such as position information or attribute information. The reference 3D point is, for example, the adjacent point described above. The 3D point is, for example, the vertex described above. The multiple 3D points in the target frame and the target 3D point are, for example, vertices constituting a 3D mesh included in the target frame. The target frame is, for example, the current frame described above. The information on the multiple 3D points and the target 3D point may not include connectivity information. In other words, the 3D point group encoded by the encoding device 100 may or may not be a 3D mesh.

[0364] It is considered that the closer the distance between the three-dimensional points, the closer the information about the three-dimensional points will be. Therefore, for example, it is considered that the prediction residual can be reduced by using a three-dimensional point that is close to the target three-dimensional point as a reference three-dimensional point to calculate a predicted value. If the prediction residual can be reduced, for example, the amount of code of a bitstream including information indicating the prediction residual can be reduced. Therefore, according to the encoding device 100, the amount of code can be reduced by selecting one or more reference three-dimensional points based on the distance between the target three-dimensional point and each of the multiple three-dimensional points.

[0365] Furthermore, for example, the encoding device 100 calculates a prediction residual, which is the difference between the value indicated by the second information and the predicted value, and generates a bitstream including prediction residual information indicating the calculated prediction residual. For example, the encoding device 100 calculates the prediction residual after performing step S202, and further generates a bitstream.

[0366] The prediction residual is, for example, the above-mentioned absolute difference Diffp, and the prediction residual information is, for example, information indicating the absolute difference Diffp.

[0367] This allows the encoding device 100 to generate a bitstream with a reduced amount of code.

[0368] Also, for example, the first information of each of the one or more reference 3D points indicates a motion vector of each of the one or more reference 3D points, and the second information indicates a motion vector of the target 3D point.

[0369] Specifically, the first information is a 3D point in a reference frame, and is information indicating a motion vector indicating the amount of change from the coordinates of a 3D point corresponding to the reference 3D point to the coordinates of the reference 3D point in the target frame. The second information is a 3D point in the reference frame, and is information indicating a motion vector indicating the amount of change from the coordinates of a 3D point corresponding to the target 3D point to the coordinates of the target 3D point in the target frame. The first information and the second information are, for example, the prediction information described above. The reference frame is, for example, the past frame described above.

[0370] This allows the encoding device 100 to encode motion vectors.

[0371] Furthermore, for example, in calculating the predicted value (S202), the encoding device 100 calculates the predicted value using inter prediction. That is, the encoding device 100 calculates the predicted value using information on a frame at a time different from that of the target frame.

[0372] This allows the encoding device 100 to calculate a predicted value.

[0373] Furthermore, for example, when selecting one or more reference 3D points, the encoding device 100 calculates the distance by calculating the difference between the coordinates of the target 3D point and the coordinates of each of the multiple 3D points.

[0374] This allows the encoding device 100 to calculate the distance between the target 3D point and each of the multiple 3D points.

[0375] Furthermore, for example, when selecting one or more reference 3D points, encoding device 100 selects, from among a plurality of 3D points, 3D points whose distance is equal to or less than a predetermined value as the one or more reference 3D points.

[0376] The predetermined value is, for example, the threshold value THd described above. The predetermined value may be determined arbitrarily in advance and is not particularly limited.

[0377] This allows the encoding device 100 to select a reference 3D point that is closest to the target 3D point from among a plurality of 3D points.

[0378] Also, for example, when selecting one or more reference three-dimensional points, the encoding device 100 selects a predetermined number of three-dimensional points from multiple three-dimensional points in order of shortest distance, thereby selecting them as one or more reference three-dimensional points.

[0379] This allows the encoding device 100 to select an appropriate number of reference 3D points for calculating the predicted value.

[0380] Furthermore, for example, the encoding device 100 generates a bitstream including predetermined number information indicating a predetermined number. For example, the encoding device 100 generates a bitstream including prediction residual information and the predetermined number information.

[0381] The predetermined number is, for example, the above-mentioned maximum number of adjacent points (NumNeiCnt).

[0382] This allows the decoding device 200 to select a reference 3D point using a predetermined amount of information acquired from the bitstream.

[0383] Furthermore, for example, when selecting one or more reference 3D points, the encoding device 100 calculates the distance using the coordinates of a 3D point in the reference frame that corresponds to the target 3D point.

[0384] This allows the decoding device 200 to calculate the distance in the same way as the encoding device 100, without decoding the coordinates of the target 3D point in the target frame.

[0385] Also, for example, the reference frame is a frame that precedes the target frame in display order.

[0386] This allows the encoding device 100 to encode the target frame using a frame that is displayed on the display before the target frame, in other words, a past frame.

[0387] Also, for example, the reference frame is a frame that precedes the target frame in coding order.

[0388] This allows the encoding device 100 to encode the current frame using an already encoded frame.

[0389] Furthermore, for example, the encoding device 100 selects one or more reference 3D points using distance and information other than distance.

[0390] The information other than the distance may be, for example, connection information (connectivity). The information other than the distance may be, for example, the threshold value THd, the NumNeiCnt, and / or information about the motion group.

[0391] According to this, by appropriately selecting information other than the distance, the encoding device 100 can further reduce the amount of code.

[0392] Furthermore, for example, the information other than the distance is connection information indicating whether the target three-dimensional point is connected to each of the multiple three-dimensional points, and for example, when selecting one or more reference three-dimensional points, the encoding device 100 selects, from the multiple three-dimensional points, three-dimensional points that are connected to the target three-dimensional point as the one or more reference three-dimensional points.

[0393] In the case of connected 3D points, the information of the 3D points is considered to be closer than that of unconnected 3D points. Therefore, for example, by using a 3D point connected to the target 3D point as a reference 3D point to calculate a predicted value, it is considered that the prediction residual can be reduced, and therefore the encoding device 100 can further reduce the amount of code.

[0394] Fig. 47 is a flow diagram showing an example of basic decoding processing according to this embodiment. For example, the circuit 251 of the decoding device 200 shown in Fig. 25 performs the decoding processing shown in Fig. 47 in operation.

[0395] The decoding device 200 executes a decoding method for decoding information on three-dimensional points in a target frame to be decoded.

[0396] First, the decoding device 200 selects one or more reference 3D points from among a plurality of 3D points in the target frame (S301).

[0397] Next, the decoding device 200 calculates a predicted value of the second information of the target 3D point to be decoded in the target frame, using the first information of each of the one or more reference 3D points (S302).

[0398] Here, in the step of selecting one or more reference 3D points (S301), decoding device 200 selects one or more reference 3D points based on the distance between the target 3D point and each of the plurality of 3D points.

[0399] It is considered that the closer the distance between the three-dimensional points, the closer the information about the three-dimensional points will be. Therefore, for example, it is considered that the prediction residual can be reduced by using a three-dimensional point that is close to the target three-dimensional point as a reference three-dimensional point to calculate a predicted value. If the prediction residual can be reduced, for example, the amount of coding of a bitstream including information indicating the prediction residual can be reduced. Therefore, according to the decoding device 200, by selecting one or more reference three-dimensional points based on the distance between the target three-dimensional point and each of the multiple three-dimensional points, it is possible to decode the information about the three-dimensional point using information with a reduced amount of coding.

[0400] Furthermore, for example, the decoding device 200 acquires prediction residual information indicating a prediction residual from the bitstream, and calculates the second information based on the prediction residual and the predicted value. For example, the decoding device 200 calculates the second information using the prediction residual and the predicted value after step S302. The timing at which the decoding device 200 acquires the prediction residual information may be any timing as long as it is before the calculation of the second information.

[0401] This allows the decoding device 200 to decode information about three-dimensional points using information about a bitstream with a reduced amount of code.

[0402] Also, for example, the first information of each of the one or more reference 3D points indicates a motion vector of each of the one or more reference 3D points, and the second information indicates a motion vector of the target 3D point.

[0403] This allows the decoding device 200 to decode the motion vector.

[0404] Furthermore, for example, in calculating the predicted value (S302), the decoding device 200 calculates the predicted value using inter prediction.

[0405] This allows the decoding device 200 to calculate a predicted value.

[0406] Furthermore, for example, when selecting one or more reference 3D points, the decoding device 200 calculates the distance by calculating the difference between the coordinates of the target 3D point and the coordinates of each of the multiple 3D points.

[0407] This allows the decoding device 200 to calculate the distance between the target 3D point and each of the multiple 3D points.

[0408] Furthermore, for example, when selecting one or more reference 3D points, decoding device 200 selects, from among a plurality of 3D points, 3D points whose distance is equal to or less than a predetermined value as the one or more reference 3D points.

[0409] This allows the decoding device 200 to select a reference 3D point that is closest to the target 3D point from among a plurality of 3D points.

[0410] Also, for example, when selecting one or more reference three-dimensional points, the decoding device 200 selects a predetermined number of three-dimensional points from a plurality of three-dimensional points in order of shortest distance, thereby selecting them as one or more reference three-dimensional points.

[0411] This allows the decoding device 200 to select an appropriate number of reference 3D points for calculating a predicted value.

[0412] Alternatively, for example, the decoding device 200 may obtain predetermined number information indicating the predetermined number from the bitstream. For example, the decoding device 200 obtains the predetermined number information from the bitstream before step S301.

[0413] This allows the decoding device 200 to select an appropriate number of reference 3D points for calculating a predicted value using the predetermined number of pieces of information acquired from the bitstream.

[0414] Furthermore, for example, when selecting one or more reference 3D points, the decoding device 200 calculates the distance using the coordinates of a 3D point in the reference frame that corresponds to the target 3D point.

[0415] This allows the decoding device 200 to calculate the distance in the same way as the encoding device 100, without decoding the coordinates of the target 3D point in the target frame.

[0416] Also, for example, the reference frame is a frame that precedes the target frame in display order.

[0417] This allows the decoding device 200 to decode the target frame using a frame that is displayed on the display device earlier than the target frame, in other words, a past frame.

[0418] Also, for example, the reference frame is a frame that precedes the target frame in decoding order.

[0419] This allows the decoding device 200 to decode the target frame using a previously decoded frame.

[0420] Furthermore, for example, when selecting one or more reference 3D points, the decoding device 200 selects one or more reference 3D points using distance and information other than distance.

[0421] According to this, by appropriately selecting information other than distance, the decoding device 200 can decode the information of the three-dimensional point using information with a further reduced amount of coding.

[0422] Furthermore, for example, the information other than the distance is connection information indicating whether the target 3D point is connected to each of the multiple 3D points, and for example, when selecting one or more reference 3D points, the decoding device 200 selects, from the multiple 3D points, 3D points that are connected to the target 3D point as the one or more reference 3D points.

[0423] In the case of connected 3D points, the information of the 3D points is considered to be closer than that of unconnected 3D points. Therefore, for example, by using a 3D point connected to the target 3D point as a reference 3D point to calculate a predicted value, it is considered possible to reduce the prediction residual, and the decoding device 200 can decode the information of the 3D point using information with a further reduced amount of code.

[0424] The method for generating LoD will be described below.

[0425] 48 and 49 are explanatory diagrams showing a method for generating LoD in this embodiment.

[0426] When encoding a motion vector of a 3D point, the encoding device may classify each 3D point into one or more layers using the position information of the 3D point before encoding. Here, each layer used for classification is called LoD (Level of Detail). An identifier (e.g., a number) that uniquely identifies the LoD is assigned to the LoD. For example, the 0th LoD is also referred to as LoD0, the 1st LoD is also referred to as LoD1, the nth LoD is also referred to as LoDn, and the (n-1)th LoD is also referred to as LoD(n-1).

[0427] The LoD generation method will be described with reference to Figures 48 and 49. In addition, if the encoding device or decoding device cannot calculate the position information or distance information of a 3D point in a frame to be encoded or decoded, the position information or distance information of a 3D point corresponding to the 3D point in a frame that has already been encoded or decoded may be used. This may enable efficient encoding by classifying the 3D points to be encoded or decoded into one or more layers.

[0428] 48 shows three-dimensional points to be encoded, namely, points a0, a1, a2, b0, b1, b2, c0, c1, and c2, where d(x, y) indicates the distance between point x and point y.

[0429] By setting the threshold value of each LoD layer larger for higher layers (layers closer to LoD0), the higher the layer, the greater the distance between three-dimensional points (also called sparse point cloud), and the lower the layer, the closer the distance between three-dimensional points (also called dense point cloud). Here, LoD0 is the top layer.

[0430] Point y belongs to the same LoD as point x if the distance d(x, y) from point x is greater than the threshold of the LoD to which point x belongs and is equal to or less than the threshold of the LoD higher than that LoD. Note that if point x belongs to LoD0, which is the highest layer, point y belongs to the same LoD as point x if the distance d(x, y) from point x is greater than the threshold of the LoD to which point x belongs.

[0431] First, the encoding device selects point a0 as the initial point and assigns it to LoD0. Next, the encoding device extracts point a1 whose distance from point a0 is greater than the threshold Thres_LoD[0] of LoD0 and assigns it to LoD0. Next, the encoding device extracts point a2 whose distance from point a1 is greater than the threshold Thres_LoD[0] of LoD0 and assigns it to LoD0. In this way, the encoding device configures LoD0 so that the distance between each point in LoD0 is greater than the threshold Thres_LoD[0].

[0432] Next, the encoding device selects point b0, which has not yet been assigned an LoD, and assigns it to LoD1. Next, the encoding device selects point b1, whose distance from point b0 is greater than the LoD1 threshold Thres_LoD[1] and whose LoD has not yet been assigned, and assigns it to LoD1. Next, the encoding device selects point b2, whose distance from point b1 is greater than the LoD1 threshold Thres_LoD[1] and whose LoD has not yet been assigned, and assigns it to LoD1. In this way, the encoding device configures LoD1 so that the distance between each point in LoD1 is greater than the threshold Thres_LoD[1].

[0433] Next, the encoding device selects point c0, which has not yet been assigned an LoD, and assigns it to LoD2. Next, the encoding device selects point c1, which has a distance from point c0 greater than the LoD2 threshold Thres_LoD[2] and has not yet been assigned an LoD, and assigns it to LoD2. Next, the encoding device selects point c2, which has a distance from point c1 greater than the LoD2 threshold Thres_LoD[2] and has not yet been assigned an LoD, and assigns it to LoD2. In this way, LoD2 is configured so that the distance between each point in LoD2 is greater than the threshold Thres_LoD[2].

[0434] The threshold value of each LoD may be added to the header of the bitstream. For example, in the case of Fig. 48, the threshold values ​​Thres_LoD[0], Thres_LoD[1], and Thres_LoD[2] may be added to the header of the bitstream.

[0435] Alternatively, all three-dimensional points that have not yet been assigned an LoD may be assigned to the lowest layer of the LoD. In this case, the threshold value of the lowest layer of the LoD is not added to the header, which has the effect of reducing the amount of coding for the header. For example, in the case of Figure 48, the encoding device may add thresholds Thres_LoD[0] and Thres_LoD[1] to the header, and Thres_LoD[2] may not be added to the header, so that the decoding device may estimate Thres_LoD[2] to be a value of 0.

[0436] Furthermore, the number of layers of the LoD may be added to the header, which allows the decoding device to determine whether the LoD is the lowest layer.

[0437] In addition, when the LoD layer is one layer, that is, when encoding the motion vector of a 3D point without generating the LoD, the encoding device may omit the LoD generation process described in the above example. Alternatively, the encoding device may apply the LoD generation method described in the above example with the LoD layer set to 1. In this case, the encoding device may perform the LoD generation process assuming that all 3D points belong to the same LoD. This allows the encoding device to reduce the processing time for generating the LoD.

[0438] The coding or decoding of the motion vector described in this embodiment may be applied to methods other than the LoD generation method described above. For example, even when the LoD layer to which a three-dimensional point belongs is determined in advance, the coding efficiency may be improved by applying the motion vector coding method or decoding method described in this embodiment.

[0439] The method of selecting the initial 3D point when constructing each LoD may depend on the encoding order when encoding the motion vector. For example, the encoding device selects the 3D point that was first encoded when encoding the motion vector as the initial point a0 of LoD0, and selects points a1 and a2 using point a0 as the base point to construct LoD0. Then, the encoding device may select, as the initial point b0 of LoD1, the 3D point for which the motion vector is encoded earliest among the 3D points that do not currently belong to LoD0. In other words, the encoding device may select, as the initial point n0 of LoDn, the 3D point for which the motion vector is encoded earliest among the 3D points that do not belong to the LoDs in the LoD(n-1) or lower hierarchy. This allows the same LoD as during encoding to be constructed using a similar initial point selection method during decoding (specifically, a method of selecting, as the initial point n0 of LoDn, the 3D point for which the motion vector is decoded earliest among the 3D points that do not belong to the LoDs in the LoD(n-1) or lower hierarchy), and the bitstream can be decoded appropriately.

[0440] FIG. 50 is an explanatory diagram showing a method for generating a predicted value of a motion vector in this embodiment.

[0441] The encoding device can generate a predicted value of a motion vector for a three-dimensional point using information about LoD.

[0442] For example, when encoding 3D points in order starting from LoD0, the encoding device may generate LoD1 using the coded and decoded motion vectors included in LoD0 and LoD1. In this way, the encoding device can generate the predicted value of the motion vector of the 3D point included in LoDn using the coded and decoded motion vector included in LoDn' (where n'≦n).

[0443] Furthermore, the predicted value of the motion vector of a 3D point can be generated by calculating the average of the motion vectors of a certain number of 3D points that are coded and decoded neighbors of the 3D point to be coded. The certain number is, for example, the number of neighbors of the 3D point to be coded (e.g., N). In this case, the value N is added to the header of the bitstream, etc.

[0444] Note that a value N indicating the number of neighboring points (i.e., N) used to calculate a predicted value may be added to each 3D point for which a predicted value is generated. This allows the encoding device to select an appropriate number of N neighboring points for each 3D point for which a predicted value is generated, thereby improving the accuracy of the predicted value and reducing the prediction residual. Alternatively, the encoding device may add the value N to a bitstream header and fix it within the bitstream (in other words, the value N may be commonly used as a fixed value in encoding the 3D points included in the bitstream). This eliminates the need for the encoding device to encode or decode the value N for each 3D point, thereby reducing the amount of processing. Alternatively, the encoding device may encode the value N separately for each LoD. This may potentially improve encoding efficiency by selecting an appropriate value N for each LoD.

[0445] Furthermore, the predicted value of the motion vector of a 3D point may be calculated from a weighted average of N adjacent points that have already been coded and decoded. For example, the coding device may perform a weighted average using distance information between the 3D point to be coded and each of the N adjacent points. This will be described with reference to FIG. 50 .

[0446] When encoding using a different value N for each LoD, the encoding device may, for example, set the value of N to be larger for higher LoD layers and set the value of N to be smaller for lower LoD layers. In higher LoD layers, the distance between 3D points belonging to the LoD is relatively large, so by setting the value of N to be large, it is possible to improve prediction accuracy by selecting and averaging a relatively large number of surrounding 3D points. In lower LoD layers, the distance between 3D points belonging to the LoD is relatively small, so setting the value of N to be small may reduce the amount of averaging processing and enable efficient prediction.

[0447] A prediction of a point P belonging to LoDN is generated from a reconstructed point P′ belonging to LoDN′ (where N′≦N), where neighbors are selected to point P′ based on connectivity and distance.

[0448] The predicted value of the motion vector may be calculated from an unweighted average value, which reduces the amount of processing.

[0449] As shown in FIG. 50 , point a2 is predicted from point a0 and point a1. Furthermore, point b2 is predicted from point a0, point a1, point a2, point b0, and point b1. Note that the points selected as adjacent points used for prediction may vary depending on the number N of adjacent points used for prediction. For example, when N=5, points a0, a1, point a2, point b0, and point b1 may be selected as adjacent points of point b2, and when N=4, points a0, a1, point a2, and point b1 may be selected based on distance information.

[0450] For example, when a weighted average of adjacent points is used for prediction, the predicted value a2p of point a2 is calculated by taking the weighted average of points a0 and a1 (see (Equation 1) and (Equation 2)), where Ai is the value of the motion vector of point ai.

[0451]

[0452] however,

[0453]

[0454] The predicted value b2p of point b2 is calculated by taking the weighted average of points a0, a1, a2, b0, and b1 (see equations 3, 4, and 5). Here, Bi is the value of the motion vector of point bi.

[0455]

[0456] however,

[0457]

[0458]

[0459] When encoding the value of a motion vector of a 3D point, the encoding device may calculate a difference value between the 3D point and a predicted value generated from points adjacent to the 3D point (also called a prediction residual, see Equations 6 and 7 below), and then quantize the calculated prediction residual to encode the value. Here, prediction residual a2r is the prediction residual of point a2, and prediction residual b2r is the prediction residual of point b2.

[0460]

[0461]

[0462] For example, the encoding device can quantize the prediction residual by dividing it by a quantization scale. In this case, the smaller the quantization scale, the smaller the error that may occur due to quantization (quantization error), and conversely, the larger the quantization scale, the larger the quantization error.

[0463] The value obtained by quantizing the prediction residual a2r is set to the quantized value a2q, and the value obtained by quantizing the prediction residual b2r is set to the quantized value b2 (see (Equation 8) and (Equation 9) below). QS_LoD0 is the quantization scale for LoD0, and QS_LoD1 is the quantization scale for LoD1.

[0464]

[0465]

[0466] The encoding device may change the value of the quantization scale for each LoD. For example, the quantization scale may be set smaller for higher LoD layers and larger for lower LoD layers. Since the value of a motion vector of a 3D point belonging to a higher layer may be used as a predicted value of a motion vector of a 3D point belonging to a lower layer, by reducing the quantization scale of the higher layer, it is possible to suppress quantization errors that may occur in the higher layer and increase the accuracy of the predicted value, thereby improving encoding efficiency. The encoding device may also add the quantization scale for each LoD to a header, etc. This allows the encoding device to contribute to the decoding device correctly decoding the quantization scale and appropriately decoding the bitstream.

[0467] The encoding device may convert the quantized prediction residual from a signed integer value to an unsigned integer value. For example, the encoding device may convert the quantized value a2q, which is a signed integer value, to the quantized value a2u, which is an unsigned integer value, as follows:

[0468] If the quantized value a2q is smaller than 0, a2u = -1 - (2 x a2q) Otherwise, a2u = 2 x a2q (Equation 10)

[0469] Furthermore, for example, the encoding device may convert the quantized value b2q, which is a signed integer value, into the quantized value b2u, which is an unsigned integer value, as follows:

[0470] If the quantized value b2q is smaller than 0, then b2u = -1 - (2 × b2q). Otherwise, b2u = 2 × b2q (Equation 11).

[0471] This has the advantage that the encoding device does not need to take into account the occurrence of negative integers when entropy encoding the prediction residual.

[0472] It should be noted that the encoding device does not necessarily have to convert from a signed integer value to an unsigned integer value, and may instead entropy encode the sign bit separately, for example.

[0473] Note that the method of encoding the prediction residual is not limited to this, and for example, the encoding device may perform arithmetic encoding for each bit of a sign bit indicating the positive or negative sign of the prediction residual and binarized data of the absolute value of the prediction residual, etc., using a context. This may enable the encoding device to improve the encoding efficiency of the prediction residual of a motion vector.

[0474] If quantization of the prediction residual of the motion vector is not necessary, this process may be skipped and the prediction residual may be directly arithmetically coded, thereby reducing the processing time.

[0475] The encoding device can decode the quantized prediction residual by inverse quantization and reconstruction, and use it for predicting subsequent three-dimensional points to be encoded. Specifically, the encoding device calculates an inverse quantization value by multiplying the quantized prediction residual by a quantization scale, and then adds the inverse quantization value and the predicted value to obtain a decoded value. For example, the encoding device can calculate the inverse quantization value a2iq from the quantization value a2q as follows, and can calculate the inverse quantization value b2iq from the quantization value b2q as follows:

[0476] a2iq=a2q×QS_LoD0 b2iq=b2q×QS_LoD1 (Formula 12)

[0477] Furthermore, the encoding device can calculate the reconstructed value a2rec from the inverse quantized value a2iq as follows, and can calculate the reconstructed value b2rec from the inverse quantized value b2iq as follows.

[0478] a2rec=a2iq+a2p b2rec=b2iq+b2p (Formula 13)

[0479] In this embodiment, a method is shown in which the encoding device constructs multiple LoDs to generate predicted values ​​of motion vectors of three-dimensional points, but this is not necessarily limited to this. For example, it may be applied to a case in which a single layer of LoDs is constructed to generate predicted values ​​of motion vectors of three-dimensional points, or to a case in which a predicted value of motion vectors of three-dimensional points is generated without generating LoDs.

[0480] In this case, since all the 3D points belong to the same LoD (for example, LoD0), when encoding or decoding in order starting from the 3D points included in LoD0, the encoding device may generate predicted values ​​of the 3D points belonging to LoD0 using the encoded and decoded motion vectors included in LoD0. In this way, the encoding device may be able to reduce processing time by encoding without generating multiple layers of LoD.

[0481] If quantization of the prediction residual of the motion vector is not necessary, the encoding device may skip the quantization and inverse quantization processes and add the arithmetically decoded prediction residual directly to the predicted value to obtain the decoded value, thereby reducing processing time.

[0482] FIG. 51 is an explanatory diagram showing an example of syntax in this embodiment.

[0483] The example syntax shown in Figure 51 shows an example of the structure of information contained in a bitstream generated by an encoding device.

[0484] The syntax shown in Figure 51 includes NumLoD, NumOfPoint[i], Thres_LoD[i], NumNeiCnt[i], THd[i] and QS[i].

[0485] NumLoD indicates the number of layers of the LoD.

[0486] NumOfPoint[i] indicates the number of 3D points belonging to layer i. Note that if the encoding device adds the total number of 3D points, AllNumOfPoint, to a separate header, NumOfPoint[NumLoD-1] (i.e., the number of 3D points belonging to the lowest layer) may not be added to the header. In this case, NumOfPoint[NumLoD-1] can be calculated using the following (Equation 14).

[0487]

[0488] Thres_LoD[i] indicates the LoD threshold of layer i. The encoding device configures LoDi so that the distance between each point in LoDi is greater than the threshold Thres_LoD[i]. Note that the value of Thres_LoD[NumLoD-1] (i.e., the LoD threshold of the lowest layer) may not be added to the header. In this case, Thres_LoD[NumLoD-1] may be estimated to be 0. This allows the amount of code in the header to be reduced.

[0489] NumNeiCnt[i] indicates the upper limit of the number of neighboring points used to generate a predicted value of a 3D point belonging to layer i. If the number of neighboring points M is less than NumNeiCnt[i] (i.e., M<NumNeiCnt[i]), the encoding device may calculate a predicted value using M neighboring points. Furthermore, if it is not necessary to change the value of NumNeiCnt[i] for each LoD, the encoding device may add one NumNeiCnt to the header.

[0490] THd[i] indicates the upper limit of the distance of a 3D point used to predict a 3D point to be coded or decoded in layer i. The coding device may not use 3D points whose distance from the 3D point to be coded or decoded is greater than THd[i] for prediction. Note that if it is not necessary to change the value of THd[i] for each LoD, one THd may be added to the header.

[0491] QS[i] denotes the quantization scale of layer i.

[0492] The encoding device may entropy-encode NumLoD, Thres_LoD[i], NumNeiCnt[i], THd[i], or QS[i] and add it to the header. For example, the encoding device may binarize each value and perform arithmetic encoding. Alternatively, the encoding device may perform fixed-length encoding to reduce the amount of processing.

[0493] It should be noted that the encoding device does not necessarily need to add NumLoD, Thres_LoD[i], NumNeiCnt[i], THd[i], or QS[i] to the header, and may be specified, for example, by a profile or level of a standard, etc. This allows the number of bits in the header to be reduced.

[0494] FIG. 52 is an explanatory diagram showing an example of syntax in this embodiment.

[0495] The example syntax shown in Figure 52 shows an example of the structure of information contained in a bitstream generated by an encoding device.

[0496] The syntax shown in FIG. 52 may include mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] for each of the 0th to NumLoDth layers of LoD (also referred to as the jth layer).

[0497] mvd_is_zero[k] is information indicating whether the absolute value of the prediction residual of the k-th component of the motion vector of the i-th 3D point (i.e., vertex[i]) included in the j-th layer of the LoD is 0. A value of 1 may indicate that the absolute value of the prediction residual of the k-th component is 0, and a value of 0 may indicate that the absolute value of the prediction residual of the k-th component is 1 or more.

[0498] mvd_is_one[k] is information indicating whether the absolute value of the prediction residual of the k-th component of the motion vector of the i-th 3D point (i.e., vertex[i]) included in the j-th layer of the LoD is 1. A value of 1 may indicate that the absolute value of the prediction residual of the k-th component is 1, and a value of 0 may indicate that the absolute value of the prediction residual of the k-th component is 2 or more.

[0499] If mvd_is_one[k] is not included in the bitstream, the decoding device may estimate its value to be 0. This makes it possible to prevent mvd_is_one[k] from being set to an indefinite value during decoding, and to perform decoding processing appropriately.

[0500] mvd_minus2[k] is information indicating the value obtained by subtracting 2 from the absolute value of the prediction residual of the kth component of the motion vector of the i-th 3D point (vertex[i]) included in the j-th layer of the LoD.

[0501] If mvd_minus2[k] is not included in the bitstream, the decoding device may estimate its value to be 0. This prevents mvd_minus2[k] from being set to an indefinite value during decoding, allowing appropriate decoding processing.

[0502] mvd_sign[k] indicates the sign bit of the k-th component of the motion vector of the i-th 3D point (vertex[i]) included in the j-th layer of the LoD. A value of 1 may indicate that the prediction residual of the k-th component is negative, and a value of 0 may indicate that the prediction residual of the k-th component is positive.

[0503] For the k-th component, if the motion vector is represented in a Cartesian coordinate system, the first component may represent the x-component, the second component may represent the y-component, and the third component may represent the z-component. Furthermore, if the motion vector is represented in a polar coordinate system, the first component may represent the distance r, the second component may represent the horizontal angle Φ, and the third component may represent the vertical angle Θ. This allows a common syntax structure to be used regardless of whether the motion vector is represented in a Cartesian coordinate system or a polar coordinate system.

[0504] The prediction residual mvd[k] of the k-th component of the motion vector of the i-th three-dimensional point (i.e., vertex[i]) may be calculated by the calculation process shown in FIG. 53 using the above information.

[0505] By introducing the syntax configuration shown in Fig. 52, when encoding a prediction residual that is likely to be mvd[k] = 0, the encoding device can reduce the frequency of encoding mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] and adding them to the bitstream, which may improve encoding efficiency. Also, when encoding a prediction residual that is likely to be mvd[k] = 1 or 0, the encoding device can reduce the frequency of encoding mvd_minus2[k] and adding it to the bitstream, which may improve encoding efficiency.

[0506] In this embodiment, an example is shown in which a case where mvd[k] is likely to be 0 or 1 as a prediction residual is assumed, but this is not necessarily limited to this, and the same processing may be applied to any mvd[k]. For example, when encoding a prediction residual where mvd[k] is likely to be 2, mvd_is_two[k] and mvd_minus3[k] may be newly introduced. This can reduce the frequency of encoding mvd_minus3[k] and adding it to the bitstream when encoding a prediction residual where mvd[k] is likely to be 2, and as a result, there is a possibility that the encoding efficiency can be improved. In this case, mvd[k] may be calculated by the calculation processing shown in FIG.

[0507] The encoding device may binarize at least one of mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k], and apply arithmetic coding using a context. For example, since mvd_is_zero[k], mvd_is_one[k], and mvd_sign[k] are each 1 bit, the encoding device may assign one context to each of them, and encode while updating the occurrence probability based on the occurrence frequency of 0 and 1. This may improve encoding efficiency. The encoding device may also binarize mvd_minus2[k] using Exponential Golomb, assign a context to each bit, and encode while updating the occurrence probability based on the occurrence frequency of 0 and 1. This may improve encoding efficiency.

[0508] The encoding device may assign different contexts to each component of mvd as the contexts to be assigned to mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k]. This may improve encoding efficiency when the value of mvd differs for each component. The encoding device may assign the same context to each component of mvd as the contexts to be assigned to mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k]. This may improve encoding efficiency when the values ​​of each component of mvd are close.

[0509] The processing of the encoding device will be described below.

[0510] FIG. 55 is a flowchart showing an example of processing performed by the encoding device according to this embodiment.

[0511] In step S1001, the encoding device generates one or more LoDs (see Figures 48 and 49).

[0512] In step S1002, the encoding device starts loop A, which repeatedly executes steps S1003 to S1011 (described later). In loop A, the encoding device focuses on each of the one or more LoDs generated in step S1001, executes processing for the focused LoD, and controls the processing so that all LoDs are ultimately processed. The focused LoD is also referred to as the "focused LoD." Loop A may also be referred to as the "LoD loop."

[0513] In step S1003, the encoding device starts a loop B, which repeatedly executes steps S1004 to S1010 (described later). In loop B, the device focuses on each of the 3D points belonging to the LoD of interest, executes processing for the 3D points of interest, and controls the device so that processing is ultimately performed for all 3D points. The 3D point of interest is also referred to as point P.

[0514] In step S1004, the encoding device searches for adjacent points of point P (see FIGS. 36 to 45).

[0515] In step S1005, the encoding device calculates a weighted average of the adjacent points found in step S1004 and sets it as a predicted value of point P (see FIG. 50).

[0516] In step S1006, the encoding device calculates the prediction residual of point P using the predicted value of point P set in step S1005 (see above (Equation 6) and (Equation 7)).

[0517] In step S1007, the encoding device calculates the quantized value of point P using the prediction residual calculated in step S1006 (see above (Equation 8) and (Equation 9)).

[0518] In step S1008, the encoding device encodes the quantized value calculated in step S1007.

[0519] In step S1009, the encoding device calculates an inverse quantized value by inverse quantizing the quantized value calculated in step S1007.

[0520] In step S1010, the encoding device calculates a reconstructed value using the inverse quantized value calculated in step S1009.

[0521] In step S1011, the encoding device performs the end process of loop B. Specifically, the encoding device determines whether the processes of S1004 to S1010 have been executed for all 3D points belonging to the LoD of interest, and if not, controls to execute the processes focusing on the 3D points that have not yet been executed.

[0522] In step S1012, the encoding device performs the process of ending loop A. Specifically, the encoding device determines whether the processes of steps S1003 to S1011 have been executed for all LoDs, and if not, controls the process to be executed focusing on the LoDs that have not yet been executed.

[0523] In addition, if the encoding device is to generate one LoD layer in step S1001, it is possible to generate a predicted value of a motion vector of a 3D point without generating LoD. In this case, the processes of steps S1001 and S1012 may be omitted. In this case, the encoding device may perform the processes of steps S1002 to S1011 assuming LoD=1 (i.e., only LoD1 exists). This allows the encoding device to reduce processing time.

[0524] If quantization of the prediction residual of the motion vector is not necessary, the encoding device may skip the quantization process (step S1007) and the inverse quantization process (step S1009) and add the arithmetically decoded prediction residual directly to the predicted value to obtain the decoded value, thereby reducing the processing time.

[0525] The decoding device may convert the decoded quantized prediction residuals from unsigned integer values ​​to signed integer values ​​in a manner opposite to that of the encoding device, thereby enabling the bitstream generated when the prediction residuals are entropy coded to be appropriately decoded without taking into account the occurrence of negative integers.

[0526] Note that it is not necessary to convert from an unsigned integer value to a signed integer value. For example, when decoding a bitstream generated by separately entropy-encoding the coded bits, the decoding device may decode the coded bits. Note that the decoding method of the prediction residual by the decoding device is not limited to this. For example, the decoding device may arithmetically decode the coded bits indicating the positive or negative sign of the prediction residual and the binarized data of the absolute value of the prediction residual, etc., for each bit using a context. This allows the decoding device to appropriately decode a bitstream with improved coding efficiency of the prediction residual of the motion vector.

[0527] The decoding device decodes the quantized prediction residual converted into a signed integer value by inverse quantization and reconstruction, and uses it for prediction of the three-dimensional point to be decoded and beyond. Specifically, the decoding device multiplies the quantized prediction residual by the decoded quantization scale to calculate an inverse quantization value, and adds the inverse quantization value and the prediction value to obtain a decoded value.

[0528] For example, the decoded unsigned quantized value a2u is converted to a signed value a2q as follows: where ">>" indicates a bit shift operation.

[0529] If the LSB (least significant bit) of a2u is 1: a2q = -((a2u + 1) >> 1) Otherwise: a2q = (a2u >> 1) (Equation 15)

[0530] Also, for example, the decoded unsigned quantized value b2u is converted into a signed value b2q as follows.

[0531] If the LSB of b2u is 1, then b2q = -((b2u + 1) >> 1). Otherwise, b2q = (b2u >> 1). (Equation 16)

[0532] The decoder calculates reconstructed values ​​after inverse quantization, which can be used for prediction of subsequent three-dimensional points to be decoded.

[0533] For example, the decoding device can calculate the inverse quantized value a2iq from the quantized value a2q as follows, and can calculate the inverse quantized value b2iq from the quantized value b2q as follows.

[0534] a2iq=a2q×QS_LoD0 b2iq=b2q×QS_LoD1 (Equation 17)

[0535] Furthermore, the decoding device can calculate the reconstructed value a2rec from the inverse quantized value a2iq as follows, and can calculate the reconstructed value b2rec from the inverse quantized value b2iq as follows.

[0536] a2rec=a2iq+a2p b2rec=b2iq+b2p (Formula 18)

[0537] FIG. 56 is a flowchart showing an example of processing performed by the decoding device according to this embodiment.

[0538] In step S1101, the decoding device generates one or more LoDs from the input bitstream (see Figures 48 and 49).

[0539] In step S1102, the decoding device performs a start process of loop A, which repeatedly executes the processes of steps S1103 to S1109 described below. In loop A, the decoding device focuses on each of the one or more LoDs generated in step S1101, executes processing for the focused LoD, and controls so that processing is ultimately performed for all LoDs. The focused LoD is also referred to as a focused LoD. Loop A may also be called a LoD loop.

[0540] In step S1103, the decoding device performs a start process of loop B, which repeatedly executes the processes of steps S1104 to S1108 described below. In loop B, the device focuses on each of the 3D points belonging to the LoD of interest, executes processing for the 3D points of interest, and controls so that processing is ultimately performed for all 3D points. The 3D point of interest is also referred to as point P.

[0541] In step S1104, the decoding device searches for adjacent points of point P (see FIGS. 36 to 45).

[0542] In step S1105, the decoding device calculates a weighted average of the adjacent points found in step S1104 and sets it as a predicted value of point P (see FIG. 50).

[0543] In step S1106, the decoding device decodes the quantized value of the point P.

[0544] In step S1107, the decoding device dequantizes the quantized value of point P decoded in step S1106 to obtain an inverse quantized value.

[0545] In step S1108, the decoding device calculates a reconstructed value of the point P using the inverse quantized value acquired in step S1107.

[0546] In step S1109, the decoding device performs the process of ending loop B. Specifically, the decoding device determines whether or not the processes of S1104 to S1108 have been executed for all 3D points belonging to the LoD of interest, and if not, controls so that the processes are executed by focusing on 3D points that have not yet been executed.

[0547] In step S1110, the decoding device performs the process of ending loop A. Specifically, the decoding device determines whether the processes of steps S1103 to S1109 have been executed for all LoDs, and if not, controls the process to be executed focusing on the LoDs that have not yet been executed.

[0548] Note that, if the decoding device is to generate one LoD layer in step S1101, it is possible to generate a predicted value of a motion vector of a 3D point without generating LoD. In this case, the processes of steps S1102 and S1110 may be omitted. In this case, the decoding device may perform the processes of steps S1103 to S1108 assuming LoD=1 (i.e., only LoD1 exists). This allows the decoding device to reduce processing time.

[0549] If the dequantization of the prediction residual of the motion vector is not necessary, the decoding device may skip the dequantization process (step S1107) and add the arithmetically decoded prediction residual directly to the predicted value to obtain the decoded value, thereby reducing the processing time.

[0550] In the above, an example was shown in which the encoding device calculates the average of the motion vectors of a certain number of three-dimensional points among the coded and decoded adjacent points of the three-dimensional point to be encoded to generate a predicted value of the motion vector of a three-dimensional point, but this is not necessarily limited to this, and the predicted value can be generated using other methods.

[0551] For example, the encoding device may use the motion vector of the closest 3D point among the 3D points that have been coded and decoded adjacent to the 3D point to be coded as the predicted value. The encoding device may also assign a prediction mode value (PredMode) to each 3D point, allowing the prediction value to be selected. For example, the encoding device may set a total number M of prediction modes, assign the average value to prediction mode 0, assign the motion vector of 3D point A to prediction mode 1, ..., assign the motion vector of 3D point Z to prediction mode M-1, and add the prediction mode used for prediction to the bitstream for each 3D point. The 3D points A to Z, to which motion vectors are assigned from prediction mode 1 to prediction mode M-1, may be used in order of proximity to the 3D point to be coded among the 3D points that have been coded and decoded adjacent to the 3D point to be coded.

[0552] Fig. 57 is an explanatory diagram showing an example of predicted value information of a motion vector in this embodiment. Fig. 58 is an explanatory diagram showing a method of generating predicted values ​​of a motion vector in this embodiment.

[0553] Figure 57 shows an example of prediction value information used to predict point b2 when the number N of adjacent three-dimensional points used for prediction is 4 and the number M of prediction modes is 5. The prediction value information includes information indicating, for each of one or more prediction modes, the predicted value to be used in that prediction mode. The example of prediction value information shown in Figure 57 is a table indicating, for each of one or more prediction modes, the predicted value to be used in that prediction mode.

[0554] In the example shown in Fig. 57, the predicted values ​​used to predict point b2 are, for example, the adjacent three-dimensional points a0, a1, a2, and b1 (see Fig. 58). Correspondingly, the "average value of points a0, a1, a2, and b1" is assigned as the predicted value for prediction mode 0.

[0555] In FIG. 57 , a "point b1" is assigned as a predicted value to prediction mode 1. A "point b2" is assigned as a predicted value to prediction mode 2. A "point a1" is assigned as a predicted value to prediction mode 3. A "point a0" is assigned as a predicted value to prediction mode 4.

[0556] Note that a numerical value that uniquely indicates a prediction mode is also referred to as a prediction mode value. Here, the following description will be given assuming that the prediction mode value of prediction mode m is m. For example, the prediction mode values ​​are used in ascending order of integer values.

[0557] The assignment of prediction mode values ​​may be determined in order of distance from the 3D point to be coded. For example, the coding device can assign a relatively smaller prediction mode value to a 3D point that is closer to the 3D point to be coded. In the above example, the 3D point that is closest to the 3D point b2 to be coded (i.e., closest to the 3D point b2) may be point b1, the 3D point that is next closest to the 3D point b2 may be point a2, the 3D point that is next closest to the 3D point b2 may be point a1, and the 3D point that is next closest to the 3D point b2 may be point a0.

[0558] This allows a smaller prediction mode value to be assigned to a point that has a relatively small distance, and therefore a relatively small difference between the motion vector and the predicted value, and is therefore more likely to be selected as a predicted value, thereby reducing the number of bits required to encode the prediction mode value.In addition, a smaller prediction mode value may be preferentially assigned to a three-dimensional point that belongs to the same LoD as the three-dimensional point to be encoded.

[0559] FIG. 59 shows an example of predicted value information used to predict point a2 when the number N of adjacent three-dimensional points used for prediction is 2 and the number M of prediction modes is 5.

[0560] In the example of predicted value information shown in Fig. 59, the predicted values ​​used to predict point a2 are, for example, adjacent three-dimensional points a0 and a1. Correspondingly, in Fig. 59, the "average value of points a0 and a1" is assigned as the predicted value for prediction mode 0.

[0561] Furthermore, "point a1" is assigned as a predicted value to prediction mode 1. "point a0" is assigned as a predicted value to prediction mode 2.

[0562] Note that, if the number of adjacent points is less than four, information indicating that the prediction mode to which a prediction value is not assigned may be set to be unavailable (denoted as "not available" in the figure).

[0563] An example of predicted value information when the motion vector is represented in an orthogonal coordinate system (XYZ coordinate system) is shown in FIG.

[0564] In the example shown in FIG. 60, the values ​​used to predict point b2 are, for example, the adjacent three-dimensional points a0, a1, a2, and b1 (see FIG. 58). Correspondingly, in FIG. 60, the coordinates (Xave, Yave, Zave) of the "average value of points a0, a1, a2, and b1" are assigned as the predicted value of prediction mode 0. Here, Xave can be calculated as the average or weighted average of Xb1, Xa2, Xa1, and Xa0. Yave can be calculated as the average or weighted average of Yb1, Yb2, Ya1, and Ya0. Zave can be calculated as the average or weighted average of Zb1, Zb2, Za1, and Za0.

[0565] Furthermore, the coordinates (Xb1, Yb1, Zb1) of "point b1" are assigned as a predicted value for prediction mode 1. The coordinates (Xa2, Ya2, Za2) of "point b2" are assigned as a predicted value for prediction mode 2. The coordinates (Xa1, Ya1, Za1) of "point a1" are assigned as a predicted value for prediction mode 3. The coordinates (Xa0, Ya0, Za0) of "point a0" are assigned as a predicted value for prediction mode 4.

[0566] For example, the encoding device may select prediction mode 2 (i.e., prediction mode value 2) and encode the X, Y, and Z components of the motion vector of the three-dimensional point to be encoded using predicted values ​​Xa2, Ya2, and Za2, respectively. In this case, the encoding device adds the prediction mode value 2 to the bitstream.

[0567] In the above example, the motion vector is in a Cartesian coordinate system, but this is not necessarily limited to this, and the present invention may be applied to a motion vector expressed in a polar coordinate system, for example.

[0568] The number of prediction modes M may be added to the bitstream. Alternatively, the number of prediction modes M may not be added to the bitstream, but may be defined by a standard profile or level. Alternatively, the number of prediction modes M may be a value calculated from the number of three-dimensional points N used for prediction (for example, M=N+1).

[0569] It should be noted that the encoding device may provide a prediction mode for each component of a quantity having multiple components, such as a motion vector, as will be described below.

[0570] For example, when a motion vector is represented in a Cartesian coordinate system (i.e., an XYZ coordinate system), the encoding device may prepare a prediction mode (also referred to as a "prediction mode (X)," a "prediction mode (Y)," and a "prediction mode (Z)," respectively) for each of the X, Y, and Z components of the motion vector, and select a prediction mode value independently for each. Examples of the prediction mode (X), the prediction mode (Y), and the prediction mode (Z) are shown in Figures 61, 62, and 63.

[0571] Fig. 61 is an explanatory diagram showing an example of predicted value information for the X component of a motion vector in this embodiment. The example of predicted value information shown in Fig. 61 indicates, for each of one or more prediction modes for the X component of a motion vector (i.e., prediction mode (X)), a predicted value to be used in the prediction mode.

[0572] In FIG. 61, Xave is calculated as a weighted average of Xb1, Xa2, Xa1, and Xa0.

[0573] The encoding device may, for example, select prediction mode 2 (in other words, a prediction mode having a prediction mode value of 2) as the prediction mode (X) and encode the X component of the motion vector of the three-dimensional point to be encoded using the predicted value Xa2. In this case, the encoding device adds the prediction mode value 2 to the bitstream.

[0574] Fig. 62 is an explanatory diagram showing an example of predicted value information for the Y component of a motion vector in this embodiment. The example of predicted value information shown in Fig. 62 indicates, for each of one or more prediction modes for the Y component of a motion vector (i.e., prediction mode (Y)), a predicted value to be used in that prediction mode.

[0575] In FIG. 62, Yave is calculated as a weighted average of Yb1, Ya2, Ya1, and Ya0.

[0576] The encoding device may, for example, select prediction mode 1 (in other words, a prediction mode whose prediction mode value is 1) as the prediction mode (Y) and encode the Y component of the motion vector of the three-dimensional point to be encoded using the predicted value Yb1. In this case, the encoding device adds the prediction mode value 1 to the bitstream.

[0577] Fig. 63 is an explanatory diagram showing an example of predicted value information for the Z component of a motion vector in this embodiment. The example of predicted value information shown in Fig. 63 indicates, for each of one or more prediction modes for the Z component of a motion vector (i.e., prediction mode (Z)), a predicted value to be used in that prediction mode.

[0578] In FIG. 63, Zave is calculated as a weighted average of Zb1, Za2, Za1 and Za0.

[0579] The encoding device may, for example, select prediction mode 4 (in other words, a prediction mode with a prediction mode value of 4) as the prediction mode (Z) and encode the Z component of the motion vector of the three-dimensional point to be encoded using the predicted value Za0. In this case, the encoding device adds the prediction mode value of 4 to the bitstream.

[0580] When selecting a prediction mode value for each of the above components, the encoding device may add the prediction mode for each component to the bitstream.

[0581] In addition, the encoding device may use the same prediction mode for some of the multiple components of a motion vector. For example, when the motion vector is represented in an orthogonal coordinate system (XYZ coordinate system), a prediction mode (X) may be prepared for the X component, and a prediction mode (YZ) may be prepared for the YZ component, and a prediction mode value may be selected independently for each. An example of the prediction mode (X) is shown in Figure 61, and an example of the prediction mode (YZ) is shown in Figure 64.

[0582] 64 is an explanatory diagram showing an example of predicted value information for the YZ components of a motion vector in this embodiment. The example of predicted value information shown in Fig. 64 indicates, for each of one or more prediction modes for the YZ components of a motion vector (i.e., prediction mode (YZ)), a predicted value to be used in the prediction mode.

[0583] 64, Yave can be calculated as the average or weighted average of Yb1, Ya2, Ya1, and Ya0, and Zave can be calculated as the average or weighted average of Zb1, Za2, Za1, and Za0.

[0584] The encoding device may, for example, select prediction mode 1 (in other words, a prediction mode whose prediction mode value is 1) as the prediction mode (YZ), and encode the Y and Z components of the motion vector of the three-dimensional point to be encoded using predicted values ​​Yb1 and Zb1.

[0585] When the encoding device selects prediction mode values ​​for the X component and the YZ component as described above, the encoding device may add the prediction mode values ​​for the X component and the YZ component to the bitstream.

[0586] The encoding device may select a prediction mode during encoding by residual optimization, as described below.

[0587] For example, the encoding device can calculate the cost (P) when various prediction modes P are selected, and select the prediction mode that minimizes cost (P). The encoding device may calculate the cost (P) using, for example, the number of bits bit (P) required to encode the prediction residual residual (P) and the prediction mode value P when the prediction value of the prediction mode P is used, and the adjustment parameter λ value, according to the following (Equation 19).

[0588] cost(P) = abs(residual(P)) + λ×bit(P) (Equation 19)

[0589] In equation (19), abs(x) means the absolute value of x. Note that the square of x may be used instead of abs(x).

[0590] By using (Equation 19), the encoding device can select a prediction mode that takes into consideration the balance between the magnitude of the prediction residual and the number of bits required to encode the prediction mode value.

[0591] The encoding device may change the value of the adjustment parameter λ depending on the value of the quantization scale. For example, when the quantization scale is small (in other words, when the bit rate is high), the encoding device may select a prediction mode that reduces the prediction residual residual (P) by reducing the λ value. This makes it possible to improve prediction accuracy as much as possible.

[0592] In addition, the encoding device may increase the λ value when the quantization scale is large (in other words, when the bit rate is low) so as to select an appropriate prediction mode while taking into account the number of bits bit(P) required to encode the prediction mode value P.

[0593] The encoding device calculates the prediction residual residual (P) by subtracting the predicted value of prediction mode P from the motion vector of the 3D point to be encoded. Note that the encoding device may quantize and then dequantize the prediction residual residual (P), instead of the prediction residual residual (P) used when calculating the cost, and add this to the predicted value to obtain a decoded value, and may reflect the difference (encoding error) between the motion vector of the original 3D point and the decoded value when prediction mode P is used in the cost value. This allows the encoding device to select a prediction mode with a small encoding error.

[0594] Note that the calculation method of the cost (P) when selecting a prediction mode is not limited to the above, and any method may be used. For example, the encoding device may set the cost (P) to a value obtained by adding the number of bits required to encode the prediction residual (P) and bit (P). This allows the encoding device to select a prediction mode that minimizes the number of bits required for encoding, thereby reducing the amount of code, and in other words, potentially improving encoding efficiency. Note that the number of bits required to encode the prediction residual (P) may also be the amount of code obtained when arithmetically encoding the binarized data of the prediction residual (P). This allows the encoding device to calculate a more accurate number of required bits using the prediction residual (P), thereby making it possible to select a more appropriate prediction mode.

[0595] For example, when the prediction mode value is binarized and encoded, the encoding device can use the number of bits after binarization as the number of bits (P) required to encode the prediction mode value P. For example, when the number of prediction modes M is 5, the prediction mode value may be binarized using a truncated unary code with a maximum value of 5, as shown in FIG. 65 . In this case, the number of bits (P) required to encode the prediction mode value 0 may be 1 bit, the number of bits (P) required to encode the prediction mode value 1 may be 2 bits, the number of bits (P) required to encode the prediction mode value 2 may be 3 bits, and the number of bits (P) required to encode the prediction mode values ​​3 and 4 may be 4 bits. By using the truncated unary code, it is possible to reduce the number of bits required to encode the prediction mode value P as the prediction mode value becomes smaller.

[0596] In other words, for example, if the average value is assigned to prediction mode 0, and for prediction modes 1 to 4, the smaller the distance from the three-dimensional point to be encoded, the smaller the prediction mode value assigned to that three-dimensional point, and there is a tendency for smaller prediction mode values ​​to be more easily selected, it may be possible to reduce the amount of coding.

[0597] Furthermore, when the maximum value of the prediction mode value has not been determined, the encoding device may binarize the prediction mode value using a unary code as shown in FIG.

[0598] Furthermore, when the occurrence probabilities of each prediction mode are relatively close, the encoding device may binarize the prediction mode value using a fixed code, as shown in Fig. 67. This may enable a reduction in the amount of code.

[0599] The encoding device may arithmetically encode the binarized data of the prediction mode P, using bit(P) as the number of bits required to encode the prediction mode P, and may set the code amount after arithmetic encoding as the value of bit(P). This allows the cost to be calculated using a more accurate number of required bits, bit(P), making it possible to select a more appropriate prediction mode.

[0600] Fig. 68 is an explanatory diagram showing examples of prediction modes and binarized data in this embodiment. Fig. 69 is a flow diagram showing an example of encoding processing of prediction mode values ​​in this embodiment. Fig. 70 is a flow diagram showing an example of decoding processing of prediction mode values ​​in this embodiment. Binarization and arithmetic coding of prediction mode values ​​will be described with reference to Figs. 68 to 70.

[0601] As shown in Fig. 69, the encoding device binarizes the prediction mode value (PredMode) (step S1201), then performs arithmetic coding (step S1202) and adds the binarized value to a bitstream. The encoding device may binarize the prediction mode value with a truncated unary code using the value of the number of prediction modes M, for example. In this case, the maximum number of bits after binarization is M-1.

[0602] As shown in FIG. 70 , the decoding device generates binarized data of a truncated unary code by arithmetically decoding a bitstream using the number of prediction modes M (step S1301), and can calculate a prediction mode value from the binarized data of the truncated unary code (step S1302).

[0603] The encoding device may perform arithmetic encoding of the binarized data using a context (also called a coding table). In this case, the encoding device may improve encoding efficiency by, for example, switching the context for each bit of the binarized data.

[0604] Furthermore, in order to reduce the number of contexts, the encoding device may encode the first bit (also referred to as "one bit" or "one bit portion") of the binarized data using context A, and encode the remaining bits (also referred to as "remaining bit" or "remaining bit portion") using context B. Context A is also referred to as a context for one bit. Context B is also referred to as a context for the remaining bit. This makes it possible to improve encoding efficiency by switching contexts according to bit positions while reducing the number of encoding tables. Note that when encoding the remaining bit, arithmetic encoding and decoding may be performed by switching contexts for each bit.

[0605] For example, the encoding device can arithmetically encode a prediction mode value binarized using a truncated unary code by switching contexts between the one-bit portion and the remaining-bit portion. The occurrence probabilities of 0 and 1 in each context may be updated according to the value of the binarized data that actually occurs. Alternatively, the occurrence probabilities of 0 and 1 in either context may be fixed, reducing the number of updates to the occurrence probabilities and reducing the amount of processing. For example, the encoding device may update the occurrence probabilities for the one-bit portion and fix the occurrence probabilities for the remaining-bit portion.

[0606] For example, as shown in FIG. 68, when prediction mode 3 is selected, one bit 1 is arithmetically coded using the context for one bit, and the remaining bit 110 is arithmetically coded using the context for the remaining bit.

[0607] When the encoding device binarizes and encodes the prediction mode value using a truncated unary code using the prediction mode number M, the prediction mode number M may be added to the header of the bitstream or the like so that the decoding device can identify the prediction mode from the decoded binarized data. The encoding device may also specify the value MaxM, which is the possible value of the prediction mode number, in a standard or the like, and add the value MaxM-M (where M≦MaxM) to the header. The encoding device may also specify the value in the profile or level of a standard or the like without adding the prediction mode number M to the stream.

[0608] Fig. 71 is an explanatory diagram showing an example of predicted value information of a motion vector in this embodiment. Fig. 72 is an explanatory diagram showing an example of a prediction mode and binarized data in this embodiment. Fig. 73 is a flow diagram showing an example of a coding process of a prediction mode value in this embodiment. Fig. 74 is a flow diagram showing an example of a decoding process of a prediction mode value in this embodiment. The binarization and arithmetic coding of prediction mode values ​​will be described with reference to Figs. 71 to 74.

[0609] Although an example of binarizing the prediction mode value (PredMode) using a truncated unary code by using the value of the number of prediction modes M has been shown, the binarization method is not necessarily limited to this. For example, the encoding device may binarize the prediction mode value by a truncated unary code by using the number L (where L≦M) of prediction modes to which prediction values ​​are assigned.

[0610] For example, when the number of prediction modes M is 5 and there are two adjacent points available for predicting the three-dimensional point to be coded, there are cases where three prediction modes are available (also referred to as "available") and two are unavailable (referred to as "not available"). In the example shown in Figure 71, prediction modes with prediction mode values ​​of 0, 1, and 2 are available, and prediction modes with prediction mode values ​​of 3 and 4 are not available.

[0611] In this case, the encoding device may be able to reduce the number of bits of binarized data more than when binarizing the prediction mode value with a truncated unary code with the number of prediction modes M set to M by binarizing the prediction mode value with a truncated unary code by using a maximum value of L, the number of prediction modes to which the prediction modes are assigned.

[0612] In the example shown in FIG. 72, three prediction modes (prediction modes 0, 1, and 2) are binarized using a truncated unary code.

[0613] In this way, the encoding device may reduce the number of bits of the binarized data of the prediction mode value by binarizing the prediction mode value using a truncated unary code with the number L of prediction modes to which the prediction value is assigned as the maximum value.

[0614] The encoding device may also perform arithmetic coding of the binarized data using a context. In this case, for example, the encoding efficiency may be improved by switching the context for each bit of the binarized data. Furthermore, in order to reduce the number of contexts, the encoding device may encode the leading bit (one bit) of the binarized data using context A and encode the remaining bits (remaining bits) using context B. For example, as shown in FIG. 72, when prediction mode 2 is selected, one bit (1) is arithmetic coded using the context for one bit, and the remaining bits (1) are arithmetic coded using the context for the remaining bits (see FIG. 72). This reduces the number of contexts while switching contexts according to the bit position, thereby improving encoding efficiency. Note that when encoding the remaining bits, arithmetic coding and decoding may be performed by switching the context for each bit.

[0615] Furthermore, when the encoding device performs binarization and encoding using a truncated unary code using the number L of prediction modes to which prediction values ​​are assigned, the decoding device may calculate the number L by assigning prediction values ​​to prediction modes in the same manner as when the encoding device performs encoding, and may decode the prediction mode values, so that the prediction mode can be identified from the decoded binarized data.

[0616] As shown in FIG. 73 , the encoding device calculates the number L of prediction modes to which prediction values ​​are assigned (step S1301), binarizes the prediction mode values ​​with a truncated unary code using the calculated L (step S1302), and arithmetically encodes the binarized data of the truncated unary code (step S1303).

[0617] As shown in FIG. 74 , the decoding device calculates the number L of prediction modes to which prediction values ​​are assigned (step S1401), generates binarized data of the truncated unary code by arithmetically decoding the bitstream using the calculated L (step S1402), and calculates a prediction mode value from the binarized data of the truncated unary code (step S1403).

[0618] Fig. 75 is a flow diagram showing an example of a process in which the encoding device in this embodiment determines a prediction mode. Fig. 76 is an explanatory diagram showing an example of a process in which the encoding device in this embodiment calculates a maximum absolute difference value of a motion vector. Fig. 77 is a flow diagram showing an example of a process in which the decoding device in this embodiment determines a prediction mode.

[0619] The encoding device does not need to add a prediction mode to every motion vector. For example, the encoding device may fix the prediction mode when a certain condition is met so as not to add the prediction mode to the bitstream, and may select a prediction mode from among multiple candidate prediction modes and encode the selected prediction mode into the bitstream when the condition is not met.

[0620] For example, when a certain condition A is satisfied, the encoding device may fix the prediction mode to prediction mode 0 and calculate a predicted value from the average value of adjacent points, while when condition A is not satisfied, the encoding device may select a prediction mode from among multiple candidate prediction modes and encode the selected prediction mode into a bitstream.

[0621] As a certain condition A, for example, a condition including the maximum absolute difference value maxdiff of the motion vectors a [0] to a [N-1] of N adjacent points (already coded and decoded) for the three-dimensional point to be coded can be used. Specifically, as a certain condition A, when the above-mentioned maximum absolute difference value maxdiff is smaller than a threshold value Thfix, the prediction mode is fixed to prediction mode 0, and when not, the prediction mode is selected from a plurality of prediction mode candidates and coded.

[0622] In this way, when the maximum absolute difference value of the motion vectors of adjacent points is smaller than the threshold value, the encoding device fixes the prediction mode to prediction mode 0 (i.e., a prediction mode that uses the average value as a predicted value) and does not encode the prediction mode, thereby making it possible to generate an appropriate predicted value without generating a code amount for encoding the prediction mode. This is based on the fact that the difference between the motion vectors of each three-dimensional point is relatively small, and even if another prediction mode is selected, the difference caused in the predicted value is considered to be relatively small.

[0623] Although the above example shows that the prediction mode is fixed to prediction mode 0, the present invention is not limited to this. For example, if a prediction mode that uses an average value as a predicted value is assigned to prediction mode 1, the prediction mode may be fixed to prediction mode 1.

[0624] The N neighboring 3D points used for prediction may be N encoded and decoded 3D points whose distance from the encoding target 3D point is less than a threshold value THd. The encoding device may add the maximum value of N to the bitstream as NumNeiCnt. Note that, if the number of neighboring points is less than the value of NumNeiCnt, the value of N does not always have to match the value of NumNeiCnt.

[0625] The processing of the encoding device will be described with reference to FIGS.

[0626] In step S1501, the encoding device calculates the maximum absolute difference value maxdiff of the motion vectors of N adjacent points adjacent to the 3D point to be encoded. An example of the process of calculating the maximum absolute difference value maxdiff using the motion vectors a[0] to a[N-1] of the N adjacent points adjacent to the 3D point to be encoded is shown in Figure 76. Note that in Figure 76, the motion vectors of the adjacent points are motion vectors that have already been coded and decoded.

[0627] In step S1502, the encoding device determines whether the maximum absolute difference value maxdiff calculated in step S1501 is smaller than the threshold value Thfix. If it is determined that the maximum absolute difference value maxdiff is smaller than the threshold value Thfix (Yes in step S1502), the process proceeds to step S1503; otherwise (No in step S1502), the process proceeds to step S1504. Note that the encoding device may encode the threshold value Thfix and add it to the header of the stream, etc.

[0628] In step S1503, the encoding device determines the prediction mode to be prediction mode 0 (i.e., a prediction mode using the average value).

[0629] In step S1504, the encoding device selects a prediction mode.

[0630] In step S1505, the encoding device arithmetically encodes the prediction mode value of the prediction mode selected in step S1504.

[0631] The encoding device may be configured to add the threshold Thfix to the header of the bitstream, or may be configured to change the threshold Thfix when encoding. For example, the encoding device may add a smaller threshold Thfix to the header when encoding at a high bitrate, thereby increasing the number of cases in which a prediction mode is selected and encoded, thereby encoding to minimize the prediction residual. Furthermore, the encoding device may add a larger threshold Thfix to the header when encoding at a low bitrate, thereby increasing the number of cases in which a prediction mode is fixed and encoded, thereby improving encoding efficiency while reducing the amount of bits required to encode the prediction mode. Furthermore, the encoding device may not add the threshold Thfix to the bitstream, but may instead specify a value in a profile or level of a standard, etc.

[0632] The processing of the decoding device will be described with reference to FIG.

[0633] In step S1601, the decoding device calculates the maximum absolute difference value maxdiff of the motion vectors of N adjacent points adjacent to the 3D point to be decoded.

[0634] In step S1602, the decoding device determines whether the maximum absolute difference value maxdiff calculated in step S1601 is smaller than the threshold value Thfix. If it is determined that the maximum absolute difference value maxdiff is smaller than the threshold value Thfix (Yes in step S1602), the process proceeds to step S1603; if not (No in step S1602), the process proceeds to step S1604. Note that the decoding device may set the threshold value Thfix by decoding a stream header or the like.

[0635] In step S1603, the decoding apparatus determines the prediction mode to be prediction mode 0 (i.e., a prediction mode that uses the average value).

[0636] In step S1604, the decoding device decodes the prediction mode value from the bitstream.

[0637] Although an example has been shown in which the prediction mode is fixed to prediction mode 0 if the maximum absolute difference value of the motion vectors of adjacent points used for prediction is smaller than the threshold value Thfix[i], the present invention is not limited to this, and the prediction mode may be fixed to any one of prediction mode 0 to prediction mode M-1. In this case, the prediction mode value of the fixed prediction mode may be added to the bit stream.

[0638] FIG. 78 is an explanatory diagram showing an example of syntax in this embodiment.

[0639] The example syntax shown in Figure 78 shows an example of the structure of information contained in a bitstream generated by an encoding device.

[0640] The syntax shown in Figure 78 includes NumLoD, NumNeiCnt[i], NumPredMode[i], Thfix[i], and NumOfPoint[i].

[0641] NumLoD indicates the number of layers of the LoD.

[0642] NumNeiCnt[i] indicates the upper limit of the number of neighboring points used to generate a predicted value of a 3D point belonging to layer i. If the number of neighboring points M is less than NumNeiCnt[i] (i.e., M<NumNeiCnt[i]), the encoding device may calculate a predicted value using M neighboring points. Furthermore, if it is not necessary to change the value of NumNeiCnt[i] for each LoD, the encoding device may add one NumNeiCnt to the header.

[0643] NumPredMode[i] indicates the total number of prediction modes (i.e., M) used to predict the motion vector of layer i. Note that the value of MaxM, which can be taken by the number of prediction modes, may be specified by standards, etc., and the encoding device may add the value of MaxM-M (where 0<M≦MaxM) to the header as NumPredMode[i], and binarize and encode the maximum value MaxM-1 using a truncated unary code. Also, the encoding device may not add the number of prediction modes NumPredMode[i] to the stream, and the value may be specified by a profile or level of standards, etc. Also, the number of prediction modes may be NumNeiCnt[i] + NumPredMode[i]. Furthermore, if it is not necessary to change the value of NumPredMode[i] for each LoD, the encoding device may add one NumPredMode to the header.

[0644] Thfix[i] indicates a threshold value of the maximum absolute difference value for determining whether to fix the prediction mode of layer i. If the maximum absolute difference value of the motion vectors of adjacent points used for prediction is smaller than Thfix[i], the prediction mode is fixed to prediction mode 0. Note that the encoding device may not add Thfix[i] to the stream, and the value may be specified in a profile or level of a standard, etc. Also, if it is not necessary to change the value of Thfix[i] for each LoD, the encoding device may add one Thfix to the header.

[0645] NumOfPoint[i] indicates the number of 3D points belonging to layer i. Note that if the encoding device adds the total number of 3D points, AllNumOfPoint, to a separate header, NumOfPoint[NumLoD-1] (the number of 3D points belonging to the lowest layer) may not be added to the header. In this case, NumOfPoint[NumLoD-1] is calculated using the following (Equation 20). This allows the amount of code in the header to be reduced.

[0646]

[0647] In addition, as a setting example of NumPredMode[i], since the distance between the three-dimensional points belonging to LoD is relatively large, the higher the layer in which the difference between the motion vector and the predicted value is relatively large, the larger the value of NumPredMode[i] is set, and the more selectable prediction modes can be increased.In addition, the lower the layer in which the difference between the motion vector and the predicted value is relatively small, the smaller the value of NumPredMode[i] is set, and the amount of bits required for encoding the prediction mode can be reduced.With these setting examples, the higher layer can increase the selectable prediction modes to reduce the prediction residual, and the lower layer can reduce the amount of coding of the prediction mode, thereby improving encoding efficiency.

[0648] In addition, as a setting example of Thfix[i], since the distance of the three-dimensional point belonging to LoD is relatively large, the higher the layer where the difference between the motion vector and the predicted value is relatively large, the smaller the value of Thfix[i] is set, and the more cases of selecting a prediction mode can be increased.In addition, the lower the layer where the difference between the motion vector and the predicted value is relatively small, the larger the value of Thfix[i] is set, and the prediction mode is fixed, and the amount of bits required for encoding the prediction mode can be reduced.With these setting examples, the higher layer increases the cases where a prediction mode is selected, reducing prediction residual, and the lower layer fixes the prediction mode, and reduces the amount of code of the prediction mode, thereby improving encoding efficiency.

[0649] The encoding device may entropy-encode the above NumLoD, NumNeiCnt[i], NumPredMode[i], Thfix[i], and NumOfPoint[i] and add them to the header. For example, each value may be binarized and then arithmetically encoded. Alternatively, fixed-length encoding may be used to reduce the amount of processing.

[0650] FIG. 79 is an explanatory diagram showing an example of syntax in this embodiment.

[0651] The example syntax shown in Figure 79 shows an example of the structure of information contained in a bitstream generated by an encoding device.

[0652] The syntax shown in FIG. 79 may include PredMode, mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] for each of the 0th to NumLoDth layers of LoD (also referred to as the jth layer).

[0653] PredMode indicates a prediction mode for encoding or decoding the motion vector of the i-th three-dimensional point, and takes a value in the range from 0 to M-1 (where M is the total number of prediction modes). If PredMode is not included in the bitstream (in other words, if the condition of the if statement "maxdiff >= Thfix[i] && NumPredMode[i] > 1" is not satisfied), PredMode may be estimated to be 0. Note that the estimated value of PredMode is not limited to 0, and may be any value in the range from 0 to M-1. In addition, the encoding device may add an estimated value when PredMode is not included in the bitstream to a separate header or the like. In addition, PredMode may be binarized using a truncated unary code and arithmetically coded using the number of prediction modes to which the predicted value is assigned.

[0654] Note that mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] are the same as the data of the same names shown in Figure 52, so detailed description will be omitted.

[0655] 80, 81, 82, and 83 are flow charts showing examples of processing performed by the encoding device in this embodiment. Examples of processing performed by the encoding device will be described with reference to FIGS. 80, 81, 82, and 83.

[0656] The processes of steps S1701 to S1704 and steps S1706 to S1712 included in the process of the encoding device shown in FIG. 80 are similar to the processes of steps S1001 to S1004 and steps S1006 to S1012 shown in FIG.

[0657] In step S1705, the encoding device determines a predicted value of point P. Details of the processing included in step S1705 will be described with reference to FIG.

[0658] In step S1801 shown in FIG. 81, the encoding device calculates a weighted average value of motion vectors of N adjacent points that can be used for prediction, and assigns the calculated weighted average value to prediction mode 0.

[0659] In step S1802, the encoding device calculates the maximum absolute difference value maxdiff of the motion vectors of N adjacent points.

[0660] In step S1803, the encoding device determines whether the maximum absolute difference value maxdiff calculated in step S1802 is smaller than the threshold value Thfix. If it is determined that the maximum absolute difference value maxdiff is smaller than the threshold value Thfix (Yes in step S1803), the process proceeds to step S1804; if not (No in step S1803), the process proceeds to step S1805.

[0661] In step S1804, the encoding device determines the prediction mode to be prediction mode 0 (i.e., a prediction mode using the average value).

[0662] In step S1805, the encoding device determines the prediction mode by selection. The processes included in step S1805 will be described in detail later.

[0663] In step S1806, the encoding device arithmetically encodes the prediction mode value of the prediction mode selected in step S1805. Note that the encoding device may binarize the prediction mode value PredMode with a truncated unary code using the number of prediction modes to which the prediction value is assigned, and then perform arithmetic encoding.

[0664] The prediction mode determined in step S1804 or S1805 is used to determine a predicted value in step S1705 (see FIG. 80).

[0665] The processing included in step S1805 will be described with reference to FIG.

[0666] In step S1811 shown in FIG. 82, the encoding device assigns the motion vectors of the N adjacent points to prediction modes 1 to N in order of shortest distance from the 3D point to be encoded. As a result, the encoding device generates N+1 prediction modes. Note that if N+1 exceeds the maximum number of prediction modes M (NumPredMode) added to the bitstream, the encoding device may generate M prediction modes from prediction mode 1 to prediction mode M (in other words, it may not generate prediction modes after prediction mode M+1).

[0667] In step S1812, the encoding device calculates the cost of each prediction mode and selects the prediction mode with the smallest cost. The prediction mode with the smallest cost is the prediction mode whose cost matches the smallest cost calculated by the process shown in Fig. 83. The selected prediction mode corresponds to the prediction mode selected in step S1805 (see Fig. 81).

[0668] The processing included in step S1812 will be described with reference to FIG.

[0669] 83 , the encoding device assigns 0 to the variable i and assigns infinity (also written as "∞") to the variable mincost. Note that when implemented as a program, infinity can be substituted with a very large numerical value (more specifically, the maximum value or a value close to the maximum value allowed for the variable type used).

[0670] In step S1822, the encoding apparatus calculates the cost cost[i] of the prediction mode value PredMode[i] of the i-th prediction mode.

[0671] In step S1823, the encoding device determines whether the cost cost[i] calculated in step S1822 is smaller than the variable mincost. If it is determined that the cost cost[i] is smaller than the variable mincost (Yes in step S1823), the encoding device proceeds to step S1824; if not (No in step S1823), the encoding device proceeds to step S1825.

[0672] In step S1824, the encoding apparatus assigns the cost cost[i] to the variable mincost, and sets the prediction mode to the prediction mode whose prediction mode value is PredMode[i].

[0673] In step S1825, the encoding device assigns a value obtained by adding 1 to the variable i to the variable i.

[0674] In step S1826, the encoding device determines whether or not the variable i is smaller than the number of prediction modes. If it is determined that the variable i is smaller than the number of prediction modes (Yes in step S1826), the process proceeds to step S1822; if not (No in step S1826), the process ends. The variable mincost at the time of ending the process shown in Fig. 83 indicates the minimum cost, and is used in step S1812 (see Fig. 82).

[0675] 84, 85 and 86 are flow charts showing an example of the processing performed by the decoding device in this embodiment. An example of the processing performed by the decoding device will be described with reference to FIGS. 84, 85 and 86.

[0676] In step S1901 shown in FIG. 84, the decoding device generates one or more LoDs from the input bitstream (see FIGS. 48 and 49).

[0677] In step S1902, the decoding device performs a start process of loop A, which repeatedly executes the processes of steps S1903 to S1909 described below. In loop A, the decoding device focuses on each of the one or more LoDs generated in step S1901, executes processing for the focused LoD, and controls so that processing is ultimately performed for all LoDs. The focused LoD is also referred to as a focused LoD. Loop A may also be called a LoD loop.

[0678] In step S1903, the decoding device starts a loop B, which repeatedly executes steps S1904 to S1908 described below. In loop B, the device focuses on each of the 3D points belonging to the LoD of interest, executes processing for the 3D points of interest, and controls the device so that processing is ultimately performed for all 3D points. The 3D point of interest is also referred to as point P.

[0679] In step S1904, the decoding device searches for adjacent points of point P (see FIGS. 36 to 45).

[0680] In step S1905, the decoding device determines a predicted value of point P. The processing included in step S1905 will be described in detail later.

[0681] In step S1906, the decoding device decodes the quantized value of the point P.

[0682] In step S1907, the decoding device calculates the inverse quantized value of the point P. Specifically, the decoding device determines the inverse quantized value of the point P by using the quantized value decoded in step S1906 as a prediction residual.

[0683] In step S1908, the decoding device calculates a reconstructed value of the point P using the inverse quantized value acquired in step S1907.

[0684] In step S1909, the decoding device performs the process of ending loop B. Specifically, the decoding device determines whether or not the processes of S1904 to S1908 have been executed for all 3D points belonging to the LoD of interest, and if not, controls so that the processes are executed by focusing on 3D points that have not yet been executed.

[0685] In step S1910, the decoding device performs the process of ending loop A. Specifically, the decoding device determines whether the processes of steps S1903 to S1909 have been executed for all LoDs, and if not, controls the process to be executed focusing on the LoDs that have not yet been executed.

[0686] The process included in step S1905 in FIG. 84 will be described in detail below.

[0687] In step S2001 shown in FIG. 85, the decoding device calculates a weighted average value of motion vectors of N adjacent points that can be used for prediction, and assigns the calculated weighted average value to prediction mode 0.

[0688] In step S2002, the decoding device calculates the maximum absolute difference value maxdiff of the motion vectors of N adjacent points.

[0689] In step S2003, the decoding device determines whether the maximum absolute difference value maxdiff calculated in step S2002 is smaller than the threshold value Thfix. If it is determined that the maximum absolute difference value maxdiff is smaller than the threshold value Thfix (Yes in step S2003), the process proceeds to step S2004. If not (No in step S2003), the process proceeds to step S2005.

[0690] In step S2004, the decoding device determines the prediction mode to be prediction mode 0 (i.e., a prediction mode that uses the average value).

[0691] In step S2005, the decoding device determines the prediction mode to be the prediction mode indicated by the prediction mode value decoded from the bitstream. The processes included in step S2005 will be described in detail later.

[0692] The prediction mode determined in step S2004 or S2005 is used to determine a prediction value in step S1905 (see FIG. 84).

[0693] The processing included in step S2005 will be described with reference to FIG.

[0694] In step S2011 shown in FIG. 86, the decoding device assigns the motion vectors of the N adjacent points to prediction modes 1 to N in order of shortest distance from the 3D point to be decoded. This allows the decoding device to generate N+1 prediction modes. Note that if N+1 exceeds the maximum number of prediction modes M (NumPredMode) added to the bitstream, the decoding device may generate M prediction modes from prediction mode 1 to prediction mode M (in other words, it may not generate prediction modes after prediction mode M+1).

[0695] In step S2012, the decoding device arithmetically decodes the prediction mode value using the number of prediction modes to which the prediction value is assigned. The prediction mode value obtained by arithmetic decoding in step S2012 corresponds to the prediction mode value obtained in step S2005 (see FIG. 85 ).

[0696] 87 is a block diagram showing an example of the configuration of the encoding unit 1000 according to this embodiment. The encoding unit 1000 is provided in an encoding device, and outputs a bitstream obtained by encoding input 3D points.

[0697] 87 , the encoding unit 1000 includes an LoD generation unit 1001, an adjacent search unit 1002, a prediction unit 1003, a residual calculation unit 1004, a quantization unit 1005, an arithmetic coding unit 1006, an inverse quantization unit 1007, a reconstruction unit 1008, and a memory 1009. At least some of the LoD generation unit 1001, the adjacent search unit 1002, the prediction unit 1003, the residual calculation unit 1004, the quantization unit 1005, the arithmetic coding unit 1006, the inverse quantization unit 1007, and the reconstruction unit 1008 can be realized by a processor (such as a CPU (Central Processing Unit)) included in the encoding device executing a program using the memory.

[0698] The LoD generation unit 1001 generates LoD using position information of a 3D point. Specifically, the LoD generation unit 1001 receives input of position information of a 3D point (also referred to as an input 3D point) and generates LoD using the position information of the input 3D point. Note that if the LoD hierarchy is one hierarchy, that is, if a predicted value of a motion vector of a 3D point is generated without generating LoD, the encoding unit 1000 does not need to include the LoD generation unit 1001. In this case, the encoding unit 1000 may perform processing with LoD = 1 in the LoD loop (see, for example, FIG. 55). In other words, the encoding unit 1000 may only perform processing with LoD 1 as the LoD of interest. This reduces processing time.

[0699] The neighbor search unit 1002 calculates neighboring points for each of the input 3D points input to the LoD generation unit 1001 .

[0700] The prediction unit 1003 generates a predicted value of a motion vector of a 3D point to be coded. The prediction unit 1003 assigns the predicted value to prediction mode 0 to prediction mode M-1 as candidate prediction modes to be used for coding. The prediction unit 1003 also selects a prediction mode to be used for predicting the 3D point to be coded from prediction mode 0 to prediction mode M-1, and provides the prediction mode value of the selected prediction mode to the arithmetic coding unit 1006 and the reconstruction unit 1008.

[0701] The residual calculation unit 1004 generates a prediction residual of the motion vector of the 3D point to be coded.

[0702] The quantization unit 1005 quantizes the prediction residual of the motion vector of the 3D point to be coded.

[0703] The arithmetic coding unit 1006 generates a bitstream by arithmetically coding the prediction residual after quantization by the quantization unit 1005. The arithmetic coding unit 1006 may binarize the prediction residual before arithmetic coding and arithmetically code the binarized prediction residual. The arithmetic coding unit 1006 outputs the generated bitstream. Note that the arithmetic coding unit 1006 may generate and code various header information. Furthermore, the arithmetic coding unit 1006 may obtain a prediction mode value of the prediction mode used for coding, arithmetically code the prediction mode value, and add it to the bitstream.

[0704] The inverse quantization unit 1007 inverse quantizes the prediction residuals quantized by the quantization unit 1005 .

[0705] The reconstruction unit 1008 reconstructs the motion vector or position information of the three-dimensional point to be coded by adding the predicted value generated by the prediction unit 1003 and the dequantized prediction residual generated by the inverse quantization unit 1007. The reconstructed motion vector or position information is also called a decoded value, and reconstructing the motion vector or position information is also called generating a decoded value. The reconstruction unit 1008 stores the generated decoded value in the memory 1009.

[0706] The memory 1009 is a storage device that stores the decoded values ​​(i.e., position information or motion vectors of 3D points) generated by the reconstruction unit 1008. The decoded values ​​stored in the memory 1009 may be used to predict 3D points that have not yet been coded.

[0707] If quantization of the prediction residual of the motion vector is not necessary, the quantization unit 1005 and the inverse quantization unit 1007 may be omitted, and the reconstruction unit 1008 may add the prediction residual generated by the residual calculation unit 1004 directly to the predicted value to obtain the decoded value, thereby reducing the processing time.

[0708] 88 is a block diagram showing an example of the configuration of the decoding unit 1100 according to this embodiment. The decoding unit 1100 is provided in a decoding device, and outputs 3D points obtained by decoding an input bitstream.

[0709] 88, the decoding unit 1100 includes an LoD generation unit 1101, an adjacent search unit 1102, a prediction unit 1103, an arithmetic decoding unit 1104, an inverse quantization unit 1105, a reconstruction unit 1106, and a memory 1107. At least some of the LoD generation unit 1101, the adjacent search unit 1102, the prediction unit 1103, the arithmetic decoding unit 1104, the inverse quantization unit 1105, and the reconstruction unit 1106 can be realized by a processor (such as a CPU) included in the decoding device executing a program using the memory.

[0710] The LoD generation unit 1101 generates LoD using position information of the three-dimensional point. Note that if the LoD layer is one layer, that is, if a predicted value of a motion vector of a three-dimensional point is generated without generating LoD, the decoding unit 1100 may not include the LoD generation unit 1101. In this case, the decoding unit 1100 may perform processing with LoD=1 in the LoD loop (see, for example, FIG. 56). In other words, the decoding unit 1100 may only perform processing with LoD1 as the LoD of interest. This can reduce processing time.

[0711] The neighbor search unit 1102 calculates neighboring points for each three-dimensional point.

[0712] The prediction unit 1103 generates a predicted value of the motion vector of the three-dimensional point to be decoded. The prediction unit 1103 obtains a prediction mode value to be used for decoding from the arithmetic decoding unit 1104.

[0713] The arithmetic decoding unit 1104 arithmetically decodes prediction residuals included in the bitstream. Note that the arithmetic decoding unit 1104 may also decode various header information. Furthermore, the arithmetic decoding unit 1104 may arithmetically decode prediction mode values ​​included in the bitstream and provide the arithmetically decoded prediction mode values ​​to the prediction unit 1103.

[0714] The inverse quantization unit 1105 inverse quantizes the prediction residuals decoded by the arithmetic decoding unit 1104 .

[0715] The reconstruction unit 1106 generates a decoded value by adding the predicted value generated by the prediction unit 1103 and the prediction residual after inverse quantization by the inverse quantization unit 1105. The reconstruction unit 1106 can output the generated decoded value as a decoded 3D point.

[0716] The memory 1107 is a storage device that stores the decoded values ​​(i.e., position information or motion vectors of decoded 3D points) generated by the reconstruction unit 1106. The decoded values ​​stored in the memory 1107 may be used to predict 3D points that have not yet been decoded.

[0717] If inverse quantization of the prediction residual of the motion vector is not necessary, the inverse quantization unit 1105 may be omitted, and the reconstruction unit 1106 may add the arithmetically decoded prediction residual directly to the predicted value to obtain the decoded value, thereby reducing the processing time.

[0718] In the explanation of Figures 75 to 77 above, an example was shown in which the encoding device and decoding device calculate the maximum absolute difference value of the motion vectors of N adjacent points that can be used for prediction, and switch between fixing the prediction mode or selecting a prediction mode from among multiple candidate prediction modes and adding it to the bitstream depending on the calculated maximum absolute difference value, but this is not necessarily limited to this.

[0719] Fig. 89 is a flow diagram showing an example of a process in which the encoding device in this embodiment determines a prediction mode. Fig. 90 is an explanatory diagram showing an example of a process in which the encoding device in this embodiment calculates a maximum absolute difference value of a motion vector. Fig. 91 is a flow diagram showing an example of a process in which the decoding device in this embodiment determines a prediction mode.

[0720] For example, the encoding device may determine whether to fix or select the prediction mode under the same conditions as above, and add the result to the bitstream as a prediction mode fixing flag. For example, the prediction mode fixing flag has a value of 1 indicating a mode in which the prediction mode is fixed, and a value of 0 indicating a mode in which the prediction mode is selected.

[0721] This allows the decoding device to determine whether the encoding device has fixed the prediction mode or selected a prediction mode and coded the prediction mode value by decoding the prediction mode fixation flag added to the bitstream. If the prediction mode is fixed by the encoding device, the decoding device can determine that the prediction mode value is not coded in the bitstream. Furthermore, if the prediction mode is selected by the encoding device, the decoding device can determine that the prediction mode of the bitstream needs to be decoded, enabling correct decoding. This allows the decoding device to arithmetically decode the prediction mode value without calculating the maximum absolute difference value of the motion vectors at the three-dimensional points of N adjacent points available for prediction, thereby enabling arithmetic decoding of the bitstream and LoD generation, etc. to be performed in parallel, thereby improving the overall throughput of the decoding process.

[0722] The encoding device may add a prediction mode fixing flag to each three-dimensional point. This allows the encoding device to switch between fixing and selecting a prediction mode for each three-dimensional point, potentially improving encoding efficiency. The encoding device may also be configured to set a prediction mode fixing flag for each LoD. For example, for an upper layer in which the difference between the motion vector and the predicted value is relatively large, the prediction mode fixing flag may be set to 0 to enable selection of a prediction mode. Furthermore, for a lower layer in which the difference between the motion vector and the predicted value is relatively small, the prediction mode fixing flag may be set to 1 to fix the prediction mode, thereby reducing the amount of code required to add the prediction mode.

[0723] The processing of the encoding device will be described with reference to FIGS.

[0724] Of the processes shown in Fig. 89, steps S2101 to S2102, S2104, and S2106 to S2107 are respectively similar to steps S1501 to S1502, S1503, and S1504 to S1505 in Fig. 75. Also, an example of the process of calculating the maximum absolute difference value maxdiff by the encoding device (see Fig. 90) is similar to the example shown in Fig. 76.

[0725] If the encoding device determines that the maximum absolute difference value maxdiff is smaller than the threshold value Thfix (Yes in step S2102), it sets the prediction mode fixation flag to 1 and performs arithmetic encoding of the prediction residual (step S2103), and then determines the prediction mode to be prediction mode 0.

[0726] Furthermore, if the encoding device determines that the maximum absolute difference value maxdiff is not smaller than the threshold value Thfix (No in step S2102), it sets the prediction mode fixation flag to 0 and performs arithmetic encoding of the prediction residual (step S2105), then selects a prediction mode and arithmetic encodes the prediction mode value of the selected prediction mode (steps S2106, S2107).

[0727] The processing of the decoding device will be described with reference to FIG.

[0728] In step S2201, the decoding apparatus arithmetically decodes the prediction mode fixing flag.

[0729] In step S2202, the decoding device determines whether or not the prediction mode fixing flag is 1. If it is determined that the prediction mode fixing flag is 1 (Yes in step S2202), the process proceeds to step S2203; otherwise (No in step S2202), the process proceeds to step S2204.

[0730] In step S2203, the decoding device determines the prediction mode to be prediction mode 0 (i.e., a prediction mode using the average value).

[0731] In step S2204, the decoding device decodes the prediction mode value from the bitstream.

[0732] Note that, although an example has been shown in which the prediction mode is fixed to prediction mode 0 if the maximum absolute difference value of the motion vectors of adjacent points used for prediction is smaller than the threshold value Thfix[i], the present invention is not limited to this, and the prediction mode may be fixed to any one of prediction mode 0 to prediction mode M-1. Furthermore, the prediction mode value of the fixed prediction mode may be added to the bitstream.

[0733] Fig. 92 is an explanatory diagram showing an example of syntax in this embodiment. The example of syntax shown in Fig. 92 is an example of syntax when a fixed prediction mode (also called a fixed prediction mode) is provided for each 3D point.

[0734] The syntax shown in FIG. 92 may include fixedPredMode, PredMode, mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] for each of the 0th to NumLoDth layers of LoD (also referred to as the jth layer).

[0735] FixedPredMode is a flag indicating whether to fix the prediction mode. For example, a value of 1 may indicate that the prediction mode is fixed, and a value of 0 may indicate that the prediction mode is selected. FixedPredMode indicates whether to fix the prediction mode for encoding or decoding the motion vector of the i-th three-dimensional point. Note that fixedPredMode may be set for each LoD layer (see FIG. 93).

[0736] PredMode indicates a prediction mode for encoding or decoding the motion vector of the i-th three-dimensional point. PredMode takes a value in the range from 0 to M-1 (where M is the total number of prediction modes). If PredMode is not included in the bitstream (in other words, if the if condition "!fixedPredMode && NumPredMode[i]>1" is not satisfied), PredMode may be estimated to be 0. Note that the estimated value of PredMode is not limited to 0, and may be any value in the range from 0 to M-1. Furthermore, the encoding device may add an estimated value when PredMode is not included in the bitstream to a separate header or the like. Furthermore, PredMode may be binarized using a truncated unary code and arithmetically coded using the number of prediction modes to which the predicted value is assigned.

[0737] The encoding device may encode the value of the total number M of prediction modes as NumPredMode in the header. This allows the decoding device to decode NumPredMode in the header to calculate the total number M of prediction modes, and decode PredMode using the total number M of prediction modes. This allows the decoding device to generate LoD, calculate adjacent points available for prediction, and perform arithmetic decoding of the bitstream without waiting for the calculation of the number of prediction modes to which prediction values ​​are assigned. This allows the decoding device to perform arithmetic decoding of the bitstream and LoD generation, etc. in parallel, thereby improving the overall throughput of the decoding process.

[0738] Since mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] are the same as the data of the same names shown in Figure 52, detailed description will be omitted.

[0739] Fig. 93 is an explanatory diagram showing an example of syntax in this embodiment. The example of syntax shown in Fig. 93 is an example of syntax when a fixed prediction mode is provided for each LoD layer.

[0740] The syntax shown in FIG. 93 may include fixedPredMode, PredMode, mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] for each of the 0th to NumLoDth layers of LoD (also referred to as the jth layer).

[0741] fixedPredMode is a flag indicating whether the prediction mode is fixed. For example, a value of 1 may indicate that the prediction mode is fixed, and a value of 0 may indicate that the prediction mode is selected. fixedPredMode is set for each LoD layer.

[0742] PredMode is the same as the data of the same name shown in FIG. 92, so a detailed description will be omitted.

[0743] Since mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] are the same as the data of the same names shown in Figure 52, detailed description will be omitted.

[0744] FIG. 94 is an explanatory diagram showing an example of the encoding process of the prediction mode value in this embodiment.

[0745] In step S2301, the encoding device uses the total number M of prediction modes to binarize the prediction mode value with a truncated unary code.

[0746] In step S2302, the encoding device arithmetically encodes the binarized data of the truncated unary code obtained by the binarization in step S2301.

[0747] In step S2303, the encoding device adds the total number M of prediction modes as NumPredMode to the header and performs encoding.

[0748] FIG. 95 is an explanatory diagram showing an example of the process of decoding prediction mode values ​​in this embodiment.

[0749] In step S2401, the decoding device decodes NumPredMode in the header of the acquired bitstream and sets the total number M of prediction modes.

[0750] In step S2402, the decoding device arithmetically decodes PredMode using the total number M of prediction modes, and generates binarized data of a truncated unary code.

[0751] In step S2403, the decoding device calculates a prediction mode value from the binarized data of the truncated unary code.

[0752] Fig. 96 is a flowchart showing an example of processing performed by the encoding device according to this embodiment, illustrating detailed processing included in step S1705 (see Fig. 80).

[0753] In step S2501, the encoding device calculates a weighted average value of motion vectors of N adjacent points that can be used for prediction, and assigns the calculated weighted average value to prediction mode 0.

[0754] In step S2502, the encoding device calculates the maximum absolute difference value maxdiff of the motion vectors of N adjacent points.

[0755] In step S2503, the encoding device determines whether the maximum absolute difference value maxdiff calculated in step S2502 is smaller than the threshold value Thfix. If it is determined that the maximum absolute difference value maxdiff is smaller than the threshold value Thfix (Yes in step S2503), the process proceeds to step S2504. If not (No in step S2503), the process proceeds to step S2506.

[0756] In step S2504, the encoding device sets the prediction mode fixing flag to 1 and performs arithmetic encoding of the prediction residual.

[0757] In step S2505, the encoding device determines the prediction mode to be prediction mode 0 (i.e., a prediction mode using the average value).

[0758] In step S2506, the encoding device sets the prediction mode fixing flag to 0 and performs arithmetic encoding of the prediction residual.

[0759] In step S2507, the encoding device determines the prediction mode by selection. The processes included in step S2507 will be described in detail later.

[0760] In step S2508, the encoding device arithmetically encodes the prediction mode value of the prediction mode selected in step S2507. Note that the encoding device may binarize the prediction mode value PredMode using a truncated unary code using the total number M of prediction modes and then perform arithmetic encoding. The encoding device may encode the total number M of prediction modes as NumPredMode and add it to the header. This allows the decoding device to correctly decode the prediction mode PredMode by decoding NumPredMode in the header. Note that when NumPredMode = 1, PredMode does not need to be encoded. This allows the amount of code when NumPredMode = 1 to be reduced.

[0761] The prediction mode determined in step S2505 or S2508 is used to determine a predicted value in step S1705 (see FIG. 80).

[0762] The processing included in step S2507 will be described with reference to FIG.

[0763] In step S2511 shown in FIG. 97, the encoding device assigns the motion vectors of the N adjacent points to prediction modes 1 to N in order of the shortest distance from the 3D point to be encoded. This allows the encoding device to generate N+1 prediction modes. Note that if N+1 exceeds the maximum number of prediction modes M (NumPredMode) added to the bitstream, the encoding device may generate M prediction modes from prediction mode 1 to prediction mode M (in other words, it may not generate prediction modes after prediction mode M+1).

[0764] In step S2512, the encoding device calculates the cost of each prediction mode and selects the prediction mode with the smallest cost. The prediction mode with the smallest cost is the prediction mode whose cost matches the smallest cost calculated by the process shown in Fig. 98. The selected prediction mode corresponds to the prediction mode selected in step S2507 (see Fig. 96).

[0765] The processing included in step S2512 will be described with reference to FIG.

[0766] 98, the encoding device assigns 0 to the variable i and assigns infinity (also written as "∞") to the variable mincost. Note that when implemented as a program, infinity can be substituted with a very large numerical value (more specifically, a value that is the maximum value or close to the maximum value allowed for the variable type used).

[0767] In step S2522, the encoding device calculates the cost cost[i] of the prediction mode value PredMode[i] of the i-th prediction mode.

[0768] In step S2523, the encoding device determines whether the cost cost[i] calculated in step S2522 is smaller than the variable mincost. If it is determined that the cost cost[i] is smaller than the variable mincost (Yes in step S2523), the encoding device proceeds to step S2524; if not (No in step S2523), the encoding device proceeds to step S2525.

[0769] In step S2524, the encoding apparatus assigns the cost cost[i] to the variable mincost, and sets the prediction mode to the prediction mode whose prediction mode value is PredMode[i].

[0770] In step S2525, the encoding device assigns the value obtained by adding 1 to the variable i to the variable i.

[0771] In step S2526, the encoding device determines whether the variable i is smaller than the number of prediction modes. If it is determined that the variable i is smaller than the number of prediction modes (Yes in step S2526), ​​the process proceeds to step S2522; if not (No in step S2526), ​​the process ends. The variable mincost at the time of ending the process shown in Fig. 98 indicates the minimum cost, and is used in step S2512 (see Fig. 97).

[0772] FIG. 99 is a flow diagram showing an example of processing by the decoding device in this embodiment.

[0773] In step S2601, the decoding apparatus arithmetically decodes the prediction mode fixing flag.

[0774] In step S2602, the decoding device determines whether the prediction mode fixing flag arithmetically decoded in step S2601 is 1. If it is determined that the prediction mode fixing flag is 1 (Yes in step S2602), the process proceeds to step S2603; otherwise (No in step S2602), the process proceeds to step S2604.

[0775] In step S2603, the decoding apparatus determines the prediction mode to be prediction mode 0 (i.e., a prediction mode using the average value).

[0776] In step S2604, the decoding device determines the prediction mode to be the prediction mode indicated by the prediction mode value decoded from the bitstream. Note that the decoding device may arithmetically decode the prediction mode value PredMode using the total number M of prediction modes obtained by decoding the header. Note that if the total number M of prediction modes is 1, the prediction mode value PredMode may not be decoded and may be estimated to be PredMode = 0.

[0777] The prediction mode determined in step S2603 or S2604 is used to determine a prediction value in step S1905 (see FIG. 84).

[0778] In the explanations referring to Figures 75 to 77, an example was shown in which the encoding device and decoding device calculate the maximum absolute difference value of motion vectors at three-dimensional points of N adjacent points available for prediction, and switch between fixing the prediction mode or selecting a prediction mode from among multiple candidate prediction modes and adding it to the bitstream depending on the calculated maximum absolute difference value, but this is not necessarily limited to this.

[0779] Fig. 100 is a flow chart showing an example of a process in which the encoding device according to this embodiment determines a prediction mode. Fig. 101 is a flow chart showing an example of a process in which the decoding device according to this embodiment determines a prediction mode.

[0780] For example, the encoding device may always select a prediction mode from among a plurality of candidate prediction modes and add that prediction mode to the bitstream (steps S2601 and S2602, see FIG. 100).

[0781] In this case, the decoding device may always decode the prediction mode from the bitstream (step S2701, see Figure 101).

[0782] This allows the decoding device to always decode the prediction mode added to the bitstream, thereby correctly decoding the bitstream. Also, since the decoding device can arithmetically decode the prediction mode without calculating the maximum absolute difference value of the motion vectors at the N adjacent three-dimensional points available for prediction, it is possible to perform arithmetic decoding of the bitstream and LoD generation, etc. in parallel, thereby improving the overall throughput of the decoding process.

[0783] When the total number of prediction modes M=1, the decoding device can estimate that the prediction mode value is 0. In this case, the encoding device may not add the prediction mode value to the bitstream. This makes it possible to reduce the amount of code when the total number of prediction modes M=1.

[0784] FIG. 102 is an explanatory diagram showing an example of syntax in this embodiment.

[0785] The example syntax shown in Figure 102 shows an example of the structure of information contained in a bitstream generated by an encoding device.

[0786] The syntax shown in FIG. 102 may include PredMode, mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] for each of the 0th to NumLoDth layers of LoD (also referred to as the jth layer).

[0787] PredMode indicates a prediction mode for encoding or decoding the motion vector of the i-th three-dimensional point, and takes a value in the range from 0 to M-1 (where M is the total number of prediction modes). If PredMode is not included in the bitstream (in other words, if the condition "NumPredMode[i]>1" is not satisfied), PredMode may be estimated to be 0. Note that the estimated value of PredMode is not limited to 0, and may be any value in the range from 0 to M-1. In addition, the encoding device may add an estimated value when PredMode is not included in the bitstream to a separate header or the like. In addition, PredMode may be binarized and arithmetically coded using a truncated unary code using the number of prediction modes to which the predicted value is assigned.

[0788] Furthermore, the encoding device may encode the value of the total number M of prediction modes as NumPredMode in the header. This allows the decoding device to decode NumPredMode in the header to calculate the total number M of prediction modes, and decode PredMode using the total number M of prediction modes. Therefore, the decoding device can generate LoD, calculate adjacent points available for prediction, and perform arithmetic decoding of the bitstream without waiting for the calculation of the number of prediction modes to which prediction values ​​are assigned. This allows the decoding device to perform arithmetic decoding of the bitstream and LoD generation, etc. in parallel, thereby improving the overall throughput of the decoding process.

[0789] Note that mvd_is_zero[k], mvd_is_one[k], mvd_minus2[k], and mvd_sign[k] are the same as the data of the same names shown in Figure 52, so detailed description will be omitted.

[0790] FIG. 103 is an explanatory diagram showing an example of the encoding process of the prediction mode value in this embodiment.

[0791] In step S2801, the encoding device uses the total number M of prediction modes to binarize the prediction mode value with a truncated unary code.

[0792] In step S2802, the encoding device arithmetically encodes the binarized data of the truncated unary code obtained by the binarization in step S2801.

[0793] In step S2803, the encoding device adds the total number M of prediction modes as NumPredMode to the header and performs encoding.

[0794] FIG. 104 is an explanatory diagram showing an example of the process of decoding prediction mode values ​​in this embodiment.

[0795] In step S2901, the decoding device decodes NumPredMode in the header of the acquired bitstream and sets the total number M of prediction modes.

[0796] In step S2902, the decoding apparatus arithmetically decodes PredMode using the total number M of prediction modes to generate binarized data of a truncated unary code.

[0797] In step S2903, the decoding apparatus calculates a prediction mode value from the binarized data of the truncated unary code.

[0798] Fig. 105 is a flowchart showing an example of processing performed by the encoding device according to this embodiment, illustrating detailed processing included in step S1705 (see Fig. 80).

[0799] In step S3001, the encoding device calculates a weighted average value of motion vectors of N adjacent points that can be used for prediction, and assigns the calculated weighted average value to prediction mode 0.

[0800] In step S3002, the encoding device determines a prediction mode by selection. The encoding device can always (in other words, without making a determination on the maximum absolute difference value of the motion vector (step S1803 (see FIG. 81) or step S2503 (see FIG. 96))) select a prediction mode from among multiple candidate prediction modes and add the selected prediction mode to the bitstream. The processing included in step S3002 will be described in detail later.

[0801] In step S3003, the encoding device arithmetically encodes the prediction mode value of the prediction mode selected in step S2507. Note that the encoding device may binarize the prediction mode value PredMode using a truncated unary code using the total number M of prediction modes and then perform arithmetic encoding. The encoding device may encode the total number M of prediction modes as NumPredMode and add it to the header. This allows the decoding device to correctly decode the prediction mode PredMode by decoding NumPredMode in the header. Note that when NumPredMode = 1, PredMode does not need to be encoded. This allows the amount of code when NumPredMode = 1 to be reduced.

[0802] The prediction mode determined in step S3003 is used to determine a prediction value in step S1705 (see FIG. 80).

[0803] The processing included in step S3002 will be described with reference to FIG.

[0804] In step S3011 shown in FIG. 106, the encoding device assigns the motion vectors of the N adjacent points to prediction modes 1 to N in order of the shortest distance from the three-dimensional point to be encoded. This allows the encoding device to generate N+1 prediction modes. Note that if N+1 exceeds the maximum number of prediction modes M (NumPredMode) added to the bitstream, the encoding device may generate M prediction modes from prediction mode 1 to prediction mode M (in other words, it may not generate prediction modes after prediction mode M+1).

[0805] In step S3012, the encoding device calculates the cost of each prediction mode and selects the prediction mode with the smallest cost. The prediction mode with the smallest cost is the prediction mode whose cost matches the smallest cost calculated by the process shown in Fig. 107. The selected prediction mode corresponds to the prediction mode selected in step S3002 (see Fig. 105).

[0806] The processing included in step S3012 will be described with reference to FIG.

[0807] 107, the encoding device assigns 0 to the variable i and assigns infinity (also written as "∞") to the variable mincost. Note that when implemented as a program, infinity can be substituted with a very large numerical value (more specifically, the maximum value or a value close to the maximum value allowed for the variable type used).

[0808] In step S3022, the encoding device calculates the cost cost[i] of the prediction mode value PredMode[i] of the i-th prediction mode.

[0809] In step S3023, the encoding device determines whether the cost cost[i] calculated in step S3022 is smaller than the variable mincost. If it is determined that the cost cost[i] is smaller than the variable mincost (Yes in step S3023), the encoding device proceeds to step S3024; if not (No in step S3023), the encoding device proceeds to step S3025.

[0810] In step S3024, the encoding apparatus assigns the cost cost[i] to the variable mincost, and sets the prediction mode to the prediction mode whose prediction mode value is PredMode[i].

[0811] In step S3025, the encoding device assigns the value obtained by adding 1 to the variable i to the variable i.

[0812] In step S3026, the encoding device determines whether the variable i is smaller than the number of prediction modes. If it is determined that the variable i is smaller than the number of prediction modes (Yes in step S3026), the process proceeds to step S3022. If not (No in step S3026), the process ends. The variable mincost at the time of ending the process shown in Fig. 107 indicates the minimum cost, and is used in step S3012 (see Fig. 106).

[0813] FIG. 108 is a flowchart showing an example of processing performed by the decoding device according to this embodiment.

[0814] In step S3101, the decoding device determines the prediction mode to be the prediction mode indicated by the prediction mode value decoded from the bitstream. The decoding device can always (in other words, without making a determination on the maximum absolute difference value of the motion vector (step S2003 (see FIG. 85)) or a determination on the prediction mode fixing flag (step S2602 (see FIG. 99))) determine the prediction mode to be the prediction mode indicated by the prediction mode value decoded from the bitstream. Note that the decoding device may arithmetically decode the prediction mode value PredMode using the total number M of prediction modes obtained by decoding the header. Note that when the total number M of prediction modes is 1, the prediction mode value PredMode may not be decoded and may be estimated to be PredMode=0.

[0815] The prediction mode determined in step S3101 is used to determine a prediction value in step S1905 (see FIG. 84).

[0816] Hereinafter, examples of predicted values ​​in the case where a prediction mode to which no predicted value is assigned occurs will be described.

[0817] When an encoding device binarizes a prediction mode value (PredMode) with a truncated unary code using the total number M of prediction modes and performs arithmetic encoding, there is a possibility that a prediction mode to which a prediction value is not assigned may occur depending on the number of encoded and decoded adjacent points available for prediction. This will be described with reference to Figures 109 to 111.

[0818] Fig. 109 is an explanatory diagram showing an example of predicted value information of a motion vector in this embodiment. Fig. 110 is an explanatory diagram showing examples of prediction modes and binarized data in this embodiment. Fig. 111 is an explanatory diagram showing an example of predicted value information of a motion vector in this embodiment.

[0819] As shown in Figure 109, for example, if the total number of prediction modes M is 5 and there are two adjacent points available for predicting the three-dimensional point to be encoded, in the prediction value information held by the encoding device, the average value is assigned as the prediction value for prediction mode 0, and two three-dimensional points (i.e., points a1 and a2) are assigned as the prediction values ​​for prediction mode 1 and prediction mode 2, and prediction mode 1 and prediction mode 2 are available for encoding.

[0820] Furthermore, prediction modes 3 and 4 are not assigned predicted values, and prediction modes 3 and 2 are not available for encoding. In this case, the encoding device may set indefinite values ​​(e.g., indefinite values ​​A and B, respectively) as predicted values ​​for prediction modes 3 and 4.

[0821] Assuming that the encoding device binarizes the prediction mode value with a truncated unary code using the value of the total number of prediction modes M (i.e., 5) and encodes the prediction residual (see FIG. 110 ), it is technically possible to select prediction mode 3 or prediction mode 4. In other words, it is not excluded but rather permissible for the encoding device to select prediction mode 3 or prediction mode 4 and add the prediction mode value of the selected prediction mode to the bitstream. Therefore, it is possible that the encoding device may select prediction mode 3 or prediction mode 4 due to, for example, a malfunction or unintended processing. When the encoding device selects prediction mode 3 or prediction mode 4, it will encode the prediction mode value 3 or prediction mode value 4 and add it to the bitstream.

[0822] In this case, the decoding device decodes the prediction mode value added to the bitstream. At this time, in the prediction value information held by the decoding device, indefinite values ​​(e.g., indefinite values ​​C and D, respectively) may be set as the predicted values ​​of prediction mode 3 and prediction mode 4 (see FIG. 111). The indefinite values ​​(e.g., indefinite values ​​C and D) in the prediction value information held by the decoding device may differ from the indefinite values ​​(e.g., indefinite values ​​A and B) in the prediction value information held by the encoding device. In this case, the motion vector decoded by the decoding device may not match the motion vector coded by the encoding device. This inconsistency also affects predictions after the prediction of the motion vector, and may prevent proper decoding of the bitstream using predictions after the prediction of the motion vector.

[0823] For example, the encoding device selects prediction mode 3 and encodes the bitstream using an undefined value A as a predicted value. The encoding device also binarizes the prediction mode value 3 with a truncated unary code using a total number of prediction modes of 5, thereby obtaining binarized data "1110." The decoding device may decode the prediction mode value 3 from the binarized data "1110" and decode the bitstream using an undefined value C as a predicted value. In this case, the undefined value C used by the decoding device as a predicted value is different from the undefined value A used by the encoding device as a predicted value, and therefore the decoding device may not be able to properly decode the bitstream.

[0824] Therefore, the encoding device and the decoding device may set a common initial value in advance as a predicted value for a prediction mode to which no predicted value is assigned. This will be described with reference to Figures 112 to 114.

[0825] Fig. 112 is an explanatory diagram showing an example of predicted value information of a motion vector in this embodiment. Fig. 113 is an explanatory diagram showing an example of predicted value information of a motion vector in this embodiment.

[0826] In the prediction value information held by the encoding device (see Figure 112) and the prediction value information held by the decoding device (see Figure 113), a common initial value is set in advance as a prediction value for a prediction mode to which no prediction value is assigned.

[0827] For example, the encoding device selects prediction mode 3 and encodes the bitstream using an initial value of 0 as a predicted value. The encoding device also binarizes the prediction mode value 3 with a truncated unary code using a total number of prediction modes of 5, thereby obtaining binarized data "1110." The decoding device decodes the prediction mode value 3 from the binarized data "1110" and decodes the bitstream using the initial value of 0 as a predicted value. In this case, the initial value of 0 used by the decoding device as a predicted value matches the initial value of 0 used by the encoding device, so the decoding device can properly decode the bitstream.

[0828] By doing this, even if the encoding device generates a prediction residual using a prediction mode to which no prediction value is assigned (in other words, a prediction mode to which an initial value is set) due to a malfunction or the like and adds that prediction mode value to the bitstream, the decoding device can decode the prediction mode value from the bitstream and obtain the decoded value using the initial value common to the encoding device, thereby correctly decoding the bitstream.

[0829] The initial value may be set to, for example, 0. That is, when the motion vector is in a Cartesian coordinate system (XYZ coordinate system), the encoding device and decoding device may set the value (0, 0, 0) as the initial value.

[0830] Although the method of setting initial values ​​has been described as setting initial values ​​to prediction modes to which no predicted values ​​have been assigned, this is not necessarily limited to this. Other examples will be described with reference to Figures 115 and 116. For example, the encoding device and decoding device may set initial values ​​(e.g., value 0) to predicted values ​​of all prediction modes in advance (see Figure 114), and then update the predicted values ​​using the average value of adjacent points available for prediction or adjacent points available for prediction (see Figure 115).

[0831] The initial value is not limited to 0, and may be any value as long as it is the same value (in other words, a common value) in the encoding device and the decoding device.

[0832] Note that a bitstream encoded by selecting a prediction mode to which no prediction value is assigned (i.e., a prediction mode set to "not available") may be defined as a violation of the standard in the standard, etc. Furthermore, when a prediction mode to which no prediction value is assigned is decoded from a bitstream, the decoding device may output a violation of the standard. In this way, it is possible to prevent the decoding device from selecting a prediction mode to which no prediction value is assigned.

[0833] Hereinafter, another example of a predicted value in a case where a prediction mode to which no predicted value is assigned occurs will be described.

[0834] As shown in FIG. 109 , when the encoding device binarizes a prediction mode value (PredMode) with a truncated unary code using the total number of prediction modes M and performs arithmetic encoding, there is a possibility that a prediction mode to which no prediction value is assigned may occur depending on the number of encoded and decoded adjacent points available for prediction.

[0835] In this case, the encoding device and the decoding device may detect a prediction mode to which no prediction value is assigned, and if there is a prediction mode to which no prediction value is assigned, may assign a new prediction value to the prediction mode to which no prediction value is assigned. This can improve encoding efficiency. This will be described with reference to Figures 116 and 117.

[0836] 116 and 117 are explanatory diagrams showing examples of predicted value information of motion vectors in this embodiment.

[0837] In the prediction value information held by the encoding device (see FIG. 116 ) and the prediction value information held by the decoding device (see FIG. 117 ), a common new prediction value (denoted as “New predictor”) is set in advance as a prediction value for prediction mode 3 to which no prediction value is assigned. The new prediction value may be any value, but may be a value different from the prediction values ​​assigned to other prediction modes.

[0838] For example, the encoding device selects prediction mode 3 and encodes the bitstream using the new predicted value. The encoding device also binarizes the prediction mode value 3 with a truncated unary code using a total number of prediction modes of 5, thereby obtaining binarized data "1110." The decoding device decodes the prediction mode value 3 from the binarized data "1110" and decodes the bitstream using the new predicted value. In this case, the new predicted value used by the decoding device matches the new predicted value used by the encoding device, so the decoding device can properly decode the bitstream.

[0839] In this case, prediction modes 0 to 3 are available (denoted as "available" in the figure).

[0840] Alternatively, the new predicted value may be, for example, the median of the range of possible values ​​of the motion vector, as will be explained with reference to Figures 118 and 119.

[0841] 118 and 119 are explanatory diagrams showing examples of predicted value information of motion vectors in this embodiment.

[0842] In the prediction value information held by the encoding device (see Figure 118) and the prediction value information held by the decoding device (see Figure 119), an intermediate value is pre-set as a common new prediction value for prediction mode 3, to which no prediction value is assigned.

[0843] For example, the encoding device selects prediction mode 3 and encodes the bitstream using the intermediate value as the predicted value. The encoding device also binarizes the prediction mode value 3 with a truncated unary code using a total number of prediction modes of 5, thereby obtaining binarized data "1110." The decoding device decodes the prediction mode value 3 from the binarized data "1110" and decodes the bitstream using the intermediate value as the predicted value. In this case, the intermediate value used by the decoding device as the predicted value matches the intermediate value used by the encoding device, so the decoding device can properly decode the bitstream.

[0844] For example, if the bit precision of the X, Y, and Z components of the motion vector is 8 bits (in other words, the range of values ​​that each component can take is 0 to 255), the intermediate value for each component may be 127, which is the intermediate value of the range of 0 to 255. In this case, the predicted value of the motion vector is expressed as (127, 127, 127).

[0845] Furthermore, for example, if the bit precision of the X, Y, and Z components of the motion vector is 10-bit precision (in other words, the range of values ​​that each component can take is 0 to 1023), the intermediate value may be 511, which is the intermediate value of the range of 0 to 1023 for each component. In this case, the predicted value of the motion vector is expressed as (511, 511, 511).

[0846] The components of the motion vector can also be various combinations of the above intermediate values ​​or 0. For example, if the bit precision of the motion vector is 8 bits, the predicted value of the motion vector can be (127, 0, 0), (0, 127, 0), or (0, 127, 127), etc.

[0847] Furthermore, when a prediction mode is determined for each component of a motion vector, a prediction mode to which a prediction value is not assigned may be detected for each component, and if there is a component of a prediction mode to which a prediction value is not assigned, a new prediction value may be assigned to the component of the prediction mode to which a prediction value is not assigned. For example, if the bit precision of each component of the motion vector is 8 bits, an intermediate value of 1...

Claims

1. An encoding method comprising: calculating the number of one or more second vertices among a plurality of vertices included in a frame to be encoded, excluding a first vertex that is set as a vertex for which information relating to that vertex is not encoded; determining the number of groups that are units for encoding information relating to the one or more second vertices in encoding the frame to be encoded; and encoding, for each of the determined number of groups, first information relating to the second vertices that belong to that group among the one or more second vertices.

2. The encoding method according to claim 1, wherein the first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not encoded, and when encoding the first information, for each of the determined number of groups, information regarding the motion vector of a second vertex among the one or more second vertices that belong to the group is encoded as the first information.

3. The encoding method according to claim 2, wherein the first vertex is a vertex among the plurality of vertices that has the same position information as a vertex for which information regarding the motion vector of the vertex has already been encoded, and is set as a vertex for which information regarding the motion vector of the vertex is not encoded.

4. The encoding method according to claim 3, wherein when encoding the first information, information (NumDupPnt) indicating the number of all vertices among the plurality of vertices that have the same position information as a vertex for which information regarding the motion vector of the vertex has already been encoded is further encoded.

5. The encoding method according to claim 3, wherein when encoding the first information, the method further comprises: encoding, for each of the plurality of vertices, information (MvSigFlag) indicating whether or not the vertex is to be encoded, the vertex having the same position information as a vertex for which information regarding the motion vector of the vertex has already been encoded.

6. The encoding method according to claim 1, wherein the first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not encoded, and when encoding the first information, for each of the determined number of groups, information regarding the skip mode of a second vertex belonging to the group among the one or more second vertices is encoded as the first information.

7. A decoding method comprising: calculating the number of one or more second vertices among a plurality of vertices included in a frame to be decoded, excluding a first vertex that is set as a vertex for which information relating to that vertex is not decoded; determining the number of groups that are units for decoding information relating to the one or more second vertices in decoding the frame to be decoded; and decoding, for each of the determined number of groups, first information relating to the second vertices belonging to that group among the one or more second vertices.

8. The decoding method according to claim 7, wherein the first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not decoded, and when decoding the first information, for each of the determined number of groups, information regarding the motion vector of a second vertex among the one or more second vertices that belongs to the group is decoded as the first information.

9. The decoding method according to claim 8, wherein the first vertex is a vertex among the plurality of vertices that has the same position information as a vertex for which information regarding the motion vector of the vertex has already been decoded, and is set as a vertex for which information regarding the motion vector of the vertex is not to be decoded.

10. A decoding method according to claim 9, wherein when decoding the first information, information (NumDupPnt) indicating the number of all vertices among the plurality of vertices that have the same position information as a vertex for which information about the motion vector of the vertex has already been decoded is further decoded.

11. The decoding method according to claim 9, further comprising the step of: when decoding the first information, decoding, for each of the plurality of vertices, information (MvSigFlag) indicating whether or not the vertex is to be decoded, the information having the same position information as a vertex for which information regarding the motion vector of the vertex has already been decoded.

12. The decoding method of claim 7, wherein the first vertex is a vertex among the plurality of vertices that is set as a vertex for which information regarding the motion vector of the vertex is not decoded, and when decoding the first information, for each of the determined number of groups, information regarding the skip mode of a second vertex belonging to the group among the one or more second vertices is decoded as the first information.

13. An encoding device comprising: a memory; and a circuit capable of accessing the memory, wherein the circuit, in its operation, calculates the number of one or more second vertices among a plurality of vertices included in a frame to be encoded, excluding first vertices that are set as vertices for which information relating to the vertices is not encoded; determines the number of groups that are units for encoding information relating to the one or more second vertices in encoding the frame to be encoded; and encodes, for each of the determined number of groups, first information relating to second vertices belonging to that group among the one or more second vertices.

14. A decoding device comprising: a memory; and a circuit capable of accessing the memory, wherein the circuit, in operation, calculates the number of one or more second vertices among a plurality of vertices included in a frame to be decoded, excluding a first vertex that is set as a vertex for which information relating to that vertex is not decoded; determines the number of groups that are units for decoding information relating to the one or more second vertices in decoding the frame to be decoded; and decodes, for each of the determined number of groups, first information relating to a second vertex belonging to that group among the one or more second vertices.

Images