Image processing device and method

Abstract

The present disclosure relates to an image processing device and method, according to which partial decoding of 3D data is facilitated. Management information is generated for a block that is a partial area of ​​a specified three-dimensional space area, the specified three-dimensional space area including 3D data representing a three-dimensional structure. Based on the management information of the block that is a partial area of ​​the specified three-dimensional space area including the 3D data representing the three-dimensional structure, a block that satisfies a condition specified by a user is selected, and encoded data of the 3D data within the selected block is decoded. The present disclosure can be applied to an image processing device, an electronic device, an image processing method, a program, etc.

Description

Technical Field

[0001] The present disclosure relates to an image processing device and method, and in particular to an image processing device and method capable of more easily performing partial decoding of 3D data. Background Art

[0002] Conventionally, for example, there is encoding using an octree (Octree) as a method of encoding 3D data representing a three-dimensional structure such as a point cloud (for example, see Non-Patent Literature 1).

[0003] The octree is obtained by converting voxels obtained by quantizing position information into a tree structure, and has dependencies between nodes. Therefore, even in the case of decoding a portion of the octree, it is necessary to cut out the desired area after searching the entire octree and reproducing the configuration of the entire octree. Therefore, the amount of processing and usage of resources such as memory are unnecessarily increased.

[0004] By the way, in the case of encoding a two-dimensional image, there is a method of dividing a frame into slices and tiles for the purpose of distributed processing. In addition, in the case of 3D data, when a three-dimensional space area is divided into small areas and each small area is independently encoded as such slices and tiles, each small area can be easily decoded independently.

[0005] Reference List

[0006] Non-patent literature

[0007] Non-patent document 1: R.Mekuria, Student Member IEEE, K.Blom, P.Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video", tcsvt_paper_submitted_february.pdf Summary of the invention

[0008] Technical issues

[0009] However, with the method of decoding each small area obtained by dividing the entire such area, there are restrictions on the position, shape, size, etc. of each small area. Therefore, it is difficult to independently decode a partial area having an optional position, an optional shape, and an optional size. In addition, since 3D data needs to be decoded in an unnecessary area, etc., the processing amount and usage of resources such as a memory may be unnecessarily increased.

[0010] The present disclosure is proposed in view of such circumstances. The present disclosure enables easier execution of partial decoding of 3D data.

[0011] Solution to the problem

[0012] An image processing device according to one aspect of the present technology is an image processing device including a management information generation section configured to generate management information of a block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure.

[0013] An image processing method according to an aspect of the present technology is an image processing method including generating management information of a block which is a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure.

[0014] An image processing device according to another aspect of the present technology is an image processing device including: a block selection unit configured to select a block satisfying a condition specified by a user based on management information of a block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure; and a decoding unit configured to decode encoded data of the 3D data within the block selected by the block selection unit.

[0015] An image processing method according to another aspect of the present technology is an image processing method comprising: selecting a block satisfying a condition specified by a user based on management information of a block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure; and decoding encoded data of the 3D data within the selected block.

[0016] In an image processing device and method according to one aspect of the present technology, management information of a block that is a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure is generated.

[0017] In an image processing device and method according to another aspect of the present technology, a block satisfying a condition specified by a user is selected based on management information of a block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure, and encoded data of the 3D data within the selected block is decoded. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] [ Figure 1 ] Figure 1 is a block diagram showing an example of a main configuration of an encoding device.

[0019] [ Figure 2 ] Figure 2 is a diagram that helps illustrate an example of a block.

[0020] [ Figure 3 ] Figure 3 is a diagram that helps explain an example of the structure of a bitstream.

[0021] [ Figure 4 ] Figure 4 is a diagram to help explain an example of the content of management information.

[0022] [ Figure 5 ] Figure 5 is a diagram to help explain an example of block information.

[0023] [ Figure 6 ] Figure 6 is a diagram that helps illustrate an example of overall area information.

[0024] [ Figure 7 ] Figure 7 is a block diagram showing an example of a main configuration of an encoding section.

[0025] [ Figure 8 ] Figure 8 is a flowchart that helps explain an example of the flow of an encoding process.

[0026] [ Fig. 9 ] Fig. 9 is a flowchart that helps explain an example of the flow of a block encoding process.

[0027] [ Fig.10 ] Fig.10 is a block diagram showing an example of a main configuration of a decoding device.

[0028] [ Fig.11 ] Fig.11 is a diagram that helps explain an example of the decoding method.

[0029] [ Fig.12 ] Fig.12 is a block diagram showing an example of a main configuration of a decoding section.

[0030] [ Fig.13 ] Fig.13 is a flowchart that helps explain an example of the flow of a decoding process.

[0031] [ Fig.14 ] Fig.14 is a flowchart that helps illustrate an example of the flow of a block decoding process.

[0032] [ Fig.15 ] Fig.15 is a diagram that helps illustrate the use case.

[0033] [ Fig.16 ] Fig.16 is a block diagram showing an example of a main configuration of an encoding device.

[0034] [ Fig.17 ] Fig.17 is a block diagram showing an example of a main configuration of an encoding device.

[0035] [ Fig.18 ] Fig.18 is a block diagram showing an example of a main configuration of a decoding device.

[0036] [ Fig.19 ] Fig.19 is a block diagram showing an example of a main configuration of a decoding device.

[0037] [ Fig. 20 ] Fig. 20 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION

[0038] Modes for carrying out the present disclosure will be described hereinafter (modes will be described as embodiments hereinafter). Incidentally, the description will be made in the following order.

[0039] 1. First Embodiment (Encoding Device)

[0040] 2. Second Embodiment (Decoding Device)

[0041] 3. Use Cases

[0042] 4. Third Embodiment (Encoding Device)

[0043] 5. Fourth Embodiment (Decoding Device)

[0044] 6. Additional Notes

[0045] <1. First Embodiment>

[0046] <Documents supporting technical content and technical terms, etc.>

[0047] The scope disclosed in the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents known at the time of filing.

[0048] Non-patent document 1: (mentioned above)

[0049] Non-Patent Literature 2: Ohji Nakagami, Phil Chou, Maja Krivokuca, Khaled Mammou, Robert Cohen, Vladyslav Zakharchenko, Gaelle Martin-Cocher, “Second Working Draft for PCC Categories 1,3,” ISO / IEC JTC1 / SC29 / WG11, MPEG 2018 / N17533, April 2018, San Diego, USA

[0050] Non-patent document 3: ITU (International Telecommunication Union) Telecommunication Standardization Sector, "Advanced video coding for generic audiovisual services", H.264, 04 / 2017

[0051] Non-patent document 4: ITU (International Telecommunication Union) Telecommunication Standardization Department, "High efficiency video coding", H.265, 12 / 2016

[0052] Non-Patent Literature 5: Jianle Chen, Elena Alshina, Gary J. Sullivan, Jens-Rainer, Jill Boyce, “Algorithm Description of Joint Exploration Test Model 4,” JVET-G1001_v1, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, Joint Video Exploration Team (JVET) Seventh Meeting: Turin, Italy, July 13-21, 2017

[0053] That is, the contents described in the above non-patent documents also constitute the basis for determining the support requirements. For example, the quadtree block structure described in non-patent document 4 and the QTBT (quadtree plus binary tree) block structure described in non-patent document 5 are both within the disclosure scope of the present technology (even if these structures are not directly described in the embodiments) and are assumed to satisfy the support requirements of the claims. In addition, for example, technical terms such as parsing, grammar, and semantics are similarly within the disclosure scope of the present technology (even if the technical terms are not directly described in the embodiments) and are assumed to satisfy the support requirements of the claims.

[0054] <Point Cloud>

[0055] Conventionally, there are 3D data such as point clouds that express a three-dimensional structure by position information and attribute information of a point group, and meshes that include vertices, edges, and faces and define a three-dimensional shape using polygonal representation.

[0056] For example, in the case of a point cloud, a three-dimensional structure (an object of a three-dimensional shape) is represented as a collection of a large number of points (a point group). That is, the data of the point cloud (this data is also called point cloud data) includes position information and attribute information (e.g., color, etc.) of each point in the point group. Therefore, the data structure is relatively simple, and an optional three-dimensional structure can be represented with sufficient accuracy by using a sufficiently large number of points.

[0057] <Quantification of position information using voxels>

[0058] Since the data volume of such point cloud data is relatively large, in order to compress the data volume by encoding, etc., an encoding method using voxels has been considered. A voxel is a three-dimensional area for quantizing the position information of the encoding target. That is, the three-dimensional area including the point cloud is divided into small three-dimensional areas called voxels, and whether a point is included is indicated in each voxel. As a result of the above, the position of each point is quantized in units of voxels. Therefore, by converting the point cloud data into such voxel data (which is also called voxel data), the increase in the amount of information can be suppressed (generally, the amount of information can be reduced).

[0059] <Octtree>

[0060] In addition, the construction of an octree using such voxel data has been considered. The octree is obtained by converting the voxel data into a tree structure. Each node of the octree includes 8 bits of data, and the value of the corresponding bit indicates the presence or absence of a point in the corresponding 2×2×2 small area. That is, the octree recursively divides the three-dimensional area and indicates the presence or absence of data in each hierarchical small area. Each bit of the node at the lowest position of the octree indicates the presence or absence of a point in the voxel. Therefore, the octree has dependencies between nodes. Therefore, even in the case of decoding a portion of the octree, after searching the entire octree and reproducing the configuration of the entire octree, it is necessary to cut out the desired area. Therefore, the usage and processing amount of resources such as memory are unnecessarily increased.

[0061] By the way, in the case of encoding a two-dimensional image, there is a method of dividing a frame into slices and tiles for the purpose of distributed processing. In addition, in the case of 3D data, when a three-dimensional space area is divided into small areas and each small area is independently encoded as such slices and tiles, each small area can be easily decoded independently.

[0062] However, with the method of decoding each small area obtained by dividing the entire such area, there are restrictions on the position, shape, size, etc. of each small area. For example, the position, shape, size, etc. of each small area may be determined by the position, shape, size, number of divisions, division direction, etc. of the entire area. Therefore, it is difficult to independently decode a partial area having an optional position, an optional shape, and an optional size.

[0063] In addition, when decoding is performed in such small area units, all small areas including the desired range are decoded. However, in this case, since 3D data outside the desired range needs to be decoded, the processing amount and usage of resources such as memory may be unnecessarily increased.

[0064] <Signaling of Management Information for Partial Areas>

[0065] Therefore, the encoder generates management information of a block as a partial area in a predetermined three-dimensional space area, the predetermined three-dimensional space area including 3D data representing a three-dimensional structure. For example, the encoder (image processing device) is provided with a management information generating unit that generates management information of a block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure.

[0066] Therefore, the encoder can set a block with an optional position, an optional shape, and an optional size, and generate management information of the block. Then, the encoder can transmit (send with a signal) the management information for managing the block to the decoding side. Therefore, the decoder can more easily decode the 3D data of the block based on the management information. In other words, the decoder can more easily decode the 3D data of the partial area with an optional position, an optional shape, and an optional size.

[0067] <Encoding device>

[0068] Figure 1 is a block diagram showing an example of the configuration of an encoding device as one mode of an image processing device to which the present technology is applied. Figure 1 The encoding device 100 shown in FIG. 1 is a device that encodes 3D data such as a point cloud using voxels and an octree.

[0069] By the way, Figure 1 The main processing parts, main data flows, etc. are shown, and Figure 1 Not all processing units, data flows, etc. are necessarily shown in the figure. That is, in the encoding device 100, there may be Figure 1 The processing section of the block in, or there may be not represented as Figure 1 The same applies to other figures and the like describing the processing units within the encoding device 100.

[0070] like Figure 1 As shown, the encoding device 100 includes a block setting section 101 , a management information generating section 102 , an encoding section 103 , and a bit stream generating section 104 .

[0071] The block setting unit 101 performs processing related to setting a block as a partial area in a predetermined three-dimensional space area, which includes 3D data representing a three-dimensional structure. The three-dimensional space area and the block will be described later. The block setting unit 101 obtains point cloud data input to the encoding device 100, for example.

[0072] In addition, the block setting section 101 receives designation information about a block, which is input, for example, from a user, a device outside the encoding device 100, etc. The designation information is information that specifies which type of block is to be set. For example, the designation information specifies a block to be set by using optional parameters such as position, size, and shape.

[0073] In addition, the block setting section 101 sets a designated block in the three-dimensional space region including the input point cloud. In addition, the block setting section 101 extracts the point cloud included in the block from the input point cloud, and provides the data of the point cloud (which data will also be referred to as block point cloud data) to the encoding section 103. In addition, the block setting section 101 provides information about the set block to the management information generation section 102. In other words, the block setting section 101 sets the block managed by the management information generated by the management information generation section 102. Incidentally, the block setting section 101 can also provide information about the three-dimensional space region including the point cloud to the management information generation section 102.

[0074] The management information generating unit 102 performs processing related to the generation of management information of the block. For example, the management information generating unit 102 obtains information related to the block from the block setting unit 101. In addition, the management information generating unit 102 generates management information based on the information. That is, the management information generating unit 102 generates management information of the block as a partial area in a predetermined three-dimensional space area, which predetermined three-dimensional space area includes 3D data representing a three-dimensional structure. The management information will be described later. By the way, in the case where information about a three-dimensional space area including a point cloud (this area will also be referred to as the entire area) is provided from the block setting unit 101, the management information generating unit 102 also generates management information using this information. In addition, the management information generating unit 102 provides the generated management information to the bitstream generating unit 104.

[0075] The encoding unit 103 performs processing related to encoding. For example, the encoding unit 103 obtains the block point cloud data provided from the block setting unit 101. In addition, the encoding unit 103 encodes the block point cloud data by a predetermined method, and thereby generates encoded data (also referred to as block point cloud encoded data). That is, the encoding unit 103 is able to encode the point cloud data of each block independently. The encoding will be described later. In addition, the encoding unit 103 provides the generated block point cloud encoded data to the bitstream generation unit 104.

[0076] The bitstream generation unit 104 performs processing related to the generation of a bitstream. For example, the bitstream generation unit 104 obtains the management information provided by the management information generation unit 102. In addition, the bitstream generation unit 104 obtains the block point cloud encoded data provided by the encoding unit 103. In addition, the bitstream generation unit 104 generates a bitstream including management information and block point cloud encoded data. That is, the bitstream generation unit 104 generates a bitstream including encoded data of point clouds in units of blocks and management information thereof. The bitstream will be described later. In addition, the bitstream generation unit 104 outputs the generated bitstream to the outside of the encoding device 100. The bitstream is transmitted to a decoder (decoding side) via a predetermined medium (e.g., a communication medium, a recording medium, both, etc.), for example.

[0077] Incidentally, these processing units (block setting unit 101, management information generating unit 102, encoding unit 103 and bit stream generating unit 104) have optional configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have, for example, a CPU (central processing unit), a ROM (read-only memory), a RAM (random access memory), etc., and the above-mentioned processing may be implemented by executing programs using the CPU, ROM, RAM, etc. Of course, each processing unit may also have a configuration of both, and implement a part of the above-mentioned processing by a logic circuit and the remaining part by executing a program. The configurations of the various processing units may be independent of each other. For example, a part of the processing unit may implement a part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and another processing unit may implement the above-mentioned processing by both the execution of the logic circuit and the program.

[0078] <block>

[0079] The following description will include blocks. The slices and tiles in the encoding of a two-dimensional image are obtained by dividing the entire image. When applied to three-dimensional space, the slices and tiles include a three-dimensional space region 121 ( Figure 2 A) is set as a processing target, and the three-dimensional space region 121 representing the three-dimensional space region of the processing target is divided into a plurality of regions. In this case, the position, shape, size, etc. of the slices and tiles depend on the position, shape, size, etc. of the three-dimensional space region 121.

[0080] Typically, the point cloud is not uniformly distributed within the three-dimensional space region 121. Therefore, the division of the three-dimensional space region 121 and the division of the point cloud do not necessarily correspond to each other. For example, the entire point cloud may be included in one slice and tile. In addition, there may be slices and tiles that do not include points. Therefore, the method of dividing the three-dimensional space region 121 in order to divide the point cloud in this way may cause inefficient control.

[0081] On the other hand, a block is a partial area having an optional position, an optional shape, and an optional size within the three-dimensional space area 121. Since the management information generation unit 102 generates management information of blocks, the block setting unit 101 can set blocks in optional positions, optional shapes, and optional sizes.

[0082] The blocks may be arranged in an optional area. Thus, for example, like slices and tiles, the blocks may be formed so as to be arranged adjacent to each other over the entire three-dimensional space area 121. Figure 2 In the case of B, the blocks 122 are arranged adjacent to each other, and are arranged to fill the entire three-dimensional space region 121. Then, as in the case of the slice and the tile, the block 122 may form a partial region obtained by dividing the three-dimensional space region 121.

[0083] In other words, in this case, the three-dimensional space region 121 includes arranging the blocks 122. Therefore, the three-dimensional space region 121 including the entire point cloud can be easily set to an optional shape. Figure 2 In the case of B, the block 122 in the portion where no point exists, such as the portion indicated by the arrow 123 and the arrow 124, can be omitted (not formed). Therefore, not only can the block 122 be formed more easily, but also the block 122 in the portion where no point exists, such as the portion indicated by the arrow 123 and the arrow 124, can be formed more easily. Figure 2 A rectangular three-dimensional space region 121 as in A can also be more easily formed into a three-dimensional space region 121 having a more complex shape.

[0084] In addition, it is not necessary to fill the entire three-dimensional space area 121 with blocks. Figure 2 In case of C, blocks 125 and 126 are formed in the three-dimensional space region 121, while no blocks exist in the other region.

[0085] In addition, a block can be set to overlap with an area of ​​another block. Figure 2 In the case of example D, the areas of block 127 and block 128 partially overlap each other (gray portion). In addition, block 128 includes block 129.

[0086] As described above, appropriate blocks can be formed only in necessary parts. Therefore, the formation of unnecessary blocks can be suppressed. Therefore, the increase of unnecessary information amount can be suppressed, and the reduction of coding efficiency can be suppressed. In addition, the increase of encoding and decoding load can be suppressed.

[0087] In addition, information of each block can be managed by management information. For example, resolution, compression ratio, etc. can be managed for each block. Since a plurality of blocks can be set in the same area, for example, scalability of parameters managed by such management information can be achieved. For example, when blocks having different values ​​for optional parameters such as resolution or compression ratio are set to the same area, the value of the parameter can be selected by block selection. That is, scalability of the parameter can be achieved.

[0088] In addition, information such as data update date and time or reliability can also be managed for each block. Since blocks can be set only in an optional position, optional shape, and optional size within a desired range, blocks that only include update information can be set. Thus, point clouds can be updated in units of blocks (replacement, addition, deletion, etc. of information). In addition, since blocks can be set in optional positions, optional shapes, and optional sizes, blocks can be set to exclude as many unnecessary areas as possible. That is, the increase in the amount of additional data and the increase in its processing load can be suppressed. In addition, even when the decoder appropriately performs decoding processing (data access), data can be updated by minimal exclusive processing.

[0089] In addition, the size of the block as a partial area is smaller than the size of the three-dimensional space area 121 as the entire area. Therefore, when decoding the block, the amount of memory (resource usage) required for encoding can be reduced compared to the case of encoding the three-dimensional space area 121.

[0090] In addition, since each block can be encoded independently, it is easy to parallelize the encoding process. In this case, since the blocks can be set in optional positions, optional shapes, and optional sizes, the distribution of the load of the encoding process can be performed more appropriately (making it closer to the desired distribution method).

[0091] In addition, since the encoding process is performed independently for each block, it is possible to perform the update of the point cloud (replacement, addition, deletion, etc. of information) in units of blocks. For example, when a new block is formed in the update part of the information, the update information can be independently encoded and decoded, and thus the data of the update information can be more easily added to the data before the update.

[0092] In addition, since encoding is performed independently for each block, the data of each block in the bit stream is independent of each other. Therefore, when decoding, only the block including the desired area needs to be decoded. Therefore, the increase in the load of the decoding process can be suppressed. For example, the processing time of the decoding process can be reduced. In addition, for example, the resource usage such as the amount of memory occupied can be reduced.

[0093] In addition, in the case where there are multiple blocks including the same area, the decoding efficiency can be improved by selecting the necessary blocks based on the management information. For example, in the case where the necessary resolution is known, the decoder can easily obtain a decoding result with a desired resolution by selecting and decoding a block including up to the resolution. In addition, in the case where the update time is known, the decoder can easily obtain a decoding result of the latest information by selecting and decoding a newer block. In addition, the decoder can select a block and decode the block based on optional parameters such as the presence or absence of color information, and can obtain more various processing results.

[0094] In addition, if Figure 2 As in the example of D, a plurality of blocks may be set so that the areas of the blocks overlap each other. In the case of a two-dimensional image, for example, when the areas thus overlap, the overlapping parts of the data of the respective areas are redundant. That is, in the case of a two-dimensional image, the information necessary at the same address is always one piece of information. Therefore, in the case of overlapping areas, a point in one of the areas is selected.

[0095] However, in the case of point cloud as 3D data, the meaning of data overlap is different. In the case of point cloud, data does not exist at all grid points within the data representation area (voxel cuboid). In addition, there may be two or more points (which represent) at the same position address (the method of using multiple points depends on the application).

[0096] Therefore, in the case where the areas of a plurality of blocks overlap with each other, data of any one block may be selected, or data of a plurality of blocks may be selected (data of all blocks whose areas overlap with each other may be selected).

[0097] For example, even when two blocks are set to the same area, the positions of the points included in the respective blocks may not coincide with each other. In such a case, for example, at the time of decoding, data of one block (points of one block) may be selected and decoded, or data of two blocks (points of two blocks) may be selected and decoded. In the case where two blocks are selected, all points included in the two blocks may be restored, or new points may be generated by integrating the points of the blocks with each other through operation.

[0098] In addition, even when the positions of points included in two blocks set to the same area coincide with each other, the attribute information of the respective points may be different from each other. In such a case, for example, at the time of decoding, the attribute information of one block (the points of one block) may be selected and decoded, or the attribute information of two blocks (the points of two blocks) may be selected and decoded. In the case where two blocks are selected, all the attribute information included in the two blocks may be restored, or new attribute information may be generated by integrating the attribute information of the blocks with each other through operation.

[0099] In addition, even when the positions of points included in two blocks set to the same area coincide with each other, the points may differ from each other in the parameters managed by the management information. In such a case, for example, at the time of decoding, one block (a point of one block in the block) may be selected and decoded based on the parameter, or two blocks (points of two blocks) may be selected and decoded. In the case where two blocks are selected, the parameters may be updated.

[0100] <bitstream>

[0101] The bit stream will be described below. The encoding apparatus 100 generates, for example, Figure 3 The bit stream of the data structure shown in A. Figure 3 In A, a bit stream 141 is a bit stream generated by the encoding device 100. The bit stream 141 has a start code (start_code), a sequence header (Seq_Header_info), and block data (Block_Data(0), Block_Data(1), ..., Block_Data(n)).

[0102] The start code (start_code) is identification information indicating the starting position of the sequence in question. A predetermined bit string is set in the start code (start_code). The sequence header (Seq_Header_info) is header information of each sequence. Information about the sequence in question is stored in the sequence header (Seq_Header_info). The data of each block is stored in the block data (Block_Data(0), Block_Data(1), ..., Block_Data(n)).

[0103] For example, the sequence header (Seq_Header_info) has Figure 3 The data structure shown in B. Figure 3 In B, the sequence header 142 has sequence information (Seq_info), a block mode flag (Block_mode_flag), and block information of each block (Block(0)_info, . . . , Block(n)_info).

[0104] Information about the sequence in question is stored in the sequence information (Seq_info). The block mode flag (Block_Mode_Flag) is flag information indicating whether a block is set in the sequence in question (i.e., whether a block mode is set). By referring to this value, it is possible to easily grasp whether a block is set in the sequence in question. In the block information (Block(0)_info, ..., Block(n)_info), information about the corresponding block is stored.

[0105] For example, block information (Block(0)_info, ..., Block(n)_info) has Figure 3 The data structure shown in C. Figure 3In C, the block information 143 has a block number (block_no), a point cloud address (Point cloud address(x, y, z)), a point cloud size (Point cloud size(x, y, z)), a data location (data_location(offset / address)) and additional information (additional_info).

[0106] The block number (block_no) is a number (identification information) for identifying the block in question. The point cloud address (Pointcloud address(x,y,z)) is the location information (x,y,z) of the block in question. The point cloud size (Point cloud size(x,y,z)) is the size information of the block in question. The data location (data_location(offset / address)) is the location information of the data of the block in question in the bit stream. The location information indicates, for example, the starting position of the data of the block in question. In the data location (data_location(offset / address)), for example, an offset or address relative to a predetermined reference position in the bit stream is stored. In addition to the above, optional information about the block is stored in additional information (additional_info).

[0107] For example, block data (Block_data(0), Block_data(1), ..., Block_data(n)) has the following structure: Figure 3 The data structure shown in D. Figure 3 In D, the block data 144 has a start code (start_code), header information (Header_info), point cloud position information (Point cloud Position info) and point cloud attribute information (Point cloudAttribute info). In addition, the block data 144 may also optionally have block information (Block_info) of the block in question.

[0108] The start code (start_code) is identification information indicating the starting position of the block in question. A predetermined bit string is set in the start code (start_code). The header information (Header_info) is header information for each block. Information about the block in question is stored in the header information (Header_info). Position information of the point cloud included in the block in question is stored in the point cloud position information (Point cloud Position info). Attribute information of the point cloud included in the block in question is stored in the point cloud attribute information (Point cloud Attribute info). Information similar to the block information 143 is stored in the block information (Block_info).

[0109] The bitstream generation section 104 is capable of storing the management information (management information for managing blocks) generated by the management information generation section 102 in an optional position of such a bitstream. The management information may be stored, for example, in the sequence information (Seq_info) of the sequence header 142. In addition, the management information may be divided for each block and stored in the additional information (additional_info) of the block information 143. In addition, the management information may be divided for each block and stored in the block information (Block_info) of the block data 144. In addition, the management information may be added to the bitstream 141 as a layer concept of an even higher level.

[0110] <Management Information>

[0111] Next, the management information generated by the management information generating unit 102 will be described. The management information may include specific area information of each block, and may also include additional information (resolution, etc.). For example, the management information may include: Figure 4 Various information shown in Table 151.

[0112] For example, the management information stores block information as information about the block. In addition, the management information may include point information as information about position information of 3D data (e.g., point cloud) included in the block. In addition, the management information may include attribute information as information about attribute information of 3D data (e.g., point cloud) included in the block. In addition, the management information may include acquisition information as information about acquisition (generation) of 3D data (e.g., point cloud).

[0113] <Block Information>

[0114] Block information is essential information. For example, block information includes Figure 4At least any one of the block identification information, block position, block size, or data position information shown in Table 151. Of course, it is sufficient that the block information is information about the block, and the block information may include information other than the above-mentioned information.

[0115] The block identification information is information for identifying a block. The block identification information includes, for example, an identification number, an index, etc. By transmitting the block identification information to the decoding side, the encoding device 100 can enable the decoder to more easily identify the block by referring to the block identification information.

[0116] The block position is information indicating the position of the block in the three-dimensional space where the point cloud exists. For example, the block position is indicated by using world coordinates. Of course, the method of expressing the block position is optional and is not limited to world coordinates.

[0117] The block size is information indicating the size, shape, etc. of the block. The method of indicating the size and shape of the block is optional. Figure 5 In the case of rectangular block 0 (block0) illustrated in the figure, for example, the size of the block can be determined by the length of each side (e.g., x0 s ), or the size of the block can be represented by vertex coordinates (e.g., (x0 a ,y0 a ,z0 a ) and (x0 b ,y0 b ,z0 b In this case, the size of the block is represented by identifying the range of the area of ​​block 0 by the coordinates of two vertices located at diagonal positions.

[0118] By transmitting the block position and block size to the decoding side, the encoding device 100 can enable the decoder to more easily perform block selection based on the specified range, for example, based on these multiple pieces of information.

[0119] The data position is information indicating the position of the data of the block in question in the bitstream. The method of indicating the data position is optional. For example, the data position can be indicated by an offset, an address, etc. By the way, this information can be included as header information in addition to the management information in the bitstream.

[0120] By transmitting the data position to the decoding side, the encoding device 100 can enable the decoder to more easily extract the data of the selected block from the bitstream based on the information.

[0121] <Point Information>

[0122] For example, point information includes Figure 4 At least any one of the information on resolution, structure or reliability shown in Table 151.

[0123] Resolution is information indicating the resolution of 3D data within a block (for example, the interval between points in the case of a point cloud). The method of expressing the resolution is optional. The resolution can be represented by a direct numerical value, or the resolution can be indirectly represented by using an index, etc. In addition, in the case where the resolution within a block is uneven, the minimum value of the resolution (the minimum value of the distance between points in the case of a point cloud), the maximum value (the maximum value of the distance between points in the case of a point cloud), the average value (the average value of the distance between points in the case of a point cloud), the median (the median value of the distance between points in the case of a point cloud), etc. can be used.

[0124] By transmitting this resolution to the decoding side, the encoding device 100 can enable the decoder to more easily perform block selection according to the necessary resolution, for example based on this information. For example, scalability of the parameter "resolution" can be achieved (this scalability will also be referred to as resolution scalability).

[0125] The structure is information describing the three-dimensional structure represented by the 3D data (e.g., point cloud) within the block. For example, the structure may be directly expressed by using text such as natural object, artificial object, moving object, stationary object, animal, plant, person, hand, foot, head, or hair, or may be indirectly expressed by using a predefined index, etc. Alternatively, the characteristics of the three-dimensional structure may be expressed, such as soft, hard, large, or small.

[0126] By transmitting this structure to the decoding side, the encoding device 100 can enable the decoder to more easily perform block selection according to the designation of the stereoscopic structure, for example, based on this information.

[0127] Reliability is information indicating the reliability of data as 3D data (e.g., point cloud) within a block. For example, reliability indicates the possibility of error according to the conditions when the data was created, degradation due to update, etc. The method of expressing reliability is optional. For example, reliability can be expressed by an absolute value, or can be expressed by a relative value. In addition, in addition to direct expression such as described above, reliability can also be indirectly expressed, for example, by using a predefined index, etc.

[0128] By transmitting this reliability to the decoding side, the encoding device 100 can enable the decoder to more easily perform block selection according to the necessary reliability, for example based on this information. For example, scalability of the parameter "reliability" can be achieved (this scalability will also be referred to as reliability scalability).

[0129] Incidentally, the point information is not necessary and can therefore be omitted. That is, a part or all of the above information can be omitted. In addition, it is sufficient that the point information is information about the position information of the point cloud included in the block, and the point information is not limited to the above information examples. For example, information other than the above information can be included.

[0130] <Attribute Information>

[0131] Attribute information includes, for example, Figure 4 At least any of the information shown in Table 151 such as color, normal, or reflectivity.

[0132] Color is flag information indicating whether the attribute information of the point cloud existing in the block includes color information. In addition, in the case of including color information, the number of colors of the color information, what color to use, etc. can be further indicated.

[0133] By transmitting this color to the decoding side, the encoding device 100 can make it easier for the decoder to make block selection according to the presence or absence of color information, for example, based on this information.

[0134] Normal is flag information indicating whether the attribute information of the point cloud existing in the block includes normal information. In addition, when normal information is included, the number and direction of normals in the normal information may be further indicated.

[0135] By transmitting the normal to the decoding side, the encoding device 100 can make it easier for the decoder to perform block selection according to the presence or absence of normal information, for example, based on the information.

[0136] The reflectivity is flag information indicating whether the attribute information of the point cloud existing in the block includes reflectivity information. In addition, when the reflectivity information is included, the number and value of the reflectivity information may be further indicated.

[0137] By transmitting the reflectivity to the decoding side, the encoding device 100 can enable the decoder to more easily perform block selection according to the presence or absence of reflectivity information, for example, based on the information.

[0138] The attribute information is not necessary and can therefore be omitted. That is, a part or all of the above information can be omitted. In addition, it is sufficient that the attribute information is information about the attribute information of the point cloud included in the block, and the attribute information is not limited to the above information examples. For example, information other than the above information can be included.

[0139] <Get information>

[0140] Obtaining information such as Figure 4At least any one of the information such as time, calculation method, sensor, acquisition location or acquirer shown in Table 151.

[0141] Time is information indicating a period of time for obtaining (generating) 3D data (e.g., point cloud) existing in a block. Time can be directly indicated by time. In this case, the unit is optional, such as year, month, day, hour, minute, or second. In addition, time can be represented by the time from a reference time (e.g., acquisition time) to the current time. In addition, time can be indirectly represented by using information other than time or period, such as version, number of updates, etc.

[0142] By transmitting this time to the decoding side, the encoding device 100 can enable the decoder to more easily select a block including the latest data or data in a specified period based on this information, for example.

[0143] The calculation method is information indicating a method of obtaining (generating) 3D data (e.g., point cloud) existing in a block. The calculation method indicates how to obtain (generate) 3D data from a stereo structure, for example, indicating that 3D data is generated from LiDAR (Light Detection and Ranging) data, or 3D data is calculated by a stereo matching method. In addition, a calculation equation, parameter values, etc. used in obtaining (generating) can be represented. The method can be directly represented by text, etc., or the method can be indirectly represented by using a predefined index, etc.

[0144] By transmitting the calculation method to the decoding side, the encoding device 100 can enable the decoder to more easily select a block including data on the specified method based on the information, for example.

[0145] The sensor is information about the sensor used when obtaining (generating) 3D data (e.g., point cloud) existing in the block. The sensor indicates, for example, the sensor type (e.g., LiDAR sensor, CMOS (Complementary Metal Oxide Semiconductor) sensor, etc.), performance (resolution, frequency, etc.), number, usage method, etc. The sensor may be directly indicated or may be indirectly indicated by using an index, etc.

[0146] By transmitting the sensor to the decoding side, the encoding device 100 can enable the decoder to more easily select a block including data obtained by a specified sensor based on the information, for example.

[0147] The acquisition position is information indicating the acquisition (generation) position of 3D data (e.g., point cloud) existing in the block. The method of expressing the acquisition position is optional. For example, the acquisition position can be expressed using GPS data. In addition, for example, the acquisition position can be expressed by a country name or a region name (e.g., Japan, the United States, or Europe). In addition, the acquisition position can be expressed by using an address, a postal code, etc. Of course, the acquisition position can be expressed by using information other than the above.

[0148] By transmitting the obtained position to the decoding side, the encoding device 100 can enable the decoder to more easily select a block including data obtained at the specified position based on the information, for example.

[0149] The acquirer is information indicating a user (acquirer) who acquires (generates) 3D data (e.g., point cloud) existing in a block. The method of expressing the acquirer is optional. For example, the acquirer may be expressed by a person's name, a job title, etc. In addition, the acquirer may be expressed indirectly by using a predefined index, etc. In addition, the rights of the acquirer (administrator, ordinary user, guest, etc.) may be expressed.

[0150] By transmitting the obtainer to the decoding side, the encoding device 100 can enable the decoder to more easily select a block including data obtained by the specified obtainer based on the information, for example.

[0151] The acquired information is not necessary and can therefore be omitted. That is, a part or all of the above information can be omitted. In addition, it is sufficient that the acquired information is information about the point cloud included in the block, and the acquired information is not limited to the above information examples. For example, information other than the above information can be included.

[0152] <Entire area information>

[0153] In addition, the management information may include entire region information as information on the entire three-dimensional space region (also referred to as the entire region) including the entire point cloud.

[0154] For example, the entire area information includes such Figure 6 At least any one of the position size, block information or common information shown in Table 161.

[0155] The position size is information indicating the size, position (coordinates), shape, etc. of the entire area. As in the case of the block position and block size described above, the method of expressing these is optional. For example, the position, shape, size, etc. can be expressed by using world coordinates. In addition, the size and shape can be expressed by using the length of each side.

[0156] The block information is information about the blocks included in the entire area, which is the block type, the total number of blocks, the number of divisions, etc. The method of expressing these is optional. The block information can be directly expressed by using text or numerical values, or can be indirectly expressed by using predefined indexes, etc.

[0157] The common information is management information common to all blocks formed in the entire area. The common information may include any content information. The information included in the common information is applied to all blocks.

[0158] <Coding unit>

[0159] Figure 7 is a diagram showing the encoding unit 103 ( Figure 1 ) is a block diagram of an example of the main configuration of Figure 7 As shown, the encoding section 103 includes a voxel generation section 181 , an octree generation section 182 , and a lossless encoding section 183 .

[0160] The voxel generation unit 181 performs processing related to the generation of voxels. For example, the voxel generation unit 181 obtains the block point cloud data provided from the block setting unit 101. In addition, the voxel generation unit 181 sets a bounding box for the area including the obtained block point cloud data, and sets voxels by further dividing the bounding box. Thus, the voxel generation unit 181 quantizes the position information of the block point cloud data. The voxel generation unit 181 provides the voxel data (also referred to as block voxel data) generated in this way to the octree generation unit 182.

[0161] The octree generation unit 182 performs processing related to the generation of the octree. For example, the octree generation unit 182 obtains the block voxel data provided from the voxel generation unit 181. In addition, the octree generation unit 182 generates an octree (also referred to as a block octree) based on the block voxel data. The octree generation unit 182 provides the data of the generated block octree (this data will also be referred to as block octree data) to the lossless encoding unit 183.

[0162] The lossless coding section 183 performs processing related to lossless coding. For example, the lossless coding section 183 obtains the block octree data provided from the octree generation section 182. The lossless coding section 183 generates coded data by encoding the block octree data. The lossless coding section 183 provides the generated coded data as block point cloud coded data to the bitstream generation section 104.

[0163] Incidentally, these processing units (voxel generation unit 181, octree generation unit 182, and lossless encoding unit 183) have optional configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have, for example, a CPU, ROM, RAM, etc., and the above-mentioned processing may be implemented by executing a program using the CPU, ROM, RAM, etc. Of course, each processing unit may also have a configuration of both, and implement part of the above-mentioned processing by a logic circuit and the remaining part by executing a program. The configurations of the various processing units may be independent of each other. For example, a part of the processing unit may implement part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and another processing unit may implement the above-mentioned processing by both the logic circuit and the execution of the program.

[0164] By having the above configuration, the encoding device 100 can set blocks at optional positions, in optional sizes and in optional shapes, and generate management information of the blocks. Therefore, the encoding device 100 can enable the decoder to more easily perform partial decoding of 3D data.

[0165] <Encoding Process Flow>

[0166] Next, we will refer to Figure 8 The flowchart in describes an example of the flow of the encoding process performed by the encoding device 100.

[0167] When the encoding process starts, the block setting section 101 obtains point cloud data in step S101 .

[0168] In step S102 , the block setting section 101 receives designation information that designates conditions of the blocks to be set (for example, conditions regarding position, size, shape, and other parameters).

[0169] In step S103 , the block setting section 101 sets blocks satisfying the conditions indicated in the designation information based on the designation information.

[0170] In step S104, the management information generation section 102 generates management information for managing the block set in step S103. That is, the management information generation section 102 generates management information for the block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure.

[0171] In step S105, the encoding section 103 performs a block encoding process. The encoding section 103 thereby encodes the data of the block set in step S103 (block point cloud data) and generates block point cloud encoded data.

[0172] In step S106 , the bitstream generation unit 104 generates a bitstream including the management information generated in step S104 and the block point cloud coded data generated in step S105 .

[0173] In step S107 , the bit stream generation unit 104 outputs the bit stream generated in step S106 to the outside of the encoding device 100 (transmits the bit stream to the decoding side).

[0174] When the process of step S107 ends, the encoding process ends.

[0175] <Flow of Block Coding Processing>

[0176] Next, we will refer to Fig. 9 The flowchart is described in Figure 8 An example of the flow of the block encoding process performed in step S105.

[0177] When the block encoding process is started, in step S121 , the voxel generation section 181 selects a processing target block from unprocessed blocks.

[0178] In step S122 , the voxel generation section 181 generates voxel data by quantizing (converting into voxels) the position information of the points included in the processing target block using the block point cloud data of the processing target block.

[0179] In step S123 , the octree generation section 182 generates an octree of the processing target block (block octree) based on the block voxel data generated in step S122 .

[0180] In step S124 , the lossless encoding section 183 generates block point cloud encoded data by encoding the octree data of the processing target block for which the octree data was generated in step S123 .

[0181] In step S125, the encoding section 103 determines whether all the set blocks have been processed. In the case where it is determined that there is an unprocessed block, the process returns to step S121 to repeat the process from step S121 onward.

[0182] In addition, in the case where it is determined in step S125 that all blocks have been processed, the block encoding process is ended, and the process returns to Figure 8 .

[0183] By performing each process as described above, the encoding device 100 can set a block at an optional position with an optional size and an optional shape, and generate management information of the block. Therefore, the encoding device 100 can enable the decoder to more easily perform partial decoding of 3D data.

[0184] <2. Second Embodiment>

[0185] <Decoding device>

[0186] Fig.10 : is a block diagram showing an example of the configuration of a decoding device as one mode of an image processing device to which the present technology is applied. Fig.10 The decoding device 200 shown in FIG. Figure 1 The decoding device corresponds to the encoding device 100 of the present invention, and is a device that reconstructs 3D data by decoding encoded data (bit stream) of 3D data (e.g., point cloud) generated by the encoding device 100, for example.

[0187] By the way, Fig.10 The main processing parts, main data flows, etc. are shown, and Fig.10 Not all processing units, data flows, etc. are necessarily shown in the figure. That is, in the decoding device 200, there may be Fig.10 The processing section of the block may exist or not represented as Fig.10 The same applies to other figures and the like describing the processing units within the decoding device 200.

[0188] like Fig.10 As shown, the decoding device 200 includes a control unit 201, a bit stream buffer 202, a decoding unit 203 and an editing unit 204.

[0189] The control section 201 performs processing related to the control of decoding. For example, the control section 201 obtains a bit stream provided from the bit stream buffer 202. The control section 201 extracts management information included in the obtained bit stream. By the way, only the management information extracted from the bit stream may be provided from the bit stream buffer 202. In this case, the control section 201 obtains the management information.

[0190] In addition, the control unit 201 receives designation information input by the user to the decoding device 200. The designation information is information set by the user and designates conditions related to the selection of the block to be decoded. For example, for a certain parameter included in the management information, the condition for selecting the block is designated by the designation information.

[0191] The control section 201 appropriately controls the bit stream buffer 202 to the editing section 204 based on these pieces of information. The control section 201 includes a block selection section 211, a decoding control section 212, and an editing control section 213, for example.

[0192] The block selection section 211 performs processing related to selection of a block to be decoded. For example, the block selection section 211 extracts management information from the bit stream obtained by the control section 201, and selects a block to be decoded based on the extracted management information (or the management information obtained by the control section 201) and the specified information obtained by the control section 201. That is, the block selection section 211 selects a block that satisfies a condition specified by a user based on the management information of the block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure.

[0193] For example, when the range of the three-dimensional space area in which the encoded data of the point cloud is decoded is specified by the designation information, the block selection unit 211 refers to the management information and selects a block including the specified range based on the "block position" and "block size" in the block information. Since the "block position" and "block size" are transmitted as the management information in this way, the block selection unit 211 can select a block including the specified range.

[0194] For example, in addition, when the necessary resolution is specified by the designation information, the block selection section 211 refers to the management information and selects a block including 3D data having a resolution equal to or higher than the specified necessary resolution based on the "resolution" in the point information. Since the "resolution" is transmitted as the management information in this way, the block selection section 211 can select a block according to the necessary resolution. For example, by setting a plurality of blocks of 3D data having different resolutions from each other in the same area and enabling the selection of a block according to such a necessary resolution, resolution scalability can be achieved.

[0195] In addition, for example, when a stereoscopic structure is specified by the designation information, the block selection unit 211 refers to the management information and selects a block of 3D data including the specified stereoscopic structure based on the "structure" in the point information. Since the "structure" is transmitted as the management information in this way, the block selection unit 211 can select a block including data of the specified stereoscopic structure.

[0196] In addition, for example, when the necessary reliability is specified by the designation information, the block selection section 211 refers to the management information and selects a block including 3D data having a reliability equal to or higher than the specified necessary reliability based on the "reliability" in the point information. Since the "reliability" is transmitted as the management information in this way, the block selection section 211 can select a block according to the necessary reliability. For example, by setting a plurality of blocks of 3D data having different reliabilities from each other in the same area and enabling the blocks to be selected according to such necessary reliability, reliability scalability can be achieved.

[0197] Furthermore, for example, when the presence or absence of color information in the attribute information of the 3D data is specified by the designation information, the block selection section 211 refers to the management information and selects a block having (or not having) the color information in the attribute information of the 3D data based on the “color” in the attribute information. Since “color” is transmitted as the management information in this way, the block selection section 211 can select a block according to the presence or absence of color information or the like.

[0198] In addition, for example, when the presence or absence of normal information in the attribute information of 3D data is specified by the designation information, the block selection section 211 refers to the management information and selects a block having (or not having) normal information in the attribute information of the 3D data based on the “normal” in the attribute information. Since the “normal” is transmitted as the management information in this way, the block selection section 211 can select a block according to the presence or absence of normal information or the like.

[0199] In addition, for example, when the presence or absence of reflectance information in the attribute information of the 3D data is specified by the designation information, the block selection section 211 refers to the management information and selects a block having (or not having) the reflectance information in the attribute information of the 3D data based on the “reflectance” in the attribute information. Since the “reflectance” is transmitted as the management information in this way, the block selection section 211 can select a block according to the presence or absence of reflectance information or the like.

[0200] In addition, for example, when a condition related to a period (time, version, etc.) is specified by the specified information, the block selection section 211 refers to the management information and selects a block that satisfies the condition specified by the specified information and is related to the acquisition period of 3D data based on the "time" in the acquisition information. Since the "time" is transmitted as the management information in this way, the block selection section 211 can select a block including the latest data, select a block including data obtained in the specified period, etc.

[0201] Furthermore, for example, when a condition related to a 3D data acquisition and calculation method is specified by the specified information, the block selection section 211 refers to the management information, and selects a block that satisfies the condition related to the 3D data acquisition and calculation method specified by the specified information based on the “calculation method” in the acquisition information. Since the “calculation method” is transmitted as the management information in this way, the block selection section 211 can select a block including 3D data obtained by the specified acquisition and calculation method, etc., for example.

[0202] Furthermore, for example, when a condition related to a sensor used when obtaining 3D data is specified by the specified information, the block selection section 211 refers to the management information and selects a block satisfying the condition specified by the specified information related to the sensor used when obtaining 3D data based on the “sensor” in the obtained information. Since the “sensor” is transmitted as the management information in this way, the block selection section 211 can select a block including 3D data obtained by the specified sensor or the like, for example.

[0203] In addition, for example, when a condition related to the 3D data acquisition position is specified by the designation information, the block selection section 211 refers to the management information and selects a block that satisfies the condition related to the acquisition position and specified by the designation information based on the “acquisition position” in the acquisition information. Since the “acquisition position” is transmitted as the management information in this way, the block selection section 211 can select, for example, a block including 3D data acquired at the designated position.

[0204] Furthermore, for example, when a condition related to an acquirer of 3D data is specified by the specified information, the block selection section 211 refers to the management information and selects a block that satisfies the condition related to the acquirer and specified by the specified information based on the “acquirer” in the acquisition information. Since the “acquirer” is transmitted as the management information in this way, the block selection section 211 can select a block including 3D data acquired by a specified user or the like, for example.

[0205] As described above, the block selection section 211 can select blocks that satisfy the conditions specified by the specification information regarding various parameters included in the management information.

[0206] When the block selection unit 211 selects a block, the block selection unit 211 identifies the identification information of the selected block based on the “block identification information” in the block information of the management information, and provides the identified identification information (i.e., the identification information of the selected block) as the block selection information to the bitstream buffer 202. In addition, the block selection unit 211 may also refer to the “data position” in the block information of the management information as needed, and provide the information indicating the data position of the selected block to the bitstream buffer 202.

[0207] Therefore, the block selection section 211 can control the bit stream buffer 202 and cause the data of the selected block to be extracted.

[0208] The decoding control section 212 performs processing related to decoding control. For example, the decoding control section 212 extracts management information from the bit stream obtained by the control section 201, and controls decoding based on the extracted management information (or the management information obtained by the control section 201) and the designated information obtained by the control section 201. That is, the decoding control section 212 controls decoding by setting a decoding method based on the management information and the designated information and providing decoding control information to the decoding section 203. For example, the decoding control section 212 controls parameters used in decoding and selects a decoding method.

[0209] The editing control section 213 performs processing related to editing of the data of the decoded block. For example, the editing control section 213 extracts management information from the bit stream obtained by the control section 201, and controls editing of the data of the decoded block based on the extracted management information (or the management information obtained by the control section 201) and the designated information obtained by the control section 201. That is, the editing control section 213 controls editing by setting editing contents based on the management information and the designated information and providing the editing control information to the editing section 204. (19)

[0211] The bitstream buffer 202 performs processing related to the bitstream. For example, the bitstream buffer 202 obtains the bitstream input to the decoding device 200, and holds the bitstream. In addition, the bitstream buffer 202 provides the held bitstream to the control section 201. Incidentally, the bitstream buffer 202 can extract management information from the held bitstream, and provide the extracted management information to the control section 201. In addition, the bitstream buffer 202 obtains block selection information provided from the block selection section 211. In addition, the bitstream buffer 202 extracts a bitstream (block bitstream) of a block (block specified by the block selection information) selected by the block selection section 211 from the held bitstream based on the block selection information, and provides the bitstream to the decoding section 203. (16)

[0213] The decoding section 203 performs processing related to decoding. For example, the decoding section 203 obtains the block bitstream provided from the bitstream buffer 202. The decoding section 203 decodes the block bitstream and thereby generates point cloud data (block point cloud data) of the block selected by the block selection section 211. Incidentally, in the case where decoding control information is provided from the decoding control section 212, the decoding section 203 obtains the decoding control information and performs decoding according to the information. The decoding section 203 provides the generated block point cloud data to the editing section 204. (18)

[0215] The editing unit 204 performs processing related to editing of the block point cloud data as a result of decoding by the decoding unit 203. For example, the editing unit 204 obtains the block point cloud data provided from the decoding unit 203. In addition, the editing unit 204 obtains the editing control information provided from the editing control unit 213. In addition, the editing unit 204 edits the block point cloud data based on the editing control information.

[0216] During the editing, the editing unit 204 deletes information from the block point cloud data, adds information to the block point cloud data, or updates information in the block point cloud data. For example, the editing unit 204 synthesizes a plurality of blocks of block point cloud data, or cuts out and deletes a portion of the block point cloud data.

[0217] The editing unit 204 outputs the appropriately edited point cloud data to the outside of the decoding device 200 .

[0218] <Decoding example>

[0219] For example, suppose you set Fig.11 The rectangular block 231 shown in A of FIG. 1 is provided by Fig.11 A three-dimensional space region is obtained by arranging 3×3×3 blocks 231 adjacent to each other as in B.

[0220] Assume that Fig.11 The partial area 233 in the three-dimensional space area 232 shown in C is decoded. The area 233 spans all blocks 231 in the three-dimensional space area 232, such as Fig.11 Therefore, in this case, the block selection unit 211 selects all blocks 231 in the three-dimensional space area 232, and the decoding unit decodes all blocks 231 in the three-dimensional space area 232. However, the necessary information is Fig.11 Therefore, in this case, the editing unit 204 extracts the 3D data in the region 233 from the 3D data of the entire three-dimensional space region 232 obtained by decoding by the decoding unit 203 .

[0221] Incidentally, when the decoding unit 203 supports partial area decoding and can decode only a partial area of ​​the block bit stream, the decoding unit 203 can decode only the 3D data in the area 233 under the control of the decoding control unit 212 .

[0222] <Decoding section>

[0223] Fig.12 It is shown Fig.10 A block diagram showing an example of the main configuration of the decoding unit 203 in FIG. Fig.12 As shown, the decoding section 203 includes a lossless decoding section 251 , an octree decoding section 252 , and a voxel decoding section 253 .

[0224] The lossless decoding section 251 performs processing related to lossless decoding of the encoded data. For example, the lossless decoding section 251 obtains a block bitstream provided from the bitstream buffer 202. In addition, the lossless decoding section 251 decodes the block bitstream and thereby generates octree data of the selected block (this data will also be referred to as block octree data). Incidentally, in the case where decoding control information is provided from the decoding control section 212, the lossless decoding section 251 performs decoding according to the decoding control information. In addition, the lossless decoding section 251 provides the block octree data to the octree decoding section 252.

[0225] The octree decoding section 252 performs processing related to decoding of the octree. For example, the octree decoding section 252 obtains the block octree data provided from the lossless decoding section 251. In addition, the octree decoding section 252 constructs an octree of the selected block based on the block octree data (the octree will also be referred to as the block octree), and generates voxel data of the selected block based on the block octree (the data will also be referred to as the block voxel data). The octree decoding section 252 provides the generated block voxel data to the voxel decoding section 253.

[0226] The voxel decoding unit 253 performs processing related to decoding of voxel data. For example, the voxel decoding unit 253 obtains the block voxel data provided by the octree decoding unit 252. In addition, the voxel decoding unit 253 reconstructs the point cloud (block point cloud data) of the selected block based on the obtained block voxel data. The voxel decoding unit 253 provides the block point cloud data generated in this way to the editing unit 204 ( Fig.10 ).

[0227] Incidentally, these processing units (lossless decoding unit 251, octree decoding unit 252, and voxel decoding unit 253) have optional configurations. For example, each processing unit may include a logic circuit that implements the above-mentioned processing. In addition, each processing unit may have, for example, a CPU, ROM, RAM, etc., and the above-mentioned processing may be implemented by executing a program using the CPU, ROM, RAM, etc. Of course, each processing unit may have a configuration of both, and implement part of the above-mentioned processing by a logic circuit and the remaining part by executing a program. The configurations of the various processing units may be independent of each other. For example, a part of the processing unit may implement part of the above-mentioned processing by a logic circuit, another part of the processing unit may implement the above-mentioned processing by executing a program, and another processing unit may implement the above-mentioned processing by both the logic circuit and the execution of the program.

[0228] By having the configuration as described above, the decoding device 200 can more easily perform partial decoding of 3D data based on the management information.

[0229] <Decoding Process Flow>

[0230] Next, we will refer to Fig.13 The flowchart describes an example of the flow of the decoding process performed by the decoding device 200.

[0231] When the decoding process starts, in step S201 , the bitstream buffer 202 of the decoding device 200 obtains a bitstream.

[0232] In step S202 , the control unit 201 receives designation information. (20)

[0234] In step S203 , the block selection section 211 selects a block to be decoded based on the management information extracted from the bit stream obtained by the process in step S201 and the designation information received in step S202 .

[0235] In step S204 , the bitstream buffer 202 extracts the bitstream of the block selected in step S203 from the bitstream obtained in step S201 .

[0236] In step S205 , the decoding control section 212 sets a decoding method based on the management information extracted from the bit stream obtained by the process in step S201 and the designation information received in step S202 . (20)

[0238] In step S206, the decoding section 203 performs a block decoding process. The decoding section 203 decodes the bit stream of the block extracted in step S204 by the decoding method set in step S205, and thereby generates point cloud data of the block.

[0239] In step S207 , the editing control section 213 sets the editing method based on the management information extracted from the bit stream obtained by the process in step S201 and the designation information received in step S202 .

[0240] In step S208 , the editing unit 204 edits the point cloud data generated in step S206 by the editing method set in step S207 .

[0241] In step S209 , the editing unit 204 outputs the point cloud data edited in step S208 to the outside of the decoding device 200 .

[0242] When the process of step S209 ends, the decoding process ends.

[0243] <Flow of Block Decoding Processing>

[0244] Will refer to Fig.14 The flowchart is described in Fig.13 An example of the flow of the block decoding process performed in step S206 in FIG.

[0245] When the block decoding process starts, in step S221 , the lossless decoding section 251 selects a processing target block from unprocessed blocks.

[0246] In step S222 , the lossless decoding section 251 decodes the bit stream of the processing target block, and generates octree data of the processing target block.

[0247] In step S223 , the octree decoding section 252 constructs an octree of the processing target block based on the octree data of the processing target block generated in step S222 , and generates voxel data of the processing target block.

[0248] In step S224 , the voxel decoding section 253 generates point cloud data of the processing target block based on the voxel data of the processing target block generated in step S223 .

[0249] In step S225, the decoding section 203 determines whether all the selected blocks have been processed. In the case where it is determined that there is an unprocessed block, the process returns to step S221 to repeat the process from step S221 onward.

[0250] In addition, in the event that it is determined in step S225 that all blocks have been processed, the block decoding process ends, and the process returns to Fig.13 .

[0251] By performing each process as described above, the decoding device 200 can more easily perform partial decoding of 3D data based on the management information.

[0252] <3. Usage>

[0253] Next, the use of the above-mentioned encoding device 100 and decoding device 200 will be described. For example, it is assumed that the following Fig.15 The blocks B0 to B4 are shown in A. Assume that the position of the block in world coordinates is B n (add)(=(x,y,z)), and the size of the block is B n (size)(=size). In this case, Fig.15 The positions and sizes of blocks B0 to B4 are set as in the corresponding equations in B. Similar equations are also shown below.

[0254] B0(add)=(0,0,0),B0(size)=10

[0255] B1(add)=(10,0,0),B1(size)=10

[0256] B2(add)=(0,10,0),B2(size)=10

[0257] B3(add)=(10,10,0),B3(size)=10

[0258] B4(add)=(5,5,5),B4(size)=10

[0259] Assume that each such block is encoded.

[0260] <Scenario 1>

[0261] For example, suppose Fig.15 The decoding of the 3D data of the area shown in C is specified by the specified information. This range is also shown below.

[0262] B d (loc) = (5,5,0-15,15,5)

[0263] First, the block selection unit 211 selects d (loc) identifies a block including the area. In this case, blocks B0 to B3 are corresponding. Then, since the areas of the blocks do not overlap with each other, the block selection unit 211 selects all of the blocks B0 to B3. The decoding unit 203 decodes the encoded data of the blocks B0 to B3 selected by the block selection unit, and thereby generates 3D data (e.g., point cloud data).

[0264] <Scenario 2>

[0265] For example, suppose Fig.15 The decoding of the 3D data of the area shown in D is specified by the specified information. This range is also shown below.

[0266] B d (loc) = (5,5,5-15,15,10)

[0267] First, the block selection unit 211 selects d (loc) identifies the block that includes the area. In this case, blocks B0 to B4 correspond. Then, block B4 overlaps with other blocks. Block B4 overlaps with block B d The ratio of (loc) is 100%, and blocks B0 to B4 are d The ratio of (loc) is 25%. Therefore, the block selection section 211 selects the block B4. The decoding section 203 decodes the encoded data of the block B4 selected by the block selection section, and thereby generates 3D data (for example, point cloud data).

[0268] <Scenario 3>

[0269] For example, suppose Fig.15The decoding of the 3D data of the area shown in E is specified by the specified information. This range is also shown below.

[0270] B d (loc) = (5,5,5-10,10,10)

[0271] First, the block selection unit 211 selects d (loc) identifies the block including the area. In this case, block B0 and block B4 are corresponding. Then, block B0 and block B4 overlap each other. The ratio of both block B0 and block B4 to Bd(loc) is 100%. Therefore, the block selection unit 211 can determine the priority order based on another element (additional information) in these corresponding blocks, and make a selection to avoid duplication. For example, a newer block can be selected according to the update time, etc. The decoding unit 203 decodes the encoded data of the block selected by the block selection unit, and thereby generates 3D data (e.g., point cloud data).

[0272] As described above, the decoding device 200 can easily decode a block at an optional position, with an optional size, and with an optional shape based on the management information. That is, the decoding device 200 can more easily perform partial decoding of 3D data based on the management information.

[0273] <4. Third Embodiment>

[0274] <Another Configuration Example 1 of Encoding Device>

[0275] Incidentally, the configuration of the encoding device 100 is optional and is not limited to the configuration example described in <1. First Embodiment> ( Figure 1 ). For example, Fig.16 As shown, the encoding device 100 may have a plurality of encoding sections 103-N capable of performing processing in parallel with each other instead of the encoding section 103. In this case, when a plurality of blocks are set in the block setting section 101, the point cloud data of each block (block point cloud data) is provided to different encoding sections 103-N from each other, and is encoded independently of each other. Then, the bitstream generation section 104 obtains the block point cloud encoded data provided from each encoding section 103-N, and generates a bitstream including the block point cloud encoded data and management information of these encoding sections 103-N.

[0276] Therefore, parallel processing of encoding is possible. Figure 1 The encoding unit 103 in may enable encoding of a plurality of pieces of point cloud data in parallel with each other.

[0277] <Another Configuration Example 2 of Encoding Device>

[0278] In addition, if Fig.17 As shown, it can also be omitted Fig.16 In this case, the block point cloud bitstreams generated in the respective encoding units 103-N and the management information generated in the management information generating unit 102 are each output as separate information.

[0279] <5. Fourth embodiment>

[0280] <Another Configuration Example 1 of Decoding Device>

[0281] As in the case of the encoding device 100, the configuration of the decoding device 200 is also optional and is not limited to the configuration example described in <2. Second Embodiment> ( Fig.10 ). For example, Fig.18 As shown, the decoding device 200 may have a plurality of decoding sections 203-N capable of performing processing in parallel with each other, instead of the decoding section 203. In this case, when a plurality of blocks are set as decoding targets in the control section 201, the bit streams (block bit streams) of the respective blocks are provided to the decoding sections 203-N different from each other, and are decoded independently of each other. Then, the editing section 204 obtains the block point cloud data provided from each of the decoding sections 203-N, and generates point cloud data by editing these block point cloud data.

[0282] Therefore, parallel processing of decoding is possible. Fig.10 The decoding section 203 in may enable decoding of a plurality of block bit streams in parallel with each other.

[0283] <Another Configuration Example 2 of Decoding Device>

[0284] In addition, if Fig.19 As shown, a plurality of block bitstreams and management information may be input to the decoding device 200. In this case, the bitstream buffer 202 obtains and holds the plurality of block bitstreams and management information input to the decoding device 200. Then, the bitstream buffer 202 provides the obtained management information to the control section 201, and the bitstream buffer 202 reads the block bitstream of the block selected by the control section 201 according to the block selection information, and provides the block bitstream to the decoding section 203-N.

[0285] <6. Additional Notes>

[0286] <Computer>

[0287] The above series of processing can be performed by hardware or can be performed by software. In the case of performing a series of processing by software, the program constituting the software is installed on a computer. Here, the computer includes a computer combined in a dedicated hardware, such as a general-purpose personal computer capable of performing various functions by installing various programs thereon, etc.

[0288] Fig. 20 : is a block diagram showing an example of the configuration of hardware of a computer that executes the above-described series of processes by a program.

[0289] exist Fig. 20 In the illustrated computer 900 , a CPU (Central Processing Unit) 901 , a ROM (Read Only Memory) 902 , and a RAM (Random Access Memory) 903 are interconnected via a bus 904 .

[0290] An input / output interface 910 is also connected to the bus 904. The input / output interface 910 is connected to an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915.

[0291] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a nonvolatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0292] In the computer configured as described above, the above series of processing is performed by the CPU 901 loading a program stored in the storage unit 913, for example, into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also appropriately stores data necessary for the CPU 901 to perform various processing and the like.

[0293] The program executed by the computer (CPU 901) can be applied in a state recorded on the removable medium 921 as a package medium, etc. In this case, by loading the removable medium 921 into the drive 915, the program can be installed in the storage unit 913 via the input output interface 910.

[0294] In addition, the program can also be provided via a wired or radio transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0295] Furthermore, the program may be installed in the ROM 902 or the storage unit 913 in advance.

[0296] <Application target of this technology>

[0297] In the above, the case where the present technology is applied to the encoding and decoding of point cloud data has been described. However, the present technology is not limited to these examples, and can also be applied to the encoding and decoding of 3D data of optional standards. That is, unless inconsistent with the above-mentioned present technology, various processes such as encoding and decoding methods and specifications of various data such as 3D data and metadata are optional. In addition, unless inconsistency with the present technology occurs, part of the above-mentioned processes and specifications can be omitted.

[0298] The present technology can be applied to optional configurations. For example, the present technology can be applied to: various electronic devices, such as transmitters and receivers in satellite broadcasting, cable broadcasting such as cable television, distribution on the Internet, distribution to terminals in cellular communications (e.g., television receivers or mobile phones); or devices for recording images on media such as optical disks, magnetic disks, and flash memories and reproducing images from these storage media (e.g., hard disk recorders or camera devices).

[0299] In addition, for example, the present technology can also be implemented as a part of the configuration of an apparatus such as a processor as a system LSI (large scale integration) (e.g., a video processor), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), a collection obtained by further adding other functions to the unit (e.g., a video collection), etc.

[0300] In addition, for example, the present technology can also be applied to a network system including a plurality of devices. For example, the present technology can be implemented as cloud computing in which a plurality of devices share and jointly perform processing via a network. For example, the present technology can be implemented in a cloud service for providing services regarding images (moving images) to optional terminals such as computers, audio-visual (AV) equipment, portable information processing terminals, or IoT (Internet of Things) devices.

[0301] Incidentally, in this specification, a system refers to a collection of multiple constituent elements (devices, modules (components), etc.), and it is not important whether all the constituent elements exist in the same housing. Therefore, multiple devices housed in separate housings and connected to each other via a network and one device having multiple modules housed in one housing are both systems.

[0302] <Fields and uses of this technology>

[0303] Systems, devices, processing units, etc. to which the present technology is applied can be used, for example, in optional fields such as transportation, medical treatment, crime prevention, agriculture, animal husbandry, mining, beauty, factories, household appliances, weather and nature monitoring. In addition, the use of the present technology is optional.

[0304] <Others>

[0305] By the way, the "flag" in this specification is information for distinguishing multiple states, and includes not only information for distinguishing two states (i.e., true (1) or false (0)), but also information capable of distinguishing three or more states. Therefore, the value that can be assumed by the "flag" can be two values ​​such as 1 / 0, or equal to or greater than three values. That is, the number of bits constituting the "flag" is optional, and the "flag" can be composed of one bit or multiple bits. In addition, it is assumed that the identification information (including the flag) is not only in the form of including the identification information in the bit stream, but also in the form of including the difference information of the identification information relative to the reference information in the bit stream. Therefore, in this specification, the "flag" and "identification information" include not only information, but also difference information relative to the reference information.

[0306] In addition, various information (metadata, etc.) related to the coded data (bitstream) can be transmitted or recorded in any form, as long as the various information (metadata, etc.) is associated with the coded data. Here, the term "association" means, for example, when processing one piece of data, so that another piece of data can be used (can be linked). That is, multiple pieces of data associated with each other can be integrated into one piece of data, or can be multiple separate pieces of data. For example, information associated with the coded data (image) can be transmitted on a transmission line different from the transmission line of the coded data (image). In addition, for example, information associated with the coded data (image) can be recorded on a recording medium different from the recording medium of the coded data (image) (or on different recording areas of the same recording medium). By the way, the "association" can be an association of a part of the data rather than the entire data. For example, an image and information corresponding to the image can be associated with each other in optional units, such as multiple frames, one frame, or a part of a frame.

[0307] By the way, in this specification, terms such as "synthesize", "multiplex", "add", "integrate", "include", "store", "put in", "insert" and "insert" refer to the integration of multiple objects into one, such as integrating encoded data and metadata into one piece of data, and refer to a method of the above-mentioned "association".

[0308] In addition, the embodiments of the present technology are not limited to the aforementioned embodiments, but various modifications can be made without departing from the spirit of the present technology.

[0309] For example, a configuration described as one device (or one processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, a configuration described above as a plurality of devices (or processing units) may be integrated and configured as one device (or one processing unit). In addition, of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). In addition, a portion of the configuration of a particular device (or a particular processing unit) may be included in the configuration of another device (or another processing unit), as long as the configuration and operation of the system as a whole are substantially the same.

[0310] In addition, for example, the above-mentioned program may be executed in an optional device. In this case, the device is only required to have necessary functions (functional blocks, etc.) and be able to obtain necessary information.

[0311] In addition, for example, each step of a flowchart may be performed by one device, or may be shared and performed by multiple devices. In addition, in the case where a step includes multiple processes, the multiple processes may be performed by one device, or may be shared and performed by multiple devices. In other words, the multiple processes included in a step may also be performed as processes of multiple steps. Conversely, the processes described as multiple steps may also be performed collectively as one step.

[0312] In addition, for example, in a program executed by a computer, the processing of the steps describing the program may be performed in time series, for example, in the order described in this specification, or may be performed in parallel or individually at a necessary timing when a call is performed. That is, unless an inconsistency occurs, the processing of each step may be performed in an order different from the above order. In addition, the processing of the steps describing a program may be performed in parallel with the processing of another program, or may be performed in combination with the processing of another program.

[0313] In addition, for example, unless there is inconsistency, multiple technologies related to the present technology can be performed separately and independently of each other. Of course, the optional multiple technologies of the present technology can also be used in combination to achieve. For example, part or all of the present technology described in one embodiment can be combined with part or all of the present technology described in another embodiment. In addition, part or all of the optional present technology described above can be combined with other technologies not described above.

[0314] Incidentally, the present technology can also adopt the following configurations. (1)

[0316] An image processing device, comprising:

[0317] A management information generation section is configured to generate management information of a block which is a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure. (2)

[0319] The image processing device according to (1), wherein:

[0320] The management information includes block information as information on the blocks. (3)

[0322] The image processing device according to (2), wherein:

[0323] The block information includes at least any one of the following items: identification information of the block, a location of the block, a size of the block, or a location of data of the block. (4)

[0325] The image processing device according to any one of (1) to (3), wherein:

[0326] The management information includes position information which is information on position information of 3D data within the block. (5)

[0328] The image processing device according to (4), wherein:

[0329] The position information includes at least any one of the following items: a resolution of the 3D data, a three-dimensional structure represented by the 3D data, or reliability of the 3D data. (6)

[0331] The image processing device according to any one of (1) to (5), wherein:

[0332] The management information includes attribute information which is information on attribute information of the 3D data within the block. (7)

[0334] The image processing device according to (6), wherein:

[0335] The attribute information includes at least any one of the following items: color, normal, or reflectivity of the 3D data. (8)

[0337] The image processing device according to any one of (1) to (7), wherein:

[0338] The management information includes acquisition information as information on acquisition of the 3D data. (9)

[0340] The image processing device according to (8), wherein:

[0341] The acquisition information includes at least any one of the following items: an acquisition time of the 3D data, a calculation method of the 3D data, a sensor used to obtain the 3D data, an acquisition position of the 3D data, or an acquirer of the 3D data. (10)

[0343] The image processing device according to any one of (1) to (9), wherein:

[0344] The management information includes entire region information which is information on the entire three-dimensional space region. (11)

[0346] The image processing device according to (10), wherein:

[0347] The entire area information includes at least any one of the following items: the position of the entire three-dimensional space area, the size of the entire three-dimensional space area, information about blocks included in the three-dimensional space area, or management information common to all blocks included in the three-dimensional space area. (12)

[0349] The image processing device according to any one of (1) to (11), further comprising:

[0350] A bitstream generating section is configured to generate a bitstream including the encoded data of the 3D data and the management information generated by the management information generating section. (13)

[0352] The image processing device according to any one of (1) to (12), further comprising:

[0353] A block setting section is configured to set the block managed by the management information generated by the management information generating section. (14)

[0355] The image processing apparatus according to (13), further comprising:

[0356] An encoding section is configured to encode the 3D data within the block set by the block setting section. (15)

[0358] An image processing method, comprising:

[0359] Management information of a block that is a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure is generated. (16)

[0361] An image processing device, comprising:

[0362] a block selection section configured to select a block satisfying a condition specified by a user based on management information of a block as a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure; and

[0363] A decoding section is configured to decode the encoded data of the 3D data within the block selected by the block selection section. (17)

[0365] The image processing apparatus according to (16), further comprising:

[0366] A decoding control section is configured to control decoding of the encoded data by the decoding section based on the instruction of the user and the management information. (18)

[0368] The image processing device according to (16) or (17), further comprising:

[0369] an editing control section configured to control editing of the 3D data based on an instruction of the user and the management information; and

[0370] An editing section is configured to edit 3D data generated by decoding the encoded data by the decoding section according to control of the editing control section. (19)

[0372] The image processing device according to any one of (16) to (18), further comprising:

[0373] an extracting unit configured to extract the encoded data of the 3D data in the block selected by the block selecting unit from the encoded data of the 3D data of the entire three-dimensional space region, wherein

[0374] The decoding section decodes the encoded data of the 3D data in the block selected by the block selection section, the encoded data being extracted by the extraction section. (20)

[0376] An image processing method, comprising:

[0377] selecting a block satisfying a condition specified by a user based on management information of a block that is a partial area in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure; and

[0378] The encoded data of the 3D data in the selected block is decoded.

[0379] Reference numerals list

[0380] 100: Encoding device

[0381] 101: Block Setting Section

[0382] 102: Management information generation department

[0383] 103: Coding Department

[0384] 104: Bitstream generation unit

[0385] 181: Voxel Generation Department

[0386] 182: Octree Generation

[0387] 183: Lossless Coding Department

[0388] 200: Decoding device

[0389] 201: Control Department

[0390] 202: Bitstream Buffer

[0391] 203: Decoding Department

[0392] 204: Editorial Department

[0393] 211: Block selection unit

[0394] 212: Decoding control unit

[0395] 213: Editorial Control Department

[0396] 251: Lossless decoding unit

[0397] 252: Octree decoding unit

[0398] 253: Voxel decoding unit

Claims

1. An image processing device, comprising: a block setting section (101) configured to allow partial areas of a plurality of blocks in a predetermined three-dimensional space area to overlap with each other, wherein the blocks include 3D data representing a three-dimensional structure; and a management information generating section (102) configured to generate management information of the plurality of blocks, wherein the management information includes information indicating the size and shape of an area of ​​each block, The block setting unit can set the blocks in an optional position, an optional shape and an optional size.

2. The image processing device according to claim 1, wherein: The management information includes block information as information about the blocks, the block information including the information indicating the size and shape of the area of ​​each block.

3. The image processing device according to claim 2, wherein: The block information further includes at least any one of the following items: identification information of the block, a location of the block, and a location of data of the block.

4. The image processing device according to claim 1, wherein: The management information includes position information which is information on position information of 3D data within the block.

5. The image processing device according to claim 4, wherein: The position information includes at least any one of the following items: a resolution of the 3D data, a three-dimensional structure represented by the 3D data, and reliability of the 3D data.

6. The image processing device according to claim 1, wherein: The management information includes attribute information which is information on attribute information of the 3D data within the block.

7. The image processing device according to claim 6, wherein: The attribute information includes at least any one of the following items: color, normal, and reflectivity of the 3D data.

8. The image processing device according to claim 1, wherein: The management information includes acquisition information as information on acquisition of the 3D data.

9. The image processing apparatus according to claim 8, wherein: The acquisition information includes at least any one of the following items: an acquisition time of the 3D data, a calculation method of the 3D data, a sensor used when acquiring the 3D data, an acquisition location of the 3D data, and an acquirer of the 3D data.

10. The image processing apparatus according to claim 1, wherein: The management information includes entire region information which is information on the entire three-dimensional space region.

11. The image processing apparatus according to claim 10, wherein: The entire area information includes at least any one of the following items: the position of the entire three-dimensional space area, the size of the entire three-dimensional space area, information about blocks included in the three-dimensional space area, and management information common to all blocks included in the three-dimensional space area.

12. The image processing apparatus according to claim 1, further comprising: A bitstream generating section is configured to generate a bitstream including the encoded data of the 3D data and the management information generated by the management information generating section.

13. The image processing apparatus according to claim 1, further comprising: An encoding section is configured to encode the 3D data within the block set by the block setting section.

14. An image processing method, comprising: Setting a plurality of blocks in a predetermined three-dimensional space region to allow partial regions of the plurality of blocks to overlap with each other, wherein the blocks include 3D data representing a three-dimensional structure; and generating management information of the plurality of blocks, wherein the management information includes information indicating a size and a shape of an area of ​​each block, Among them, the blocks can be set in an optional position, an optional shape and an optional size.

15. An image processing device, comprising: a block selection section configured to select a block satisfying a condition specified by a user based on management information of a plurality of blocks in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure, wherein the management information includes information indicating a size and a shape of an area of ​​each block, partial areas of the plurality of blocks can be set to overlap each other, and the blocks can be set in an optional position, an optional shape, and an optional size; and A decoding section is configured to decode the encoded data of the 3D data within the block selected by the block selection section.

16. The image processing apparatus according to claim 15, further comprising: A decoding control section is configured to control decoding of the encoded data by the decoding section based on the instruction of the user and the management information.

17. The image processing apparatus according to claim 15, further comprising: an editing control section configured to control editing of the 3D data based on an instruction of the user and the management information; as well as An editing section is configured to edit 3D data generated by decoding the encoded data by the decoding section according to control of the editing control section.

18. The image processing apparatus according to claim 15, further comprising: an extracting unit configured to extract the encoded data of the 3D data in the block selected by the block selecting unit from the encoded data of the 3D data of the entire three-dimensional space region, wherein The decoding section decodes the encoded data of the 3D data in the block selected by the block selection section, the encoded data being extracted by the extraction section.

19. An image processing method, comprising: selecting a block satisfying a condition specified by a user based on management information of a plurality of blocks in a predetermined three-dimensional space area including 3D data representing a three-dimensional structure, wherein the management information includes information indicating a size and a shape of an area of ​​each block, partial areas of the plurality of blocks can be set to overlap each other, and the blocks can be set in an optional position, an optional shape, and an optional size; and The encoded data of the 3D data in the selected block is decoded.

Images