Encoding device, encoding system, encoding method, and encoding program

The encoding system addresses the inefficiency in existing techniques by adapting spatial quantization based on resolution and density, enhancing encoding efficiency and image quality for 3D point cloud data.

WO2024232304A1PCT designated stage expired Publication Date: 2024-11-14SONY SEMICON SOLUTIONS CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/016626
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-04-30
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing encoding techniques for 3D point cloud data, such as G-PCC, apply the same spatial quantization value across all resolution levels, leading to reduced encoding efficiency and subjective image quality due to unnecessary high precision in low-resolution data.

Method used

An encoding system that determines spatial quantization values based on the resolution and density of each point cloud data set, allowing for adaptive quantization scales across different resolution levels, thereby optimizing encoding efficiency and image quality.

Benefits of technology

Improves encoding efficiency by approximately 10% while maintaining subjective image quality by assigning appropriate spatial quantization scales to each resolution level, reducing data size without deteriorating image quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024016626_14112024_PF_FP_ABST
    Figure JP2024016626_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The objective of the present invention is to enable an improvement in encoding efficiency. This encoding device comprises: an input accepting unit that accepts input of a plurality of items of point cloud data; a determining unit that determines a spatial quantization value of each of the plurality of items of point cloud data on the basis of at least one of a resolution and a density of each of the plurality of items of point cloud data of which input has been accepted by the input accepting unit; and an encoding unit that encodes each of the plurality of items of point cloud data on the basis of the spatial quantization value determined by the determining unit.
Need to check novelty before this filing date? Find Prior Art

Description

Encoding device, encoding system, encoding method, and encoding program

[0001] The present invention relates to an encoding device, an encoding system, an encoding method, and an encoding program.

[0002] Techniques for encoding point cloud data are known (see, for example, Patent Documents 1 to 3). Techniques for encoding point cloud data include techniques for compressing 3D point cloud data such as meshes and point clouds in which terrain data, architectural structures, civil engineering structures, etc. are represented as 3D (Dimensional) structures. Techniques for compressing 3D point cloud data such as point clouds include, for example, Geometry-based Point Cloud Compression (G-PCC) and Video-based Point Cloud Compression (V-PCC), which are standardized by the Moving Picture Experts Group (MPEG) (see, for example, Non-Patent Document 1).

[0003] For example, G-PCC is a method for compressing 3D point cloud data by dividing the space to be encoded, which contains the point cloud data, into uniformly sized voxel data, spatially quantizing the voxel data, and then dividing it into octree structure data with a recursive octree structure, and encoding this octree structure data.

[0004] International Publication No. 2020 / 071116 International Publication No. 2019 / 078000 International Publication No. 2020 / 162495

[0005] ISO / IEC JTC 1 / SC 29 / WG 11 N18189 "G-PCC codec description v2"

[0006] However, the above-described techniques may not be able to improve the coding efficiency. For example, when encoding a plurality of point cloud data, if the same spatial quantization value is applied to all the point cloud data, the low-resolution point cloud data may be quantized in the spatial direction with more precision than necessary, which may result in a decrease in coding efficiency. One aspect of the present disclosure enables the improvement of coding efficiency.

[0007] An encoding device according to one aspect of the present disclosure includes an input receiving unit that receives input of a plurality of point cloud data, a determination unit that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data received as input by the input receiving unit, and an encoding unit that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit.

[0008] An encoding system according to one aspect of the present disclosure includes an encoding device and a playback device, wherein the encoding device includes an input receiving unit that receives input of a plurality of point cloud data, a determination unit that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data received as input by the input receiving unit, and an encoding unit that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit, and the playback device includes a request unit that requests a playback target from the plurality of point cloud data encoded by the encoding unit to be played back by the playback device.

[0009] An encoding method according to one aspect of the present disclosure is an encoding method executed by an encoding device, and includes an input receiving step of receiving input of a plurality of point cloud data, a determination step of determining a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data whose input is received by the input receiving step, and an encoding step of encoding each of the plurality of point cloud data based on the spatial quantization value determined by the determination step.

[0010] An encoding program according to one aspect of the present disclosure causes a computer installed in an encoding device to execute an input reception process for receiving input of a plurality of point cloud data, a determination process for determining a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data whose input is received by the input reception process, and an encoding process for encoding each of the plurality of point cloud data based on the spatial quantization value determined by the determination process.

[0011] FIG. 1 is a diagram illustrating an example of a schematic configuration of a coding system according to a first embodiment. FIG. 2 is a diagram for explaining the coding system according to the first embodiment. FIG. 3 is a diagram for explaining the coding system according to the first embodiment. FIG. 4 is a diagram for explaining the coding system according to the first embodiment. FIG. 5 is a diagram for explaining the coding system according to the first embodiment. FIG. 6 is a diagram for explaining the coding device according to the first embodiment. FIG. 7 is a diagram for explaining the coding device according to the first embodiment. FIG. 8 is a flowchart illustrating an example of processing of the coding device according to the first embodiment. FIG. 9 is a flowchart illustrating an example of processing of the coding device according to the first embodiment. FIG. 10 is a flowchart illustrating an example of processing of the coding device according to the first embodiment. FIG. 11 is a flowchart illustrating an example of processing of the coding device according to the first embodiment. FIG. 12 is a flowchart illustrating an example of processing of the playback device according to the first embodiment. FIG. 13 is a flowchart illustrating an example of processing of the playback device according to the first embodiment. FIG. 14 is a diagram illustrating an example of processing of the playback device according to the first embodiment. FIG. 15 is a diagram illustrating an example of processing of the playback device according to the first embodiment. FIG. 16 is a diagram illustrating an example of processing of the playback device according to the first embodiment. Fig. 1 is a diagram showing an example of a hardware configuration of a device. Fig. 2 is a diagram showing an example of a schematic configuration of a coding device according to a reference technology. Fig. 3 is a diagram for explaining a coding device according to a reference technology. Fig. 4 is a diagram for explaining a coding device according to a reference technology. Fig. 5 is a diagram showing an example of a configuration of a control unit of a coding device according to a reference technology.

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same elements are denoted by the same reference numerals, and redundant description may be omitted.

[0013] The present disclosure will be described in the following order: 0. Introduction 1. First embodiment 2. Second embodiment 3. Example of hardware configuration 4. Example of effects

[0014] 0. Introduction As described above, techniques for encoding point cloud data include, for example, techniques for compressing 3D (Dimensions) point cloud data, such as G-PCC (Geometry-based compression approach). Before describing embodiments of the present disclosure, an example of a reference technique related to a technique for compressing 3D point cloud data using G-PCC will be described with reference to FIGS. 24 to 27. FIG. 24 is a diagram illustrating an example of a schematic configuration of an encoding device according to the reference technique. FIGS. 25 and 26 are diagrams for explaining an encoding device according to the reference technique. FIG. 27 is a diagram illustrating an example of the configuration of a control unit of an encoding device according to the reference technique.

[0015] The encoding device 10' shown in FIG. 24 encodes point cloud data. As described above, in the example shown in FIG. 24, the encoding device 10' encodes 3D point cloud data because it is a reference technology related to the technology for compressing 3D point cloud data using G-PCC. However, the point cloud data is not particularly limited as long as it is point cloud data within a predetermined space, and may be, for example, 2D point cloud data within a 2D space. Therefore, the encoding device 10' may encode 2D point cloud data. Examples of the encoding device 10' include a cloud and a terminal. Below, an overview of the encoding device 10' will be described using FIG. 25.

[0016] The encoding device 10' divides a space to be encoded, which includes point cloud data, into uniformly sized voxel data. The encoding device 10' spatially quantizes the voxel data, then divides it into octree structure data 101 having a recursive octree structure as shown in FIG. 25 , and performs arithmetic coding on the octree structure data 101 to convert it into binary data 102 as shown in FIG. 25 , which is expressed in binary format. The encoding device 10' then stores (preserves) the binary data 102 as encoded point cloud data (compressed point cloud data) such as a bit stream. Next, returning to FIG. 24 , an example of the configuration of the encoding device 10' will be described. In the example shown in FIG. 24 , the encoding device 10' includes an input receiving unit 11 and a control unit 12'.

[0017] The input accepting unit 11 accepts input of a plurality of point cloud data. In the example shown in Fig. 24, the input accepting unit 11 accepts input of a plurality of point cloud data including 3D point cloud data. The input accepting unit 11 may accept input of a plurality of point cloud data including 2D point cloud data. The input accepting unit 11 is not particularly limited as long as it accepts input of a plurality of point cloud data. Examples of the input accepting unit 11 include a UI (User Interface) such as buttons displayed on a display unit such as a desktop or touch panel of a personal computer.

[0018] The control unit 12' controls the entire encoding device 10'. For example, the control unit 12' is configured with one or more processors having internal memories storing programs defining each processing procedure and control data, and the processors execute each process using the programs and internal memories. Examples of the control unit 12' include electronic circuits such as a central processing unit (CPU), a micro processing unit (MPU), and a graphics processing unit (GPU), as well as integrated circuits such as an application specific integrated circuit (ASIC) and a field programmable gate array (FPGA). In the example shown in FIG. 24, the control unit 12' includes a division unit 121, a determination unit 122', and an encoding unit 123'.

[0019] The dividing unit 121 divides each of the plurality of point cloud data into a plurality of divided point cloud data based on at least one of the space in which each of the plurality of point cloud data is included and the resolution of each of the plurality of point cloud data. For example, the dividing unit 121 divides each of the plurality of point cloud data into a plurality of divided point cloud data by dividing the space in which each of the plurality of point cloud data is included into a plurality of divided spaces. Furthermore, the dividing unit 121 divides each of the plurality of point cloud data into a plurality of divided point cloud data for each resolution level of the plurality of point cloud data.

[0020] An example of division by the division unit 121 will be described below with reference to Fig. 26. As shown in the spatial division diagram of Fig. 26, the division unit 121 divides a space containing each of the multiple point cloud data into multiple divided spaces, such as multiple 100-meter square tiles 201, thereby dividing each of the multiple point cloud data into multiple divided point cloud data. Next, as shown in the resolution division diagram of Fig. 26, the division unit 121 divides each of the multiple divided point cloud data into multiple divided point cloud data 301 to 305 for each resolution of the multiple divided point cloud data.

[0021] In the example of resolution division shown in Fig. 26, the division unit 121 divides each of the multiple divided point cloud data into five pieces: low-resolution level divided point cloud data 301 to high-resolution level divided point cloud data 305. However, the number of divisions based on resolution by the division unit 121 is not particularly limited, and may be, for example, three pieces: low-resolution level divided point cloud data, medium-resolution level divided point cloud data, and high-resolution level divided point cloud data, or more. Similarly, the number of divisions based on space by the division unit 121 is not particularly limited.

[0022] The determination unit 122′ determines the spatial quantization value of each of the plurality of point cloud data whose input is accepted by the input accepting unit 11. The spatial quantization value refers to a value used to determine the accuracy of quantization when a plurality of point cloud data in a predetermined space is quantized in the spatial direction. For example, if the plurality of point cloud data includes 3D point cloud data, the spatial quantization value refers to a value used to determine the accuracy of quantization when the 3D point cloud data in the 3D space is quantized in the spatial direction. If the plurality of point cloud data includes 2D point cloud data, the spatial quantization value refers to a value used to determine the accuracy of quantization when the 2D point cloud data in the 2D space is quantized in the spatial direction. In the example shown in FIG. 24 , the determination unit 122′ determines the same spatial quantization value for each of the plurality of divided point cloud data divided by the dividing unit 121. For example, the determination unit 122′ determines the same spatial quantization value for each of the plurality of divided point cloud data 301 to 305 shown in FIG. 26 .

[0023] The encoding unit 123′ encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit 122′. In the example shown in FIG. 24, the encoding unit 123′ encodes each of the plurality of divided point cloud data based on the spatial quantization value of each of the plurality of divided point cloud data divided by the division unit 121. As an example, since the spatial quantization values ​​of each of the plurality of divided point cloud data 301 to 305 shown in FIG. 26 are determined to be the same by the determination unit 122′, the encoding unit 123′ encodes each of the plurality of divided point cloud data 301 to 305 with the same precision. The configuration of the encoding unit 123′ will be described below with reference to FIG. 27. In the example shown in FIG. 27, the encoding unit 123′ includes a spatial quantization unit 1231, an octree encoding unit 1232, an attribute conversion unit 1233, and an arithmetic encoding unit 1234.

[0024] The spatial quantization unit 1231 quantizes each of the multiple pieces of divided point cloud data divided by the dividing unit 121. For example, the spatial quantization unit 1231 converts each of the multiple pieces of divided point cloud data into voxel data by quantizing spatial position information of each of the multiple pieces of divided point cloud data (by performing rounding processing to lattice points) using the spatial quantization values ​​of each of the multiple pieces of divided point cloud data.

[0025] The octree encoding unit 1232 converts the voxel data converted by the spatial quantization unit 1231 into octree structure data. For example, the octree encoding unit 1232 converts the voxel data converted by the spatial quantization unit 1231 into octree structure data 101 shown in FIG. 25 .

[0026] If the octree structure data divided by the octree encoding unit 1232 is attribute data with attributes such as color information, the attribute conversion unit 1233 converts the format of the attributes included in the attribute data.

[0027] The arithmetic coding unit 1234 performs arithmetic coding processing on the octree structure data converted by the octree coding unit 1232 or the attribute data whose attribute format, etc. has been converted by the attribute conversion unit 1233. For example, as shown in Fig. 25 , the arithmetic coding unit 1234 performs arithmetic coding processing on octree structure data 101 to convert it into binary data 102. The arithmetic coding unit 1234 stores the binary data 102 in the storage unit 13 as encoded point cloud data such as a bit stream.

[0028] Returning to the description of FIG. 24 , the storage unit 13 stores various data such as a plurality of point cloud data, a plurality of divided point cloud data, a plurality of encoded point cloud data such as a plurality of encoded divided point cloud data, an OS (Operating System), various programs, etc. For example, the storage unit 13 stores each of the plurality of encoded (compressed) point cloud data for each resolution (resolution level). Examples of the storage unit 13 include storage devices such as HDDs (Hard Disk Drives), SSDs (Solid State Drives), and optical disks, as well as data-rewritable semiconductor memories such as RAMs (Random Access Memory), flash memories, and NVSRAMs (Non-Volatile Static Random Access Memory).

[0029] In order to quickly display, process, and operate large-scale 3D point cloud data such as topographical data, it is conceivable to display the entire space not as a single large-scale point cloud data, but as spatially divided and resolution divided data as shown in Fig. 26. This makes it possible to extract point cloud data of the required spatial area with the required resolution.

[0030] However, the above-described reference techniques may not be able to improve the encoding efficiency in some cases. For example, as shown in the resolution division diagram in Fig. 26 , when encoding a plurality of divided point cloud data divided for each resolution, if the same spatial quantization value is applied to the plurality of divided point cloud data 301 to 305 in all layers, the low-resolution point cloud data will be quantized in the spatial direction with more precision than necessary, which poses a problem of reduced encoding efficiency.

[0031] One aspect of the present disclosure has been made in view of this problem, and makes it possible to improve coding efficiency. Specific techniques will be described in the following embodiments.

[0032] 1. First Embodiment Fig. 1 is a diagram showing an example of the schematic configuration of an encoding system according to a first embodiment. The encoding system 1 receives input of a plurality of point cloud data, determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data, and encodes each of the plurality of point cloud data based on the spatial quantization value. The point cloud data and spatial quantization value are similar to those in the above-mentioned reference technology, and therefore description thereof will be omitted.

[0033] The encoding system 1 includes an encoding device 10 that receives input of a plurality of point cloud data and performs recording-system processing such as encoding on the received input of the plurality of point cloud data, and a playback device 20 that requests playback targets from the encoded point cloud data to the encoding device 10, decodes the playback targets received from the encoding device 10, and plays back the playback targets. In other words, the encoding system 1 is a system in which the encoding device 10 and the playback device 20 are linked together.

[0034] First, an overview of the encoding system 1 will be described with reference to Figures 2 to 4. Figures 2 to 4 are diagrams for explaining the encoding system according to the first embodiment.

[0035] In the example shown in FIG. 2 , the encoding device 10 receives input of a plurality of point cloud data. Based on the space in which each of the plurality of point cloud data is included and the resolution of each of the plurality of point cloud data, the encoding device 10 divides each of the plurality of point cloud data into a plurality of divided point cloud data, namely, one slice of 2k points, one slice of 0.2M points, and 20 slices of 20M points. The encoding device 10 determines a spatial quantization value for each of the plurality of divided point cloud data based on at least one of the resolution and density of each of the plurality of divided point cloud data. For example, the encoding device 10 determines the spatial quantization value of one slice of 2k points to be q1, q1', or q1", where q1 > q1' > q1". The encoding device 10 determines the spatial quantization value of one slice of 0.2M points to be q2, q2', or q2", where q2 > q2' > q2". The encoding device 10 determines the spatial quantization value of 20 slices of 20M points to be q3, q3', or q3'', which satisfies the relationship q3>q3'>q3'', for each slice.

[0036] The encoding device 10 encodes each of the plurality of divided point cloud data based on the spatial quantization value of each of the determined plurality of divided point cloud data. The encoding device 10 determines, based on a request from the playback device 20, from the plurality of encoded point cloud data to be reproduced by the playback device 20.

[0037] In the example shown in FIG. 2 , the playback device 20 requests a playback target to be played back by the playback device 20 from among the multiple point cloud data encoded by the encoding device 10. For example, the playback device 20 acquires at least one of communication information, region information, and viewpoint information, and if the range indicated by the acquired information is within a predetermined range, requests a playback target having a smaller spatial quantization value than if it is outside the predetermined range. The communication information refers to information regarding communication between the encoding device 10 and the playback device 20. The communication information may include communication bandwidth information regarding the communication bandwidth between the encoding device 10 and the playback device 20, such as the available communication bandwidth of a network line that allows communication between the encoding device 10 and the playback device 20. The region information refers to information regarding the region in which the playback target requested by the playback device 20 will be played back. The viewpoint information refers to information regarding the viewpoint from which the playback target will be played back by the playback device 20. The viewpoint information may include information regarding the viewpoint as well as the field of view, camera parameters, and the like.

[0038] As an example, when the range indicated by the acquired information of the communication information, area information, and viewpoint information is within a predetermined range, the playback device 20 requests high-quality point cloud data having a spatial quantization value q2" that is smaller than q2 and q2' as the data to be played back. When the range of the acquired information of the communication information, area information, and viewpoint information is not within the predetermined range, the playback device 20 requests normal-quality point cloud data having a spatial quantization value q2' that is larger than q2" or low-quality point cloud data having a spatial quantization value q2 that is larger than q2' as the data to be played back.

[0039] 3, the reference technology applies a point cloud data structure with the same spatial quantization scale to all resolution levels from point cloud data at level 0 (low resolution level) to level 30 (high resolution level), and simply determines the same spatial quantization value for each resolution level. In this way, if the spatial quantization value of each of multiple point cloud data is simply determined to be the same for each resolution level, encoding efficiency may decrease and subjective degradation of image quality may become noticeable.

[0040] In contrast, the encoding system 1 according to this embodiment can improve coding efficiency by performing quantization control that determines spatial quantization values ​​based on at least one of the resolution and density of each of multiple point cloud data. For example, in addition to the function of GPCC, the encoding system 1 performs adaptive spatial quantization using an adaptive scale that uses a data structure of point cloud data divided by resolution and to which different spatial quantization scales (adaptive scales) are applied. This allows the encoding system 1 to improve coding efficiency. For example, the encoding system 1 can improve coding efficiency by approximately 10%.

[0041] As an example, as shown in FIG. 4 , the encoding system 1 assigns different spatial quantization scales to the resolution levels of the multiple divided point cloud data divided based on the spaces (tiles) containing each of the multiple point cloud data and the resolutions (resolution levels) of each of the multiple point cloud data. Tile N is low-density point cloud data, Tile N+1 is high-density point cloud data, and Tile N+2 is medium-density point cloud data. Furthermore, the encoding system 1 divides each of the multiple point cloud data of Tile N to Tile N+2 based on the resolutions of Level K to Level K+4, and represents them as multiple divided point cloud data included in a portion of a divided space divided into 5x3 grids. That is, the multiple divided point cloud data are divided into five hierarchical levels of resolution and three spaces, and each of the multiple divided point cloud data is represented as a black point cloud in a portion of the divided space divided into 5x3 grids. Each point data included in the point cloud data is represented as a single black dot.

[0042] The encoding system 1 performs encoding (compression) on each of these multiple division point cloud data. In the example shown in FIG. 4 , for multiple division point cloud data with a resolution level K, the encoding system 1 assigns qk as a spatial quantization scale corresponding to the level K as the spatial quantization scale during encoding (compression). The spatial quantization scale refers to a degree of magnitude used to determine the accuracy of quantization when multiple point cloud data in a predetermined space are quantized in the spatial direction. The spatial quantization scale includes, for example, a spatial quantization value. In other words, the spatial quantization value is one of the spatial quantization scales. For multiple division point cloud data with a level K+1, the encoding system 1 assigns q(k+1) as a spatial quantization scale corresponding to the level K+1. For multiple division point cloud data with a level K+2, the encoding system 1 assigns q(k+2) as a spatial quantization scale corresponding to the level K+2. For multiple division point cloud data with a level K+3, the encoding system 1 assigns q(k+3) as a spatial quantization scale corresponding to the level K+3. The encoding system 1 assigns q(k+4) as the spatial quantization scale corresponding to level K + 4 to the plurality of division point group data at level K + 4. Here, the spatial quantization scale is assumed to be q(k+4)<q(k+3)<q(k+2)<q(k+1)<qk.

[0043] In this way, the encoding system 1 assigns a smaller spatial quantization scale to high-resolution point cloud data and a larger spatial quantization scale to low-resolution point cloud data. This allows the encoding system 1 to encode each of the multiple point cloud data using an appropriate spatial quantization scale for each resolution level. As a result, the encoding system 1 improves the image quality of the high-resolution point cloud data and reduces the overall amount of encoding processing by not encoding the low-resolution point cloud data with unnecessary high precision, thereby improving encoding efficiency while maintaining subjective image quality. Note that, as shown in FIG. 4 , the point data included in the point cloud data at each resolution level may or may not be uniformly distributed (at equal intervals).

[0044] Next, returning to Fig. 1, a description will be given of the configuration of the encoding device 10. In the example shown in Fig. 1, the encoding device 10 has a control unit 12 instead of the control unit 12' of the encoding device 10' according to the reference technology, and further has a storage unit 13 and a communication unit 14.

[0045] The control unit 12 has a determination unit 122 and an encoding unit 123 instead of the determination unit 122' and the encoding unit 123' in the control unit 12' according to the reference technology. Except for this, the control unit 12 is similar to the control unit 12' according to the reference technology.

[0046] The determination unit 122 determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data received as input by the input receiving unit 11. In the example shown in FIG. 1 , the determination unit 122 determines a spatial quantization value for each of the plurality of divided point cloud data based on at least one of the resolution and density of each of the plurality of divided point cloud data divided by the dividing unit 121. An example of the configuration of the determination unit 122 will be described below with reference to FIG. 5 . FIG. 5 is a diagram showing an example of the configuration of a control unit of the encoding device according to the first embodiment. In the example shown in FIG. 5 , the determination unit 122 includes a density calculation unit 1221, a spatial quantization value determination unit 1222, and a reproduction target determination unit 1223.

[0047] The density calculation unit 1221 calculates the density of each of the plurality of point cloud data. In this case, the density calculation unit 1221 calculates the density of each of the plurality of point cloud data by calculating the density of the point cloud data within a predetermined space for each of the plurality of point cloud data. As an example, the density calculation unit 1221 calculates the density of each of the plurality of point cloud data whose input has been accepted by the input accepting unit 11. As another example, the density calculation unit 1221 calculates the density of each of the plurality of divided point cloud data divided by the dividing unit 121.

[0048] The spatial quantization value determination unit 1222 determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution of each of the plurality of point cloud data received as input by the input receiving unit 11 and the density calculated by the density calculation unit 1221. In the example shown in Fig. 1 , the spatial quantization value determination unit 1222 determines a spatial quantization value for each of the plurality of divided point cloud data based on at least one of the resolution of each of the plurality of divided point cloud data obtained by division by the division unit 121 and the density calculated by the density calculation unit 1221.

[0049] As one example, the spatial quantization value determination unit 1222 changes the spatial quantization value during compression in accordance with the resolution of each of the multiple divided point cloud data of the same scene. As another example, the spatial quantization value determination unit 1222 changes the spatial quantization value during compression in accordance with the density of point cloud data within a predetermined space for each of the multiple divided point cloud data of the same scene. As another example, the spatial quantization value determination unit 1222 changes the spatial quantization value during compression in accordance with the resolution of each of the multiple divided point cloud data of the same scene and the density of point cloud data within a predetermined space.

[0050] 6 to 9, an example of determining spatial quantization values ​​by the spatial quantization value determination unit 1222 will be described below. Figures 6 to 9 are diagrams for explaining the encoding device according to the first embodiment.

[0051] 6 , the higher the resolution of the point cloud data, the smaller the spatial quantization value determined by the spatial quantization value determination unit 1222. For example, the spatial quantization value determination unit 1222 determines the spatial quantization value to be 1 / 2 times as the resolution level of the point cloud data increases by 1 (becomes finer), so the higher the resolution level, the smaller the spatial quantization value determined by the spatial quantization value determination unit 1222. Furthermore, the spatial quantization value determination unit 1222 determines the spatial quantization value to be a power of a predetermined value. Specifically, the spatial quantization value determination unit 1222 determines the spatial quantization value to be a power of 2.

[0052] 7, the spatial quantization value determination unit 1222 determines a smaller spatial quantization value as the density of the point cloud data increases. The spatial quantization value determination unit 1222 sets the spatial quantization value of tile N, which has the highest density of point cloud data among tiles N to N+2 at level K+1, to the spatial quantization value of tiles N+1 and N+2, which is 2 α+3 Less than 2 α The spatial quantization value determination unit 1222 determines the spatial quantization value of tile N+2, which has a higher density of point cloud data among tiles N+1 and N+2 at the same resolution level K+2, as the spatial quantization value of tile N+1, 2 α+2 Less than 2 α is decided.

[0053] The spatial quantization value determination unit 1222 may determine the same spatial quantization value for the divided point cloud data with the highest resolution in each of the multiple divided spaces divided by the division unit 121. In other words, the spatial quantization value determination unit 1222 may prioritize the spatial quantization value of the divided point cloud data with the finest resolution in each of tile N, tile N+1, and tile N+2.

[0054] For example, the spatial quantization value determination unit 1222 sets the spatial quantization values ​​of the divided point cloud data of the highest resolution level K+1 in tile N, the divided point cloud data of the highest resolution level K+4 in tile N+1, and the divided point cloud data of the highest resolution level K+2 in tile N+2 to the same value 2 α is decided.

[0055] 8, even when β>α, the spatial quantization value determination unit 1222 may prioritize the spatial quantization value of the finest level of divided point cloud data in each of multiple tiles. In the example shown in FIG. 8, the spatial quantization value of the finest level of divided point cloud data in the same tile is significantly larger than the spatial quantization value of the other divided point cloud data. Even in such a case, the spatial quantization value determination unit 1222 may prioritize the spatial quantization value of the finest level of divided point cloud data in each tile. αBy determining the above, the image quality of the point cloud data at a high resolution level can be improved, and the encoding efficiency can be improved while maintaining the subjective image quality.

[0056] Furthermore, the spatial quantization value determination unit 1222 may determine the same spatial quantization value for the divided point cloud data with the highest density at each resolution level among the multiple divided point cloud data divided for each resolution level by the dividing unit 121. In other words, the spatial quantization value determination unit 1222 may prioritize the spatial quantization value of the divided point cloud data with the highest density at each resolution.

[0057] For example, the spatial quantization value determination unit 1222 sets the spatial quantization values ​​of the divided point cloud data of tile N having the highest density at level K+1, the divided point cloud data of tile N+2 having the highest density at level K+2, and the divided point cloud data of tile N+1 having the highest density at level K+4 to the same value 2. α is decided.

[0058] Furthermore, in the above example, the spatial quantization value determination unit 1222 determines the spatial quantization value to be a power of two, but the spatial quantization value may be set to a value other than a power of two, such as a general-purpose value. For example, the spatial quantization value determination unit 1222 determines the spatial quantization value of each of the multiple divided point cloud data to be one of q1 to q5, which satisfies the relational expression q5≦q4≦q3≦q2≦q1. In the example shown in FIG. 9 , the spatial quantization value determination unit 1222 determines the spatial quantization value of each of the multiple divided point cloud data to be one of q1 to q5, so that the higher the resolution of the point cloud data, the smaller the spatial quantization value, and the higher the density of the point cloud data, the smaller the spatial quantization value. Even in such a case, the spatial quantization value determination unit 1222 can determine the spatial quantization value of each of the multiple point cloud data according to the resolution and density of each of the multiple point cloud data.

[0059] Returning to the description of Fig. 5, in the example shown in Fig. 5, the reproduction target determination unit 1223 determines a reproduction target to be reproduced by the reproduction device 20 from among the plurality of point cloud data encoded by the encoding unit 123, based on a request from the reproduction device 20. For example, in response to the request from the reproduction device 20, the reproduction target determination unit 1223 determines a reproduction target to be reproduced at a predetermined resolution, in a predetermined region, at a predetermined viewpoint, and in a predetermined field of view, and to be transmitted (transmitted) to the reproduction device 20, from among the plurality of point cloud data encoded in advance using a plurality of spatial quantization values ​​for each resolution.

[0060] As an example, when low-quality point cloud data is requested by the playback device 20 as the data to be played back, the playback object determination unit 1223 determines that the low-quality point cloud data to be played back is low-quality point cloud data that has been coded by a low-level coding unit in the coding unit 123 (described later) using the spatial quantization value of q1, which is the highest value among q1, q1', and q1". The playback object determination unit 1223 determines that the low-quality point cloud data to be played back is low-quality point cloud data that has been coded by a medium-level (normal level) coding unit in the coding unit 123 (described later) using the spatial quantization value of q2, which is the highest value among q2, q2', and q2". The reproduction object determination unit 1223 determines, as the reproduction object, the low-quality point cloud data that has been coded using the spatial quantization value of q3, which is the highest value among q3, q3', and q3", by a high-level coding unit in the coding unit 123, which will be described later. In this way, when low-quality point cloud data is requested by the playback device 20, the reproduction object determination unit 1223 determines, as the reproduction object, the low-quality point cloud data that has been coded using the spatial quantization values ​​of q1, q2, and q3, respectively.

[0061] The encoding unit 123 encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit 122. For example, the encoding unit 123 encodes each of the plurality of divided point cloud data with accuracy corresponding to the spatial quantization value of each of the plurality of point cloud data. In the example shown in FIG. 1 , the encoding unit 123 encodes each of the plurality of divided point cloud data based on the spatial quantization value of each of the plurality of divided point cloud data determined by the determination unit 122. For example, the encoding unit 123 encodes each of the plurality of divided point cloud data with accuracy corresponding to the spatial quantization value of each of the plurality of divided point cloud data.

[0062] The encoding unit 123 has a plurality of encoding units corresponding to the respective resolution levels of the plurality of point cloud data. For example, the encoding unit 123 has a plurality of encoding units corresponding to the respective resolution levels of the plurality of divided point cloud data divided by the dividing unit 121. Below, an example in which the encoding unit 123 has a plurality of encoding units will be described, but the number of the plurality of encoding units included in the encoding unit 123 is not particularly limited and may be any number corresponding to the number of resolution levels, for example.

[0063] As an example, when the resolution levels of the plurality of division point cloud data are three levels, namely, low level, medium level, and high level, the encoding unit 123 has a low-level encoding unit, a medium-level encoding unit, and a high-level encoding unit. Each of the low-level encoding unit, medium-level encoding unit, and high-level encoding unit receives input of level information regarding the resolution level and a plurality of division point cloud data corresponding to each level information. The low-level encoding unit encodes (compresses) the low-level division point cloud data, the medium-level encoding unit encodes the medium-level division point cloud data, and the high-level encoding unit encodes the high-level division point cloud data.

[0064] In this case, for example, the low-level encoding unit encodes each of the multiple low-level point cloud data with an accuracy corresponding to one of the spatial quantization values ​​q1, q1', and q". When the relationship q1 > q1' > q1" exists, data encoded with the spatial quantization value of q1 becomes low-quality point cloud data, data encoded with the spatial quantization value of q1' becomes medium-quality (normal-quality) point cloud data, and data encoded with the spatial quantization value of q1" becomes high-quality point cloud data. The medium-level and high-level encoding units also encode the multiple point cloud data in the same way as the low-level encoding unit. The encoding unit 123 may encode each of the multiple point cloud data so that the data volume of the encoded point cloud data increases in the order of high-quality point cloud data, normal-quality point cloud data, and low-quality point cloud data.

[0065] As another example, when the resolution levels of the multiple division point cloud data are five levels from level K to level K+4, the encoding unit 123 has a level K encoding unit, a level K+1 encoding unit, a level K+2 encoding unit, a level K+3 encoding unit, and a level K+4 encoding unit. Each of the level K encoding unit to level K+4 encoding unit accepts input of level information related to the resolution level and multiple division point cloud data corresponding to each level information. Each of the level K encoding unit to level K+4 encoding unit encodes (compresses) the division point cloud data of the level corresponding to each of the level K encoding unit to level K+4 encoding unit, out of the division point cloud data of level K to level K+4.

[0066] The communication unit 14 communicates with the communication unit 21 of the playback device 20. For example, the communication unit 14 transmits the playback target determined by the playback target determination unit 1223 to the communication unit 21 of the playback device 20 via a telecommunications line such as a LAN (Local Area Network) or the Internet. An example of the communication unit 14 is a NIC (Network Interface Card).

[0067] 1, a description will be given of the configuration of the playback device 20. In the example shown in FIG.

[0068] The communication unit 21 communicates with the communication unit 14 of the encoding device 10. For example, the communication unit 21 receives the playback target determined by the playback target determination unit 1223 from the communication unit 14 of the encoding device 10 via a telecommunications line such as a LAN or the Internet. An example of the communication unit 21 is a NIC.

[0069] The storage unit 23 stores playback targets, communication information, area information, viewpoint information, an OS, various programs, etc. Examples of the storage unit 23 include storage devices such as HDDs, SSDs, and optical discs, as well as data-rewritable semiconductor memories such as RAMs, flash memories, and NVSRAMs.

[0070] The control unit 22 controls the entire playback device 20. For example, the control unit 22 is configured with one or more processors having an internal memory in which programs defining each processing procedure and control data are stored, and the processor executes each process using the programs and internal memory. Examples of the control unit 22 include electronic circuits such as a CPU, MPU, and GPU, as well as integrated circuits such as an ASIC and FPGA. In the example shown in FIG. 1 , the control unit 22 has an acquisition unit 221, a request unit 222, and a playback unit 223.

[0071] The acquisition unit 221 acquires at least one of communication information related to communication between the encoding device 10 and the playback device 20, region information related to a region in which a playback target requested by the playback device 20 is to be played back, and viewpoint information related to a viewpoint from which the playback target is to be played back by the playback device 20. In the example shown in Fig. 1 , the acquisition unit 221 acquires the communication information, region information, viewpoint information, and the playback target received by the communication unit 21. The method by which the acquisition unit 221 acquires the communication information, region information, and viewpoint information is not particularly limited. For example, the acquisition unit 221 may acquire the communication information, region information, and viewpoint information via the communication unit 21 and the communication unit 14 of the encoding device 10, or may acquire the communication information, region information, and viewpoint information from outside the encoding system 1.

[0072] The request unit 222 requests a reproduction target to be reproduced by the reproduction device 20 from among the plurality of point cloud data encoded by the encoding unit 123. The request unit 222 may request a reproduction target based on the information acquired by the acquisition unit 221 from among the communication information, the area information, and the viewpoint information.

[0073] As one example, when the communication bandwidth indicated by the communication information acquired by the acquisition unit 221 is equal to or greater than a threshold, the request unit 222 requests a playback target with a smaller spatial quantization value than when the communication bandwidth is not equal to or greater than the threshold. As another example, when the region indicated by the region information acquired by the acquisition unit 221 is within a region of interest (ROI), the request unit 222 requests a playback target with a smaller spatial quantization value than when the region is not within the region of interest. As another example, when the distance between the viewpoint indicated by the viewpoint information acquired by the acquisition unit 221 and the playback target is within a predetermined range, the request unit 222 requests a playback target with a smaller spatial quantization value than when the distance is outside the predetermined range. For example, when the playback target played back by the playback device 20 is viewed from a nearby viewpoint, the request unit 222 requests high-quality point cloud data, such as high-density point cloud data, with a smaller spatial quantization value than when the playback target is viewed from a distant viewpoint.

[0074] When requesting high-quality point cloud data as the reproduction target, the requesting unit 222 may determine point cloud data up to a medium-level resolution as the reproduction target instead of point cloud data up to a high-level resolution. However, in this case, the requesting unit 222 determines point cloud data up to a medium-level resolution that satisfies a predetermined spatial resolution as the reproduction target.

[0075] The playback unit 223 plays the playback target. For example, the playback unit 223 decodes the playback target acquired by the acquisition unit 221 and displays the decoded playback target on a display unit, thereby playing back the playback target. As shown in Fig. 1 , if the playback device 20 does not have a display unit, the playback unit 223 displays the decoded playback target on a display unit external to the playback device 20. However, if the playback device 20 has a display unit, the playback unit 223 may display the decoded playback target on a display unit internal to the playback device 20.

[0076] Next, an example of each process executed by the encoding device 10 according to the first embodiment will be described with reference to Fig. 10 to Fig. 14. Fig. 10 to Fig. 14 are flow charts showing an example of the process of the encoding device according to the first embodiment. First, an example of the overall process executed by the encoding device 10 will be described with reference to Fig. 10.

[0077] In step S1, the input receiving unit 11 receives input of a plurality of point cloud data.

[0078] In step S2, the spatial quantization value determination unit 1222 determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data whose input has been accepted by the input accepting unit 11.

[0079] In step S3 , the encoding unit 123 encodes each of the plurality of point cloud data based on the spatial quantization value determined by the spatial quantization value determination unit 1222 .

[0080] Next, an example of the spatial quantization value determination process corresponding to step S2 in FIG. 10 will be described in detail with reference to FIG.

[0081] In step S21, the density calculation unit 1221 calculates the density of each of the plurality of point cloud data.

[0082] In step S22, the spatial quantization value determination unit 1222 determines the spatial quantization value of each of the plurality of point cloud data.

[0083] Next, an example of the density calculation process corresponding to step S21 in FIG. 11 will be described in detail with reference to FIG.

[0084] In step S211, the density calculation unit 1221 receives input of each of the plurality of point cloud data from the input receiving unit 11.

[0085] In step S212, the density calculation unit 1221 calculates the number of point data for each of the plurality of point cloud data whose input has been accepted.

[0086] In step S213, the density calculation unit 1221 outputs the number of point data for each of the calculated plurality of point cloud data. For example, the density calculation unit 1221 outputs information on the number of point data for each of the calculated plurality of point cloud data to the spatial quantization value determination unit 1222 as density calculation processing result information on the result of the density calculation processing.

[0087] Next, an example of the spatial quantization value determination process corresponding to step S22 in FIG. 11 will be described in detail with reference to FIG.

[0088] In step S221, the spatial quantization value determination unit 1222 receives an input of the number of point data for each of the plurality of point cloud data from the density calculation unit 1221. For example, the spatial quantization value determination unit 1222 receives an input of the number of point data for each of the plurality of point cloud data indicated by the density calculation processing result information output from the density calculation unit 1221.

[0089] In step S222, the spatial quantization value determination unit 1222 determines whether the number of point data in each of the plurality of point cloud data is equal to or less than a threshold value. That is, the spatial quantization value determination unit 1222 determines whether the number of point data in each of the plurality of point cloud data is equal to or less than a threshold value.

[0090] If it is determined that the number of point data of the point cloud data included in the plurality of point cloud data is equal to or less than the threshold value (Yes in step S222), in step S223, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data in which the number of point data is equal to or less than the threshold value to be a value greater than the spatial quantization value of the point cloud data in which the number of point data is greater than the threshold value. For example, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data in which the number of point data is equal to or less than the threshold value to be a spatial quantization value for sparse data, which is low-density point cloud data, greater than the spatial quantization value of the point cloud data in which the number of point data exceeds the threshold value.

[0091] If it is determined that the number of point data of the point cloud data included in the plurality of point cloud data is not equal to or less than the threshold value (No in step S222), in step S224, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data in which the number of point data is not equal to or less than the threshold value to be a value smaller than the spatial quantization value of the point cloud data in which the number of point data is equal to or less than the threshold value. For example, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data in which the number of point data exceeds the threshold value to be a spatial quantization value for dense data, which is high-density point cloud data, smaller than the spatial quantization value of the point cloud data in which the number of point data is equal to or less than the threshold value.

[0092] In step S225, the spatial quantization value determination unit 1222 outputs the determined spatial quantization value.

[0093] Next, another example of the spatial quantization value determination process corresponding to step S22 in Fig. 11 will be described in detail with reference to Fig. 14. Step S225 in Fig. 14 is similar to step S225 in Fig. 13, and therefore description thereof will be omitted.

[0094] In step S221X, the spatial quantization value determination unit 1222 receives an input of the resolution of each of the plurality of point cloud data. As one example, the spatial quantization value determination unit 1222 receives an input of level information relating to the resolution level of each of the plurality of point cloud data from the input receiving unit 11. As another example, the spatial quantization value determination unit 1222 receives an input of level information relating to the resolution level of each of the plurality of divided point cloud data from the dividing unit 121.

[0095] In step S222X, the spatial quantization value determination unit 1222 determines whether the resolution of each of the plurality of point cloud data is equal to or less than a threshold value. That is, the spatial quantization value determination unit 1222 determines whether the resolution of each of the plurality of point cloud data is equal to or less than a threshold value.

[0096] If it is determined that the resolution of point cloud data included in the plurality of point cloud data is equal to or less than the threshold (Yes in step S222X), in step S223X, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data having a resolution equal to or less than the threshold to be a value greater than the spatial quantization value of the point cloud data having a resolution greater than the threshold. For example, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data having a resolution equal to or less than the threshold to be a spatial quantization value for sparse data that is greater than the spatial quantization value of the point cloud data having a resolution greater than the threshold.

[0097] If it is determined that the resolution of point cloud data included in the plurality of point cloud data is not equal to or less than the threshold (No in step S222X), in step S224X, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data whose resolution is not equal to or less than the threshold to be a value smaller than the spatial quantization value of the point cloud data whose resolution is equal to or less than the threshold. For example, the spatial quantization value determiner 1222 determines the spatial quantization value of the point cloud data whose resolution exceeds the threshold to be a spatial quantization value for dense data that is smaller than the spatial quantization value of the point cloud data whose resolution is equal to or less than the threshold.

[0098] In the examples shown in Figures 13 and 14, the spatial quantization value determiner 1222 determines the spatial quantization value based on one of the density and resolution of the point cloud data, but may determine the spatial quantization value based on both. For example, if both the number of points and the resolution of the point cloud data exceed a threshold, the spatial quantization value determiner 1222 may determine the spatial quantization value of the point cloud data to be that for high-quality point cloud data (for ultra-dense data). If either the number of points or the resolution of the point cloud data exceeds a threshold, the spatial quantization value determiner 1222 may determine the spatial quantization value of the point cloud data to be that for normal-quality point cloud data (for dense data). If both the number of points and the resolution of the point cloud data are equal to or less than a threshold, the spatial quantization value determiner 1222 may determine the spatial quantization value of the point cloud data to be that for low-quality point cloud data (for sparse data).

[0099] Next, an example of each process executed by the playback device 20 according to the first embodiment will be described with reference to Figures 15 to 19. Figures 15 to 19 are flow charts showing an example of the process executed by the playback device according to the first embodiment. First, an example of request processing executed by the playback device 20 will be described with reference to Figure 15.

[0100] 15 , the acquiring unit 221 acquires communication information including communication bandwidth information such as the available communication bandwidth of a network line that allows communication between the encoding device 10 and the reproduction device 20.

[0101] In step S42, the request unit 222 determines whether the width of the communication band indicated by the communication information acquired by the acquisition unit 221 is equal to or greater than a low threshold.

[0102] If it is determined that the communication bandwidth is equal to or greater than the low threshold (Yes in step S42), in step S43, the request unit 222 requests normal quality point cloud data with a smaller spatial quantization value as the reproduction target than when it is determined that the communication bandwidth is not equal to or greater than the low threshold.

[0103] If it is determined that the communication bandwidth is not greater than or equal to the low threshold (No in step S42), in step S44, the request unit 222 requests low-quality point cloud data with a larger spatial quantization value as the reproduction target than when it is determined that the communication bandwidth is greater than or equal to the low threshold.

[0104] Next, another example of the request process executed by the playback device 20 will be described with reference to Fig. 16. Step S41 in Fig. 16 is the same as step S41 in Fig. 15.

[0105] In step S42X, the request unit 222 determines whether the width of the communication band indicated by the communication information acquired by the acquisition unit 221 is equal to or greater than the high threshold.

[0106] If it is determined that the communication bandwidth is equal to or greater than the high threshold (Yes in step S42X), in step S43X, the requesting unit 222 requests, as the reproduction target, high-quality point cloud data having a smaller spatial quantization value than when it is determined that the communication bandwidth is not equal to or greater than the high threshold. In this way, if the communication bandwidth is equal to or greater than the high threshold and is sufficiently wide, the requesting unit 222 requests high-quality point cloud data as the reproduction target in order to improve the quality of the reproduction target.

[0107] If it is determined that the communication bandwidth is not greater than or equal to the high threshold (No in step S42X), in step S44X, the request unit 222 requests normal quality point cloud data with a larger spatial quantization value as the reproduction target than when it is determined that the communication bandwidth is greater than or equal to the high threshold.

[0108] 15 and 16, the request unit 222 separately performs the request process shown in Fig. 15 and the request process shown in Fig. 16, but these request processes may be performed together. For example, if the answer is Yes in step S42, the request unit 222 may perform the processes of steps S42X to S44X instead of step S43. In this way, by performing a two-stage determination, the request unit 222 can request a playback target that is more appropriate for the communication bandwidth.

[0109] Next, another example of the request process executed by the playback device 20 will be described with reference to FIG.

[0110] In step S61, the acquisition unit 221 acquires area information.

[0111] In step S62, the request unit 222 determines whether the area indicated by the area information acquired by the acquisition unit 221 is within the range of the attention area. For example, the request unit 222 determines whether the area indicated by the area information acquired in advance by the acquisition unit 221 and to be requested by the request unit 222 is within the range of the attention area.

[0112] If it is determined that the region is within the range of the region of interest (Yes in step S62), in step S63, the request unit 222 requests high-quality point cloud data to be reproduced, the high-quality point cloud data having a smaller spatial quantization value than if it were determined that the region is not within the range of the region of interest.

[0113] If it is determined that the area is not within the area of ​​interest (No in step S62), in step S64, the requesting unit 222 requests normal quality point cloud data to be reproduced, which has a larger spatial quantization value than if it were determined that the area was within the area of ​​interest.

[0114] Next, another example of the request process executed by the playback device 20 will be described with reference to Fig. 18. Steps S61 and S62 in Fig. 18 are similar to steps S61 and S62 in Fig. 17.

[0115] If it is determined that the area is within the range of the area of ​​interest (Yes in step S62), in step S63X, the requesting unit 222 requests normal quality point cloud data to be reproduced, which has a smaller spatial quantization value than if it were determined that the area is not within the range of the area of ​​interest.

[0116] If it is determined that the area is not within the area of ​​interest (No in step S62), in step S64X, the requesting unit 222 requests low-quality point cloud data to be reproduced that has a larger spatial quantization value than if it were determined that the area was within the area of ​​interest.

[0117] 17 and 18 , when a region is determined to be within the region of interest, the reproduction target requested by the requesting unit 222 is not particularly limited as long as the spatial quantization value is smaller than when the region is determined to be outside the region of interest. For example, the reproduction target may be high-quality point cloud data or normal-quality point cloud data.

[0118] Next, another example of the request process executed by the playback device 20 will be described with reference to FIG.

[0119] In step S81, the acquisition unit 221 acquires viewpoint information.

[0120] In step S82, the request unit 222 determines whether the distance between the viewpoint indicated by the viewpoint information acquired by the acquisition unit 221 and the playback target is within a predetermined range.

[0121] If it is determined that the distance between the viewpoint and the object to be reproduced is within a predetermined range (Yes in step S82), in step S83, the request unit 222 requests high-quality point cloud data as the object to be reproduced, the high-quality point cloud data having a smaller spatial quantization value than when it is determined that the distance is not within the predetermined range.

[0122] If it is determined that the distance between the viewpoint and the object to be reproduced is not within a specified range (No in step S82), in step S84, the requesting unit 222 requests normal quality point cloud data as the object to be reproduced, the point cloud data having a larger spatial quantization value than when it is determined that the distance is within the specified range.

[0123] Next, another example of the request process executed by the playback device 20 will be described with reference to Fig. 20. Steps S81 and S82 in Fig. 20 are similar to steps S81 and S82 in Fig. 19.

[0124] If it is determined that the distance between the viewpoint and the object to be reproduced is within a predetermined range (Yes in step S82), in step S83X, the requesting unit 222 requests normal quality point cloud data as the object to be reproduced, the normal quality point cloud data having a smaller spatial quantization value than if it is determined that the distance is not within the predetermined range.

[0125] If it is determined that the distance between the viewpoint and the object to be reproduced is not within a specified range (No in step S82), in step S84X, the requesting unit 222 requests low-quality point cloud data as the object to be reproduced, the low-quality point cloud data having a larger spatial quantization value than when it is determined that the distance is within the specified range.

[0126] 19 and 20 , when it is determined that the distance between the viewpoint and the reproduction target is within a predetermined range, the reproduction target requested by the request unit 222 is not particularly limited as long as its spatial quantization value is smaller than that when it is determined that the distance is not within the predetermined range. For example, the reproduction target may be high-quality point cloud data or normal-quality point cloud data.

[0127] 2. Second Embodiment An encoding system according to one aspect of the present disclosure may further include a 3D sensing processing unit instead of or in addition to the above-described division unit. Hereinafter, an encoding system 1X according to the second embodiment will be described with reference to FIGS. 21 and 22 . FIG. 21 is a diagram illustrating an example of a schematic configuration of an encoding system according to the second embodiment. FIG. 22 is a diagram illustrating an example of a configuration of a control unit of an encoding device according to the second embodiment.

[0128] In the example shown in Fig. 21 , the encoding system 1X has an encoding device 10X instead of the encoding device 10 according to the first embodiment. Except for this, the encoding system 1X is similar to the encoding system 1 according to the first embodiment. The encoding device 10X has a control unit 12X instead of the control unit 12 according to the first embodiment, and further has a 3D sensor 15. Except for this, the encoding system 10X is similar to the encoding device 10 according to the first embodiment.

[0129] The 3D sensor 15 generates a plurality of point cloud data including 3D point cloud data, and inputs the generated plurality of point cloud data to the input receiving unit 11. In the example shown in Fig. 21 , the encoding device 10X includes the 3D sensor 15, but may not include the 3D sensor 15.

[0130] The control unit 12X has a determination unit 122X instead of the dividing unit 121 and the determination unit 122 in the first embodiment. Except for this, the control unit 12X is the same as the control unit 12 in the one-to-one embodiment. Next, the configuration of the determination unit 122X will be described with reference to FIG. 22 .

[0131] 22, the determination unit 122X has a 3D sensing processing unit 1221X and a spatial quantization value determination unit 1222X instead of the density calculation unit 1221 and the spatial quantization value determination unit 1222. Except for this, the determination unit 122X is similar to the determination unit 122 in the first embodiment.

[0132] The 3D sensing processing unit 1221X detects at least one of the resolution and density of the 3D point cloud data based on the 3D point cloud data input accepted by the input accepting unit 11. The 3D sensing processing unit 1221X may detect the resolution of the 3D point cloud data based on the 3D point cloud data input accepted by the input accepting unit 11, and divide the 3D point cloud data into a plurality of divided point cloud data based on the detected resolution of the 3D point cloud data. In this case, the 3D sensing processing unit 1221X divides the 3D point cloud data into a plurality of divided point cloud data by classifying the resolution of the detected 3D point cloud data based on information regarding at least one of the area and resolution of the space when encoding the point cloud data set by the control unit 12X that controls the 3D sensing processing unit 1221X, for example.

[0133] The spatial quantization value determination unit 1222X determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of the 3D point cloud data detected by the 3D sensing processing unit 1221X. The spatial quantization value determination unit 1222X may determine a spatial quantization value for each of the plurality of divided point cloud data divided by the 3D sensing processing unit 1221X based on at least one of the resolution and density of the 3D point cloud data detected by the 3D sensing processing unit 1221X.

[0134] 3. Example of Hardware Configuration The various devices described above, such as the encoding device 10, the encoding device 10X, and the playback device 20, may be configured to include a computer. An example will be described with reference to FIG.

[0135] 23 is a diagram showing an example of the hardware configuration of the device. The illustrated computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0136] The CPU 1100 operates and controls each component based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0137] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0138] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records programs for executing each operation related to the present disclosure, which are an example of program data 1450.

[0139] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0140] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), magneto-optical recording media such as an MO (Magneto-Optical Disk), tape media, magnetic recording media, and semiconductor memories.

[0141] At least some of the functions of the encoding device 10, encoding device 10X, playback device 20, etc. described above may be realized, for example, by the CPU 1100 of the computer 1000 executing a program loaded onto the RAM 1200. Furthermore, the HDD 1400 stores programs and the like according to the present disclosure. The CPU 1100 reads and executes program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from another device via an external network 1550.

[0142] 1 to 22 etc., one of the disclosed techniques is an encoding device 10 etc. The encoding device 10 has an input receiving unit 11 that receives input of a plurality of point cloud data, a determination unit 122 (spatial quantization value determination unit 1122) that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data received as input by the input receiving unit 11, and an encoding unit 123 that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit 122.

[0143] The encoding device 10 determines the spatial quantization value of each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data, thereby reducing the data size of each of the plurality of point cloud data during compression (encoding). This allows the encoding device 10 to improve encoding efficiency. For example, the encoding device 10 can improve encoding efficiency by approximately 10%. Furthermore, compared to when the same spatial quantization value is simply determined for each resolution level, the encoding device 10 can perform encoding without degrading subjective image quality while reducing data size.

[0144] 1 to 22, the input receiving unit 11 may receive input of a plurality of point cloud data including 3D point cloud data. The encoding device 10 can particularly improve the encoding efficiency of 3D point cloud data.

[0145] As described with reference to Figures 1 to 22, the determination unit 122 may determine a smaller spatial quantization value as the resolution of the point cloud data increases. This allows the encoding device 10 to encode each of the multiple point cloud data at an appropriate spatial quantization scale corresponding to each resolution level. As a result, the encoding device 10 improves the image quality of the high-resolution point cloud data and reduces the overall amount of encoding processing by not encoding the low-resolution point cloud data with unnecessary high precision, thereby improving encoding efficiency while maintaining subjective image quality.

[0146] 1 to 22, the determining unit 122 may determine a smaller spatial quantization value as the density of the point cloud data increases. The higher the density of the point cloud data, the more noticeable and easily perceptible the degradation of subjective image quality becomes. However, the encoding device 10 can improve the image quality of high-density point cloud data, making the degradation of subjective image quality less noticeable.

[0147] 1 to 22, the determination unit 122 may determine the spatial quantization value as a power of a predetermined value, which enables the encoding device 10 to determine a spatial quantization value close to the spatial quantization value of the GPCC octree.

[0148] 1 to 22, the determination unit 122 may determine the spatial quantization value as a power of 2. This allows the encoding device 10 to determine a spatial quantization value that is closer to the spatial quantization value of the GPCC octree.

[0149] As described with reference to FIGS. 1 to 22 , the determination unit 122 (playback target determination unit 1223) may determine, based on a request from the playback device 20, a playback target to be played back by the playback device 20 from among the multiple point cloud data encoded by the encoding unit 123. This allows the encoding device 10 to appropriately switch the quality and playback speed of the playback target in response to a request from the playback device 20. Therefore, the encoding device 10 can, for example, improve the image quality of the playback target by the playback device 20 in response to a request, or reduce a decrease in the playback speed of the playback target by the playback device 20 due to an increase in the amount of low-resolution encoded point cloud data transmitted. In this way, the encoding device 10 can improve the convenience of playback of the playback target by the playback device 20.

[0150] 1 to 22 , etc., the encoding device 10 may further include a division unit 121 that divides each of the plurality of point cloud data into a plurality of divided point cloud data based on at least one of the space in which each of the plurality of point cloud data is included and the resolution of each of the plurality of point cloud data, a determination unit 122 that determines a spatial quantization value for each of the plurality of divided point cloud data based on at least one of the resolution and density of each of the plurality of divided point cloud data divided by the division unit 121, and an encoding unit 123 that encodes each of the plurality of divided point cloud data based on the spatial quantization value for each of the plurality of divided point cloud data determined by the determination unit 122. This allows the encoding device 10 to determine and encode an appropriate spatial quantization value for each of the plurality of divided point cloud data.

[0151] As described with reference to Figures 1 to 22, the division unit 121 divides a space including each of the plurality of point cloud data into a plurality of divided spaces, thereby dividing each of the plurality of point cloud data into a plurality of divided point cloud data, and the determination unit 122 may determine the same spatial quantization value for the divided point cloud data with the highest resolution in each of the plurality of divided spaces divided by the division unit 121. This allows the encoding device 10 to improve the image quality of the divided point cloud data with the highest resolution in each of the plurality of divided spaces. Furthermore, even if the spatial quantization value of the point cloud data with the highest resolution is significantly higher than the spatial quantization value of other divided point cloud data in the same divided space, the encoding device 10 can improve the image quality of the high-resolution point cloud data, thereby improving encoding efficiency while maintaining subjective image quality.

[0152] 1 to 22 , the dividing unit 121 may divide each of the plurality of point cloud data into a plurality of divided point cloud data for each resolution level of the plurality of point cloud data, and the determining unit 122 may determine, as the same value, the spatial quantization values ​​of the divided point cloud data with the highest density at each resolution level among the plurality of point cloud data divided for each resolution level by the dividing unit 121. This enables the encoding device 10 to improve the image quality of the divided point cloud data with the highest density, thereby improving encoding efficiency while maintaining subjective image quality.

[0153] As described with reference to Figures 1 to 22, the encoding device 10 may further include a 3D sensing processing unit 1221X that detects at least one of the resolution and density of the 3D point cloud data based on the 3D point cloud data input received by the input receiving unit 11, and the determination unit 122 may determine a spatial quantization value for each of the multiple point cloud data based on at least one of the resolution and density of the 3D point cloud data detected by the 3D sensing processing unit 1221X. This allows the encoding device 10 to improve the encoding efficiency of the 3D point cloud data. Furthermore, the encoding device 10 can calculate the density of each of the multiple point cloud data using the 3D sensing processing unit 1221X even without including the density calculation unit 1221.

[0154] 1 to 22 , etc., the 3D sensing processing unit 1221X may detect the resolution of the 3D point cloud data based on the 3D point cloud data input received by the input receiving unit 11, and divide the 3D point cloud data into multiple divided point cloud data based on the detected resolution of the 3D point cloud data, the determination unit 122 may determine a spatial quantization value for each of the multiple divided point cloud data divided by the 3D sensing processing unit 1221X based on at least one of the resolution and density of the 3D point cloud data detected by the 3D sensing processing unit 1221X, and the encoding unit 123 may encode each of the multiple divided point cloud data based on the spatial quantization value for each of the multiple divided point cloud data determined by the determination unit 122. In this way, the encoding device 10 can determine an appropriate spatial quantization value for each of the multiple divided point cloud data and encode it without having a dividing unit 121.

[0155] The encoding system described with reference to Figures 1 to 22 is also one of the disclosed technologies. The encoding system 1 includes an encoding device 10 and a playback device 20. The encoding device 10 includes an input receiving unit 11 that receives input of a plurality of point cloud data, a determination unit 122 that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data received as input by the input receiving unit 11, and an encoding unit 123 that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit 122. The playback device 20 includes a request unit 222 that requests a reproduction target to be reproduced by the playback device 20 from the plurality of point cloud data encoded by the encoding unit 123. As described above, such an encoding system 1 can also improve encoding efficiency.

[0156] 1 to 22, the determination unit 122 may determine the playback target from among the plurality of point cloud data encoded by the encoding unit 123, based on a request from the playback device 20. As described above, such an encoding system 1 can also appropriately switch the quality and playback speed of the playback target in response to a request.

[0157] 1 to 22 , etc., the playback device 20 further includes an acquisition unit 221 that acquires at least one of communication information related to communication between the encoding device 10 and the playback device 20, region information related to a region in which a playback target requested by the playback device 20 is to be played back, and viewpoint information related to a viewpoint from which the playback target is to be played back by the playback device 20, and the request unit 222 may request a playback target based on the information acquired by the acquisition unit 221 out of the communication information, region information, and viewpoint information. This allows the encoding system 1 to request an appropriate playback target according to at least one of the communication information, region information, and viewpoint information.

[0158] As described with reference to Figures 1 to 22, the acquisition unit 221 acquires communication information, and when the communication bandwidth indicated by the communication information acquired by the acquisition unit 221 is equal to or greater than a threshold, the request unit 222 may request a playback target with a smaller spatial quantization value than when the communication bandwidth is less than the threshold. This allows the encoding system 1 to request an appropriate playback target according to the communication bandwidth indicated by the communication information. For example, even when the communication bandwidth of the network line between the encoding device 10 and the playback device 20 fluctuates, the encoding system 1 can request an appropriate playback target according to the communication bandwidth by changing the playback target to another playback target with different quality in accordance with the fluctuation in communication bandwidth. The encoding system 1 can save communication bandwidth by requesting a playback target with a smaller spatial quantization value only when the communication bandwidth is equal to or greater than a threshold, compared to when the communication bandwidth is less than the threshold. Furthermore, the encoding system 1 can play back the playback target at a higher speed when the communication bandwidth is greater than the threshold.

[0159] As described with reference to Figures 1 to 22, the acquisition unit 221 acquires region information, and the request unit 222 may request a reproduction target with a smaller spatial quantization value when the region indicated by the region information acquired by the acquisition unit 221 is within the region of interest, compared to when the region is not within the region of interest. This allows the encoding system 1 to request an appropriate reproduction target according to the region indicated by the region information. The encoding system 1 can save communication bandwidth by requesting a reproduction target with a smaller spatial quantization value only when the region is within the region of interest, compared to when the region is not within the region of interest. Furthermore, the encoding system 1 can reproduce the reproduction target at high speed when the region is not within the region of interest.

[0160] As described with reference to Figures 1 to 22, the acquisition unit 221 acquires viewpoint information, and the request unit 222 may request a playback target with a smaller spatial quantization value when the distance between the viewpoint indicated by the viewpoint information acquired by the acquisition unit 221 and the playback target is within a predetermined range, compared to when the distance is outside the predetermined range. This allows the encoding system 1 to request an appropriate playback target according to the distance between the viewpoint indicated by the viewpoint information and the playback target. The encoding system 1 can save communication bandwidth by requesting a playback target with a smaller spatial quantization value only when this distance is within the predetermined range, compared to when this distance is outside the predetermined range. Furthermore, the encoding system 1 can play back the playback target at a high speed when this distance is longer than the predetermined range.

[0161] The encoding method described with reference to Figures 1 to 22 is also one of the disclosed techniques. The encoding method is an encoding method executed by an encoding device 10, and includes an input receiving step (step S1) of receiving input of a plurality of point cloud data, a determination step (step S2) of determining a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data received in the input receiving step, and an encoding step (step S3) of encoding each of the plurality of point cloud data based on the spatial quantization value determined in the determination step. As described above, such an encoding method can also improve encoding efficiency.

[0162] The encoding program described with reference to Figures 1 to 22 is also one of the disclosed technologies. The encoding program causes a computer 1000 installed in an encoding device 10 to execute an input reception process for receiving input of a plurality of point cloud data, a determination process for determining a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data received by the input reception process, and an encoding process for encoding each of the plurality of point cloud data based on the spatial quantization value determined by the determination process. As described above, such an encoding program can also improve encoding efficiency.

[0163] The effects described in this disclosure are merely examples and are not limited to the disclosed contents. Other effects may also be present.

[0164] The above describes embodiments of the present disclosure, but the technical scope of the present disclosure is not limited to the above-described embodiments as they are, and various modifications are possible within the scope that does not deviate from the gist of the present disclosure, and different components across various modifications may be combined as appropriate.

[0165] This technology may be related to Goal 9, "Industry, innovation, infrastructure," of the Sustainable Development Goals (SDGs) adopted at the United Nations Summit in 2015.

[0166] The present technology may also be configured as follows. (1) An encoding device including: an input receiving unit that receives input of a plurality of point cloud data; a determination unit that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of a resolution and a density of each of the plurality of point cloud data received by the input receiving unit; and an encoding unit that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit. (2) The encoding device described in (1), wherein the input receiving unit receives input of the plurality of point cloud data including 3D (Dimensions) point cloud data. (3) The encoding device described in (1) or (2), wherein the determination unit determines the spatial quantization value to be smaller as the resolution of the point cloud data increases. (4) The encoding device described in any of (1) to (3), wherein the determination unit determines the spatial quantization value to be smaller as the density of the point cloud data increases. (5) The encoding device according to any one of (1) to (4), wherein the determination unit determines the spatial quantization value to be a power of a predetermined value. (6) The encoding device according to (5), wherein the determination unit determines the spatial quantization value to be a power of 2. (7) The encoding device according to any one of (1) to (6), wherein the determination unit determines, based on a request from a playback device, which of the plurality of point cloud data encoded by the encoding unit is to be played back by the playback device. (8) The encoding device according to (1), further comprising a division unit that divides each of the plurality of point cloud data into a plurality of divided point cloud data based on at least one of a space including each of the plurality of point cloud data and a resolution of each of the plurality of point cloud data, wherein the determination unit determines a spatial quantization value for each of the plurality of divided point cloud data based on at least one of a resolution and a density of each of the plurality of divided point cloud data divided by the division unit, and wherein the encoding unit encodes each of the plurality of divided point cloud data based on the spatial quantization value for each of the plurality of divided point cloud data determined by the determination unit.(9) The encoding device according to (8), wherein the dividing unit divides a space including each of the plurality of point cloud data into a plurality of divided spaces to divide each of the plurality of point cloud data into a plurality of divided point cloud data, and the determining unit determines the spatial quantization value of the divided point cloud data having the highest resolution in each of the plurality of divided spaces divided by the dividing unit to be the same value. (10) The encoding device according to (8), wherein the dividing unit divides each of the plurality of point cloud data into a plurality of divided point cloud data for each resolution level of the plurality of point cloud data, and the determining unit determines the spatial quantization value of the divided point cloud data having the highest density in each resolution level among the plurality of divided point cloud data divided by the dividing unit for each resolution level to be the same value. (11) The encoding device according to (2), further comprising a 3D sensing processing unit that detects at least one of a resolution and a density of the 3D point cloud data based on the 3D point cloud data input received by the input receiving unit, and the determination unit determines a spatial quantization value of each of the plurality of point cloud data based on at least one of the resolution and the density of the 3D point cloud data detected by the 3D sensing processing unit. (12) The encoding device according to (11), wherein the 3D sensing processing unit detects a resolution of the 3D point cloud data based on the 3D point cloud data input received by the input receiving unit, and divides the 3D point cloud data into a plurality of divided point cloud data based on the detected resolution of the 3D point cloud data; the determination unit determines a spatial quantization value of each of the plurality of divided point cloud data divided by the 3D sensing processing unit based on at least one of the resolution and density of the 3D point cloud data detected by the 3D sensing processing unit; and the encoding unit encodes each of the plurality of divided point cloud data based on the spatial quantization value of each of the plurality of divided point cloud data determined by the determination unit.(13) An encoding system comprising an encoding device and a playback device, wherein the encoding device comprises: an input receiving unit that receives input of a plurality of point cloud data, a determination unit that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of a resolution and a density of each of the plurality of point cloud data whose input is received by the input receiving unit, and an encoding unit that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit, and the playback device comprises a request unit that requests a playback target from the plurality of point cloud data encoded by the encoding unit to be played back by the playback device. (14) The encoding system according to (13), wherein the determination unit determines the playback target from the plurality of point cloud data encoded by the encoding unit based on a request from the playback device. (15) The encoding system according to (13) or (14), wherein the playback device further includes an acquisition unit that acquires at least one of communication information related to communication between the encoding device and the playback device, area information related to an area in which the playback target requested by the playback device is to be played back, and viewpoint information related to a viewpoint from which the playback target is to be played back by the playback device, and the request unit requests the playback target based on the information acquired by the acquisition unit among the communication information, the area information, and the viewpoint information. (16) The encoding system according to (15), wherein the acquisition unit acquires the communication information, and when a communication bandwidth indicated by the communication information acquired by the acquisition unit is equal to or greater than a threshold, the request unit requests the playback target with a smaller spatial quantization value than when the communication bandwidth is not equal to or greater than the threshold. (17) The encoding system according to (15), wherein the acquisition unit acquires the region information, and the request unit requests the reproduction target having a smaller spatial quantization value when the region indicated by the region information acquired by the acquisition unit is within a range of a region of interest than when the region is not within the range of the region of interest.(18) The encoding system according to (15), wherein the acquisition unit acquires the viewpoint information, and the request unit requests the reproduction target having the spatial quantization value smaller when a distance between the viewpoint indicated by the viewpoint information acquired by the acquisition unit and the reproduction target is within a predetermined range than when the distance is outside the predetermined range. (19) An encoding method executed by an encoding device, comprising: an input accepting step of accepting input of a plurality of point cloud data, a determination step of determining a spatial quantization value for each of the plurality of point cloud data based on at least one of a resolution and a density of each of the plurality of point cloud data accepted by the input accepting step, and an encoding step of encoding each of the plurality of point cloud data based on the spatial quantization value decided by the determination step. (20) An encoding program that causes a computer mounted on an encoding device to execute the following: an input reception process that receives input of a plurality of point cloud data; a determination process that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data whose input is received by the input reception process; and an encoding process that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination process.

[0167] REFERENCE SIGNS LIST 1 Encoding system 10 Encoding device 11 Input receiving unit 12, 22 Control unit 13, 23 Storage unit 14, 21 Communication unit 20 Playback device 121 Division unit 122 Determination unit 123 Encoding unit 221 Acquisition unit 222 Request unit 223 Playback unit

Claims

1. An encoding device having an input receiving unit that receives input of a plurality of point cloud data; a determination unit that determines a spatial quantization value of each of the plurality of point cloud data based on at least one of a resolution and a density of each of the plurality of point cloud data whose input is received by the input receiving unit; and an encoding unit that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit.

2. The encoding device according to claim 1, wherein the input receiving unit receives input of the plurality of point cloud data including 3D (Dimensions) point cloud data.

3. The encoding device according to claim 1, wherein the determination unit determines the spatial quantization value to be a smaller value as the resolution of the point cloud data is higher.

4. The encoding device according to claim 1, wherein the determination unit determines the spatial quantization value to be a smaller value as the density of the point cloud data increases.

5. The encoding device according to claim 1, wherein the determination unit determines the spatial quantization value to be a power of a predetermined value.

6. The encoding device according to claim 5, wherein the determination unit determines the spatial quantization value to be a power of two.

7. The encoding device according to claim 1, wherein the determination unit determines, based on a request from a playback device, from among the multiple point cloud data encoded by the encoding unit, a playback target to be played back by the playback device.

8. The encoding device of claim 1, further comprising a division unit that divides each of the plurality of point cloud data into a plurality of divided point cloud data based on at least one of a space in which each of the plurality of point cloud data is included and a resolution of each of the plurality of point cloud data, wherein the determination unit determines a spatial quantization value of each of the plurality of divided point cloud data based on at least one of a resolution and a density of each of the plurality of divided point cloud data divided by the division unit, and the encoding unit encodes each of the plurality of divided point cloud data based on the spatial quantization value of each of the plurality of divided point cloud data determined by the determination unit.

9. The encoding device according to claim 8, wherein the division unit divides a space including each of the plurality of point cloud data into a plurality of divided point cloud data by dividing the space into a plurality of divided spaces, and the determination unit determines the spatial quantization value of the divided point cloud data having the highest resolution in each of the plurality of divided spaces divided by the division unit to be the same value.

10. The encoding device according to claim 8, wherein the division unit divides each of the plurality of point cloud data into a plurality of split point cloud data for each resolution level of the plurality of point cloud data, and the determination unit determines the spatial quantization values ​​of the split point cloud data having the highest density at each resolution level among the plurality of split point cloud data divided by the division unit for each resolution level to be the same value.

11. The encoding device of claim 2, further comprising a 3D sensing processing unit that detects at least one of a resolution and a density of the 3D point cloud data based on the 3D point cloud data input accepted by the input accepting unit, and the determination unit determines a spatial quantization value of each of the multiple point cloud data based on at least one of the resolution and the density of the 3D point cloud data detected by the 3D sensing processing unit.

12. The encoding device described in claim 11, wherein the 3D sensing processing unit detects a resolution of the 3D point cloud data based on the 3D point cloud data input received by the input receiving unit, and divides the 3D point cloud data into a plurality of split point cloud data based on the detected resolution of the 3D point cloud data; the determination unit determines a spatial quantization value of each of the plurality of split point cloud data divided by the 3D sensing processing unit based on at least one of the resolution and density of the 3D point cloud data detected by the 3D sensing processing unit; and the encoding unit encodes each of the plurality of split point cloud data based on the spatial quantization value of each of the plurality of split point cloud data determined by the determination unit.

13. An encoding system comprising an encoding device and a playback device, wherein the encoding device has an input receiving unit that receives input of a plurality of point cloud data, a determination unit that determines a spatial quantization value for each of the plurality of point cloud data based on at least one of a resolution and a density of each of the plurality of point cloud data whose input is received by the input receiving unit, and an encoding unit that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination unit, and the playback device has a request unit that requests a playback target to be reproduced by the playback device from among the plurality of point cloud data encoded by the encoding unit.

14. The encoding system according to claim 13, wherein the determination unit determines, based on a request from the playback device, the playback target from among the multiple point cloud data encoded by the encoding unit.

15. The encoding system of claim 13, further comprising an acquisition unit that acquires at least one of communication information regarding communication between the encoding device and the playback device, area information regarding an area in which the playback target requested by the playback device is to be played, and viewpoint information regarding a viewpoint from which the playback target is to be played by the playback device, and the request unit requests the playback target based on the information acquired by the acquisition unit among the communication information, the area information, and the viewpoint information.

16. The encoding system of claim 15, wherein the acquisition unit acquires the communication information, and the request unit requests the playback target having a smaller spatial quantization value when the communication band width indicated by the communication information acquired by the acquisition unit is equal to or greater than a threshold value, compared to when the communication band width is not equal to or greater than the threshold value.

17. The encoding system of claim 15, wherein the acquisition unit acquires the region information, and the request unit requests the reproduction target having a smaller spatial quantization value when the region indicated by the region information acquired by the acquisition unit is within a range of a region of interest, compared to when the region is not within the range of the region of interest.

18. The encoding system of claim 15, wherein the acquisition unit acquires the viewpoint information, and the request unit requests the reproduction target having a smaller spatial quantization value when the distance between the viewpoint indicated by the viewpoint information acquired by the acquisition unit and the reproduction target is within a predetermined range, compared to when the distance is not within the predetermined range.

19. An encoding method executed by an encoding device, comprising: an input receiving step of receiving input of a plurality of point cloud data; a determination step of determining a spatial quantization value of each of the plurality of point cloud data based on at least one of a resolution and a density of each of the plurality of point cloud data whose input is received by the input receiving step; and an encoding step of encoding each of the plurality of point cloud data based on the spatial quantization value determined by the determination step.

20. An encoding program that causes a computer mounted on an encoding device to execute the following: an input reception process that receives input of a plurality of point cloud data; a determination process that determines a spatial quantization value of each of the plurality of point cloud data based on at least one of the resolution and density of each of the plurality of point cloud data whose input is received by the input reception process; and an encoding process that encodes each of the plurality of point cloud data based on the spatial quantization value determined by the determination process.

Citation Information

Patent Citations

  • Fast Region of Interest Encoding Using Multi-Segment Resampling

    JP2021521744A

  • Information processing device and method

    JP2022051968A

  • Information processing device and method

    WO2020189296A1

  • Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

    WO2020189709A1