Encoding method and decoding method, and information processing system

Abstract

An encoding method according to the present disclosure comprises an encoding step of encoding first data, which includes coordinate information of a three-dimensional point cloud including coordinates of three axes represented by integers for each point and attribute information including correction information of each of the three axes for correcting each of the coordinates of the three axes, into second data including the coordinate information of the three-dimensional point cloud and the correction information which is common among the three axes. The encoding step adds the correction information of each of the three axes to the second data.

Description

Encoding method, decoding method, and information processing system

[0001] The present disclosure relates to encoding and decoding methods and information processing systems.

[0002] A technology called digital twin is known, which collects various data from the real world and recreates it on a computer as if it were a twin. One proposed application of this digital twin technology is the use of 3D (three-dimensional) point cloud data collected from a wide range of environments using drones or other devices in agriculture, surveying, construction, and other fields. The collected 3D point cloud data is transmitted to a server, for example, via a communication network such as the Internet. The server recreates the environment based on the received 3D point cloud data and performs analysis, etc.

[0003] Because 3D point cloud data collected from a wide area can be enormous, it must be compressed and encoded before being sent to a server. Existing technologies typically divide the target space into uniformly sized voxels, perform spatial quantization, and then encode the data using techniques such as octree partitioning. Furthermore, a method for encoding and decoding attribute information associated with 3D point clouds has been standardized by the Moving Picture Experts Group-Immersive (MPEG-I) as Geometry-based Point Cloud Compression (G-PCC) (referred to as the G-PCC standard).

[0004] International Publication No. 2019 / 012975

[0005] In conventional compression encoding processing of 3D point cloud data, it has been difficult to suppress errors caused by quantization associated with converting coordinate information into integers in the upstream stage of the encoder that performs the encoding processing.

[0006] Furthermore, since the G-PCC standard assumes that coordinate information is based on a planar rectangular coordinate system, if the 3D point cloud data is collected using another coordinate system, such as a latitude-longitude ellipsoidal coordinate system, a coordinate system conversion process is required when performing compression encoding. This increases the processing time required for the conversion, and makes it difficult to suppress errors caused by the coordinate system conversion.

[0007] Therefore, an object of the present disclosure is to provide an encoding method, a decoding method, and an information processing system that enable encoding and decoding processes of 3D point cloud data to be performed with higher accuracy.

[0008] The encoding method according to the present disclosure includes an encoding step of encoding first data having coordinate information of a three-dimensional point cloud, each of which includes coordinates of three axes expressed as integers for each point, and attribute information having correction information for each of the three axes for correcting the coordinates of each of the three axes, into second data having the coordinate information of the three-dimensional point cloud and the correction information common to the three axes, and the encoding step adds the correction information for each of the three axes to the second data.

[0009] 1 is a schematic diagram showing the configuration of an example of an information processing system applicable to the present disclosure. FIG. 2 is a block diagram of an example of the hardware configuration of a data collection device applicable to the present disclosure. FIG. 3 is a block diagram of an example of the hardware configuration of a terminal device applicable to the present disclosure. FIG. 4 is a schematic diagram showing an example of acquiring wide-area 3D point cloud data based on images captured by a camera mounted on a drone, applicable to the present disclosure. FIG. 5 is a functional block diagram of an example for explaining the function of a point cloud data encoding device that performs G-PCC encoding according to existing technology. FIG. 6 is a schematic diagram showing an example of the configuration of compressed point cloud data. FIG. 7 is a functional block diagram of an example for explaining the function of a point cloud data decoding device that performs G-PCC decoding according to existing technology. FIG. 8 is a schematic diagram for explaining the process of G-PCC encoding 3D point cloud data managed in an LAS file according to existing technology. FIG. 9 is a schematic diagram showing the configuration of an LAS file. FIG. 10 is a schematic diagram showing an example of the fields of the public header area of ​​an LAS file. FIG. 11 is a schematic diagram showing an example of the fields of the point data record area of ​​an LAS file. FIG. 12 is a schematic diagram showing the function of a file conversion SW in more detail. FIG. 1 is a schematic diagram showing an example of a configuration when coordinate information is transferred to a point cloud data encoding device without going through a PLY file. FIG. 1 is a schematic diagram for explaining a point cloud data encoding process according to the present disclosure. FIG. 2 is a functional block diagram of an example for explaining the function of a point cloud data encoding device according to an embodiment. FIG. 3 is a functional block diagram of an example for explaining the function of a point cloud data decoding device according to an embodiment. FIG. 4 is a schematic diagram showing an encoding / decoding process according to an embodiment in comparison with an encoding / decoding process according to existing technology. FIG. 5 is a schematic diagram outlining examples of file formats and attributes of three-dimensional point cloud data. FIG. 6 is a schematic diagram for explaining an encoding / decoding process according to an embodiment. FIG. 7 is a schematic diagram for explaining a field selection process by a field data selection unit according to an embodiment. FIG. 8 is a schematic diagram for explaining a scale factor value and an offset value. FIG. 9 is a schematic diagram for explaining processing by the field data selection unit according to an embodiment. FIG. 10 is a flowchart of an example of a field selection process according to an embodiment. FIG. 11 is a flowchart of an example of a determination of whether or not an LAS file can be directly INT encoded based on a scale factor value according to an embodiment.FIG. 1 is a flowchart illustrating an example of determining whether or not to perform direct INT encoding of an LAS file based on a scale factor value according to an embodiment. FIG. 2 is a flowchart illustrating an example of a first selection method for selecting an encoding method for an LAS file according to an embodiment. FIG. 3 is a flowchart illustrating an example of a second selection method for selecting an encoding method for an LAS file according to an embodiment. FIG. 4 is a schematic diagram illustrating G-PCC encoding / decoding according to existing technology. FIG. 5 is a schematic diagram illustrating G-PCC encoding / decoding according to an embodiment. FIG. 6 is a schematic diagram illustrating a comparison of the representation of scale factor values, offset values, and coordinate values ​​of each point between a G-PCC file and an LAS file.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are denoted by the same reference numerals, and redundant description will be omitted.

[0011] Hereinafter, embodiments of the present disclosure will be described in the following order: 1. Overview of an information processing system applicable to the present disclosure 2. Regarding existing technologies 2-1. Regarding GPCC encoding / decoding processing 2-2. Regarding point cloud data management 2-3. Regarding differences between G-PCC encoding and processing of scale factor values ​​and offset values ​​in LAS files 2-4. Regarding encoding processing proposed in the present disclosure 3. Embodiments of the present disclosure 3-1. Overview of encoding / decoding processing according to embodiments 3-2. More specific explanation of encoding / decoding processing according to embodiments 3-2-1. Regarding details of encoding / decoding processing 3-2-2. Regarding specific examples of encoding / decoding processing 3-2-3. Regarding selection of encoding method 4. Comparison of embodiments of the present disclosure with existing technologies

[0012] (1. Overview of Information Processing System Applicable to the Present Disclosure) First, an overview of an information processing system applicable to the present disclosure will be described. The information processing system applicable to the present disclosure may handle relatively large-scale three-dimensional point cloud data, and encodes / decodes three-dimensional point cloud data acquired by, for example, photography or ranging.

[0013] Fig. 1 is a schematic diagram showing the configuration of an example of an information processing system applicable to the present disclosure. In the example of Fig. 1, the information processing system 1 includes a data collection device 10, a terminal device 20, and a server 30. In the information processing system 1, the data collection device 10, the terminal device 20, and the server 30 are connected to each other so as to be able to communicate with each other via a communication network 2 such as the Internet.

[0014] 1, the server 30 is shown as being configured in a cloud network 3 connected to the communication network 2. However, the server 30 is not limited to this and may be configured as a single computer or may be configured as being distributed across multiple computers.

[0015] The data collection device 10 is also called an edge box, and collects measurement data obtained using, for example, a drone 4. The data collection device 10 may also have a function of setting a flight plan for the drone 4.

[0016] The drone 4 is equipped with a ranging mechanism such as a camera or LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), and while flying along a preset route, photographs the ground and measures distances at one or more positions, acquiring the photographed images and distance data as measurement data. The data collection device 10 collects the measurement data acquired by the drone 4 via a storage medium such as an SD memory card or via wired or wireless communication.

[0017] If the measurement data acquired by the drone 4 is a photographed image captured by a camera with R (red), G (green), and B (blue) color information for each pixel, the data collection device 10 performs point cloud matching based on the collected measurement data using, for example, Structure from Motion (SFM) to generate three-dimensional point cloud data. The generated three-dimensional point cloud data is uncompressed data with three-dimensional coordinate values ​​for each point on the x, y, and z axes and RGB color information.

[0018] The data collection device 10 transmits the generated 3D point cloud data to the server 30 via the communication network 2. At this time, the data collection device 10 compresses and encodes the 3D point cloud data using a predetermined method, thereby reducing the bandwidth required for data transmission over the communication network 2. In other words, the data collection device 10 functions as an encoding device that encodes the 3D point cloud data. The data collection device 10 transmits the compressed and encoded 3D point cloud data (hereinafter referred to as compressed point cloud data as appropriate) to the server 30 via the communication network 2. The server 30 stores the compressed point cloud data transmitted from the data collection device 10.

[0019] The terminal device 20 may be, for example, a general-purpose computer, and may acquire compressed point cloud data stored in the server 30 via the communication network 2. The terminal device 20 decodes the compressed point cloud data acquired from the server 30 to restore the original three-dimensional point cloud data. In other words, the terminal device 20 functions as a decoding device that decodes the compressed and encoded three-dimensional point cloud data. The terminal device 20 may perform analysis processing or the like on the restored three-dimensional point cloud data using a predetermined application program.

[0020] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the data collection device 10 applicable to the present disclosure.

[0021] In FIG. 2 , the data collection device 10 includes a CPU (Central Processing Unit) 1000, a ROM (Read Only Memory) 1001, a RAM (Random Access Memory) 1002, a DSP (Digital Signal Processor) 1003, a storage device 1004, a UI (User Interface) section 1005, a data I / F (Interface) 1006, and a communication I / F 1007, and each of these sections is connected to each other via a bus 1010 so that they can communicate with each other.

[0022] The storage device 1004 is a non-volatile storage medium such as a flash memory or a hard disk drive. The CPU 1000 operates in accordance with programs stored in the ROM 1001 or the storage device 1004, using the RAM 1002 as a work memory, and controls the overall operation of the data collection device 10.

[0023] The DSP 1003 performs calculations in accordance with instructions from the CPU 1000. The DSP 1003 may execute a process of combining each piece of measurement data, a process of generating three-dimensional point cloud data based on the combined measurement data, and a process of compressing and encoding the generated three-dimensional point cloud data in accordance with instructions from the CPU 1000. If the CPU 1000 has sufficient processing power, the DSP 1003 may be omitted and the compression and encoding process may be executed by the CPU 1000.

[0024] The UI unit 1005 includes an input unit that accepts user operations and a display unit that presents information to the user. The UI unit 1005 may be configured using a touch panel in which the input unit and the display unit are integrally formed. The CPU 1000 may control the operation of the data collection device 10 in accordance with a program in response to user operations on the UI unit 1005, and may present information to the user in accordance with the operation.

[0025] The data I / F 1006 is an interface for inputting and outputting data to and from an external device. For example, an SD memory card slot may be used as the data I / F 1006. Alternatively, a USB (Universal Serial Bus) connector or Bluetooth (registered trademark) may be used as the data I / F 1006. For example, measurement data acquired by the drone 4 is transferred to the data collection device 10 via the data I / F 1006.

[0026] The communication I / F 1007 is an interface that controls communication via the communication network 2. The communication I / F 1007 may be connected to the communication network 2 via wired communication or wireless communication.

[0027] FIG. 3 is a block diagram illustrating an example of a hardware configuration of the terminal device 20 applicable to the present disclosure.

[0028] 3 , the terminal device 20 includes a CPU (Central Processing Unit) 2000, a ROM 2001, a RAM 2002, a display control unit 2003, a storage device 2004, an input device 2005, a data I / F 2006, and a communication I / F 2007, and these units are communicably connected to each other via a bus 2010. In this manner, the terminal device 20 may be configured using a general-purpose computer. However, the terminal device 20 is not limited to this, and may also be a tablet computer or a smartphone.

[0029] The storage device 2004 is a non-volatile storage medium such as a flash memory or a hard disk drive. The CPU 2000 operates in accordance with programs stored in the ROM 2001 or the storage device 2004, using the RAM 2002 as a work memory, and controls the overall operation of the terminal device 20.

[0030] The display control unit 2003 generates a display signal that can be displayed by the display device 2020, based on a display control signal passed from the CPU 2000, for example. The display device 2020 displays a screen according to the display signal supplied from the display control unit 2003.

[0031] The input device 2005 accepts user operations. A pointing device such as a mouse or a keyboard may be used as the input device 2005. The input device 2005 may also be a touch panel formed integrally with the display device 2020.

[0032] The data I / F 2006 is an interface for inputting and outputting data to and from an external device. A USB (Universal Serial Bus) connector, Bluetooth (registered trademark), or the like may be used as the data I / F 2006. The communication I / F 2007 is an interface for controlling communication via the communication network 2. The communication I / F 2007 may be connected to the communication network 2 via wired communication or wireless communication.

[0033] The CPU 2000 may acquire compressed data from the server 30 via the communication I / F 2007. The CPU 2000 may decode the compressed data acquired from the server 30 to restore the original three-dimensional point cloud data. The CPU 2000 may execute predetermined processing, such as analysis processing or visualization processing, on the restored three-dimensional point cloud data in accordance with the operation of an application program according to the intended use of the three-dimensional point cloud data, for example.

[0034] The server 30 can have a configuration similar to that of a general computer (for example, the terminal device 20), and therefore a detailed description thereof will be omitted here.

[0035] (2. Existing Technology) Next, existing technology related to the present disclosure will be described.

[0036] (2-1. Regarding G-PCC Encoding / Decoding Processing) Figure 4 is a schematic diagram showing an example of acquiring 3D point cloud data from a wide range based on images captured by a camera mounted on a drone 4, which is applicable to the present disclosure. In Figure 4, the camera mounted on the drone 4 captures images of a relatively wide range, such as farmland, a construction site, a civil engineering work site, or the inside of a large warehouse, and obtains multiple captured images 40a, 40b, 40c, and 40d. These multiple captured images 40a, 40b, 40c, and 40d are transferred from the drone 4 to a data collection device 10 (not shown).

[0037] The data collection device 10 generates uncompressed 3D data 41 using, for example, SFM based on multiple captured images (e.g., captured image 40a) captured in the same area. The data collection device 10 compresses and encodes the generated uncompressed 3D data 41 so that it can be used as G-PCC data, and outputs it as G-PCC compressed data 42.

[0038] (3D Point Cloud Data Encoding Process Using Existing Technology) Fig. 5 is a functional block diagram of an example for explaining the functions of a point cloud data encoding device that performs G-PCC encoding using existing technology. In Fig. 5, the point cloud data encoding device 500 includes a spatial quantization processing unit 510, an octree encoding unit 511, an arithmetic encoding unit 512, an attribute data conversion unit 513, and a syntax encoding unit 514.

[0039] 3D point cloud data 501 is input to a point cloud data encoding device 500. Hereinafter, unless otherwise specified, 3D point cloud data will be simply referred to as point cloud data. The point cloud data 501, which is 3D point cloud data, may be written in various known data formats. Examples of data formats include the PLY (Polygon File) format and the LAS (LiDAR Data Exchange) format. In existing technology, tools that perform G-PCC encoding generally assume data in the PLY format as input data. Therefore, here, the point cloud data 501 is assumed to be data written in accordance with the PLY format.

[0040] The point cloud data 501 generally includes header data 502 and body data 503. The header data 502 is data that is added to the point cloud data 501 separately from the body data 503, and describes, for example, various types of information related to the body data 503. The header data 502 may include scale information for the coordinate information of each point included in the body data 503. The scale information is correction information for correcting the coordinate information.

[0041] The body data 503 is data that describes the actual content of the point cloud. In the example of Fig. 5, the body data 503 is data that can be separated into geometry data 504 and attribute data 505 during encoding (more specifically, G-PCC encoding). The geometry data 504 is position data that indicates the spatial position of each point of the point cloud data, and identifies each point, for example, using a three-axis coordinate system consisting of x, y, and z axes. The attribute data 505 is data that indicates the attributes of this point cloud data 501, and may be, for example, the color, reflectance, normal, or class of each point included in the point cloud data 501.

[0042] The spatial quantization processing unit 510 spatially quantizes the body data 503 in the point cloud data 501. The spatial quantization processing unit 510 performs processing such as dividing the target space of the body data 503 into voxels of uniform size and converting coordinates into integers. The spatial quantization processing unit 510 outputs the spatially quantized body data 503 as spatial quantized data.

[0043] The octree encoding unit 511 performs octree encoding on the spatial quantized data output from the spatial quantization processing unit 510. The octree encoding unit 511 performs octree encoding by dividing each voxel of the spatial quantized data into eight voxels. The octree encoding unit 511 outputs the data obtained by octree encoding the spatial quantized data as octree data.

[0044] The attribute data conversion unit 513 performs data conversion to obtain attribute data 505 associated with the geometry data 504 in the body data 503. This conversion is performed based on octree data, etc. The attribute data conversion unit 513 outputs the data resulting from the conversion of the attribute data 505 as converted attribute data.

[0045] The arithmetic coding unit 512 arithmetically codes the octree data and the transformed attribute data. The arithmetic coding unit 512 may apply entropy coding as an example of arithmetic coding. The arithmetic coding unit 512 outputs data obtained by performing arithmetic coding on the octree data and the transformed attribute data as arithmetically coded data.

[0046] The syntax encoding unit 514 generates compressed point cloud data 506 including header data 502 and arithmetically coded data. More specifically, the syntax encoding unit 514 syntax-codes the arithmetically coded data and the header data 502 to generate the compressed point cloud data 506. The compressed point cloud data 506 is data obtained by compressing and coding the point cloud data 501 using G-PCC coding as described above. The compressed point cloud data 506 is data corresponding to the G-PCC compressed data 42 described with reference to FIG. 4. Note that the syntax encoding unit 514 may include data included in the header data 502 as user data in the compressed point cloud data 506.

[0047] FIG. 6 is a schematic diagram showing an example of the configuration of compressed point cloud data 506. The compressed point cloud data 506 shown in FIG. 6 has a G-PCC syntax structure. The compressed point cloud data 506 includes a header area 51, a user data area 52, and an entity data area 53, and also includes multiple units. Examples of the multiple units include an AU (Access Unit) header included in the header area 51, a UD (User Data Unit) included in the user data area 52, and an SPS (Sequence Parameter Set Data Unit), GPS (Geometry Parameter Set Data Unit), GDU (Geometry Data Unit), APS (Attribute Parameter Set Data Unit), and ADU (Attribute Data Unit) included in the entity data area 53. Hereinafter, when there is no particular distinction between these, they will be simply referred to as units.

[0048] Data related to the body data 503 after arithmetic coding is mainly included in GPS, GDU, APS, and ADU. GPS and GDU are related to geometry data 504. For example, GPS includes data related to parameters of the geometry data 504. GDU includes the geometry data 504 after arithmetic coding. APS and ADU are related to attribute data 505. For example, APS includes data related to parameters of the attribute data 505. ADU includes the attribute data 505 after arithmetic coding.

[0049] The syntax encoding unit 514 includes (places or records) the header data 502 in one of the multiple units of the compressed point cloud data 506. In this example, the syntax encoding unit 514 includes the header data 502 in a UD. More specifically, the UD includes a user data header and a user data payload. The syntax encoding unit 514 includes the header data 502 in the user data payload.

[0050] The user data payload, for example, assigns an ID to data in byte units and describes its contents. The user data payload may include N pieces of data (N is an integer equal to or greater than 1). The ID of each piece of data is indicated as user data ID_0 to ID_N, and the corresponding data contents are indicated as user data byte ID_0 to user data byte ID_N. For example, the data in the first row of the header data 502 may correspond to user data ID_0 and user data byte ID_0. The same applies to the data in the second and subsequent rows of the header data 502.

[0051] 5 , for example, the above-described compressed point cloud data 506 is generated by the syntax encoding unit 514. The generated compressed point cloud data 506 may be output at an appropriate timing. An example of output is transmission of the compressed point cloud data 506 from the data collection device 10 to the server 30 via the communication network 2, as previously described with reference to FIG.

[0052] (3D Point Cloud Data Decoding Process Using Existing Technology) Fig. 7 is a functional block diagram of an example for explaining the functions of a point cloud data decoding device that performs G-PCC decoding using existing technology. In Fig. 7, a point cloud data decoding device 520 includes a syntax decoding unit 530, an arithmetic decoding unit 531, an octree decoding unit 532, an attribute data inverse conversion unit 533, and a spatial inverse quantization processing unit 534.

[0053] Compressed point cloud data 506 is input to the point cloud data decoding device 520. The compressed point cloud data 506 may be compressed point cloud data 506 obtained by compressing and encoding the point cloud data 501 by the point cloud data encoding device 500 described with reference to Fig. 5. The point cloud data decoding device 520 passes the input compressed point cloud data 506 to a syntax decoding unit 530.

[0054] The syntax decoding unit 530 syntax-decodes the compressed point cloud data 506 and extracts header data 502 from the compressed point cloud data 506. The syntax decoding unit 530 passes the data of the compressed point cloud data 506 other than the header data 502 as arithmetically coded data to the arithmetic decoding unit 531. The point cloud data decoding device 520 holds the header data 502 extracted by the syntax decoding unit 530 as header data 502'.

[0055] The arithmetic decoding unit 531 decodes the arithmetically coded data passed from the syntax decoding unit 530 to obtain octree data and transformed attribute data. The arithmetic decoding unit 531 passes the octree data to an octree decoding unit 532. The arithmetic decoding unit 531 also passes the transformed attribute data and octree data to an attribute data inverse conversion unit 533.

[0056] The attribute data inverse conversion unit 533 performs inverse conversion of data to obtain attribute data 505, based on the octree data and converted attribute data passed from the arithmetic decoding unit 531. The octree decoding unit 532 performs octree decoding based on the octree data passed from the arithmetic decoding unit 531 and the result of the inverse conversion by the attribute data inverse conversion unit 533, to obtain spatial quantized data. The octree decoding unit 532 passes the spatial quantized data to a spatial inverse quantization processing unit 534.

[0057] The spatial inverse quantization processing unit 534 decodes the spatial quantized data passed from the octree decoding unit 532 to obtain body data 503' obtained by restoring the body data 503. The body data 503' includes geometry data 504' obtained by restoring the geometry data 504, and attribute data 505' obtained by restoring the attribute data 505.

[0058] The point cloud data decoding device 520 generates point cloud data 501′ using header data 502′ extracted and stored by the syntax decoding unit 530 and body data 503′ obtained by restoring body data 503 by the spatial inverse quantization processing unit 534.

[0059] (2-2. Point Cloud Data Management) Next, the management of point cloud data will be briefly described. As described above, the data collection device 10 generates three-dimensional point cloud data based on measurement data using, for example, SFM. Here, the data collection device 10 can manage the generated three-dimensional point cloud data using an LAS file.

[0060] LAS is a standard format for point cloud data measured by LiDAR, defined by the American Society for Photogrammetry and Remote Sensing (ASPRS). An LAS file stores coordinates of three axes (x, y, and z) in integer format (INT type) as coordinate information, and can manage scale factor values ​​(scale information) and offset values ​​(offset information) for the coordinates individually for each of the three axes. The scale factor value and offset value are correction information used to correct the coordinate values ​​of each point during decoding. The structure of an LAS file will be described later.

[0061] Fig. 8 is a schematic diagram for explaining the process of G-PCC encoding of 3D point cloud data managed in an LAS file according to existing technology. The encoding method shown in Fig. 8 is based on a simulation method generally used in the G-PCC standard.

[0062] In FIG. 8, an LAS file 60 is converted into a PLY file 70 using a file conversion SW (software) 80 .

[0063] Here, we will explain the structure of the LAS file 60. Fig. 9 is a schematic diagram showing the structure of the LAS file 60. The LAS file 60 includes a public header area 61, a variable-length record area 62, a point data record area 63, and an extended variable-length record area 64.

[0064] The public header area 61 stores a public header that sets attribute areas for the entire LAS file 60. The variable length record area 62 is an area for variable length records that follows the public header area 61. The variable length record area 62 stores CRS (Coordinate Reference System) information for the LAS.

[0065] The point data record area 63 is an area where the actual point cloud data is stored, and stores coordinate information including the three-dimensional coordinates of each point included in the point cloud of the point cloud data and attribute information such as color information.

[0066] The extended variable length record area 64 includes the same header items as the variable length record area 62 .

[0067] The public header area 61 and the point data record area 63 of the LAS file 60 will be described in more detail with reference to FIGS. 10A and 10B.

[0068] 10A is a schematic diagram showing an example of each field of the public header area 61 of the LAS file 60. The field "File Signature ("LASF")" indicates the signature of this file. The field "File Source ID" indicates the ID of the source of this file. The field "Global Encoding" indicates certain global properties related to this file.

[0069] The fields "Project ID - GUID Data 1" to "Project ID - GUID Data 4" are reserved for use as project IDs. The fields "Project ID - GUID Data 1" to "Project ID - GUID Data 4" are optional.

[0070] The fields "Version Major" and "Version Minor" indicate the version number applied to this LAS file 60. The field "System Identifier" indicates the identifier of the system that generated this LAS file 60. The field "Generating Software" indicates information describing the software that generated this LAS file 60. The fields "File Creation Day of Year" and "File Creation Year" indicate the date on which this LAS file 60 was created. The field "Header Size" indicates size information of this public header area 61 itself.

[0071] The field "Offset to Point Data" indicates the number of bytes from the beginning of this LAS file 60 to the beginning of the point data record area 63. When data is added to or deleted from the variable-length record area 62, the value of this field "Offset to Point Data" is updated.

[0072] The field “Number of Variable Length Records” indicates the number of variable length records stored in the variable length record area 62 of this LAS file 60 .

[0073] The field "Point Data Record Format" indicates the type of point cloud data stored in the point data record area 63 of this LAS file 60. The value indicated in this field "Point Data Record Format" may be a value indicating a format ID for identifying the format of this LAS file 60. The field "Point Data Record Length" indicates the length of the point cloud data stored in the point data record area 63 of this LAS file 60.

[0074] The fields "Legacy Number of Point Records" and "Legacy Number of Points by Return" each indicate information about compatibility with older files.

[0075] The fields "X Scale Factor", "Y Scale Factor", and "Z Scale Factor" respectively indicate the scale factor values ​​(scale information) for each of the three axes (x, y, and z axes) for the point cloud data stored in this LAS file 60. These fields "X Scale Factor", "Y Scale Factor", and "Z Scale Factor" make it possible to manage the scale information for each of the three axes individually.

[0076] The fields "X Offset," "Y Offset," and "Z Offset" each indicate the offset values ​​(offset information) for each of the three axes for the point cloud data stored in this LAS file 60. For example, to calculate the x coordinate from the position data of a point in the point cloud data stored in the point data record area 63, the x coordinate of the point is multiplied by the scale factor value for the x axis indicated in the field "X Scale Factor," and the value of the field "X Offset" is added to the result.

[0077] The fields "Max X" and "Min X", "Max Y" and "Min Y", and "Max Z" and "Min Z" respectively indicate the actual (unscaled) extents of the point cloud data specified in the coordinate system of this LAS file 60.

[0078] The field "Start of Waveform Data Packet Record" indicates the offset in bytes from the beginning of the LAS file 60 to the beginning of the waveform data when waveform data is stored in this LAS file 60. The field "Start of First Extended Variable Length Record" indicates the number of bytes from the beginning of this LAS file 60 to the beginning of the extended variable length record area 64. When data is added to or deleted from the variable length record area 62 or the point data record area 63, the value of this field "Start of First Extended Variable Length Record" is updated.

[0079] The field "Number of Point Records" indicates the total number of points in the point cloud data included in the point data record area 63 of this LAS file 60. The field "Number of Points by Return" indicates an array of records of the total number of points for each return.

[0080] Fig. 10B is a schematic diagram showing an example of each field in the point data record area 63 of the LAS file 60. Each field may be generated for each point of the point cloud data. Fig. 10B shows the field configuration defined as point data record format 4. Each field in the point data record area 63 may be generated for each point of the point cloud data.

[0081] The fields "X", "Y", and "Z" respectively indicate the x-coordinate, y-coordinate, and z-coordinate of a point in the point cloud data. The fields "X", "Y", and "Z" are of Long data type and are expressed as integer values. The values ​​indicated in the fields "X", "Y", and "Z" are combined with the values ​​of the fields "X Scale Factor", "Y Scale Factor", and "Z Scale Factor" and the fields "X Offset", "Y Offset", and "Z Offset" in the public header area 61, respectively, to determine the coordinates of each point.

[0082] The field "Intensity" indicates the magnitude of the pulse return value. The field "Intensity" is optional. The fields "Return Number" and "Number of Returns (Given Pulse)" indicate information about the return for the pulse. The field "Scan Direction Flag" indicates the direction in which the scan mirror was moving when the pulse was output. The field "Edge of Flight Line Flag" indicates whether the point is at the end of the scan. The field "Classification" indicates the class attribute of the point. The field "Scan Angle Rank (-90 to +90) - Left Side" indicates the angle at which the laser light is emitted from the laser system, with an effective range of -90° to +90°, including the roll of the air vehicle such as drone 4.

[0083] The field "User Data" is a field that can be optionally used by the user.

[0084] The field "Point Source ID" indicates information identifying the source that generated this point. The field "GPS Time" indicates the time when this point was measured.

[0085] The fields "Red", "Green" and "Blue" indicate the R (red), G (green) and B (blue) image channel values ​​for this point, respectively.

[0086] The fields "Wave Packet Descriptor Index," "Byte Offset to Waveform Data," "Waveform Packet Size in Bytes," and "Return Point Waveform Location," as well as the fields "Parametric dx," "Parametric dy," and "Parametric dz," each indicate information related to the waveform packet (waveform data) at this point.

[0087] Of the fields shown in FIG. 10B, in the present disclosure, the fields "X" to "Blue" indicated by a diagonal line under the item "Required" in the figure may be used.

[0088] Returning to Fig. 8 , the point cloud data encoding device 500 acquires point cloud data 540 from the PLY file 70. The PLY file 70 can have coordinate information 541 in Float or Double data type. In G-PCC encoding, the coordinate information 541 must be input in integer type (INT type). Therefore, the point cloud data encoding device 500 converts the coordinate information 541 into integers through integer conversion processing 542.

[0089] The point cloud data encoding device 500 removes offset information when converting the coordinate information 541 into integers in the integer conversion process 542. In addition, the point cloud data encoding device 500 performs rounding using scale information 543 included in the PLY file 70, and retains information after the decimal point of the coordinate information 541.

[0090] The point cloud data encoding device 500 performs a compression encoding process 544 on INT type coordinate information 546, which is obtained by converting Float type or Double type coordinate information 541 into integers. The compression encoding process 544 is similar to the process performed by the spatial quantization processing unit 510 to the syntax encoding unit 514 described with reference to FIG. 5, and therefore a description thereof will be omitted here.

[0091] The point cloud data encoding device 500 may output the compressed point cloud data 506 generated by the compression encoding process 544, for example, as a stream 548. At this time, the point cloud data encoding device 500 adds the scale information 543 and the offset information 545 removed in the integer conversion process 542 to the stream 548 as an upper header. Alternatively, the point cloud data encoding device 500 may output the compressed point cloud data 506 as a file (G-PCC file). The scale information 543 and offset information 545 are stored as an upper header in the file.

[0092] Fig. 11 is a schematic diagram showing in more detail the function of the file conversion SW 80. As shown in Fig. 11, the file conversion SW 80 may be configured to include a scale / offset decoding unit 610 and a coordinate system conversion unit 611a. Note that the configuration of the point cloud data encoding device 500 in Fig. 11 is the same as the configuration of the point cloud data encoding device 500 shown in Fig. 8, so description thereof will be omitted here.

[0093] Here, it is assumed that the coordinates of points stored in the fields "X", "Y" and "Z" of the point data record area 63 in the LAS file 60 use the Long type, WGS84 coordinate system.

[0094] Using a tool such as an LAS reader, scale factor values ​​and offset values ​​for each axis are extracted from the fields “X Scale Factor,” “Y Scale Factor,” and “Z Scale Factor,” as well as the fields “X Offset,” “Y Offset,” and “Z Offset,” in the public header area 61 of the LAS file 60. Similarly, coordinate values ​​along the x, y, and z axes (referred to as x, y, and z coordinate values) of each point in the point cloud data are extracted from the point data record area 63 of the LAS file 60.

[0095] The scale / offset decoding unit 610 converts each coordinate value from a Long type value to a Double type (or Float type) value using the scale factor value and offset value for each axis extracted from the LAS file 60. The coordinate system conversion unit 611a converts the coordinate system of each coordinate value converted by the scale / offset decoding unit 610 from the WGS84 coordinate system to a planar rectangular coordinate system using Gauss-Krüger projection. The coordinate system conversion unit 611a stores the x, y, and z coordinate values ​​converted from the coordinate system into the planar rectangular coordinate system in the PLY file 70.

[0096] In the example of Fig. 11, the point cloud data encoding device 500 acquires coordinate information via the PLY file 70. On the other hand, a configuration is also possible in which the point cloud data encoding device 500 acquires coordinate information without going through the PLY file 70. Fig. 12 is a schematic diagram showing an example of a configuration in which coordinate information is transferred to the point cloud data encoding device 500 without going through the PLY file 70.

[0097] 12 , the coordinate system conversion unit 611b includes, for example, a buffer memory 612, and temporarily stores the x, y, and z coordinate values ​​of each point, obtained by converting the coordinate system into a planar rectangular coordinate system, in the buffer memory 612. The point cloud data encoding device 500 reads the x, y, and z coordinate values ​​of each point from the buffer memory 612 and processes them as point cloud data 540. However, the coordinate system conversion unit 611b may output the x, y, and z coordinate values ​​of each point, obtained by converting the coordinate system into a planar rectangular coordinate system, as a stream to the point cloud data encoding device 500.

[0098] Now consider the case where the 3D point cloud data stored in the LAS file 60 is G-PCC encoded. As described above, the LAS file 60 has coordinate information in INT type, whereas the PLY file 70 has coordinate information in Float type or Double type. Therefore, it is necessary to convert the coordinate information of the LAS file 60 into the coordinate information of the PLY file 70.

[0099] As described above, conversion from the LAS file 60 to the PLY file 70 is generally performed using a predetermined file conversion SW 80. Here, if the ASCii format is used to express coordinate values ​​in the coordinate information in the LAS file 60, the coordinate values ​​will be expressed in decimal numbers. On the other hand, if the file conversion SW 80 internally handles coordinate values ​​in binary numbers, the precision will change when converting from decimal numbers to binary numbers.

[0100] When coordinate values ​​are processed in binary, they are expressed in Float or Double format. In Float or Double format, the mantissa has a fixed length of 23 bits for Float and 52 bits for Double. Therefore, when the coordinate value becomes very large, the precision after the decimal point changes depending on the value.

[0101] For example, in Japan, 19 reference points for the geodetic reference system have been established. The distance from these reference points ranges from several tens of kilometers to 100 or 200 kilometers, and even 500 kilometers on remote islands. When the distance from the reference point becomes so large that distances are expressed with millimeter (mm) precision, for example, the length of the mantissa in the Float or Double format may be insufficient. If the length of the mantissa is insufficient, the desired calculation accuracy may not be achieved in the floating-point calculation when converting the value to an integer. In other words, the calculation accuracy changes depending on the distance from the reference point.

[0102] In principle, 3D point cloud data generated using SFM has different accuracy for each of the three axes (x, y, z). There are demands for accuracy for each of the three axes of 3D point cloud data, and there is a need to manage and store the accuracy for each of the three axes. Meanwhile, the volume of 3D data is on the rise, and there is a demand for reducing data volume through data compression such as G-PCC encoding.

[0103] (2-3. Differences in the processing of scale factor values ​​and offset values ​​between G-PCC encoding and LAS files) Here, we will explain the differences in the processing of scale factor values ​​and offset values ​​between G-PCC encoding and LAS files 60. Equation (1) shows the coordinate representation in G-PCC encoding and the calculation formula used during decoding. Furthermore, equation (2) shows the coordinate representation in LAS files 60 and the calculation formula used during decoding.

[0104]

[0105]

[0106] In equations (1) and (2), Xval, Yval, and Zval respectively represent the x-, y-, and z-coordinate values ​​of each point in the point cloud data. Xval, Yval, and Zval are each Float or Double data types. Xint, Yint, and Zint respectively represent the x-, y-, and z-coordinate values ​​of each point in the point cloud data stored in the LAS file 60 or the PLY file 70. Xint, Yint, and Zint are each Long integer data types. Xoffset, Yoffset, and Zoffset are offset values ​​relative to the x-, y-, and z-coordinate values ​​in the G-PCC encoding or the LAS file 60, respectively, and are each Double data types and are fixed values ​​within the scope.

[0107] In formula (1), CommonScale indicates a scale factor value common to the x-coordinate, y-coordinate, and z-coordinate. CommonScale has a double data type and is a fixed value within its scope. In G-PCC encoding, during decoding, as shown in formula (1), Xint, Yint, and Zint are multiplied by the common scale factor value indicated in CommonScale, and offset values ​​for the x-coordinate, y-coordinate, and z-coordinate are added to calculate Xval, Yval, and Zval.

[0108] To be precise, in the G-PCC standard, the Common Scale is expressed as an integer of "numerator / denominator." For simplification, in equation (1), the Common Scale is expressed as one variable of double type.

[0109] Meanwhile, in equation (2), Xscale, Yscale, and Zscale represent scale factor values ​​for the x-coordinate, y-coordinate, and z-coordinate, respectively. Xscale, Yscale, and Zscale are each double data type and are fixed values ​​within their scope. In the LAS file 60, during decoding, as shown in equation (2), Xint, Yint, and Zint are multiplied by Xscale, Yscale, and Zscale, respectively, and offset values ​​for the x-coordinate, y-coordinate, and z-coordinate are added to calculate Xval, Yval, and Zval.

[0110] As shown in equations (1) and (2), the scale factor values ​​for the three axes, i.e., the x-coordinate, y-coordinate, and z-coordinate, are common in G-PCC encoding, but can be managed individually for each of the x-coordinate, y-coordinate, and z-coordinate in the LAS file 60. For example, in the SmartConstruction system, the LAS file 60 or a compressed LAZ file is used as a file format that can record and manage coordinate system information for three-dimensional data and the precision for each coordinate axis.

[0111] However, in G-PCC encoding, precision is managed using an LAS file, and scale factor values ​​are stored separately for each of the x, y, and z coordinates. Even though the coordinate values ​​are stored as integer values, the coordinate values ​​must first be converted from integer values ​​to floating-point values ​​before G-PCC encoding, and then voxelized using a scale factor value common to all three axes. Therefore, with existing technology, it is difficult to suppress the occurrence of quantization errors in G-PCC encoding, and managing precision for each axis is cumbersome.

[0112] (2-4. Regarding the encoding process proposed in the present disclosure) Next, the encoding process proposed in the present disclosure will be described. Fig. 13 is a schematic diagram for explaining the point cloud data encoding process according to the present disclosure. Note that in Fig. 13, the configuration of the point cloud data encoding device 500a is substantially the same as the configuration of the point cloud data encoding device 500 shown in Fig. 8, and therefore detailed description thereof will be omitted here.

[0113] 13 , the x, y, and z coordinate values ​​of each point stored as integer values ​​in the LAS file 60 (first data) are directly subjected to a compression encoding process 544 using G-PCC encoding to generate compressed point cloud data 506 (second data). In the compressed point cloud data 506, the x, y, and z coordinate values ​​of each point in the point cloud data are stored in the entity data area 53.

[0114] Furthermore, in the present disclosure, the scale factor values ​​stored in the fields “X Scale Factor,” “Y Scale Factor,” and “Z Scale Factor” in the LAS file 60, and the offset values ​​stored in the fields “X Offset,” “Y Offset,” and “Z Offset” are stored in the user data area 52 of the compressed point cloud data 506.

[0115] In the example of Figure 13, the processing by the scale / offset (S / O) decoding unit 610 and coordinate system conversion unit 611b shown in Figure 11, etc., and the integer conversion processing 542 in the point cloud data encoding device 500a are omitted.

[0116] (3. Embodiments of the Present Disclosure) Next, embodiments of the present disclosure will be described.

[0117] (3-1. Overview of Encoding / Decoding Process According to Embodiment) First, an overview of encoding and decoding processes according to an embodiment of the present disclosure will be described. Fig. 14 is a functional block diagram of an example for explaining the function of a point cloud data encoding device according to an embodiment. The point cloud data encoding device 100 shown in Fig. 14 may be included in the data collection device 10 shown in Fig. 1, for example.

[0118] 14, the point cloud data encoding device 100 includes a data selecting unit 110, a spatial quantization processing unit 111, an octree encoding unit 112, an arithmetic encoding unit 113, and a syntax encoding unit 114.

[0119] The data selection unit 110, spatial quantization processing unit 111, octree coding unit 112, arithmetic coding unit 113, and syntax coding unit 114 may be configured by running a coding program according to the embodiment on the CPU 1000 (see FIG. 2 ). However, without being limited to this, some or all of the data selection unit 110, spatial quantization processing unit 111, octree coding unit 112, arithmetic coding unit 113, and syntax coding unit 114 may be configured by hardware circuits that operate in cooperation with each other.

[0120] The LAS file 60 is input to the point cloud data encoding device 100 and passed to the data selecting unit 110. The data selecting unit 110 selects each field of the LAS file 60 according to the frequency of point-to-point changes in the field value. In general, the data selecting unit 110 passes fields that do not change point-to-point to the syntax encoding unit 114 as fixed-value fields, and passes fields that change point-to-point to the spatial quantization processing unit 111.

[0121] Fixed value fields include the fields "X Scale Factor", "Y Scale Factor", and "Z Scale Factor", and the fields "X Offset", "Y Offset", and "Z Offset", while fields with point-by-point variation may include, for example, the fields "X", "Y", and "Z".

[0122] The spatial quantization processing unit 111 divides the target space including each point represented by the fields "X," "Y," and "Z" into voxels of uniform size and performs spatial quantization. The spatial quantization processing unit 111 passes the spatial quantized data obtained by spatially quantizing the target space to the octree encoding unit 112.

[0123] The octree encoding unit 112 performs octree encoding by dividing each voxel of the spatial quantization data into eight. The octree encoding unit 112 passes the octree data obtained by octree-encoding the spatial quantization data to the arithmetic encoding unit 113. Note that the number of divisions into voxels by the octree encoding unit 112 may be a number other than eight, and in this sense, "octree" may simply be read as "tree." Furthermore, since the G-PCC standard allows selection of a predictive encoding method in addition to octree encoding, the octree encoding unit 112 may perform predictive encoding instead of octree encoding.

[0124] The arithmetic coding unit 113 arithmetically codes the octree data. The arithmetic coding unit 512 may apply entropy coding as an example of arithmetic coding. The arithmetic coding unit 113 passes the arithmetically coded data obtained by performing arithmetic coding on the octree data to the syntax coding unit 114.

[0125] The syntax encoding unit 114 generates compressed point cloud data 506 that includes the arithmetically coded data passed from the arithmetic coding unit 113 and the fixed value fields passed from the data selecting unit 110. More specifically, the syntax encoding unit 114 records the values ​​of at least the fields "X Scale Factor," "Y Scale Factor," and "Z Scale Factor" of the fixed value fields, as well as the fields "X Offset," "Y Offset," and "Z Offset," in the user data area 52. The syntax encoding unit 114 also records the values ​​of the fields "X," "Y," and "Z" in the entity data area 53.

[0126] That is, the syntax encoding unit 114 adds three-axis scale information and offset information based on the x, y, and z axes to the coordinate information based on the x, y, and z coordinate values ​​as the encoded entity, thereby generating compressed point cloud data 506.

[0127] In the data collection device 10, for example, the CPU 1000 executes the encoding program according to the embodiment, thereby configuring the above-mentioned data selection unit 110, spatial quantization processing unit 111, octree encoding unit 112, arithmetic encoding unit 113 and syntax encoding unit 114, for example, as modules in the main memory area of ​​the RAM 1002.

[0128] The encoding program can be obtained from outside via the communication network 2, for example, by communication via the communication I / F 1007, or can be obtained from a storage medium connected to the data I / F 1006 and installed on the data collection device 10.

[0129] 15 is a functional block diagram illustrating an example of a point cloud data decoding device 200 according to an embodiment. The point cloud data decoding device 200 may be included in the terminal device 20 illustrated in FIG. 1, for example.

[0130] 15, the point cloud data decoding device 200 includes a syntax decoding unit 210, an arithmetic decoding unit 211, an octree decoding unit 212, a spatial inverse quantization processing unit 213, and a file generation unit 214.

[0131] The syntax decoding unit 210, the arithmetic decoding unit 211, the octree decoding unit 212, the spatial inverse quantization processing unit 213, and the file generation unit 214 may be configured by a decoding program according to the embodiment running on the CPU 2000 (see FIG. 3 ). However, this is not limiting, and some or all of the syntax decoding unit 210, the arithmetic decoding unit 211, the octree decoding unit 212, the spatial inverse quantization processing unit 213, and the file generation unit 214 may be configured by hardware circuits that operate in cooperation with each other.

[0132] Compressed point cloud data 506 is input to the point cloud data decoding device 200. The compressed point cloud data 506 may be the compressed point cloud data 506 generated by the point cloud data encoding device 100 described with reference to Fig. 14. The point cloud data decoding device 200 passes the input compressed point cloud data 506 to the syntax decoding unit 210.

[0133] The syntax decoding unit 210 syntax-decodes the compressed point cloud data 506 and extracts from the compressed point cloud data 506 the AU header included in the header area 51, the UD included in the user data area 52, and each unit included in the entity data area 53. The syntax decoding unit 210 passes each unit of the compressed point cloud data 506, for example, included in the entity data area 53, as arithmetically coded data to the arithmetic decoding unit 211. The syntax decoding unit 210 passes the AU header extracted from the header area 51 of the compressed point cloud data 506 and each unit extracted from the user data area 52 to the file generation unit 214.

[0134] The arithmetic decoding unit 211 decodes the arithmetically coded data passed from the syntax decoding unit 210 to obtain octree data. The arithmetic decoding unit 211 passes the octree data to the octree decoding unit 212.

[0135] The octree decoding unit 212 obtains spatial quantized data by performing octree decoding on the octree data passed from the arithmetic decoding unit 211. The octree decoding unit 212 passes the spatial quantized data to a spatial inverse quantization processing unit 213.

[0136] The spatial inverse quantization processing unit 213 decodes the spatial quantization data passed from the octree decoding unit 212 and passes the decoded data to the file generation unit 214 .

[0137] The file generation unit 214 generates an LAS file 60′ using the AU header passed from the syntax decoding unit 210, each unit extracted from the user data area 52, and the data passed from the spatial inverse quantization processing unit 213. The point cloud data decoding device 200 may output the generated compressed point cloud data 506 at an appropriate timing.

[0138] In the terminal device 20, for example, the CPU 2000 executes the decoding program according to the embodiment, thereby configuring the above-mentioned syntax decoding unit 210, arithmetic decoding unit 211, octree decoding unit 212, spatial inverse quantization processing unit 213 and file generation unit 214, for example, as modules, in the main memory area of ​​the RAM 2002.

[0139] The decryption program can be obtained from outside via the communication network 2, for example, by communication via the communication I / F 2007, or can be obtained from a storage medium connected to the data I / F 2006, and then installed on the terminal device 20.

[0140] FIG. 16 is a schematic diagram illustrating the encoding / decoding process according to the embodiment in comparison with the encoding / decoding process according to the existing technology.

[0141] In existing technology, the G-PCC encoder 120 has the configuration of the point cloud data encoding device 500 described with reference to Fig. 5, for example, and receives a PLY file 70 as input, G-PCC-encoding the input PLY file 70 to generate a G-PCC stream 55. The G-PCC stream 55 may be stream data that conforms to the same syntax as the compressed point cloud data 506. The G-PCC decoder 220 has the configuration of the point cloud data decoding device 520 described with reference to Fig. 7, for example, and decodes the G-PCC stream 55 to output a PLY file 70' in which the PLY file 70 has been restored.

[0142] In contrast, in an embodiment of the present disclosure, the G-PCC encoder 120 has the functionality of the point cloud data encoding device 100 described using Fig. 14, for example, and inputs the LAS file 60 and G-PCC-encodes the input LAS file 60 to generate a G-PCC stream 55. Furthermore, the G-PCC decoder 220 has the functionality of the point cloud data decoding device 200 described using Fig. 15, and decodes the G-PCC stream 55 and outputs an LAS file 60' in which the LAS file 60 is restored.

[0143] That is, in the embodiment of the present disclosure, during G-PCC encoding, the LAS file 60 is directly G-PCC encoded without going through the PLY file 70 to generate the G-PCC stream 55. Furthermore, during G-PCC decoding, the G-PCC stream 55 is G-PCC decoded without going through the PLY file 70 to restore the LAS file 60'.

[0144] 17 is a schematic diagram showing an example of file formats and attributes of three-dimensional point cloud data. In the example of FIG. 17, three types of file formats are shown: PLY, LAS, and G-PCC.

[0145] The PLY file format corresponds to the JGD2011 (Japanese Geodetic System 2011) as the ellipsoid representing the Earth's surface. Since the PLY file format does not have a field for recording the coordinate system, the corresponding ellipsoid is implicitly set to JGD2011, and the coordinate system is set to a plane rectangular coordinate system. The unit is implicitly set to meters. The internal representation of the geometry type is that the x, y, and z coordinate values ​​are each set to Float or Double type.

[0146] The LAS file format supports JGD2011 and WGS (World Geodetic System) 84 ellipsoids. When the ellipsoid is JGD2011, the coordinate system is a planar rectangular coordinate system with meters as the unit. When the ellipsoid is WGS84, the coordinate system is latitude-longitude-ellipsoid height with meters as the unit for latitude, longitude, and altitude. The internal representation of the geometry type is common to both JGD2011 and WGS84, with x, y, and z coordinate values ​​each being Long type, and with scale factor values ​​and offset values ​​for each of the x, y, and z axes.

[0147] The G-PCC file format implicitly corresponds to JGD2011 as an ellipsoid, and the coordinate system is a planar rectangular coordinate system with units in meters. The coordinate system may be other than a planar rectangular coordinate system. The internal representation of the geometry type is Unit type for each of the x, y, and z coordinate values, and each of the x, y, and z coordinate values ​​has an offset value, but the scale factor value is a common value for the x, y, and z axes.

[0148] (3-2. More Specific Description of Encoding / Decoding Process According to Embodiment) Next, the encoding / decoding process according to the embodiment will be described more specifically.

[0149] (3-2-1. Details of Encoding / Decoding Process) Fig. 18 is a schematic diagram for explaining the encoding / decoding process according to the embodiment. Fig. 18 shows the encoding / decoding process according to the embodiment, focusing on the header information of each file.

[0150] 18, it is assumed that the G-PCC encoding process is performed by a G-PCC encoder (not shown) that includes the functions of the point cloud data encoding device 100 described with reference to FIG. 14, for example. The G-PCC encoder G-PCC-encodes the LAS file 60 to generate a G-PCC file 50 that corresponds to the compressed point cloud data 506. That is, the G-PCC file 50 includes a header area 51, a user data area 52, and an entity data area 53. The G-PCC stream 55 may correspond to the entity data area 53.

[0151] In the LAS file 60, the public header area 61, the variable-length record area 62, and the extended variable-length record area 64 are set to fixed values ​​within the file. On the other hand, the point data record area 63 contains field data for each point of the point cloud data.

[0152] The G-PCC decoding process is assumed to be performed by a G-PCC decoder (not shown) that includes the functions of the point cloud data decoding device 200 described with reference to Fig. 15. The G-PCC decoder decodes the G-PCC file 50 and generates an LAS file 60' in which the LAS file 60 is restored.

[0153] First, the processing during G-PCC encoding according to the embodiment will be described. The G-PCC encoder according to the embodiment includes a field data selection unit 130. During G-PCC encoding, the field data selection unit 130 selects each field from the point data record area 63 of the LAS file 60 and places the fields in the G-PCC stream 55. The field data selection unit 130 may be included in the data selection unit 110 described using FIG. 14, for example.

[0154] 19 is a schematic diagram for explaining the field selection process by the field data selection unit 130 according to the embodiment. For the sake of explanation, it is assumed here that the LAS file 60 is converted into the PLY file 70.

[0155] The left side of Fig. 19 shows the fields of the point data record area 63 of the LAS file 60. Here, an example is shown in which the format ID of the LAS file 60 is "3," and the fields (fields "X" to "Blue") with diagonal lines past the item "Required" in Fig. 10A are shown. The total size of each point in the point cloud data due to these fields is 34 bytes in binary conversion.

[0156] The format ID is stored in the field "Point Data Record Format" among the fields of the public header area 61 in the LAS file 60, for example.

[0157] The right side of Figure 19 shows fields in the PLY file 70 that correspond to each field of the LAS file 60 shown on the left. The data "X", "Y", and "Z" in the PLY file 70 correspond to the fields "X", "Y", and "Z" in the LAS file 60, respectively. The data "Color_RGB" in the PLY file 70 includes values ​​for "Red", "Green", and "Blue", and corresponds to the fields "Red", "Green", and "Blue" in the LAS file 60. Furthermore, the data "reflectance" in the PLY file 70 corresponds to the field "Intensity" in the LAS file 60.

[0158] The data "X", "Y", and "Z" in the PLY file 70 each have a Float data type and a data size of 12 bytes. The data "Color_RGB" has a uchar data type and a data size of 3 bytes. The data "reflectance" has a ushort data type. The total size of the PLY file 70 is 15 bytes in binary conversion.

[0159] 19, of the members (fields) of the point cloud data to be encoded in the LAS file 60, seven fields are used: fields "X," "Y," and "Z," field "Intensity," and fields "Red," "Green," and "Blue," and the other nine fields are discarded. In existing technologies, the nine fields discarded during encoding cannot be restored even when decoding is performed.

[0160] Returning to FIG. 18 , the field data selection unit 130 places, for example, fields "X," "Y," and "Z" in the point data record area 63 of the LAS file 60 as geometry in the G-PCC stream 55. At this time, if the coordinate system of the LAS file 60 is a planar rectangular coordinate system, coordinate conversion of the values ​​of the fields "X," "Y," and "Z" is not necessary. The field data selection unit 130 places the fields "Red," "Green," and "Blue" as attribute information in the G-PCC stream 55. The field data selection unit 130 also places the field "Intensity" in the G-PCC stream 55 as reflectance.

[0161] The G-PCC encoder places each field of the public header area 61 of the LAS file 60 in the user data area 52 of the G-PCC file 50 as relevant information related to G-PCC encoding. That is, the G-PCC encoder places each field of the public header area 61 in the user data area 52 as an LAS Header. The LAS Header includes the fields "X Scale Factor," "Y Scale Factor," and "Z Scale Factor" in the public header area 61, and the fields "X Offset," "Y Offset," and "Z Offset."

[0162] Next, the process of decoding G-PCC according to the embodiment will be described. The G-PCC decoder includes a field data decoding unit 230. The field data decoding unit 230 may be included in the functions of the point cloud data decoding device 200 described with reference to FIG. 15, for example.

[0163] The field data decoder 230 decodes the G-PCC stream 55 and restores each field in the point data record area 63 in the decoded LAS file 60'. The G-PCC decoder also records each piece of data stored in the user data area 52 in the public header area 61 of the LAS file 60'.

[0164] In an embodiment, by performing G-PCC encoding and decoding of the LAS file 60 as described above, the scale factor values ​​and offset values ​​for each of the three axes (X, Y, and Z axes) can be restored in the decoded LAS file 60'.

[0165] The scale factor value and the offset value will now be described with reference to Fig. 20, which is a schematic diagram for explaining the scale factor value and the offset value.

[0166] In Figure 20, the coordinate origin (0,0,0) is, for example, the reference point of a geodetic reference system. Each point of the point cloud data acquired by measurement is distributed between the minimum value min(x0,y0,z0) and the maximum value max(x1,y1,z1) of the coordinate values ​​(x,y,z) in the figure. By subtracting the minimum value min(x0,y0,z0) from the coordinate values ​​(x,y,z) of each point within the range as offset value components for the three axes, it is possible to reduce the absolute value components of the x,y,z coordinate values ​​of each point.

[0167] The space represented by the minimum value min(x0, y0, z0) and the maximum value max(x1, y1, z1) is called a BBOX (Bounding Box). When a plane rectangular coordinate system is used as the coordinate system, the distance from the reference point to the minimum value min(x0, y0, z0) is expected to be several hundred kilometers, based on the distance between the reference points and the range covered by the reference points. The size (amplitude) of the BBOX (Bounding Box) is expected to be several kilometers.

[0168] The x, y, and z coordinate values ​​of each point in the point cloud data are converted to integers, for example, as follows: The offset values ​​for the x, y, and z axes are subtracted from the x, y, and z coordinate values ​​of each point. Then, to maintain decimal precision, the x, y, and z coordinate values ​​of each point are multiplied by a scale factor value, and then rounded to convert to integers. In the LAS file 60, the scale factor values ​​are managed for each of the three axes, allowing for more accurate conversion to integers.

[0169] 19, in the existing technology, fields discarded during encoding are not restored even when decoding is performed. In the embodiment, the field selection process of the field data selection unit 130 makes it possible to perform G-PCC encoding without discarding any fields of the LAS file 60.

[0170] Fig. 21 is a schematic diagram for explaining the processing of the field data selection unit 130 according to the embodiment. Fig. 21 is an example illustrating Fig. 18 described above, focusing on the processing of the field data selection unit 130.

[0171] First, the G-PCC encoding process according to the embodiment will be described. In Fig. 21, the field data selection unit 130 selects an encoding method for each field based on the change in each point in the point cloud data for each field in the point data record area 63 of the LAS file 60. It selects whether each field should be placed in the user data area 52 of the G-PCC file 50 or in the G-PCC stream 55.

[0172] More specifically, the field data selection unit 130 places fields "X," "Y," and "Z" in the G-PCC stream 55 as normal G-PCC data. The field data selection unit 130 places fields that change at each point other than fields "X," "Y," and "Z" in the G-PCC stream 55 as additional G-PCC attributes. For fields that change but are minimal, the field data selection unit 130 divides the point cloud data and places fields with the same value in the same group. Furthermore, fields that do not change at each point are recorded as fixed values ​​in the user data area 52. The field data selection unit 130 also records field data encoding method information corresponding to these selection results in the user data area 52.

[0173] As described above, the G-PCC encoding unit records each field of the public header area 61 in the LAS file 60 in the user data area 52 of the G-PCC file 50 (LAS Header). The G-PCC encoding unit may also record each field of the variable-length record area 62 and the extended variable-length record area 64 in the LAS file 60 in the user data area 52.

[0174] Next, the process of decoding the G-PCC according to the embodiment will be described. The G-PCC decoder extracts each field (LAS Header) of the public header area 61 in the LAS file 60 from the user data area 52 of the G-PCC file 50.

[0175] The G-PCC decoder records the extracted LAS Header in the public header area 61 of the decoded LAS file 60'. Furthermore, if the fields of the variable length record area 62 and the extended variable length record area 64 in the LAS file 60 before encoding are recorded in the user data area 52, the G-PCC decoder extracts the fields of the variable length record area 62 and the extended variable length record area 64 from the user data area 52 and records them in the variable length record area 62 and the extended variable length record area 64, respectively, of the decoded LAS file 60'.

[0176] In the G-PCC decoder, the field data decoding unit 230 references the encoding method information for the field data recorded in the user data area 52 of the G-PCC file 50, acquires the data recorded in the G-PCC file 50 for each field, and restores each field. The field data decoding unit 230 records each restored field in the point data record area 63 in the decoded LAS file 60'.

[0177] As described above, in the G-PCC encoding according to the embodiment, fields that are defined in the LAS file 60 but not in the G-PCC file 50 are recorded in the user data area 52 of the G-PCC file 50. Also, in the G-PCC decoding according to the embodiment, the fields of the LAS file 60' after decoding are restored using the data that was recorded in the user data area 52 of the G-PCC file 50 during encoding.

[0178] Therefore, by using the G-PCC encoding and decoding according to the embodiment, it is possible to avoid a situation where, when converting the LAS file 60 to a file of another format, fields that are defined in the LAS file 60 but not defined in the converted file of another format are lost, as described with reference to Fig. 19. Therefore, by applying the embodiment, lossless encoding in the G-PCC encoding of the LAS file 60 is possible.

[0179] (3-2-2. Specific Examples of Encoding / Decoding Processes) Next, specific examples of encoding / decoding processes according to the embodiment will be described.

[0180] 22 is a flowchart illustrating an example of a field selection process according to an embodiment, where section (a) of FIG. 22 shows an overview of the field selection process, and section (b) of FIG. 22 shows the field selection process in more detail.

[0181] In section (a) of FIG. 22, in step S10, the field data selection unit 130 acquires the format ID of the LAS file 60 to be encoded.

[0182] In the next step S11, the field data selection unit 130 measures the amount of change for each field in the LAS file 60. That is, the field data selection unit 130 compares each field of the LAS file 60 with points in the point cloud data for each field, and measures the amount of change between the points in the field. In the next step S12, the field data selection unit 130 determines the encoding method for each field based on the amount of change for each field measured in step S11.

[0183] In the next step S13, the field data selection unit 130 records information indicating the coding method for each field determined in step S12 in the user data area 52 of the G-PCC file 50.

[0184] In section (b) of Figure 22, steps S110 to S113 on the left side show the process of step S11 in section (a) in more detail, and steps S120 to S123 on the right side show the process of step S12 in section (a) in more detail.

[0185] In section (b) of FIG. 22, in step S110, the field data selection unit 130 acquires the maximum and minimum values ​​for each field from all point data of the point cloud data included in the LAS file 60.

[0186] In the next step S111, the field data selection unit 130 determines for each field whether the maximum value and minimum value match. For a field where the maximum value and minimum value match (step S111, "Yes"), the field data selection unit 130 records the value of that field in the user data area 52 of the G-PCC file 50 as a fixed value indicating the attributes of the entire file in step S120, and then ends the series of processes according to this flowchart.

[0187] In step S111, for the remaining fields determined in step S111, i.e., fields whose maximum and minimum values ​​do not match (step S111, "No"), the field data selection unit 130 performs a sort (e.g., Radix Sort) on all points for each of the remaining fields in step S112.

[0188] In the next step S113, the field data selection unit 130 determines, for each field, whether the frequency is equal to or less than a predetermined value. Here, frequency refers to the number of point data items with different values ​​in the target field. For example, a case where half of the points included in the target point cloud data have the same value and the remaining points have other values ​​is considered to be more frequent than a case where all but one point have the same value.

[0189] If the field data selection unit 130 determines in step S113 that the frequency in the target field is equal to or less than a predetermined value ("Yes" in step S113), it proceeds to step S121, where it divides and encodes the field based on the field value. More specifically, the field data selection unit 130 divides each field into groups with the same point value, and encodes the groups by tile division or slice division. In the next step S122, the field data selection unit 130 records the field value of the field in the user data area 52 of the G-PCC file 50, and ends the series of processes according to this flowchart.

[0190] A slice is a G-PCC coding unit obtained by dividing and grouping point cloud data, and includes a header section and a substance section. A tile is management information for grouping and managing slices, and includes a header section and slice management information. One tile is composed of (references) one or more slices.

[0191] If the field data selection unit 130 determines in step S113 that the frequency in the target field exceeds a predetermined value (step S113, "No"), it proceeds to step S123. In step S123, the field data selection unit 130 G-PCC-encodes the remaining fields, i.e., fields whose values ​​differ from the maximum and minimum values ​​of each point and whose frequencies exceed a predetermined value, as additional attributes and records them point by point in the G-PCC stream 55. After recording in the G-PCC stream 55, the G-PCC encoder terminates the series of processes according to this flowchart.

[0192] Next, a description will be given of a process for determining whether direct INT encoding of the LAS file 60, i.e., direct encoding of integer values, is possible, according to an embodiment. Figures 23A and 23 are flowcharts illustrating an example of a process for determining whether direct INT encoding of the LAS file 60 is possible based on a scale factor value, according to an embodiment. Figure 23A shows an overview of the process for determining whether direct INT encoding of the LAS file 60 is possible, and Figure 23B shows the process in more detail.

[0193] 23A, in step S20, the G-PCC encoder obtains the scale factor values ​​and offset values ​​for each of the three X, Y, and Z axes from the fields "X Scale Factor," "Y Scale Factor," and "Z Scale Factor," as well as the fields "X Offset," "Y Offset," and "Z Offset" in the LAS Header (public header area 61) of the pre-encoding LAS file 60. The G-PCC encoder may also obtain these scale factor values ​​and offset values ​​from the user data area 52 of the G-PCC file 50.

[0194] In the next step S21, the G-PCC encoder obtains coordinate system information from the LAS Header. More specifically, the G-PCC encoder obtains coordinate reference system information recorded in the variable-length record area 62 of the LAS file 60. The coordinate system is either a latitude-longitude ellipsoidal coordinate system or a planar rectangular coordinate system. The G-PCC encoder may obtain the coordinate reference system information from the user data area 52 of the G-PCC file 50.

[0195] In the next step S22, the G-PCC encoder obtains the maximum and minimum values ​​for each of the LAS fields X, Y, and Z. More specifically, the G-PCC encoder obtains the maximum and minimum values ​​of each point included in the point cloud data for each of the fields "X," "Y," and "Z" from the point data record area 63 of the LAS file 60.

[0196] In the next step S23, the G-PCC encoder determines whether direct INT encoding of the target LAS file 60 is possible based on the data acquired in steps S20 to S22.

[0197] The processing of step S23 will be explained in more detail using Fig. 23B. In step S200, the G-PCC encoder determines whether the coordinate system applied to the target LAS file 60 is a planar rectangular coordinate system or a latitude-longitude ellipsoidal coordinate system, based on the coordinate reference system information acquired in step S21 of Fig. 23A.

[0198] When the G-PCC encoder determines that the coordinate system applied to the target LAS file 60 is a planar rectangular coordinate system, if the target region is Japan, EPSG (European Petroleum Survey Group): 6669-6687, JGD2011 is applied, as shown in step S201.

[0199] In the next step S202, the G-PCC encoder determines whether the scale factor values ​​for each of the three axes X, Y, and Z acquired in step S20 in Fig. 23A are all equal to or greater than a first predetermined value (assumed to be 0.001). Since the scale factor values ​​are in meters, if the first predetermined value is 0.001, it will determine whether the scale factor value is equal to or greater than 1 mm.

[0200] If the G-PCC encoder determines that all of the scale factor values ​​for the three axes X, Y, and Z are equal to or greater than the first predetermined value ("Yes" in step S202), it proceeds to step S212. On the other hand, if the G-PCC encoder determines that at least one of the scale factor values ​​for the three axes x, y, and z is less than the first predetermined value ("No" in step S202), it proceeds to step S203.

[0201] In step S203, the G-PCC encoder calculates the difference between the maximum and minimum values ​​for each LAS field XYZ based on the values ​​acquired in step S22 of Fig. 23A, and determines whether the calculated difference is 2^21 (21 bits) or more. In other words, in step S203, the G-PCC encoder determines whether the range of BBOX is expressed using 21 bits or more.

[0202] If the G-PCC encoder determines in step S203 that the difference is less than 2^21 (step S203, "<2^21"), it proceeds to step S213. In step S213, the G-PCC encoder estimates that the target LAS file 60 is probably capable of direct INT encoding, and ends the series of processes according to this flowchart.

[0203] On the other hand, if the G-PCC encoder determines in step S203 that the difference is 2^21 or greater (step S203, "≧2^21"), it proceeds to step S214. In other words, if the G-PCC encoder determines that the BBOX range is expressed using 21 bits or more, it proceeds to step S214.

[0204] In this case, the G-PCC encoder may divide the BBOX into tiles so that each range is a space less than 2^21. Furthermore, the G-PCC encoder may divide each tile into slices if the number of points included in the tile exceeds 1E+6. In this way, dividing the BBOX according to its size makes it possible to support lossless encoding.

[0205] In the above-mentioned step S200, if it is determined that the coordinate system applied to the target LAS file 60 is a latitude-longitude ellipsoidal coordinate system based on WGS84, and if the target region is Japan, EPSG:4326 is applied as shown in step S210.

[0206] In the next step S211, the G-PCC encoder determines whether the scale factor values ​​for the two axes, the x-axis and the y-axis, acquired in step S20 of Fig. 23A are all equal to or greater than a second predetermined value (assumed to be 1E-10), and whether the scale factor value for the z-axis (height direction) is equal to or greater than the first predetermined value (0.001) described above. If the G-PCC encoder determines that the scale factor values ​​for the three axes, the x-axis, y-axis, and z-axis, satisfy these conditions ("Yes" in step S211), it proceeds to step S212.

[0207] In step S212, the G-PCC encoder determines that the target LAS file 60 can be directly INT encoded, and ends the series of processes according to this flowchart.

[0208] If the G-PCC encoder determines in step S211 that the scale factor values ​​of the three axes (x, y, and z) do not satisfy the above-described condition (step S211, "No"), it proceeds to step S214. In step S214, the G-PCC encoder estimates that re-integerization processing is required after decimalization, and ends the series of processing steps according to this flowchart.

[0209] If the coordinate information is coded in units smaller than mm (millimeters), for example, μm (micrometers), problems may occur. By executing the process according to the flowchart in Fig. 23B, such problems can be avoided.

[0210] (3-2-3. Selection of Encoding Method) Next, a method for selecting an encoding method according to the embodiment will be described.

[0211] 24 is a flowchart illustrating an example of a first selection method according to an embodiment for selecting an encoding method for the LAS file 60. The first selection method is an example in which the same LAS file 60 is actually encoded using a plurality of encoding methods, the file sizes of the generated compressed encoded files are compared, and the encoding method to be adopted is selected.

[0212] 24, in step S30, the G-PCC encoder performs compression encoding using the G-PCC compression process described above on the target LAS file 60, generating a G-PCC file 50. In addition, in step S31, the G-PCC encoder performs LAZ compression encoding on the LAS file 60, generating an LAZ file. LAZ compression is a compression encoding method applied to LAS files, and a standard library is distributed.

[0213] In step S32, the G-PCC encoder compares the file size of the G-PCC file 50 generated in step S30 with the file size of the LAZ file generated in step S31.

[0214] If the G-PCC encoder determines that the file size of G-PCC file 50 is equal to or less than the file size of the LAZ file (step S32, "LAZ≧G-PCC"), it proceeds to step S33 and selects G-PCC file 50 as the output file. On the other hand, if the G-PCC encoder determines that the file size of G-PCC file 50 exceeds the file size of the LAZ file (step S32, "LAZ<G-PCC"), it proceeds to step S34 and selects the LAZ file as the output file.

[0215] 25 is a flowchart illustrating an example of a second selection method for selecting an encoding method for the LAS file 60 according to an embodiment. The second selection method is an example in which an encoding method to be adopted is selected from a plurality of encoding methods according to the coordinate system of the target LAS file 60.

[0216] In step S40, the G-PCC encoder obtains coordinate reference system (CRS) information from the variable length record area 62 of the target LAS file 60 and determines whether the coordinate system applied to the LAS file 60 is a planar rectangular coordinate system or a latitude-longitude ellipsoidal coordinate system.

[0217] When the G-PCC encoder determines that the coordinate system applied to the target LAS file 60 is a planar rectangular coordinate system, if the target region is Japan, EPSG: 6669-6687, JGD2011 is applied, as shown in step S41.

[0218] In the next step S42, the G-PCC encoder acquires the scale factor values ​​for each of the three axes, x, y, and z, from the fields "X Scale Factor," "Y Scale Factor," and "Z Scale Factor" in the public header area 61 of the target LAS file 60. The G-PCC encoder determines whether the acquired scale factor values ​​for each of the three axes are all equal to or greater than a first predetermined value (assumed to be 0.001).

[0219] If the G-PCC encoder determines that the acquired scale factor values ​​for each of the three axes are all equal to or greater than the first predetermined value (step S42, "Yes"), it proceeds to step S43.

[0220] In step S43, the G-PCC encoder adopts the compression encoding method using the G-PCC compression process described above as the encoding method to be applied to the target LAS file 60. The G-PCC encoder performs G-PCC compression encoding on the target LAS file 60 to generate a G-PCC file 50. In the next step S44, the G-PCC encoder selects the G-PCC file 50 generated in step S43 as the output file, and terminates the series of processes according to the flowchart in FIG.

[0221] On the other hand, if the G-PCC encoder determines in step S42 that at least one of the scale factor values ​​for each of the three axes acquired is less than the first predetermined value (step S42, "No"), it transitions the processing to step S45.

[0222] In step S45, the G-PCC encoder adopts the LAZ compression encoding method as the encoding method to be applied to the target LAS file 60. The G-PCC encoder generates an LAZ file by performing LAZ compression encoding on the target LAS file 60. In the next step S46, the G-PCC encoder selects the LAZ file generated in step S45 as the output file, and ends the series of processes according to the flowchart in FIG.

[0223] (4. Comparison of the embodiment according to the present disclosure with the existing technology) Next, the embodiment according to the present disclosure will be compared with the existing technology to explain the advantages of the configuration according to the embodiment.

[0224] FIG. 26 is a schematic diagram for explaining G-PCC encoding / decoding according to existing technology.

[0225] In the existing technology, during G-PCC encoding, before processing by the G-PCC encoding unit 140, the values ​​of the fields "X," "Y," and "Z" are extracted from the point data record area 63 in the LAS file 60 before encoding, and the BBOX is searched for. The G-PCC encoding unit 140 also extracts scale factor values ​​(scale XYZ) for the x, y, and z axes from the fields "X Scale Factor," "Y Scale Factor," and "Z Scale Factor" in the public header area 61 in the LAS file 60. Similarly, the G-PCC encoding unit 140 extracts offset values ​​(offset XYZ) for the x, y, and z axes from the fields "X Offset," "Y Offset," and "Z Offset" in the public header area 61 in the LAS file 60.

[0226] The G-PCC encoding unit 140 calculates "coordinate value x scale factor value + offset value" for each of the x, y, and z axes for each point included in the BBOX (see equation (2) above), and converts the coordinate value of each point into a double-type value. The G-PCC encoding unit 140 places the coordinate value of each point whose data type has been converted into double-type in the main body of the G-PCC stream 55.

[0227] The G-PCC encoding unit 140 stores the offset values ​​for each of the x, y, and z axes (offset XYZ) in the header area 51 of the G-PCC file 50. Furthermore, the G-PCC encoding unit 140 generates a single scale factor value that is common to the x, y, and z axes based on the scale factor values ​​for each of the x, y, and z axes, and stores this in the header area 51 of the G-PCC file 50.

[0228] In existing technology, when decoding G-PCC, the G-PCC decoding unit 240 obtains the x, y, and z axes (point XYZ) of each point of the point cloud data from the main body of the G-PCC stream 55 of the G-PCC file 50. The G-PCC decoding unit 240 also obtains the offset values ​​(offset XYZ') for each of the x, y, and z axes and the scale factor value (scale XYZ') common to the x, y, and z axes from the header area 51 of the G-PCC file 50.

[0229] The G-PCC decoding unit 240 converts the coordinate values ​​of each point in the LAS file 60, which were converted to double type and decimalized during G-PCC encoding, into integers. Specifically, the G-PCC decoding unit 240 performs subtraction (offset assignment) by subtracting offset values ​​(offset XYZ) for each of the x, y, and z axes from the coordinate values ​​of each point along the x, y, and z axes. The G-PCC decoding unit 240 divides the subtraction results obtained for each of the x, y, and z axes by a scale factor value (scale XYZ') common to the x, y, and z axes to restore the coordinate values ​​for each of the x, y, and z axes. The G-PCC decoding unit 240 records the restored coordinate values ​​for each of the x, y, and z axes as the values ​​of the fields "X," "Y," and "Z," respectively, in the point data record area 63 of the decoded LAS file 60'.

[0230] Thus, during G-PCC encoding according to the existing technology, G-PCC encoding unit 140 decimalizes the coordinate values ​​of each point stored as integer values ​​in LAS file 60 before encoding and stores them in G-PCC file 50. At the same time, G-PCC encoding unit 140 converts the scale factor values ​​stored as values ​​for each of the x, y, and z axes in LAS file 60 before encoding into scale factor values ​​common to the x, y, and z axes and stores them in G-PCC file 50. During G-PCC decoding, G-PCC decoding unit 240 converts the coordinate values ​​of each point into integers by reflecting the scale factor value common to the x, y, and z axes on the result of assigning offset values ​​for each of the x, y, and z axes to the coordinate values ​​of each point.

[0231] As described above, in G-PCC encoding using existing technology, the coordinate values ​​of each point in the LAS file 60 before encoding are first converted to decimals, then converted back to integers, and recorded in the decoded LAS file 60'. Also, in G-PCC decoding, the scale factor values ​​for the x, y, and z axes in the LAS file 60 before encoding are converted into a single common scale factor value, and this is recorded in the public header area 61 of the decoded LAS file 60' together with the offset values ​​for the x, y, and z axes. Therefore, there is a risk that the accuracy of the coordinate values ​​of each point in the decoded LAS file 60' will be degraded compared to the coordinate values ​​of each point in the LAS file 60 before encoding.

[0232] FIG. 27 is a schematic diagram for explaining G-PCC encoding / decoding according to the embodiment.

[0233] In the embodiment, during G-PCC encoding, the G-PCC encoding unit 150 extracts the fields “X Scale Factor,” “Y Scale Factor,” and “Z Scale Factor,” and the fields “X Offset,” “Y Offset,” and “Z Offset” from the public header area 61 in the LAS file 60 before encoding, and records the scale factor values ​​(scale XYZ) and offset values ​​(offset XYZ) for each of the x, y, and z axes in the user data area 52 in the G-PCC file 50.

[0234] In addition, the G-PCC encoding unit 150 extracts the fields "X", "Y", and "Z" from the point data record area 63 in the LAS file 60 before encoding, and places the x, y, and z coordinate values ​​as they are, i.e., integer values ​​with a data type of Long, into the main body of the G-PCC stream 55.

[0235] In the embodiment, during G-PCC decoding, the G-PCC decoding unit 250 extracts the x, y, and z coordinate values ​​of each point of the point cloud data from the main body of the G-PCC stream 55 of the G-PCC file 50. The G-PCC decoding unit 250 records the x, y, and z coordinate values ​​of each extracted point as coordinate values ​​of Long data type in the point data record area 63 in the decoded LAS file 60′.

[0236] Furthermore, the G-PCC decoding unit 250 extracts the scale factor values ​​(scale XYZ') and offset values ​​(scale XYZ') for each of the x, y, and z axes from the user data area 52 of the G-PCC file 50. The G-PCC decoding unit 250 records the extracted scale factor values ​​and offset values ​​for each of the x, y, and z axes in the fields "X Scale Factor," "Y Scale Factor," and "Z Scale Factor," as well as the fields "X Offset," "Y Offset," and "Z Offset," respectively, in the public header area 61 of the decoded LAS file 60'.

[0237] In this way, the G-PCC encoding unit 150 according to the embodiment records the values ​​of the fields "X Scale Factor," "Y Scale Factor," and "Z Scale Factor," as well as the fields "X Offset," "Y Offset," and "Z Offset," in the public header area 61 of the LAS file 60 before encoding, directly into the user data area 52 of the G-PCC file 50. Similarly, the G-PCC encoding unit 150 records the values ​​of the fields "X," "Y," and "Z" in the point data record area 63 of the LAS file 60 before encoding, as integer values ​​of Long data type, directly into the user data area 52 of the G-PCC file 50. This prevents deterioration in the accuracy of the coordinate values ​​of each point in the decoded LAS file 60' compared to the coordinate values ​​of each point in the LAS file 60 before encoding.

[0238] FIG. 28 is a schematic diagram showing a comparison of the representation of scale factor values, offset values, and coordinate values ​​of each point between the G-PCC file 50 and the LAS file 60.

[0239] In the G-PCC file 50, the scale factor value (scale) is common to the x, y, and z axes and is composed of a numerator and a denominator. On the other hand, in the LAS file 60, the scale factor value is separate for the x, y, and z axes and is expressed as a double data type.

[0240] In the G-PCC file 50, the offset values ​​(offsets) are individual for the x, y, and z coordinates and are converted into integers using a common scale value for the x, y, and z axes. On the other hand, in the LAS file 60, the offset values ​​are individual for the x, y, and z axes and each is expressed as a Double data type.

[0241] In the G-PCC file, the x, y, and z coordinates for each point are separated into x, y, and z coordinates, each of which has a Long data type and a range Duration of less than 2^21. On the other hand, in the LAS file 60, the x, y, and z coordinates for each point are separated into x, y, and z coordinates, each of which has a Long data type and a range Duration of 2^32.

[0242] As described above, in an embodiment according to the present disclosure, the x, y, and z coordinate values ​​are used as the x, y, and z coordinate values ​​during G-PCC encoding while remaining in the Long type internal representation in the LAS file 60. This eliminates the need for coordinate system conversion processing and quantization operations, eliminating the errors that result from these and enabling lossless compression. Furthermore, since the internal representation of the x, y, and z coordinate values ​​in the LAS file 60 can be reproduced during decoding, lossless compression is possible.

[0243] Furthermore, in the present disclosure, the internal integer representation using the scale factor value and offset value stored in the LAS file 60 can be reproduced as is during decoding, making it possible to apply the present disclosure regardless of the coordinate system. Also, it is possible to omit the complex calculation process required to convert the WGS84 latitude-longitude ellipsoidal coordinate system into a planar rectangular coordinate system.

[0244] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0245] Note that the present technology can also be configured as follows. (1) An encoding method including: an encoding step of encoding first data having coordinate information of a three-dimensional point cloud including, for each point, coordinates of three axes, each expressed as an integer, and attribute information including correction information for each of the three axes for correcting the coordinates of the three axes, into second data having the coordinate information of the three-dimensional point cloud and the correction information common to the three axes, wherein the encoding step adds the correction information for each of the three axes to the second data. (2) The encoding method according to (1), wherein the encoding step selects fields in the first data based on changes at each point. (3) The encoding method according to (2), wherein the encoding step adds the fields selected as not changing at each point to the second data as fixed-value fields. (4) The encoding method according to (2) or (3), wherein the encoding step includes encoding the fields selected as having a frequency of changes at each point exceeding a threshold, by including them in the coordinate information. (5) The encoding method according to (4), wherein the encoding step adds fields selected as having a frequency of change for each point greater than or equal to 1 and less than the threshold to the second data for each block obtained by dividing the three-dimensional point cloud. (6) The encoding method according to (5), wherein the encoding step divides the three-dimensional point cloud based on the selected fields. (7) The encoding method according to any one of (2) to (6), wherein the encoding step adds information indicating the selection method for each field to the second data. (8) The encoding method according to any one of (1) to (7), wherein the encoding step estimates whether coordinate values ​​in the coordinate information can be encoded as integers as they are, based on a coordinate system of the coordinate information, the correction information, and the maximum and minimum values ​​for each of the three axes, each included in the first data.(9) The encoding method according to any one of (1) to (8), wherein the first data is data conforming to an LAS (LASer) file format, and the second data is data encoded by an encoding method conforming to GPCC (Geometry-based point cloud compression). (10) The encoding method according to (9), wherein the encoding step adopts data of a smaller size between an LAZ file obtained by losslessly compressing the first data and the second data. (11) The encoding method according to (9), wherein the encoding step determines which of data obtained by encoding the first data into an LAZ file or the second data to adopt, based on a coordinate system of coordinate information included in the first data and the correction information. (12) The encoding method according to any one of (1) to (11), wherein the correction information includes scale information. (13) A decoding method according to (13), comprising: a decoding step of decoding second data encoded with first data including coordinate information of a three-dimensional point cloud including, for each point, coordinates of three axes each expressed as an integer, and correction information for each of the three axes for correcting the coordinates of the three axes, wherein the second data has the coordinate information of the three-dimensional point cloud and the correction information common to the three axes, and the correction information for each of the three axes is added, and the decoding step arranges the correction information for each of the three axes added to the second data in a predetermined area of ​​the first data different from an area where the coordinate information is arranged. (14) The decoding method according to (13), wherein the decoding step arranges the coordinate information decoded from the second data in an area where coordinate information is arranged in the first data, and arranges the correction information for each of the three axes added to the second data in an area where the correction information for each of the three axes is arranged in the first data. (15) The decoding method according to (13) or (14), wherein the decoding step determines a field in the first data in which to place data decoded from the second data, based on coding information for each field that is added to the second data together with the correction information.(16) The decoding method according to (15), wherein the decoding step arranges, based on the encoding information, from among the coordinate information included in the second data, coordinate information selected as having a frequency of change per point exceeding a threshold, in an area in the first data that includes the coordinate information. (17) The decoding method according to (15) or (16), wherein, based on the encoding information, from among the coordinate information included in the second data, coordinate information selected as having a frequency of change per point that is equal to or greater than 1 and equal to or less than the threshold, in an area in the first data that includes the coordinate information, for each block obtained by dividing the three-dimensional point cloud. (18) The decoding method according to any one of (15) to (17), wherein the correction information includes scale information. (19) An information processing system including: an encoding device that encodes first data having coordinate information of a three-dimensional point cloud, the coordinates of which are each expressed as an integer along three axes for each point, and attribute information including correction information for each of the three axes for correcting the coordinates of each of the three axes, into second data having the coordinate information of the three-dimensional point cloud and the correction information common to the three axes; and a decoding device that decodes the second data, wherein the encoding device adds the attribute information for each of the three axes to the second data, and the decoding device places the correction information for each of the three axes added to the second data in a predetermined area of ​​the first data different from an area where the coordinate information is placed.

[0246] 1 Information processing system 2 Communication network 4 Drone 10 Data collection device 20 Terminal device 30 Server 40a, 40b, 40c, 40d Photographed image 41 Uncompressed 3D data 42 G-PCC compressed data 50 G-PCC file 51 Header area 52 User data area 53 Entity data area 60, 60' LAS file 61 Public header area 62 Variable length record area 63 Point data record area 64 Extended variable length record area 70 PLY file 80 File conversion SW 100, 500, 500a Point cloud data encoding device 110 Data selection unit 111, 510 Spatial quantization processing unit 112, 511 Octree encoding unit 113, 512 Arithmetic encoding unit 114, 514 Syntax encoding unit 120 G-PCC encoder 130 Field data selection unit 140, 150 G-PCC encoding unit 200, 520 Point cloud data decoding device 210, 530 Syntax decoding unit 211, 531 Arithmetic decoding unit 212, 532 Octree decoding unit 213, 534 Spatial inverse quantization processing unit 214 File generation unit 220 G-PCC decoder 230 Field data decoding unit 240, 250 G-PCC decoding unit 501, 501' Point cloud data 502, 502' Header data 503, 503' Body data 506 Compressed point cloud data 513 Attribute data conversion unit 533 Attribute data inverse conversion unit 540 Point cloud data 541, 546 Coordinate information 542 Integer conversion processing 543 Scale information 544 Compression encoding process 545 Offset information 548 Stream 610 Scale / offset decoding unit 611a, 611b Coordinate system conversion unit

Claims

1. An encoding method comprising: an encoding step of encoding first data having coordinate information of a three-dimensional point cloud, each of which contains coordinates on three axes expressed as integers for each point, and attribute information including correction information for each of the three axes for correcting the coordinates of each of the three axes, into second data having the coordinate information of the three-dimensional point cloud and the correction information common to the three axes, wherein the encoding step adds the correction information for each of the three axes to the second data.

2. The encoding method according to claim 1, wherein said encoding step selects fields in said first data based on said point-by-point changes.

3. The encoding method according to claim 2, wherein the encoding step adds the field selected as not changing from point to point to the second data as a fixed value field.

4. The encoding method according to claim 2, wherein the encoding step encodes the field selected as having a frequency of change for each point that exceeds a threshold value, by including it in the coordinate information.

5. The encoding method according to claim 4, wherein the encoding step adds a field selected as having a frequency of change for each point that is greater than or equal to 1 and less than or equal to the threshold value to the second data of each block obtained by dividing the three-dimensional point cloud.

6. The encoding method according to claim 5, wherein said encoding step divides said three-dimensional point cloud based on said selected fields.

7. The encoding method according to claim 2, wherein said encoding step adds information indicating the method of selecting for each field to said second data.

8. The encoding method according to claim 1, wherein the encoding step estimates whether or not the coordinate values ​​in the coordinate information can be encoded as integers based on the coordinate system of the coordinate information, the correction information, and the maximum and minimum values ​​for each of the three axes, each of which is included in the first data.

9. The encoding method according to claim 1, wherein the first data is data conforming to the LAS (LASer) file format, and the second data is data encoded using an encoding method conforming to GPCC (Geometry based point cloud compression).

10. The encoding method according to claim 9, wherein the encoding step employs the smaller data of the LAZ file obtained by losslessly compressing the first data or the second data.

11. The encoding method according to claim 9, wherein the encoding step determines whether to adopt data obtained by encoding the first data into an LAZ file or the second data, based on the coordinate system of the coordinate information contained in the first data and the correction information.

12. The encoding method according to claim 1, wherein the correction information includes scale information.

13. A decoding method comprising: a decoding step of decoding second data encoded with first data including coordinate information of a three-dimensional point cloud, each point having coordinates on three axes expressed as integers, and correction information for each of the three axes for correcting the coordinates of each of the three axes; wherein the second data includes the coordinate information of the three-dimensional point cloud and the correction information common to the three axes, and the correction information for each of the three axes is added; and the decoding step places the correction information for each of the three axes added to the second data in a predetermined area of ​​the first data different from the area where the coordinate information is placed.

14. A decoding method as described in claim 13, wherein the decoding step places the coordinate information decoded from the second data in an area in which coordinate information is placed in the first data, and places the correction information for each of the three axes added to the second data in an area in which the correction information for each of the three axes is placed in the first data.

15. The decoding method according to claim 13, wherein the decoding step determines a field in the first data in which to place the data decoded from the second data, based on encoding information for each field that is added to the second data together with the correction information.

16. The decoding method according to claim 15, wherein the decoding step, based on the encoding information, places coordinate information selected from the coordinate information contained in the second data as having a frequency of change per point exceeding a threshold in an area containing coordinate information in the first data.

17. The decoding method according to claim 16, wherein the decoding step, based on the encoding information, places coordinate information included in the second data that has been selected as having a frequency of change per point of 1 or more and less than the threshold value in an area containing coordinate information in the first data for each block obtained by dividing the three-dimensional point cloud.

18. The decoding method according to claim 15, wherein the correction information includes scale information.

19. An information processing system comprising: an encoding device that encodes first data having coordinate information of a three-dimensional point cloud, each point having coordinates on three axes expressed as integers, and attribute information including correction information for each of the three axes for correcting the coordinates of each of the three axes, into second data having the coordinate information of the three-dimensional point cloud and the correction information common to the three axes; and a decoding device that decodes the second data, wherein the encoding device adds the attribute information for each of the three axes to the second data, and the decoding device places the correction information for each of the three axes added to the second data in a predetermined area of ​​the first data different from the area where the coordinate information is placed.

Images