Point cloud data processing device, point cloud data processing method and program

Abstract

This point cloud data processing device comprises: a point cloud data storage unit which stores a plurality of pieces of point cloud data measured at an arbitrary measurement time at a plurality of measurement positions in a three-dimensional space; and a merged point cloud data generation unit which generates merged point cloud data in time series order on the basis of the point cloud data stored in the point cloud data storage unit, the merged point cloud data including location data that indicates the locations of points when the point cloud data corresponding to any one measurement time is integrated, in time series order, with point cloud data measured prior to the corresponding measurement time, and reference data that is imparted to each piece of the location data and specifies point cloud data including points indicated by the location data.

Description

[Technical field]

[0001] The present invention relates to a point cloud data processing device, a point cloud data processing method, and a program. [Background technology]

[0002] In recent years, point cloud data has been used in various fields, and in order to handle point cloud data with a common standard, for example, a standard called G-pcc (Geometry based Point Cloud Compression) is currently being considered. G-pcc has the following two standards: n ×2 n ×2 n The occupancy state of points included in the point cloud data is expressed in voxel units in an Octree (hereinafter referred to as "octree") structure, and the expressed Octree structure is the subject of encoding (see, for example, Non-Patent Document 1). This makes it easy to perform various processes on points included in the point cloud data under a common standard.

[0003] Regarding point cloud data, there is a demand to integrate multiple point cloud data acquired by measuring at different times and different positions. For example, there is a demand to acquire point cloud data in the direction of a building from positions on all four sides of the building, and to integrate the acquired four point cloud data to generate point cloud data of the building and surrounding structures. There is also a demand to merge already acquired point cloud data with newly acquired point cloud data to update it to the latest point cloud data. There is also a demand to generate point cloud data at any time or at any measurement position from already acquired point cloud data, and to use the generated point cloud data to observe changes in buildings and towns over time. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] “Information technology - MPEG I (Coded Representation of Immersive Media) - Part9: Geometry-based Point Cloud Compression)”, ISO / IEC 23090-9, ISO / IEC JTC 1 / SC 29 / WG 11, 2019. Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned G-pcc does not take into consideration where and when the point cloud data was acquired, as in image coding standards such as HEVC (High Efficiency Video Coding). Therefore, there is a problem that the above-mentioned requirements cannot be met.

[0006] Point cloud data is not acquired at all measurement positions at all times. Therefore, when observing the above-mentioned temporal changes, a large amount of point cloud data is collected, and point cloud data for each time is generated by integrating multiple point cloud data before each time. In this case, it is necessary to read multiple point cloud data each time and arrange them in chronological order by referring to the measurement time, which is a problem that this process takes a lot of time.

[0007] In view of the above circumstances, an object of the present invention is to provide a technique that makes it possible to shorten the time required to integrate multiple point cloud data acquired at any time and any measurement position. [Means for solving the problem]

[0008] One aspect of the present invention is a point cloud data processing device comprising: a point cloud data storage unit that stores multiple point cloud data measured at multiple measurement positions in three-dimensional space at arbitrary measurement times; and a merged point cloud data generation unit that generates merged point cloud data in chronological order based on the point cloud data stored in the point cloud data storage unit, the merged point cloud data including: location data indicating the location of a point when the point cloud data corresponding to any one of the measurement times and the point cloud data measured before that measurement time are integrated in chronological order; and reference data assigned to each of the location data, the reference data identifying the point cloud data that included the point indicated by the location data.

[0009] One aspect of the present invention is a point cloud data processing method, in which a point cloud data storage unit stores multiple point cloud data measured at multiple measurement positions in three-dimensional space at arbitrary measurement times, and a merged point cloud data generation unit generates merged point cloud data in chronological order based on the point cloud data stored in the point cloud data storage unit, the merged point cloud data including location data indicating the location of a point when the point cloud data corresponding to any one of the measurement times and the point cloud data measured before that measurement time are integrated in chronological order, and reference data assigned to each of the location data, the reference data identifying the point cloud data that included the point indicated by the location data.

[0010] One aspect of the present invention is a program for causing a computer to function as a point cloud data storage means for storing multiple point cloud data measured at multiple measurement positions in three-dimensional space at arbitrary measurement times, and a merged point cloud data generation means for generating merged point cloud data in chronological order based on the point cloud data stored in the point cloud data storage means, the merged point cloud data including: location data indicating the location of a point when the point cloud data corresponding to any one of the measurement times and the point cloud data measured before that measurement time are integrated in chronological order; and reference data assigned to each of the location data, the reference data identifying the point cloud data in which the point indicated by the location data was included. Effect of the Invention

[0011] According to the present invention, it is possible to reduce the time required to integrate a plurality of point cloud data acquired at any time and any measurement position. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration of a point cloud data processing device according to a first embodiment. [Diagram 2] 4A to 4C are diagrams illustrating an example of a file name and a data format of point cloud data stored in a point cloud data storage unit according to the first embodiment. [Diagram 3] FIG. 4 is a diagram illustrating an example of a data format of merged point cloud data in the first embodiment. [Figure 4] 4 is a diagram showing an example of a data format of reference list data in the first embodiment. FIG. [Diagram 5] FIG. 11 is a diagram illustrating a flow of a process for generating merged point cloud data and reference list data according to the first embodiment. [Figure 6] FIG. 11 is a diagram (part 1) illustrating an example of a process for generating merged point cloud data and reference list data according to the first embodiment. [Figure 7] FIG. 11 is a diagram (part 2) illustrating an example of the process of generating merged point cloud data and reference list data according to the first embodiment. [Figure 8] 10A to 10C are diagrams for explaining repetitive processing in the generation processing of merged point cloud data and reference list data according to the first embodiment. [Figure 9] FIG. 4 is a diagram showing a flow of point cloud data generation processing according to the first embodiment. [Figure 10] FIG. 4 illustrates an example of a generation process of point cloud data according to the first embodiment; [Figure 11] FIG. 11 is a block diagram showing the configuration of a point cloud data processing device according to a second embodiment. [Figure 12] FIG. 11 is a diagram for explaining octree encoding performed by an encoding processing unit of the second embodiment. [Figure 13] FIG. 11 is a diagram illustrating an example of a data format of encoded point group data in the second embodiment. [Figure 14]FIG. 11 is a diagram illustrating an example of a file name of point cloud data stored in a point cloud data storage unit according to the second embodiment. [Figure 15] 13A to 13C are diagrams illustrating an example of a process for generating merged point cloud data and reference list data according to the second embodiment. [Figure 16] FIG. 11 is a diagram (part 1) showing the flow of point cloud data generation processing according to the second embodiment. [Figure 17] FIG. 13 is a diagram (part 2) showing the flow of the point cloud data generation process according to the second embodiment. [Figure 18] FIG. 11 is a diagram showing the flow of a subroutine of a tile merging process performed in the point cloud data generation process according to the second embodiment. [Figure 19] FIG. 11 is a diagram (part 1) showing an example of a generation process of point cloud data according to the second embodiment. [Figure 20] FIG. 13 is a diagram (part 2) illustrating an example of the point cloud data generation process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (First embodiment) Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram showing the configuration of a point cloud data processing device 1 according to a first embodiment. The point cloud data processing device 1 includes a point cloud data acquisition unit 11, a point cloud data storage unit 12, a merged point cloud data generation unit 13, a merged point cloud data storage unit 14, a reference list generation unit 15, a reference list storage unit 16, and a point cloud data generation unit 17.

[0014] The point cloud data acquisition unit 11 is connected to a measuring device such as a three-dimensional laser scanner that generates point cloud data, and acquires point cloud data specified by the user from the measuring device in response to an operation by the user of the point cloud data processing device 1. The point cloud data storage unit 12 stores the point cloud data acquired by the point cloud data acquisition unit 11. As shown in FIG. 2, the point cloud data storage unit 12 stores a plurality of point cloud data in the form of a file. FIG. 2 shows, as an example, an example in which the point cloud data storage unit 12 stores point cloud data files with the file names "Pointcloud_DATA1", "Pointcloud_0001", and "Pointcloud_File1", respectively.

[0015] The box with the reference numeral 30 indicates data contained in the file "Pointcloud_DATA1", the box with the reference numeral 31 indicates data contained in the file "Pointcloud_0001", and the box with the reference numeral 32 indicates data contained in the file "Pointcloud_File1". As indicated by the boxes with the reference numerals 30, 31, and 32, the point cloud data includes measurement time data indicating the time when the measurement was performed, and coordinate data indicating the position of each of the multiple points included in the point cloud data by coordinate values ​​on the X-axis, Y-axis, and Z-axis of a three-dimensional coordinate system. In the example shown in FIG. 2, for example, "t0" in the box with the reference numeral 30, "t1" in the box with the reference numeral 31, and "t2" in the box with the reference numeral 32 are measurement time data. "x" in the box with the reference numeral 30 indicates the time when the measurement was performed, and "x" in the box with the reference numeral 32 indicates the time when the measurement was performed. 01 ,y 01 ,z 01 ", "x 02 ,y 02 ,z 02 ",..., "x" in the box of 31 11 ,y 11 ,z 11 ", "x 12 ,y 12 ,z 12 ",..., "x" in the box with the code 32 21 ,y 21 ,z 21 ", "x 22 ,y 22 ,z 22 ",... are coordinate data.

[0016] The measurement time data is data indicating the time expressed in the year, month, day, hour, minute, and second, and hereinafter, when the term "time" is used, it refers to the time expressed in the year, month, day, hour, minute, and second. The coordinate values ​​of the X-axis, Y-axis, and Z-axis indicating the positions of the points included in each of the multiple point cloud data are coordinate values ​​in the same three-dimensional coordinate system.

[0017] The time indicated by the measurement time data included in the point cloud data does not strictly indicate the time when each point included in the point cloud data was measured because the point cloud data includes multiple points, but indicates, for example, the time when the file of the point cloud data was generated in a measuring device. Each of the point cloud data may be point cloud data generated by a different measuring device. Therefore, there may be point cloud data whose measurement time is the same. The size of the range in which the points included in the point cloud data exist in three-dimensional space (hereinafter referred to as the range of the point cloud data) is often different in each of the point cloud data, but it may be the same size. The position of the range of the point cloud data may be the same position or may be different positions. If it is different positions, a part of the range of the point cloud data may overlap with the range of other point cloud data. For example, when measuring the change over time of the shape of the same building, the measurement is performed so that the size of the range measured by a measuring device installed at a fixed position is the same. In this case, the multiple point cloud data obtained from the measuring device are multiple point cloud data with different measurement times, and the range of each point cloud data is point cloud data of the same position and the same size.

[0018] As the file format of the point cloud data file, for example, a ".ply" format or a ".csv" format can be applied, and any one of the file formats is predetermined as the file format to be processed by the point cloud data processing device 1. A different file name is given to each of the multiple point cloud data files. FIG. 2 shows, as an example, an example in which the characters "DATA1", "0001", and "File1" are added to the common character string "Pointcloud_".

[0019] As shown in FIG. 2, the file names of the point cloud data stored in the point cloud data storage unit 12 are various, so when arranging the point cloud data in the order of measurement time, it is necessary to read a plurality of point cloud data and refer to the measurement time indicated by the measurement time data included in each of the point cloud data. When the merged point cloud data generation unit 13 receives generation instruction data including information indicating full merging, it arranges the point cloud data in chronological order by referring to the measurement time data included in all the point cloud data stored in the point cloud data storage unit 12. Here, the chronological order means a chronological order starting from the point cloud data generated at the earliest time, i.e., the oldest point cloud data, and hereinafter, the same order is indicated when referring to the chronological order. The merged point cloud data generation unit 13 repeatedly integrates, i.e., merges, the point cloud data one by one in chronological order with the point cloud data at the beginning in chronological order, and generates merged point cloud data every time one point cloud data is merged. When the merge point cloud data generation unit 13 imports generation instruction data including information indicating an individual merge associated with the file name of one piece of point cloud data, it generates new merged point cloud data by merging the point cloud data with the file name associated with the information indicating the individual merge included in the generation instruction data into the already generated merged point cloud data.

[0020] The merged point cloud data storage unit 14 stores the merged point cloud data generated by the merged point cloud data generation unit 13. The data format of the merged point cloud data is, for example, the data format shown in FIG. 3, and has items of "merge reference time", "coordinate data", and "reference data". The time indicated by the measurement time data included in the last merged point cloud data is written in the "merge reference time". The coordinate data of the merged point cloud data is written in the "coordinate data" item. The reference data is data that can specify which point cloud data file each of the merged coordinate data was included in. FIG. 3 shows an example in which the remaining character strings of "DATA1", "0001", and "File1" are written as reference data, excluding the common character string "Pointcloud_" in the file names of the point cloud data. The reference data may be any data as long as it is data that can specify which point cloud data file it was included in. For example, the entire file name of the point cloud data may be used as reference data.

[0021] The reference list generating unit 15 generates reference list data from the merge point cloud data stored in the merge point cloud data storage unit 14. The data format of the reference list data is, for example, the data format shown in Fig. 4, and has items of "merge reference time" and "reference data". In the "merge reference time" item, the time written in the "merge reference time" item of the merge point cloud data referred to when the reference list generating unit 15 generated the reference list data is written. In the "reference data" item, different reference data from among the reference data written in the "reference data" item of the merge point cloud data referred to when the reference list generating unit 15 generated the reference list data is written in chronological order.

[0022] In the case of the merged point cloud data shown in FIG. 3, the reference list generating unit 15 selects different reference data in chronological order, for example, as follows. The reference list generating unit 15 selects the reference data "DATA1" in the first row of the "reference data" field of the merged point cloud data as the first reference data, and then moves downward line by line starting from the second row of the "reference data" field of the merged point cloud data to detect reference data different from the reference data selected immediately before. When the reference list generating unit 15 detects different reference data, it selects the detected reference data as the second reference data. In the example of FIG. 3, the reference list generating unit 15 selects "0001" as the second reference data. After that, the reference list generating unit 15 repeats the same process up to the last row. As a result, the reference data selected by the reference list generating unit 15 are "DATA1", "0001", and "File1", each of which is different reference data and arranged in chronological order.

[0023] The reference list storage unit 16 stores the reference list data generated by the reference list generation unit 15. The point cloud data generation unit 17 receives an operation by a user of the point cloud data processing device 1 and imports time designation data designated by the user. The point cloud data generation unit 17 detects reference list data corresponding to the time indicated by the imported time designation data from the reference list storage unit 16. The point cloud data generation unit 17 reads out point cloud data from the point cloud data storage unit 12 in chronological order based on the reference data written in the "reference data" field of the detected reference list data. The point cloud data generation unit 17 merges the read out point cloud data in chronological order to generate point cloud data corresponding to the time designated by the time designation data.

[0024] (Generation process of merged point cloud data and reference list data in the first embodiment) Next, a process for generating merged point cloud data and reference list data by the point cloud data processing device 1 will be described with reference to Fig. 5 to Fig. 8. Fig. 5 is a flowchart showing the flow of the process for generating merged point cloud data and reference list data by the point cloud data processing device 1.

[0025] 5 is started, the point cloud data acquisition unit 11 acquires N+1 pieces of point cloud data and writes the acquired point cloud data into the point cloud data storage unit 12. Here, N is an integer equal to or greater than 1.

[0026] The merge point cloud data generating unit 13 receives the operation of the user of the point cloud data processing device 1 and imports the generation instruction data specified by the user (step Sa1). Here, it is assumed that the user provides the generation instruction data including information indicating full merging to the merge point cloud data generating unit 13. The merge point cloud data generating unit 13 determines whether the information included in the imported generation instruction data is information indicating full merging or information indicating individual merging (step Sa2).

[0027] As described above, since the generation instruction data includes information indicating full merging, the merged point cloud data generation unit 13 determines that the information included in the generation instruction data is information indicating full merging (step Sa2, full merge). The merged point cloud data generation unit 13 reads out all point cloud data, i.e., N+1 pieces of point cloud data, from the point cloud data storage unit 12. The merged point cloud data generation unit 13 arranges the point cloud data in chronological order by referring to the measurement time data included in the read N+1 pieces of point cloud data (step Sa3). Here, the merged point cloud data generation unit 13 refers to the N+1 pieces of point cloud data arranged in chronological order as point cloud data A0, A1, ..., A N In this case, the first point cloud data A0 is the point cloud data generated at the earliest time. Note that some of the N+1 point cloud data may have the same measurement time, in which case the point cloud data with the same measurement time will be lined up consecutively.

[0028] The merged point cloud data generating unit 13 generates merged point cloud data B1 by merging the first two point cloud data A0 and the point cloud data A1 in chronological order. For example, the point cloud data A0 and the point cloud data A1 are assumed to be point cloud data as shown in FIG. 6(a). The file name of the point cloud data A0 is "Pointcloud_DATA1" and the measurement time data is "t0". The point cloud data A0 includes one point p 01 contains the point p 01 The coordinate data is "x 01 ,y 01 ,z 01 The file name of the point cloud data A1 is "Pointcloud_0001", and the measurement time data is "t1". Note that, although t1>t0 here, as described above, the measurement time data included in the point cloud data may indicate the same time, so t0=t1 may also be true. The point cloud data A1 is made up of three points p 11 ,p 12 ,p 13 contains the point p 11 The coordinate data is "x 11 ,y 11 ,z 11 " and point p 12 The coordinate data of "x 12 ,y 12 ,z 12 " and point p 13 The coordinate data of "x 13 ,y 13 ,z 13 "

[0029] Here, point p 11 ,p 12 ,p 13 A point p is placed in a range of a predetermined size centered on each position of 01 In this case, the merged point cloud data B1 generated by the merged point cloud data generating unit 13 by merging the point cloud data A0 and the point cloud data A1 is a set of four points p 01 ,p 11 ,p 12 ,p 13Therefore, as shown in FIG. 6C, the merge point cloud data generating unit 13 writes the time "t1" indicated by the measurement time data of the point cloud data A1, which is the later in the chronological order, in the "merge reference time" field, and writes the four points p 01 ,p 11 ,p 12 ,p 13 The merge point cloud data generator 13 writes the coordinate data of point p in the “reference data” field. 01 ,p 11 ,p 12 ,p 13 The reference data "DATA1", "0001", "0001", and "0001" that identify the point cloud data that contained each of the above are written, and new merged point cloud data B1 is generated.

[0030] When the merge point cloud data generation unit 13 generates the merge point cloud data B1, it writes and stores the generated merge point cloud data B1 in the merge point cloud data storage unit 14. When the merge point cloud data generation unit 13 finishes writing the merge point cloud data B1 to the merge point cloud data storage unit 14, it outputs a reference list generation start signal to the reference list generation unit 15 (step Sa4).

[0031] When the reference list generating unit 15 receives a reference list generation start signal from the merge point cloud data generating unit 13, it reads out the merge point cloud data B1 from the merge point cloud data storage unit 14. The reference list generating unit 15 generates reference list data R1 by referring to the read out merge point cloud data B1. As shown in FIG. 6(d), the reference list generating unit 15 writes the time "t1" written in the "merge reference time" field of the merge point cloud data B1 to be referenced in the data format of the reference list data in the "merge reference time" field. The reference list generating unit 15 writes two different pieces of reference data, "DATA1" and "0001", written in the "reference data" field of the merge point cloud data B1 to be referenced, in chronological order in the "reference data" field. In this way, the reference list generating unit 15 generates reference list data R1.

[0032] When the reference list generation unit 15 generates the reference list data R1, it writes and stores the generated reference list data R1 in the reference list storage unit 16. When the reference list generation unit 15 finishes writing the reference list data R1 in the reference list storage unit 16, it outputs a reference list generation end signal to the merged point cloud data generation unit 13 (step Sa5).

[0033] When the merge point cloud data generation unit 13 receives a reference list generation end signal from the reference list generation unit 15, it next N The process of steps Sa6 to Sa8 is repeated for each of the merged point cloud data B n-1 Here, n is an integer between 2 and N inclusive.

[0034] The merged point cloud data generator 13 reads out the merged point cloud data B n-1 And point cloud data A n and merge them into new merged point cloud data B n For example, when n=2, the merged point cloud data generating unit 13 generates merged point cloud data B2 by merging the merged point cloud data B1 and the point cloud data A2. Here, the file name of the point cloud data A2 is "Pointcloud_File1" and the measurement time data is "t2". Note that t2≧t1. The point cloud data A2 is composed of three points p 21 ,p 22 ,p 23 contains the point p 21 The coordinate data is "x 21 ,y 21 ,z 21 " and point p 22 The coordinate data of "x 22 ,y 22 ,z 22 " and point p 23 The coordinate data of "x 23 ,y 23 ,z 23 "

[0035] As described above, the merged point cloud data B1 is obtained by merging the point cloud data A0 and the point cloud data A1. FIG. 7(a) is a diagram in which the point cloud data A0, the point cloud data A1, and the point cloud data A2 are arranged in chronological order. Here, the point p 23 Coordinate data of "x 23 ,y 23 ,z 23 The point p of the point cloud data A0 is located within a predetermined range centered on the position of 01 Coordinate data of "x 01 ,y 01 ,z 01 " exists, and point p 22 Coordinate data of "x 22 ,y 22 ,z 22 The point p of the point group data A1 is located within a range of a predetermined size centered on the position of 12 Coordinate data of "x 12 ,y 12 ,z 12 " position exists.

[0036] In this way, when an earlier point in the chronological order exists within a range of a certain size determined in advance and centered on a later point in the chronological order, the merge point cloud data generation unit 13 does not target the coordinate data of the earlier point in the chronological order for merging, but targets the coordinate data of the later point in the chronological order for merging. Therefore, as shown in FIG. 7(b), the merge point cloud data generation unit 13 merges the merge point cloud data B1 and the point cloud data A2 to generate point p 11 ,p 13 ,p 21 ,p 22 ,p 23 The merge point cloud data generator 13 generates merge point cloud data B2, for example, in the following manner.

[0037] The merge point cloud data generation unit 13 extracts coordinate data “x 01 ,y 01 ,z 01 ", "x 12,y 12 ,z 12 The merge point cloud data generation unit 13 deletes each of the points p of the point cloud data A2 to be merged in the “coordinate data” field of the merge point cloud data B1 and the corresponding reference data “DATA1” and “0001”. 21 ,p 22 ,p 23 The coordinate data of "x 21 ,y 21 ,z 21 ", "x 22 ,y 22 ,z 22」 , "x 23 ,y 23 ,z 23 " to the already written "x 11 ,y 11 ,z 11 ", "x 13 ,y 13 ,z 13 " and add it so that it is the line after ".

[0038] The merge point cloud data generator 13 adds the point p 21 ,p 22 ,p 23 7(c) from the merge point cloud data B1 (step Sa6). The merge point cloud data generator 13 writes "File1", which is reference data for each of the coordinate data, into the row of the corresponding "Reference Data" item. The merge point cloud data generator 13 rewrites the "Merge Reference Time" item of the merge point cloud data B1 to the time "t2" indicated by the measurement time data included in the point cloud data A2. In this way, the merge point cloud data generator 13 generates the merge point cloud data B2 shown in FIG. 7(c) from the merge point cloud data B1 (step Sa6).

[0039] The merge point cloud data generator 13 generates the merge point cloud data B n When the merge point cloud data storage unit 14 generates the merge point cloud data B n-1 The merged point cloud data B is generated by deleting n In this way, the merge point cloud data stored in the merge point cloud data storage unit 14 is written as merge point cloud data B n-1 Merge point cloud data from Bn The merge point cloud data generator 13 replaces the merge point cloud data B n When writing of the merged point cloud data into the merged point cloud data storage unit 14 is completed, a reference list generation start signal is output to the reference list generation unit 15 (step Sa7).

[0040] When the reference list generating unit 15 receives a reference list generation start signal from the merge point cloud data generating unit 13, the reference list generating unit 15 reads the merge point cloud data B n The reference list generating unit 15 reads out the merged point cloud data B n Refer to the reference list data R n When n=2, the reference list generation unit 15 writes the time "t2" written in the "Merge Reference Time" field of the reference merge point cloud data B2 to the "Merge Reference Time" field in the data format of the reference list data as shown in FIG. 7(d). The reference list generation unit 15 writes two different pieces of reference data, "0001" and "File1", written in the "Reference Data" field of the reference merge point cloud data B2 to the "Reference Data" field in chronological order. In this way, the reference list generation unit 15 generates reference list data R2.

[0041] The reference list generating unit 15 generates the reference list data R n When you generate the reference list data R n The reference list generating unit 15 writes and stores the reference list data R n When writing to the reference list storage unit 16 is completed, a reference list generation end signal is output to the merged point cloud data generator 13 (step Sa8).

[0042] When the merge point cloud data generation unit 13 receives a reference list generation end signal from the reference list generation unit 15, it adds "1" to the current value of "n" to set the new value of "n", and then repeats the processing of steps Sa6 to Sa8 as shown in FIG. 8 until n=N (loop La1s to La1e). When "n" becomes "N+1", the processing ends. When the processing ends, the reference list storage unit 16 stores the reference list data R1 to R N The merge point cloud data storage unit 14 stores the merge point cloud data B N The following will be memorized.

[0043] (Processing when adding one new point cloud data in the first embodiment) The point cloud data acquisition unit 11 receives an operation by a user of the point cloud data processing device 1, and acquires one new point cloud data A designated by the user. M The point cloud data acquisition unit 11 acquires the acquired point cloud data A M is written and stored in the point cloud data storage unit 12.

[0044] The merge point cloud data generation unit 13 receives an operation by the user of the point cloud data processing device 1 and generates the newly added point cloud data A M The merge point cloud data generating unit 13 retrieves generation instruction data including information on an individual merge associated with the file name (step Sa1). The merge point cloud data generating unit 13 determines whether the information included in the retrieved generation instruction data indicates a full merge or an individual merge (step Sa2).

[0045] As described above, since the generation instruction data includes information indicating an individual merge, the merge point cloud data generation unit 13 determines that the information included in the generation instruction data is information indicating an individual merge (step Sa2, individual merge).

[0046] The merge point cloud data generating unit 13 generates point cloud data A corresponding to a file name associated with information indicating an individual merge included in the generation instruction data.M The merge point cloud data generator 13 reads out the merge point cloud data B stored in the merge point cloud data storage unit 14. N Read out the merged point cloud data B N Read the time written in the "Merge Reference Time" field of the merge point cloud data B. N The time written in the "Merge Reference Time" field is "t N The merged point cloud data generator 13 generates point cloud data A M The measurement time data included in the point cloud data A is read out. M The time indicated by the measurement time data included in M "

[0047] The merge point cloud data generator 13 generates the merge point cloud data B N The merge reference time "t N " is point cloud data A M The measurement time data indicates the time "t M " or before, i.e., t M ≧t N It is determined whether or not this is the case (step Sa9).

[0048] Usually, the point cloud data A newly added to the point cloud data storage unit 12 M is the time t N For example, if the user selects t N Point cloud data A was generated before the time M If the measurement device had been stored in N Point cloud data A was generated before the time M It is also assumed that the merged point cloud data B N The merge reference time "t N " is point cloud data A included in the generation instruction data M The measurement time data indicates the time "tM ", i.e., not before t M <t N If it is determined that the measurement time is the latest merged point cloud data B N An error message is output to the effect that point cloud data that is earlier than the merge reference time has been added, and the user is prompted to generate reference list data anew (step Sa10).

[0049] If the user refers to the output error message and determines that it is necessary to generate reference list data again, the user provides the merge point cloud data generating unit 13 with generation instruction data including information indicating a full merge again. This causes the processing of steps Sa3 to Sa8 to be performed, and the point cloud data A M In contrast, if the user refers to the output error message and determines that there is no need to generate new reference list data, the user can delete the point cloud data A that was erroneously recorded in the point cloud data storage unit 12. M is deleted from the point cloud data storage unit 12.

[0050] On the other hand, the merge point cloud data generation unit 13 generates merge point cloud data B N The merge reference time "t N " is point cloud data A M The time indicated by the measurement time data is "t M " before t M ≧t N If it is determined that the merged point cloud data B N And point cloud data A M and merge them to create new merged point cloud data B M (Step Sa11). Here, as a criterion for the determination of step Sa9, t M =t N The merged point cloud data B N The merge reference time "t N " and merge point cloud data B N Even if you merge with B, the merged point cloud dataN This is because the chronological order of the reference data written in the "Reference Data" field is maintained in the correct order.

[0051] The merge point cloud data generator 13 generates the merge point cloud data B M When the merge point cloud data storage unit 14 generates the merge point cloud data B N The merged point cloud data B is generated by deleting M The merge point cloud data stored in the merge point cloud data storage unit 14 is then written to the merge point cloud data B N Merge point cloud data from B M The merge point cloud data generator 13 replaces the merge point cloud data B M When writing of the merged point cloud data into the merged point cloud data storage unit 14 is completed, a reference list generation start signal is output to the reference list generation unit 15 (step Sa12).

[0052] When the reference list generating unit 15 receives a reference list generation start signal from the merge point cloud data generating unit 13, the reference list generating unit 15 reads the merge point cloud data B M The reference list generating unit 15 reads out the merged point cloud data B M , and the reference list data R M The reference list generating unit 15 generates the reference list data R M When you generate the reference list data R M is written and stored in the reference list storage unit 16 (step Sa13), and then the process ends.

[0053] (Processing for generating point cloud data in the first embodiment) Next, the processing by the point cloud data generating unit 17 will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is a flowchart showing the flow of processing by the point cloud data generating unit 17. Before the processing shown in Fig. 9 is performed, the processing shown in Fig. 5 is completed, and the reference list storage unit 16 stores at least N pieces of reference list data R1 to R NIn response to an operation by a user of the point cloud data processing device 1, the point cloud data generating unit 17 retrieves time designation data designated by the user (step Sb1).

[0054] The point cloud data generating unit 17 selects a point cloud data set R from the reference list data R, in which the time that matches the time indicated by the imported time designation data or the time closest to the time is written in the "merge reference time" field. i is detected from the reference list storage unit 16 as the reference list data corresponding to the time indicated by the imported time designation data (step Sb2). Here, i is an integer between 1 and N. Note that, among the times before and after the time indicated by the time designation data, the reference list data R that has the closest time to the time indicated by the time designation data written in the "merge reference time" field is i If there are two or more points, the point cloud data generating unit 17 may select one of them, or may output a message to prompt the user to select one of them.

[0055] Here, the time indicated by the time specification data imported by the point cloud data generation unit 17 is “t2”, and the point cloud data generation unit 17 has detected reference list data R2 from the reference list memory unit 16 in which “t2” is written in the “Merge reference time” item shown in Figure 10(a).

[0056] The point cloud data generating unit 17 extracts the detected reference list data R iThe reference data written in the "Reference Data" field is selected in order from the first line to the last line, and the point cloud data corresponding to each of the reference data is read from the point cloud data storage unit 12 in the order of the selected reference data. As a result, the point cloud data generation unit 17 acquires point cloud data arranged in chronological order (step Sb3). In the case of the reference list data R2, the point cloud data generation unit 17 first reads out from the point cloud data storage unit 12 the point cloud data A1 having the file name "Pointcloud_0001" corresponding to the reference data "0001" in the first line of the "Reference Data" field. The point cloud data generation unit 17 then reads out from the point cloud data storage unit 12 the point cloud data A2 having the file name "Pointcloud_File1" corresponding to the reference data "File1" in the second line of the "Reference Data" field.

[0057] The point cloud data generating unit 17 merges the read point cloud data in chronological order to generate point cloud data corresponding to the time indicated by the time designation data, similar to the merging process performed by the merged point cloud data generating unit 13 (step Sb4). In the case of the reference list data R2, the point cloud data generating unit 17 reads out the point cloud data A1 and the point cloud data A2, as shown in FIG. 10(b). 22 Within a range of a predetermined size centered on the position of 12 Therefore, when merging the point cloud data A1 and the point cloud data A2, the point cloud data generating unit 17 merges the point p 22 As shown in Fig. 10(c), the point p 11 , point p 13 , point p 21 , point p 22 , point p 23 point cloud data including the above is generated as point cloud data corresponding to time t2.

[0058] In the point cloud data processing device 1 of the first embodiment, the point cloud data storage unit 12 stores a plurality of point cloud data measured at a plurality of measurement positions in a three-dimensional space at any measurement time. The merged point cloud data generating unit 13 generates merged point cloud data in chronological order based on the point cloud data stored in the point cloud data storage unit 12, the merged point cloud data including location data indicating the location of a point when the point cloud data corresponding to any one of the measurement times and the point cloud data measured before the measurement time are integrated in chronological order, i.e., coordinate data indicating the position of the point, and reference data given to each location data and specifying the point cloud data including the point indicated by the location data. The reference list generating unit 15 selects different reference data in chronological order from the reference data included in the merged point cloud data generated by the merged point cloud data generating unit 13, and generates reference list data including the reference data selected in chronological order for each measurement time of the plurality of point cloud data.

[0059] If all point cloud data before the time indicated by the time designation data given to the point cloud data generating unit 17 are merged, it is possible to generate point cloud data corresponding to the time indicated by the time designation data. However, in this case, the more recent the time indicated by the time designation data, the more the number of point cloud data to be merged increases, and the more time is required for merging. In contrast, in the point cloud data processing device 1 of the first embodiment, reference list data indicating the point cloud data to be merged and the order of merging is generated in advance for each measurement time of the point cloud data, and further, reference list data is generated each time point cloud data is added. Therefore, when it is desired to generate point cloud data corresponding to an arbitrary time, the point cloud data required for merging and the order of merging can be specified by selecting reference list data corresponding to the arbitrary time from the generated reference list data. Therefore, in the point cloud data processing device 1 of the first embodiment, it is possible to reduce the time required to integrate multiple point cloud data acquired at an arbitrary time and at an arbitrary measurement position.

[0060] In addition, the point cloud data processing device 1 of the first embodiment has the following advantages. As described above, data indicating the location of a point at any time in any region can be generated by merging multiple point cloud data. However, in this case, it is not efficient to store data indicating the location of all combinations of points. In addition, sensor data such as point cloud data accumulates over time and continues to increase in volume, so in the conventional general handling of sensor data, the sensor data is discarded after extracting necessary features and performing necessary processing. In the conventional general handling of sensor data, the merged point cloud data generated by the point cloud data processing device 1 of the first embodiment is also generally handled in such a way that the original point cloud data is discarded after generating merged point cloud data at any time in any region. In contrast, the point cloud data processing device 1 of the first embodiment does not store data indicating the location of a point at any time in any region, i.e., the merged point cloud data itself, but stores the original point cloud data and then stores the relationship of the point cloud data constituting the data indicating the location of a point at any time in any region as reference list data. By adopting such a configuration, in the first embodiment, in addition to being able to shorten the processing time as described above, the original point cloud data and the reference list data are stored, so that it is not necessary to store data indicating the locations of all combinations of points. Also, for example, even when the merging method is changed in accordance with changes in conditions such as changes in the application of use, the configuration of the first embodiment is advantageous over the conventional approach in that it is possible to generate desired data from the stored original point cloud data.

[0061] Second Embodiment 11 is a block diagram showing the configuration of a point cloud data processing device 1a according to the second embodiment. In the second embodiment, the same components as those in the first embodiment are given the same reference numerals, and different components will be described below. The point cloud data processing device 1a includes a point cloud data acquisition unit 11, an encoding processing unit 18, a point cloud data storage unit 12a, a merged point cloud data generation unit 13a, a merged point cloud data storage unit 14a, a reference list generation unit 15a, a reference list storage unit 16, a point cloud data generation unit 17a, and a decoding processing unit 19.

[0062] The encoding processing unit 18 is, for example, an encoder that generates encoded point cloud data by encoding the point cloud data with octree coding. When the encoding processing unit 18 receives an operation by a user of the point cloud data processing device 1a and imports definition data specified by the user, the encoding processing unit 18 performs octree coding on the point cloud data acquired and output by the point cloud data acquisition unit 11 according to the imported definition data.

[0063] The octree encoding process performed by the encoding processing unit 18 will be described with reference to Fig. 12. As shown in Fig. 12(a), a three-dimensional space indicated by the reference numeral 100 (hereinafter referred to as the three-dimensional space 100) is divided into cubes of equal size without gaps. Hereinafter, each of these cubes is referred to as a tile. Note that, although Fig. 12(a) shows the three-dimensional space 100 with 36 tiles of 4 x 4 x 4, it is assumed that the tiles continue infinitely in all directions.

[0064] The position of a tile is represented by, for example, coordinate values ​​of a three-dimensional coordinate system of X-axis, Y-axis, and Z-axis that are predetermined in the three-dimensional space 100. Specifically, the position of one predetermined vertex of a cube that is the shape of the tile, or the position of the center of the cube, is determined as a tile reference position that indicates the position of the tile. Note that, in the definition data that the encoding processing unit 18 imports, coordinate data that indicates the tile reference position of each tile (hereinafter, referred to as tile reference position data) and data that indicates the size of the tile, that is, data that indicates the length of one side of the cube that constitutes one tile, are defined. The encoding processing unit 18 can specify the tile reference position of each tile and the size of one tile by referring to the definition data.

[0065] Each point included in the point cloud data output by the point cloud data acquisition unit 11 is present at a position in the three-dimensional space 100, and the range in which this point cloud data exists is the range indicated by the reference numeral 40 (hereinafter, this point cloud data is referred to as point cloud data 40). As shown in FIG. 12(b), the encoding processing unit 18 selects a tile group 101 including all or part of the range of the point cloud data 40. Since the shape of the point cloud data 40 varies, the shape of the tile group 101 selected by the encoding processing unit 18 corresponds to the shape of the point cloud data 40. As shown in FIG. 12(b), when the shape of the tile group 101 is a 3×3×3 tile with the top three tiles missing, the tile group 101 includes 24 tiles. The encoding processing unit 18 performs octree encoding, which will be described below, on each of the 24 tiles included in the tile group 101.

[0066] For example, when performing octree coding on tile 101-1, which is one of the tiles included in tile group 101, encoding processing unit 18 divides tile 101-1 into eight cubes of the same size by halving each side, as shown in Fig. 12(c). Hereinafter, each of the cubes obtained by dividing the tile into eight will be referred to as a block, and each of the cubes obtained by further dividing a block described below will also be referred to as a block. Tile 101-1 will be divided into eight blocks, namely blocks 101-1-1 to 101-1-8.

[0067] The encoding processing unit 18 further divides, into eight, blocks 101-1-1 to 101-1-8 that include any point of the point cloud data 40. For example, when block 101-1-5 includes any point of the point cloud data 40, the encoding processing unit 18 further divides the block 101-1-5 into eight, as shown in FIG. 12(d). In this manner, the encoding processing unit 18 repeatedly recursively divides the block including the point of the point cloud data 40 into eight until the block includes only one point, and the position of the one point becomes the position of the representative point of the block. Here, the position of the representative point of the block is, for example, the position of the center of the block.

[0068] As shown in Fig. 12(d), it is assumed that the encoding unit 18 further divides the block 101-1-5 into eight blocks, and the points contained in block 101-1-5-3, one of the blocks, are reduced to one, and the position of the one point becomes the position of the representative point of block 101-1-5-3. In this case, the positions of the points contained in block 101-1-5-3 can be represented by an octree structure rooted at root node 201-1 corresponding to tile 101-1, as shown in Fig. 12(e).

[0069] Block 101-1-5-3 is block 101-1-5, which is the fifth block of tile 101-1, and is the third block of block 101-1-5. If this position is expressed as an eight-bit string with the bit of the block position where the point exists being 1, then internal node 201-1-5 corresponding to block 101-1-5 in the first layer is expressed as [00010000], with the fifth bit being 1. Leaf node 201-1-5-3 corresponding to block 101-1-5-3 in the second layer is expressed as [00000100], with the third bit being 1. [00010000] in the first layer is expressed as "16" in decimal. [00000100] in the second layer is expressed as "4" in decimal.

[0070] In reality, since there are many points included in the point cloud data 40, the points will exist in some of the blocks 101-1-1 to 101-1-8 in the first layer, and also in some of the blocks in the second layer and below. For example, if points exist in all of the blocks 101-1-1 to 101-1-8 in the first layer, the bit string will be "11111111", which is "255" in decimal. Therefore, when expressing the location of a point in the first layer in decimal, it will be expressed by any value from 1 to 255. The location of a point in the second layer and below will also be expressed by any value from 1 to 255, just like the first layer. In this way, by repeatedly expressing the internal nodes or leaf nodes in each layer up to the lowest layer by any value from 1 to 255, the location of all points in the point cloud data 40 can be expressed by any combination of values ​​from 1 to 255.

[0071] The encoding processing unit 18 generates encoded data of octree encoding for the tile 101-1 (hereinafter, referred to as octree encoded data) by performing variable-length encoding on combinations of values ​​from 1 to 255 in sequence starting from the first layer. In G-pcc, when performing octree encoding on a certain block, the encoding table is switched according to the status of octree encoding on neighboring blocks that have already been encoded, and octree encoding is performed. Therefore, when performing octree encoding according to the provisions of G-pcc, the encoding processing unit 18 needs to consider blocks that do not contain points, i.e., nodes with a value of "0", and performs variable-length encoding on combinations of values ​​from 0 to 255, including 0, in sequence starting from the first layer. The encoding processing unit 18 can generate encoded point cloud data for the point cloud data 40 by performing the above-mentioned octree encoding on all tiles included in the tile group 101. The encoded point cloud data generated by the encoding processing unit 18 has a reduced amount of data compared to the coordinate data representing the points of the point cloud data 40 output by the point cloud data acquisition unit 11.

[0072] The encoding processing unit 18 encodes, for example, the point cloud data A output by the point cloud data acquisition unit 11. nWhen the above-mentioned octree coding is performed, the encoded point group data C in the data format shown in FIG. n Here, n is an integer between 0 and N. n has the following items: "measurement time", "point cloud data header information", "tile header information", and "encoded data".

[0073] In the "Measurement time" field, point cloud data A n The time indicated by the measurement time data included in the point cloud data 40 is written in the "point cloud data header information" field. Data indicating the range of tiles including all or part of the range of the point cloud data 40 is written in the "tile header information". The "tile header information" field contains "tile C" information that enables the encoding processing unit 18 to identify each tile that has been subjected to octree encoding. n,1 " and the coordinate value indicating the position of the tile indicated by the tile identification information (x T-01 ,y T-01 ,z T-01 ) is written in association with tile reference position data such as the tile position data.

[0074] In the "encoded data" field, octree-encoded data for each tile that has been octree-encoded by the encoding processing unit 18 is written. Each piece of octree-encoded data is associated with tile identification information in the "tile header information" field. For example, the encoded point group data C n In this case, the tile identification information "Tile C n,1 By specifying ", "Tile C" in the "Encoded Data" item n,1 It is possible to detect octree encoded data corresponding to ".

[0075] The point cloud data storage unit 12a stores the encoded point cloud data generated by the encoding processing unit 18. As shown in FIG. 14, the point cloud data storage unit 12a stores a plurality of encoded point cloud data in the form of files. FIG. 14 shows an example in which the point cloud data storage unit 12 stores encoded point cloud data files with file names "Coded_DATA1", "Coded_0001", and "Coded_File1". For example, it is assumed that "Coded_DATA1" is the file name of encoded point cloud data obtained by octree-encoding the point cloud data with the file name "Pointcloud_DATA1" in the first embodiment, "Coded_0001" is the file name of encoded point cloud data obtained by octree-encoding the point cloud data with the file name "Pointcloud_0001" in the first embodiment, and "Coded_File1" is the file name of encoded point cloud data obtained by octree-encoding the point cloud data with the file name "Pointcloud_File1" in the first embodiment.

[0076] When the merge point cloud data generating unit 13a receives generation instruction data including information indicating all merging, the unit arranges the encoded point cloud data in chronological order by referring to the time written in the "measurement time" field of all encoded point cloud data stored in the point cloud data storage unit 12a. The merge point cloud data generating unit 13a repeatedly merges encoded point cloud data one by one in chronological order with the encoded point cloud data at the top of the chronological order, and generates merge point cloud data every time one encoded point cloud data is merged. When the merge point cloud data generating unit 13a receives generation instruction data including information indicating an individual merge associated with the file name of one encoded point cloud data, the unit merges the encoded point cloud data of the file name associated with the information indicating the individual merge included in the generation instruction data into the already generated merge point cloud data to generate new merge point cloud data.

[0077] The merge point cloud data storage unit 14a stores the merge point cloud data generated by the merge point cloud data generation unit 13a. The reference list generation unit 15a generates reference list data from the merge point cloud data stored in the merge point cloud data storage unit 14a. The data format of the reference list data generated by the reference list generation unit 15a is the same as the data format of the reference list data generated by the reference list generation unit 15 of the first embodiment shown in FIG.

[0078] The point cloud data generating unit 17a receives the operation of the user of the point cloud data processing device 1a and imports the definition data, time specification data, and range specification data specified by the user. The point cloud data generating unit 17a detects the reference list data corresponding to the time indicated by the imported time specification data from the reference list storage unit 16. The point cloud data generating unit 17a reads out the encoded point cloud data from the point cloud data storage unit 12a in chronological order based on the reference data written in the "reference data" item of the detected reference list data. The point cloud data generating unit 17a, according to the imported definition data, selects the encoded point cloud data that includes points within the range indicated by the range specification data as the merging target among the read encoded point cloud data. The point cloud data generating unit 17a merges the encoded point cloud data selected as the merging target in chronological order to generate encoded point cloud data corresponding to the time specified by the time specification data and the range indicated by the range specification data.

[0079] The decoding processing unit 19 is a decoder that decodes the original point cloud data from the encoded point cloud data that has been octree encoded, for example. When the decoding processing unit 19 receives an operation by a user of the point cloud data processing device 1a and imports definition data specified by the user, the decoding processing unit 19 performs a decoding process to restore the original point cloud data from the encoded point cloud data generated by the point cloud data generating unit 17a according to the imported definition data.

[0080] (Generation process of merged point cloud data and reference list data in the second embodiment) The process of generating merged point cloud data and reference list data in the second embodiment is performed on the premise that N+1 pieces of encoded point cloud data encoded by the encoding processing unit 18 are written in the point cloud data storage unit 12a. In the process of generating merged point cloud data and reference list data in the second embodiment, except for the following differences, the merged point cloud data generating unit 13a performs a process similar to the process of generating merged point cloud data performed by the merged point cloud data generating unit 13 in the first embodiment shown in Fig. 5, and the reference list generating unit 15a performs a process similar to the process of generating reference list data performed by the reference list generating unit 15 in the first embodiment.

[0081] The difference between the first and second embodiments is that in the first embodiment, the object to be merged is point cloud data, whereas in the second embodiment, the object to be merged is encoded point cloud data. In both the first and second embodiments, the object to be merged is common in that both are data indicating the location of points of the point cloud data, but in the first embodiment, the process of merging is performed with the coordinate data indicating the position of the points of the point cloud data as the object. In contrast, in the second embodiment, the process of merging is performed with the data indicating the range written in the "point cloud data header information" field of the encoded point cloud data, the tile identification information and tile reference position data written in the "tile header information" field, and the octree encoded data written in the "encoded data" field.

[0082] In the process of step Sa3 in the second embodiment, the merge point cloud data generator 13a performs a process of reading all the encoded point cloud data, i.e., N+1 pieces of encoded point cloud data, from the point cloud data storage unit 12a, and arranging the encoded point cloud data in chronological order by referring to the time written in the "measurement time" field of the read N+1 pieces of encoded point cloud data. Here, the N+1 pieces of encoded point cloud data arranged in chronological order are treated as encoded point cloud data C0, C1, . . . , C N As described above, the encoded point group data C0, C1, . . . , C NEach of the point cloud data A0, A1, ..., A N Each of these is data that has been octree coded by the coding processing unit 18.

[0083] For example, it is assumed that the encoded point cloud data C0 and C1 are the encoded point cloud data C0 and C1 shown in FIG. 15(a), and t1≧t0. It is assumed that the file names of the encoded point cloud data C0 and C1 are “Coded_DATA1” and “Coded_0001”, respectively. In the process of step Sa4 in the second embodiment, the merged point cloud data generating unit 13a performs a process of merging the encoded point cloud data C0 and C1 to generate the merged point cloud data D1 shown in FIG. 15(b). As shown in FIG. 15(b), the merged point cloud data D1 has the items of “merge reference time”, “point cloud data header information”, “tile header information”, “reference data”, and “encoded data”.

[0084] The "merge reference time" is written with the time in the "measurement time" field of the last encoded point cloud data merged. Therefore, in the case of merged point cloud data D1, "t1" written in the "measurement time" field of encoded point cloud data C1 is written. The "point cloud data header information" field is written with the time of tile C written in the "point cloud data header information" field of encoded point cloud data C0. 0,1 The range of tile C written in the "Point Cloud Data Header Information" item of the encoded point cloud data C1 1,1 ,C 1,2 ,C 1,3 The range and integration of tiles C 0,1 ,C 1,1 ,C 1,2 ,C 1,3 In the "tile header information" field, the tile identification information and tile reference position data written in the "tile header information" field of the encoded point cloud data C0, and the tile identification information and tile reference position data written in the "tile header information" field of the encoded point cloud data C1 are written in chronological order.

[0085] In the "Reference Data" section, tile C is identified by the tile identification information.0,1 ,C 1,1 ,C 1,2 ,C 1,3 Reference data that can specify which file of encoded point group data each of the above was included in is written. FIG. 15(b) shows an example in which the characters "DATA1" and "0001", which are the remaining characters in the file names of the encoded point group data C0 and C1 excluding the common character "Coded_", are written as reference data. The reference data may be any data as long as it can specify which file of encoded point group data each of the above was included in. For example, all of the file names of the encoded point group data may be used as reference data. In the "encoded data" field, the octree encoded data written in the "encoded data" field of the encoded point group data C0 and the octree encoded data written in the "encoded data" field of the encoded point group data C1 are written.

[0086] In the process of step Sa5 in the second embodiment, the reference list generating unit 15a performs a process of generating reference list data R1 shown in FIG. 15(c) from the "merge reference time" item and the "reference data" item of the merge point cloud data D1. As shown in FIG. 15(c), the reference list generating unit 15a writes the time written in the "merge reference time" item of the merge point cloud data D1 in the data format of the reference list data in the "merge reference time" item. The reference list generating unit 15a writes different reference data selected from the "tile header information" item of the merge point cloud data D1 in chronological order in the "reference data" item by the same procedure as the reference list generating unit 15 in the first embodiment. In this way, the reference list generating unit 15a generates the reference list data R1. That is, the reference list data R1 generated by the reference list generating unit 15a records information on which encoded point cloud data includes the point represented by the merge point cloud data D1 at time t1.

[0087] In the second embodiment, the loop La1s to La1e is repeated to process the encoded point group data C2 to C NThe process of step Sa6 in the second embodiment is performed on each of the encoded point group data C n And merged point cloud data D n-1 Merge the point cloud data D n The process is to generate the following.

[0088] That is, the merge point cloud data generating unit 13a generates the merge point cloud data D n-1 In the "Tile Header Information" and "Encoded Data" items, enter the encoded point cloud data C n The merge point cloud data generation unit 13a adds the data written in the "tile header information" and "encoded data" items in chronological order. The merge point cloud data generation unit 13a writes reference data corresponding to the added tile identification information and tile reference position data in the corresponding row of the "reference data" item. The merge point cloud data generation unit 13a writes the merge point cloud data D n-1 The data indicating the range written in the "Point Cloud Data Header Information" item and the encoded point cloud data C n The merged point cloud data generating unit 13a writes data indicating a range obtained by integrating the data indicating the range written in the "point cloud data header information" field of the encoded point cloud data C. n The merge point cloud data generator 13a writes the time written in the "measurement time" field of the new merge point cloud data D n Generate.

[0089] In addition, when the above-mentioned merged point group data generating unit 13a merges the encoded point group data C0 and the encoded point group data C1 in the process of step Sa4, and when the above-mentioned merged point group data D n-1 and the encoded point cloud data C nWhen merging two pieces of data to be merged, the tile reference position data written in the "tile header information" field of the two pieces of data to be merged may indicate the same position. In this case, the merge point cloud data generation unit 13a merges the tile reference position data written in the "tile header information" field of the data that is later in chronological order and the tile identification information corresponding to the tile reference position data. The merge point cloud data generation unit 13a further merges the octree encoded data corresponding to the tile identification information to be merged. The merge point cloud data generation unit 13a generates merge point cloud data so as to include the tile reference position data, tile identification information, and octree encoded data to be merged.

[0090] For example, tile C of the encoded point group data C1 in FIG. 1,1 The tile reference position data is (x T-02 ,y T-02 ,z T-02 ) but tile C of the encoded point cloud data C0 0,1 The same tile reference position data as (x T-01 ,y T-01 ,z T-01 In this case, the merge point cloud data D1 generated by the merge point cloud data generating unit 13a includes, in the “tile header information” field, tile C 0,1 The tile identification information and the tile reference position data (x T-01 ,y T-01 ,z T-01 ) in the "Reference Data" item, and does not include the reference data "DATA1" in the "Encoded Data" item. 0,1 The item "Point Cloud Data Header Information" does not include (tile C 1,1 ,C 1,2 ,C 1,3 (data indicating the range of

[0091] In the process of step Sa7 in the second embodiment, the merge point cloud data generating unit 13a generates the merge point cloud data D stored in the merge point cloud data storage unit 14a. n-1The newly generated merged point cloud data D n In the process of step Sa8 in the second embodiment, the reference list generating unit 15a replaces the merge point cloud data D n Reference list data from R n This process is performed in the same manner as the process in which the merge point cloud data generator 13a generates the reference list data R1 from the merge point cloud data D1.

[0092] When a user of the point cloud data processing device 1a provides the merged point cloud data generating unit 13a with generation instruction data including information indicating individual merging, the user selects the encoded point cloud data C newly added to the point cloud data storage unit 12a in response to the information indicating individual merging. M The file name of the encoded point group data C M For example, point cloud data A M is the encoded point group data that has been octree encoded by the encoding processing unit 18. In the process of step Sa9 in the second embodiment, the merge point group data generating unit 13a generates the merge point group data D N The time written in the "Merge Reference Time" field of the encoded point cloud data C M It is determined whether the measured time is before the time written in the "measurement time" field ...

[0093] That is, in the process of step Sa11 in the second embodiment, the merge point cloud data generator 13a generates the merge point cloud data D stored in the merge point cloud data storage unit 14a. N and the encoded point cloud data C M Merge the point cloud data D M In the process of step Sa12 in the second embodiment, the merge point cloud data generator 13a generates the merge point cloud data D stored in the merge point cloud data storage unit 14a. NThe newly generated merged point cloud data D M In the process of step Sa13 in the second embodiment, the reference list generating unit 15a replaces the merged point group data D M Reference list data from R M The process of generating the

[0094] (Processing for generating point cloud data in the second embodiment) Next, the processing by the point cloud data generation unit 17a and the decoding processing unit 19 will be described with reference to Fig. 16 to Fig. 20. Fig. 16 and Fig. 17 are flowcharts showing the flow of processing by the point cloud data generation unit 17a and the decoding processing unit 19. Fig. 18 is a flowchart showing the flow of processing of a subroutine of the tile merge process performed in the processing shown in Fig. 16 and Fig. 17.

[0095] Before the process shown in FIG. 16 is performed, the process of generating merged point cloud data and reference list data in the second embodiment described above is completed, and the reference list storage unit 16 stores at least N pieces of reference list data R1 to R N It is assumed that the following is stored.

[0096] The point cloud data generating unit 17a and the decoding processing unit 19 receive definition data specified by the user of the point cloud data processing device 1a in response to the user's operation. The point cloud data generating unit 17a further receives time specification data and range specification data specified by the user in response to the user's operation of the point cloud data processing device 1a (step Sc1).

[0097] The point cloud data generating unit 17a selects a point cloud data set R from the reference list data R, in which the time that matches the time indicated by the imported time designation data or the time closest to the time is written in the "merge reference time" field. iis detected from the reference list storage unit 16 as the reference list data corresponding to the time indicated by the imported time designation data (step Sc2). Here, i is an integer between 1 and N. Note that, among the times before and after the time indicated by the time designation data, the reference list data R that has the closest time to the time indicated by the time designation data written in the "merge reference time" field is i When there are two or more points, the point cloud data generating unit 17a may select one of them, or may output a message to prompt the user to select one of them.

[0098] Here, the time indicated by the time specification data imported by the point cloud data generation unit 17a is “t2”, and the point cloud data generation unit 17a has detected reference list data R2 from the reference list memory unit 16 in which “t2” is written in the “merge reference time” field shown in Figure 19(a).

[0099] The point cloud data generating unit 17a detects the reference list data R i The encoded point cloud data corresponding to the reference data written in the "reference data" field of the point cloud data generator 17a is read out from the point cloud data storage unit 12a in chronological order (step Sc3). The encoded point cloud data read out from the point cloud data storage unit 12a in chronological order by the point cloud data generator 17a is stored as encoded point cloud data C i,0 ,C i,1 ,…,C i,K Here, K is an integer equal to or greater than 0.

[0100] In the case of the reference list data R2, the point cloud data generating unit 17a first reads out the encoded point cloud data C0 having the file name "Coded_DATA1" corresponding to the reference data "DATA1" in the first line of the "reference data" item from the point cloud data storage unit 12a. The point cloud data generating unit 17a then reads out the encoded point cloud data C1 having the file name "Coded_0001" corresponding to the reference data "0001" in the second line of the "reference data" item from the point cloud data storage unit 12a. The point cloud data generating unit 17a further reads out the encoded point cloud data C2 having the file name "Coded_File1" corresponding to the reference data "File1" in the third line of the "reference data" item from the point cloud data storage unit 12a. Here, it is assumed that the encoded point cloud data C0 and C1 are the data shown in Figs. 19(b) and (c), respectively, and the encoded point cloud data C2 is the data shown in Fig. 20(b).

[0101] The point cloud data generator 17a initializes k to "0" (step Sc4). i,k ,C i,k+1 (Step Sc5). The point group data generating unit 17a selects the encoded point group data C i,k ,C i,k+1 Based on the data indicating the range written in each "point cloud data header information" item, the encoded point cloud data C i,k ,C i,k+1 It is determined whether or not a part or all of each of the ranges is present within the range indicated by the range designation data (step Sc6).

[0102] The point group data generating unit 17a generates the encoded point group data C i,k ,C i,k+1 If it is determined that a part or the whole of both of the ranges does not exist in the range specified by the range designation data (step Sc6, neither exists), the value k is increased by "2" to obtain a new k (step Sc7). i,k ,C i,k+1 When both of the ranges are not present in the range specified by the range specification data, the encoded point group data C i,kThere is no overlapping range between the range of the range specified by the range specification data, and the encoded point group data C i,k+1 This is the case when there is no overlapping range between the range specified by the range specification data and the range specified by the range specification data.

[0103] The point cloud data generating unit 17a judges whether the new k is k>K (step Sc8). If the point cloud data generating unit 17a judges that k>K (step Sc8, Yes), it outputs a message indicating that there is no target, that is, that there is no point cloud data corresponding to both the time specification data and the range specification data (step Sc9), and ends the process.

[0104] On the other hand, if the point group data generation unit 17a determines that k>K is not true (step Sc8, No), the process proceeds to step Sc5, and the next two encoded point group data C i,k ,C i,k+1 In addition, when k=K, the point group data generating unit 17a selects the encoded point group data C i,K+1 does not exist, so the encoded point group data C i,K In this case, the point group data generating unit 17a selects only the encoded point group data C i,K The point group data generating unit 17a performs a process of determining whether or not a part or the whole of the range of the encoded point group data C exists in the area specified by the range designation data. i,K If it is determined that a part or the whole of the range of the encoded point group data C exists within the range specified by the range specification data, the process proceeds to step Sc10. i,K If it is determined that a part or all of the range does not exist within the range specified by the range designation data, the process proceeds to step Sc7, and then the process ends.

[0105] In the process of step Sc6, the point group data generator 17a generates the encoded point group data C i,k ,C i,k+1If it is determined that a part or the whole of either of the ranges exists within the range specified by the range specification data (step Sc4, one exists), the encoded point group data existing within the range specified by the range specification data is designated as encoded point group data E (step Sc10). Then, the process proceeds to step Sc13 in FIG. 17, as indicated by the connection point A in FIG. 16 and FIG. 17.

[0106] In the process of step Sc6, the point group data generator 17a generates the encoded point group data C i,k ,C i,k+1 If it is determined that a part or all of both of the ranges are present in the range specified by the range specification data (step Sc4, both exist), the process is a subroutine shown in FIG. 18, and the encoded point group data C i,k ,C i,k+1 Then, the tile merge process is performed for the tile (step Sc11).

[0107] Referring to the flowchart shown in FIG. 18, the encoded point group data C i,k ,C i,k+1 The following describes the tile merge process for merging two pieces of encoded point group data to be processed. i,k ,C i,k+1 Based on the tile header reference position data in the "Tile Header Information" item and the tile size indicated by the definition data, a combination of the tile identification information and the tile reference position data of the tile whose range is partly or entirely within the range indicated by the range specification data is stored in the encoded point group data C i,k ,C i,k+1 The first step is to find all of the "tile header information" items for each of the tiles (step Sd1).

[0108] The point cloud data generating unit 17a judges whether or not there is a combination of the detected tile identification information and the tile reference position data in which the positions indicated by the tile reference position data are the same (step Sd2). When the point cloud data generating unit 17a judges that there is a combination in which the positions indicated by the tile reference position data are the same (step Sd2, Yes), the point cloud data generating unit 17a selects the encoded point cloud data that is later in time series among the combinations in which the positions indicated by the tile reference position data are the same, that is, the encoded point cloud data C i,k+1 The encoded point group data C is generated so as to include a combination of the tile identification information and the tile reference data, and octree encoded data corresponding to the tile identification information. i,k and the encoded point cloud data C i,k+1 and generate new encoded point group data (step Sd3).

[0109] On the other hand, when it is determined that there is no combination in which the positions indicated by the tile reference position data are the same (step Sd2, No), the point cloud data generating unit 17a generates the encoded point cloud data C i,k and the encoded point cloud data C i,k+1 and generate new encoded point group data (step Sd4).

[0110] The point cloud data generating unit 17a generates the new encoded point cloud data C i,k+1 The point cloud data generator 17a writes the time written in the "measurement time" field in step Sd11 (step Sd5), and ends the processing of the subroutine. The point cloud data generator 17a sets the newly generated encoded point cloud data in the processing of the subroutine of step Sc11 as encoded point cloud data E (step Sc12). Thereafter, the processing proceeds to the processing of step Sc13 in FIG. 17, as shown by the connection point A in FIG. 16 and FIG. 17.

[0111] For example, in the case of the encoded point group data C0 shown in FIG. 19(b) and the encoded point group data C1 shown in FIG. 19(c), the process of step Sc11, that is, the tile merge process shown in FIG. 18 is performed on the encoded point group data C0 and the encoded point group data C1. Here, if the tile C 0,1 and tile C in the "Tile Header Information" section of the encoded point cloud data C1 1,1 ,C 1,2 Assume that a part or the whole of the range of tile C exists within the range specified by the range specification data. 1,3 In this case, the point cloud data generating unit 17a performs the process of step Sd1 to determine whether or not the tile C 0,1 And Tile C 1,1 And Tile C 1,2 and tile C 1,3 will not be detected.

[0112] The point cloud data generator 17a detects the tile C 0,1 And Tile C 1,1 And Tile C 1,2 In step Sd4 after step Sd2, the point group data generator 17a determines whether the tile C of the encoded point group data C0 has a tile reference position different from the tile reference position of the encoded point group data C0. 0,1 and tile C of the encoded point cloud data C1 1,1 ,C 1,2 After merging these, the process of step Sd5 is performed to generate the encoded point group data E shown in FIG. 20(a).

[0113] As shown in Fig. 20(a), in the encoded point cloud data E in which the encoded point cloud data C0 and C1 are merged by the tile merge process, "t1" is written in the "measurement time" field of the encoded point cloud data C1, which is the later in chronological order. 0,1 ,C 1,1 ,C 1,2 Data indicating the range is written.

[0114] In the “tile header information” field, the point cloud data generating unit 17a detects the tile C in the process of step Sd1. 0,1 ,C 1,1 ,C 1,2 The tile identification information and the tile reference position data of each of the tiles are written in the "encoded data" field. 0,1 ,C 1,1 ,C 1,2 Octree encoded data corresponding to each tile identification information is written.

[0115] Next, the process from step Sc13 onward will be described with reference to FIG. 17. The point cloud data generating unit 17a adds "1" to k to set a new "k" (step Sc13). The point cloud data generating unit 17a determines whether the new k is k>K or not (step Sc14). If it is determined that k>K is not true (step Sc14, No), the point cloud data generating unit 17a selects the encoded point cloud data C based on the new k. i,k Refer to the data indicating the range written in the "Point Cloud Data Header Information" item of the encoded point cloud data C i,k It is determined whether or not a part or the whole of the range is present within the range indicated by the range specification data (step Sc15).

[0116] The point group data generating unit 17a generates the encoded point group data C i,kIf it is determined that a part or the whole of the range does not exist within the range specified by the range designation data (step Sc15, No), the process proceeds to step Sc13. i,k If it is determined that a part or all of the range of the encoded point group data C exists within the range specified by the range specification data (step Sc15, Yes), the process of the subroutine shown in FIG. i,k , E (step Sc16). The point cloud data generator 17a sets the newly generated encoded point cloud data in the subroutine processing of step Sc16 as encoded point cloud data E (step Sc17). Then, the process proceeds to step Sc13.

[0117] For example, in the case of the encoded point group data E shown in FIG. 20(a) and the encoded point group data C2 shown in FIG. 20(b), the process of step Sc16, i.e., the tile merge process shown in FIG. 18 is performed on the encoded point group data E and the encoded point group data C2. Here, the tile C in the "tile header information" field of the encoded point group data C2 is 2,1 ,C 2,2 In this case, the point group data generating unit 17a performs the process of step Sd1 to generate the tile C of the encoded point group data E. 0,1 ,C 1,1 ,C 1,2 and tile C of the encoded point cloud data C2 2,1 ,C 2,2 The above will be detected.

[0118] The point cloud data generator 17a detects the tile C 1,1 The position indicated by the reference position data and tile C 2,1 The position indicated by the tile reference position data is (x T-02 ,y T-02 ,z T-02 ), the result of the determination in step Sd2 is “Yes.” In the process of step Sd3 that is performed thereafter, the point cloud data generator 17a selects the tile C2,1 The encoded point group data E and the encoded point group data C2 are merged so as to include the encoded point group data E, and then the processing of step Sd5 is performed to generate the encoded point group data E shown in Figure 20(c).

[0119] As shown in Fig. 20(c), in the encoded point cloud data E obtained by merging the encoded point cloud data E shown in Fig. 20(a) and the encoded point cloud data C1 shown in Fig. 20(b) by the tile merging process, "t2" is written in the "measurement time" field of the encoded point cloud data C2, which is the later in chronological order. 0,1 ,C 1,1 ,C 1,2 ,C 2,1 C 2,2 From the tile C that was not subject to merging in the process of step Sd3 1,1 The remaining tiles except for C 0,1 ,C 1,2 ,C 2,1 ,C 2,2 The data indicating the range of tile C is written in the "Tile Header Information" field. 0,1 ,C 1,2 ,C 2,1 ,C 2,2 The tile identification information and the tile reference position data of each of the tiles are written in the "encoded data" field. 0,1 ,C 1,2 ,C 2,1 ,C 2,2 Octree encoded data corresponding to each tile identification information is written.

[0120] In the process of step Sc14, when the point cloud data generating unit 17a determines that k>K (step Sc14, Yes), the point cloud data generating unit 17a outputs the last generated encoded point cloud data E to the decoding processing unit 19. The decoding processing unit 19 takes in the encoded point cloud data E output by the point cloud data generating unit 17a, and performs a decoding process on the taken-in encoded point cloud data E according to the definition data. By performing the decoding process, the coordinate values ​​of the X-axis, Y-axis, and Z-axis of each point indicated by the octree encoded data included in the encoded point cloud data E are restored. The decoding processing unit 19 generates point cloud data including the restored coordinate values ​​of the X-axis, Y-axis, and Z-axis of each point and the time written in the "measurement time" field of the encoded point cloud data, and outputs the generated point cloud data to the outside (step Sc18). The point cloud data output to the outside becomes point cloud data corresponding to the time specification data and the range specification data.

[0121] In the point cloud data processing device 1a of the above-mentioned second embodiment, the point cloud data storage unit 12a stores encoded point cloud data in which each point is encoded by octree encoding for each tile whose range is partially or completely included in the point cloud data, and the encoded point cloud data has tile header information including tile reference position data which is location data and indicates the position of a tile containing a point included in the point cloud data.

[0122] As a result, the point cloud data processing device 1a of the second embodiment achieves the following effects in addition to the effects achieved by the point cloud data processing device 1 of the first embodiment. That is, in the point cloud data processing device 1a, the encoded point cloud data obtained by octree-encoding the point cloud data is the object of the merge process. Since the encoded point cloud data is octree-encoded, the range in which the points exist can be specified for each tile. Therefore, the point cloud data processing device 1a narrows down the encoded point cloud data to be merged using the reference list data corresponding to the time indicated by the time designation data, and further narrows down the encoded point cloud data in which the points exist within the range indicated by the range designation data based on the point cloud data header information indicating the range of the tiles in which the points exist in the encoded point cloud data. Then, the point cloud data processing device 1a narrows down the tiles in the narrowed down encoded point cloud data to tiles in which the points exist within the range indicated by the range designation data.

[0123] When tiles at the same position exist among the narrowed down tiles, the point cloud data processing device 1a generates merged encoded point cloud data by merging tiles on a tile-by-tile basis, with the tile at the later time-series order as the merging target. Therefore, the point cloud data processing device 1a of the second embodiment can reduce the amount of data to be merged compared to the point cloud data processing device 1 of the first embodiment. In other words, the point cloud data processing device 1a of the second embodiment does not perform merging based on the coordinate data of each of the multiple points included in the point cloud data as the point cloud data processing device 1 of the first embodiment does, but performs merging on a tile-by-tile basis based on the tile reference position data indicating the position of the tile indicated in the tile header information. Since each tile often includes multiple points, merging on a tile-by-tile basis also means merging multiple points together. In this case, the point cloud data processing device 1a of the second embodiment merges multiple points together, so that it is possible to perform merging in less time than the point cloud data processing device 1 of the first embodiment, which performs merging on a point-by-point basis. The fact that the time required for merging can be reduced means that the time required for generating reference list data and the time required for generating point cloud data based on the reference list data can be reduced. Therefore, the point cloud data processing device 1a of the second embodiment can reduce the time required for integrating multiple point cloud data acquired at any time and any measurement position more than the point cloud data processing device 1 of the first embodiment.

[0124] In the second embodiment, the encoding processing unit 18 repeats dividing the block into eight parts until the number of points included in the block becomes one, and the position of the one point becomes the position of the representative point of the block. Alternatively, the encoding processing unit 18 may divide the block into eight parts until the number of points included in the block becomes a predetermined number, and generate encoded point group data by associating coordinate data indicating the relative coordinates of each point included in the block in the lowest layer in which the number of points has become the predetermined number with octet-tree encoded data indicating the block in the lowest layer. The octet-tree encoding described in the second embodiment is octet-tree encoding of lossless encoding in which the positions of the points are completely reproduced by decoding. Alternatively, the encoding processing unit 18 may perform octet-tree encoding of lossy encoding. When performing octet-tree encoding of lossy encoding, the encoding processing unit 18 quantizes the block in the lowest layer in which the number of points has become the predetermined number by further dividing it into multiple cubes of the same size, and approximates the positions of each point included in the block in the lowest layer to the representative points of the quantized cubes, for example, the positions of the centers of the cubes. The encoding processing unit 18 generates encoded point group data by associating the information indicating the approximate position with the octree encoded data indicating the block in the lowest layer.

[0125] In the second embodiment, the merge point cloud data generating unit 13a generates merge point cloud data in the data format shown in FIG. 15(b). In contrast, the merge point cloud data generating unit 13a may generate merge point cloud data in the data format generated by the merge point cloud data generating unit 13 in the first embodiment, that is, merge point cloud data in the data format shown in FIG. 3. When generating merge point cloud data in the data format shown in FIG. 3, the tile reference position data in the item "tile header information" in the merge point cloud data in the data format shown in FIG. 15(b) is written in the item "coordinate data" of the merge point cloud data. In this case, the point cloud data processing device 1a is provided with the reference list generating unit 15 of the first embodiment instead of the reference list generating unit 15a of the second embodiment, thereby making it possible to generate reference list data.

[0126] In the above-described second embodiment, the definition data is provided to the point cloud data generation unit 17a, the encoding processing unit 18, and the decoding processing unit 19. Alternatively, the definition data may be determined in advance, and the predetermined definition data may be stored in an internal storage area of ​​each of the point cloud data generation unit 17a, the encoding processing unit 18, and the decoding processing unit 19, so that the point cloud data generation unit 17a, the encoding processing unit 18, and the decoding processing unit 19 refer to the definition data stored in each internal storage area.

[0127] In the above second embodiment, the encoding processing unit 18 generates, for example, encoded point group data in the data format shown in Figure 13. However, the data format shown in Figure 13 is only an example, and the encoding processing unit 18 may also generate, for example, encoded point group data in the G-pcc data format.

[0128] (Processing when measurement time data is not included in point cloud data (part 1)) In the first and second embodiments described above, the data format of each of the point cloud data acquired by the point cloud data acquisition unit 11 is a data format including measurement time data, as shown in FIG. 2. In contrast, the data format of the point cloud data may be a data format not including measurement time data. In this case, for example, the user of the point cloud data processing device 1 may own another file in which the file name of the point cloud data and the time indicated by the measurement time data are associated with each other, and when performing an operation to specify the file of the point cloud data acquired by the point cloud data acquisition unit 11 while referring to the file, the user may specify data indicating the measurement time corresponding to the point cloud data and provide the data to the point cloud data acquisition unit 11. This enables the point cloud data acquisition unit 11 to associate the point cloud data with data indicating the measurement time of the point cloud data. This association may be an association in which the measurement time data corresponding to the point cloud data is written at the beginning of the point cloud data as shown in FIG. 2, or an association in which the point cloud data and the measurement time data corresponding to the point cloud data are combined into one file.

[0129] (Processing when measurement time data is not included in point cloud data (part 2)) In the case where the point cloud data does not include measurement time data, the following processing may be performed in the first and second embodiments. First, the processing in the case of the first embodiment will be described. The point cloud data storage unit 12 stores point cloud data that does not include measurement time data. The user provides the merged point cloud data generating unit 13 with generation instruction data including information indicating full merging, further including data indicating the chronological order of each of the point cloud data. The merged point cloud data generating unit 13 imports the generation instruction data, and when it is determined in the processing of step Sa2 that the information included in the imported generation instruction data is information indicating full merging, it reads out data indicating the chronological order of each of the point cloud data included in the generation instruction data.

[0130] The merged point cloud data generating unit 13 reads out the point cloud data from the point cloud data storage unit 12 in the order indicated by the data indicating the time series order of each of the read out point cloud data, and arranges the read out point cloud data in time series order. Then, the processing from step Sa4 onwards is performed. In the processing of steps Sa5 and Sa8, the reference list generating unit 15 uses the reference list data R i (where i is an integer from 1 to N) is generated, the reference list data R i When the user refers to the message, the reference list data R i In the "Merge Reference Time" field, enter the reference list data R i The corresponding time is written.

[0131] In this case, merged point cloud data B1 to B NTherefore, when the merge point cloud data generation unit 13 determines in the process of step Sa2 that the information included in the generation instruction data is information indicating an individual merge, the merge point cloud data generation unit 13 does not perform the processes of steps Sa9 and Sa10. Instead of performing the processes of steps Sa9 and Sa10, the merge point cloud data generation unit 13 writes data to the point cloud data A corresponding to the file name associated with the information indicating the individual merge. M The measurement time of the merged point cloud data B stored in the merged point cloud data storage unit 14 N Merged point cloud data A1~A N In this case, the reference list generating unit 15 performs the processes of steps Sa11 to Sa13 by regarding the reference list data R M When generating the reference list data R M When the user refers to the message, the reference list data R M In the "Merge Reference Time" field, enter the reference list data R M The corresponding time is written.

[0132] Next, the process of the second embodiment will be described. The encoding processing unit 18 generates encoded point cloud data in which no data is written in the "measurement time" field because the point cloud data output by the point cloud data acquisition unit 11 does not include measurement time data, and writes the data to the point cloud data storage unit 12a. As in the first embodiment, the user provides the merged point cloud data generating unit 13a with data indicating the time series order of each piece of encoded point cloud data in addition to the generation instruction data including information indicating full merging. The merged point cloud data generating unit 13a retrieves the generation instruction data, and when it is determined in the process of step Sa2 that the information included in the retrieved generation instruction data is information indicating full merging, it reads out data indicating the time series order of each piece of encoded point cloud data included in the generation instruction data.

[0133] The merged point cloud data generator 13a reads out the encoded point cloud data from the point cloud data storage unit 12a in the order indicated by the data indicating the time series order of each of the read encoded point cloud data, and arranges the read encoded point cloud data in time series order. After that, the processing from step Sa4 onward in the second embodiment is carried out. In the subsequent processing, similar to the processing in the first embodiment described above, the reference list generator 15a reads out the reference list data R i Each time you generate a list of reference data, R i When the user refers to the message, the reference list data R i In the "Merge Reference Time" field, enter the reference list data R i The corresponding time is written.

[0134] In the second embodiment, when the generation instruction data includes information indicating individual merging, the merge point cloud data generation unit 13a does not perform the processes of steps Sa9 and Sa10, as in the first embodiment, and generates the encoded point cloud data C corresponding to the file name associated with the information indicating individual merging. M The measurement time of the merge point cloud data D stored in the merge point cloud data storage unit 14a N The encoded point cloud data C1 to C are merged into N In the first and second embodiments, the reference list generating unit 15, 15a performs the processes of steps Sa11 to Sa13 in the second embodiment by regarding the reference list data R i When outputting a message urging the user to add data indicating the time to the reference list data R i The reference data in the last line of the "Reference Data" item in the message may be included in the message and output. In this way, the user can output the reference list data R based on the reference data included in the message. i This makes it possible to easily identify the file name of the point cloud data or encoded point cloud data corresponding to the time written in the "merge reference time" field.

[0135] (Other configuration examples of the first and second embodiments) In the above first and second embodiments, the time indicated by the measurement time data included in the point cloud data is, for example, the time when the point cloud data was generated in a measuring device. In contrast, the time indicated by the measurement time data included in the point cloud data may be the following times. For example, depending on the point cloud data, the measurement time when the point was measured may be associated with each piece of coordinate data of the point. In such a case, the merge point cloud data generating unit 13 may set the earliest time, the latest time, or the average time of the earliest and latest times among the measurement times of each point included in the point cloud data file as the measurement time of the point cloud data file.

[0136] In the above first and second embodiments, the file format of the point cloud data acquired by the point cloud data acquisition unit 11 is predefined, and the merged point cloud data generation unit 13, the point cloud data generation unit 17, and the encoding processing unit 18 refer to the data included in the point cloud data according to the predefined file format. In contrast, when the point cloud data acquisition unit 11 acquires point cloud data in a plurality of file formats, the merged point cloud data generation unit 13, the point cloud data generation unit 17, and the encoding processing unit 18 may acquire the data included in the point cloud data by performing different read processes according to the extensions given to the file names, such as ".ply" or ".csv".

[0137] In the above first and second embodiments, for example, a transcoder is applied as the merge point cloud data generator 13, 13a, the reference list generator 15, 15a, and the point cloud data generator 17, 17a. When a transcoder is applied, in the first embodiment, the merge point cloud data generator 13 and the reference list generator 15 may be realized by one transcoder, and in the second embodiment, the merge point cloud data generator 13a and the reference list generator 15a may be realized by one transcoder.

[0138] In the first and second embodiments, the point cloud data includes coordinate data indicating the positions of the points, but may further include data indicating the colors of the points, for example, data indicating the light intensity values ​​of red, blue, and green, data indicating reflection intensity, etc. Note that the merge point cloud data generating unit 13, the point cloud data generating unit 17, the encoding processing unit 18, and the decoding processing unit 19 process points that are different in color or reflection intensity as different points even if the positions indicated by the coordinate data are the same.

[0139] As described above, in both the first and second embodiments, the objects to be merged are common in that they are data indicating the location of points in point cloud data. That is, in the above-mentioned first embodiment, coordinate data indicating the position of a point is applied as location data indicating the location of a point, and reference data is associated with the coordinate data as attribute information. In the second embodiment, tile reference position data indicating the position of a tile including a point is applied as location data indicating the location of a point, and reference data is associated with the tile reference position data as attribute information. In addition to these, data indicating the position of a structure such as a mesh or polygon including a point may be applied as location data indicating the location of a point, and reference data may be associated with the data as attribute information.

[0140] In the first embodiment, when two point cloud data are merged, the merge point cloud data generation unit 13 and the point cloud data generation unit 17 perform a selection process to select the point of the later point cloud data in the chronological order as the merge target if a point of the earlier point cloud data in the chronological order exists within a range of a predetermined size centered on a point of the later point cloud data in the chronological order. By the merge point cloud data generation unit 13 performing the selection process, the number of reference data included in the reference list data generated by the reference list generation unit 15 can be reduced, and there is an effect that the point cloud data generation unit 17 can reduce the number of point cloud data when merging point cloud data. However, even if the same process as described above is not performed in both the merge point cloud data generation unit 13 and the point cloud data generation unit 17, the user can obtain the point cloud data desired by performing the process in the point cloud data generation unit 17. Therefore, the merge point cloud data generation unit 13 may not perform the selection process. Also in the second embodiment, when two encoded point cloud data are merged, if the two encoded point cloud data contain tiles with the same tile reference position, the merge point cloud data generation unit 13a performs a selection process to select the later tile in chronological order as the merge target. However, for the same reasons as above, the merge point cloud data generation unit 13a may not perform this selection process.

[0141] In the first and second embodiments described above, "time" indicates time expressed in years, months, days, hours, minutes, and seconds, but it may also indicate time expressed in years, months, and days without including the units of hours, minutes, and seconds. In this case, data indicating time, such as measurement time data, will be data including values ​​in units of years, months, and days.

[0142] The point cloud data processing device 1, 1a in the above-mentioned embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0143] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Industrial Applicability]

[0144] This technology can be applied to processing multiple point cloud data. [Explanation of symbols]

[0145] 1 ... point cloud data processing device, 11 ... point cloud data acquisition unit, 12 ... point cloud data storage unit, 13 ... merged point cloud data generation unit, 14 ... merged point cloud data storage unit, 15 ... reference list generation unit, 16 ... reference list storage unit, 17 ... point cloud data generation unit

Claims

1. a point cloud data storage unit that stores a plurality of point cloud data measured at a plurality of measurement positions in a three-dimensional space at any measurement time; a merged point cloud data generating unit that generates merged point cloud data in chronological order based on the point cloud data stored in the point cloud data storage unit, the merged point cloud data including: location data indicating the location of a point when the point cloud data corresponding to any one of the measurement times and the point cloud data measured before the measurement time are integrated in chronological order; and reference data assigned to each of the location data, the reference data specifying the point cloud data that included the point indicated by the location data; Equipped with The merged point cloud data generation unit, when generating the merged point cloud data, if a point included in the point cloud data measured before the measurement time exists within a predetermined range of a certain size centered on a point included in the point cloud data corresponding to a later measurement time in chronological order, does not treat the coordinate data of the point included in the point cloud data measured before the measurement time as the target for merging, but instead treats the coordinate data of the point included in the point cloud data corresponding to the later measurement time as the target for merging.

2. a reference list generating unit that selects, in chronological order, different reference data from the reference data included in the merged point cloud data generated by the merged point cloud data generating unit, and generates reference list data including the reference data selected in chronological order for each of the measurement times of the plurality of point cloud data. The point cloud data processing device according to claim 1 , further comprising:

3. The three-dimensional space is divided into cubes of a predetermined equal size, and each of the divided cubes is predetermined as a tile; The point cloud data stored in the point cloud data storage unit is encoded point cloud data in which each point is encoded by octree encoding for each of the tiles that are partly or entirely included in the range of the point cloud data, and the encoded point cloud data has tile header information including tile reference position data that indicates the position of the tile that includes the point included in the point cloud data, the tile reference position data being the location data. The point cloud data processing device according to claim 2 .

4. a point cloud data generation unit that receives time designation data that designates a time and range designation data that designates a range, selects the reference list data for a measurement time corresponding to the time indicated by the received time designation data, reads out from the point cloud data storage unit the encoded point cloud data that corresponds to the reference data included in the selected reference list data, and integrates the read encoded point cloud data in chronological order on a tile basis based on the tile header information included in the read encoded point cloud data and the range designation data to generate encoded point cloud data that corresponds to the time indicated by the time designation data and the range designated by the range designation data; a decoding processing unit that generates point cloud data by decoding the encoded point cloud data generated by the point cloud data generating unit; The point cloud data processing device according to claim 3 , further comprising:

5. the encoded point cloud data stored in the point cloud data storage unit further includes point cloud data header information indicating a range of the encoded point cloud data, The point cloud data generation unit selecting, from the encoded point cloud data read from the point cloud data storage unit, encoded point cloud data that includes points within the range indicated by the range indication data, based on the point cloud data header information and the range indication data, and integrating the selected encoded point cloud data in chronological order on a tile-by-tile basis, based on the tile header information and the range indication data, to generate encoded point cloud data that corresponds to the time indicated by the time indication data and the range indicated by the range indication data. The point cloud data processing device according to claim 4 .

6. The point cloud data generation unit When integrating the encoded point cloud data in chronological order on a tile-by-tile basis, if there is a tile having the same position indicated by the tile reference position data, a tile included in the encoded point cloud data that is later in chronological order is selected as a merge target, and the encoded point cloud data is integrated in chronological order on a tile-by-tile basis. The point cloud data processing device according to claim 4 or 5.

7. A plurality of point cloud data measured at a plurality of measurement positions in a three-dimensional space at any measurement time is stored in a point cloud data storage unit; a merged point cloud data generating unit generates merged point cloud data in chronological order based on the point cloud data stored in the point cloud data storage unit, the merged point cloud data including location data indicating the location of a point when the point cloud data corresponding to any one of the measurement times and the point cloud data measured before the measurement time are integrated in chronological order, and reference data assigned to each of the location data, the reference data specifying the point cloud data in which the point indicated by the location data was included; When generating the merged point cloud data, if a point included in the point cloud data measured before the measurement time exists within a range of a predetermined size centered on a point included in the point cloud data corresponding to the measurement time that is later in chronological order, the coordinate data of the point included in the point cloud data measured before the measurement time is not subject to merging, but the coordinate data of the point included in the point cloud data corresponding to the measurement time that is later in chronological order is subject to merging. Point cloud data processing methods.

8. Computer a point cloud data storage means for storing a plurality of point cloud data measured at a plurality of measurement positions in a three-dimensional space at any measurement time; a merged point cloud data generating means for generating, in chronological order, merged point cloud data based on the point cloud data stored in the point cloud data storage means, the merged point cloud data including: location data indicating the location of a point when the point cloud data corresponding to any one of the measurement times and the point cloud data measured before the said measurement time are integrated in chronological order; and reference data assigned to each of the location data, the reference data specifying the point cloud data in which the point indicated by the location data was included; Function as a A program for merging, in the merged point cloud data generation means, when generating the merged point cloud data, if a point included in the point cloud data measured before the measurement time exists within a predetermined range of a certain size centered on a point included in the point cloud data corresponding to a later measurement time in chronological order, the coordinate data of the point included in the point cloud data measured before the measurement time is not treated as a target for merging, but the coordinate data of the point included in the point cloud data corresponding to the later measurement time in chronological order is treated as a target for merging.

Images