Annotation device, annotation method, and program

Abstract

The purpose of the present disclosure is to provide an annotation device capable of improving workability in annotation. An annotation device (1) according to one aspect of the present disclosure comprises: a cluster generation unit (11) which generates a plurality of clusters by grouping point cloud data corresponding to three-dimensional position information on a measurement target; a presentation order determination unit (12) which determines a presentation order of a plurality of clusters, on the basis of the similarity of the generated plurality of clusters; a cluster presentation unit (13) which presents the plurality of clusters in order on the basis of the determined presentation order; and a labeling unit (14) which assigns a predetermined label to each cluster presented in order.

Description

[Technical field]

[0001] The present disclosure relates to an annotation apparatus, an annotation method, and a non-transitory computer-readable medium. [Background technology]

[0002] In recent years, technology has been developed to obtain point cloud data, which is three-dimensional position information of a measurement target, using a three-dimensional distance sensor such as LIDAR (Light Detection And Ranging). For example, technology has been developed to automatically identify equipment, parts, etc. using such point cloud data of a measurement target and manage facilities, etc. In such technology, in order to automatically identify equipment, parts, etc., it is necessary to create training data by adding a predetermined label to the point cloud data in advance.

[0003] Patent Document 1 discloses a technology related to an object recognition device that can recognize objects with high accuracy by using point cloud data provided by a LIDAR, while Patent Document 2 discloses a technology related to a method for adding annotations to point cloud data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-185347 A [Patent Document 2] JP 2018-63236 A Summary of the Invention [Problem to be solved by the invention]

[0005] As described above, in order to automatically identify a specific object using point cloud data of a measurement target, it is necessary to create training data by assigning a specific label to the point cloud data in advance. The annotation work of assigning a specific label to the point cloud data is performed, for example, by a user assigning a specific label to each point of the point cloud data. Specifically, points that are close to each other are grouped together to generate a cluster, and all of the generated clusters are displayed on a display unit. Then, the user assigns a label to each cluster while checking the clusters displayed on the display unit.

[0006] However, if a user performs annotation work while checking all the clusters displayed on the display unit on the screen, there is a problem that the annotation work becomes complicated because the point cloud data of the measurement target is enormous.

[0007] An object of the present disclosure is to provide an annotation device, an annotation method, and a non-transitory computer-readable medium that can improve annotation workability. [Means for solving the problem]

[0008] An annotation device according to one aspect of the present disclosure includes a cluster generation unit that generates a plurality of clusters by grouping point cloud data corresponding to three-dimensional position information of a measurement object, a presentation order determination unit that determines an order of presentation of the plurality of clusters based on similarities between the generated plurality of clusters, a cluster presentation unit that presents the plurality of clusters in sequence based on the determined presentation order, and a label assignment unit that assigns a predetermined label to each of the clusters presented in the sequence.

[0009] An annotation method according to one aspect of the present disclosure groups point cloud data corresponding to three-dimensional position information of a measurement object to generate a plurality of clusters, determines an order of presentation of the plurality of clusters based on the similarity of the generated plurality of clusters, presents the plurality of clusters in sequence based on the determined presentation order, and assigns a predetermined label to each of the clusters presented in the sequence.

[0010] A non-transitory computer-readable medium according to one embodiment of the present disclosure is a non-transitory computer-readable medium having stored thereon a program for causing a computer to execute an annotation process including a process of grouping point cloud data corresponding to three-dimensional position information of a measurement target to generate a plurality of clusters, a process of determining an order of presentation of the plurality of clusters based on similarities between the generated plurality of clusters, a process of presenting the plurality of clusters in sequence based on the determined presentation order, and a process of assigning a predetermined label to each of the clusters presented in the sequence. Effect of the Invention

[0011] The present disclosure makes it possible to provide an annotation device, an annotation method, and a non-transitory computer-readable medium that can improve annotation workability. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram illustrating a configuration example of an annotation device according to an embodiment. [Diagram 2] 11 is a flowchart for explaining an example of an operation of the annotation device according to the embodiment. [Diagram 3] FIG. 11 is a diagram for explaining an example of an operation for generating a plurality of clusters. [Figure 4A] 11 is a diagram for explaining an example of an operation for determining a presentation order of clusters. FIG. [Figure 4B] 11 is a diagram for explaining an example of an operation for determining a presentation order of clusters. FIG. [Figure 4C] 11 is a diagram for explaining an example of an operation for determining a presentation order of clusters. FIG. [Figure 4D] 11 is a diagram for explaining an example of an operation for determining a presentation order of clusters. FIG. [Diagram 5] 11 is a diagram for explaining an example of an operation for determining a presentation order of clusters. FIG. [Figure 6]FIG. 2 is a diagram illustrating an example of point cloud data of a measurement target. [Figure 7A] FIG. 11 is a diagram for explaining an example of annotation processing. [Figure 7B] FIG. 11 is a diagram for explaining an example of annotation processing. [Figure 7C] FIG. 11 is a diagram for explaining an example of annotation processing. [Figure 7D] FIG. 11 is a diagram for explaining an example of annotation processing. [Figure 8] 1 is a block diagram showing an example of a hardware configuration including an annotation device according to an embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a block diagram showing a configuration example of an annotation device according to an embodiment. As shown in Fig. 1, the annotation device 1 according to the embodiment includes a cluster generation unit 11, a presentation order determination unit 12, a cluster presentation unit 13, and a label assignment unit 14.

[0014] The annotation device 1 according to the present embodiment is a device that performs annotation processing to assign a predetermined label to point cloud data corresponding to three-dimensional position information of a measurement target. The three-dimensional position information of the measurement target can be acquired in advance using a three-dimensional distance sensor such as a LIDAR.

[0015] The cluster generating unit 11 groups point cloud data corresponding to the three-dimensional position information of the measurement target to generate a plurality of clusters. That is, the cluster generating unit 11 groups each point included in the point cloud data to generate a plurality of clusters. For example, the point cloud data corresponding to the three-dimensional position information of the measurement target is stored in a point cloud data storage unit 70 (see FIG. 8). The point cloud data stored in the point cloud data storage unit is supplied to the cluster generating unit 11.

[0016] For example, the cluster generation unit 11 generates clusters such that points corresponding to the same structure are included in the same cluster. In other words, the cluster generation unit 11 generates clusters such that different structures are not included in the same cluster. In this embodiment, the same structure may be divided into fine clusters, in which case clusters are generated such that the same structure is included in multiple clusters.

[0017] Specifically, the cluster generating unit 11 can generate a plurality of clusters using graph cut segmentation, region growing, supervoxel division, and the like. Graph cut segmentation is a method of creating a graph based on the neighborhood relationship of each point of point cloud data, and performing graph cutting based on the value of shape feature amount. For example, the shape feature amount can be obtained using Principal Component Analysis (PCA) or the like. Region growing is a method of generating clusters by repeating random selection of seed points and expansion to neighboring points having similar information. Supervoxel segmentation is a method of generating clusters by dividing the entire space into fine cubic lattice shapes (voxels) and fine-tuning the shape of each voxel so as to follow the contour or object boundary. Note that the method of generating clusters in this embodiment is not limited to these, and clusters may be generated using other existing methods.

[0018] The presentation order determination unit 12 determines the presentation order of the clusters based on the similarity of the clusters generated by the cluster generation unit 11. Specifically, the presentation order determination unit 12 determines the presentation order of the clusters based on at least one of the spatial similarity and the appearance similarity of the clusters. In other words, the presentation order determination unit 12 determines the presentation order of the clusters such that the clusters similar in appearance and space are arranged in order. At this time, the presentation order determination unit 12 may determine that the closer the spatial positional relationship of the clusters is, the higher the spatial similarity is. Also, the presentation order determination unit 12 may determine that the closer the appearance of the clusters is, the higher the appearance similarity is. Here, the appearance similarity refers to shape similarity, color (RGB information) similarity, reflection intensity similarity, and the like. For example, the presentation order determination unit 12 may determine that the more similar the shapes of the clusters are, the higher the appearance similarity is. Also, the presentation order determination unit 12 may determine that the more similar the colors (RGB information) of the clusters are, the higher the appearance similarity is. The presentation order determination unit 12 may determine that the more similar the reflection intensities of the clusters are, the higher the appearance similarity is. Note that a method for determining the presentation order of the clusters by the presentation order determination unit 12 will be described in detail later.

[0019] The cluster presentation unit 13 presents the multiple clusters in order based on the presentation order determined by the presentation order determination unit 12. For example, the cluster presentation unit 13 presents the multiple clusters to the user in order by displaying each cluster in order on a separately provided display unit 50 (see FIG. 8 ).

[0020] The label assignment unit 14 assigns a predetermined label to each cluster presented in order by the cluster presentation unit 13. Specifically, the label assignment unit 14 assigns a label to each cluster in response to a predetermined input. For example, the label assignment unit 14 assigns a label to each cluster in response to an input result from a separately provided input unit 60 (see FIG. 8).

[0021] For example, the cluster presentation unit 13 sequentially presents each of the multiple clusters in order based on the presentation order determined by the presentation order determination unit 12. The label assignment unit 14 assigns a label to each of the sequentially presented clusters in response to a predetermined input.

[0022] The annotation device 1 according to this embodiment can perform annotation processing on point cloud data by repeating a process in which the cluster presenting unit 13 sequentially presents each of a plurality of clusters in order, and a process in which the label assignment unit 14 assigns a label to each cluster in accordance with a specified input.

[0023] Next, the operation (annotation method) of the annotation device according to the present embodiment will be described. FIG. 2 is a flowchart for explaining the operation of the annotation device according to the present embodiment. In the present embodiment, before performing annotation processing using the annotation device 1, point cloud data corresponding to three-dimensional position information of a measurement target is acquired in advance using a three-dimensional distance sensor such as LIDAR. FIG. 6 shows an example of point cloud data 40. As shown in FIG. 6, the point cloud data 40 corresponds to the three-dimensional position information of the measurement target, and includes information such as the distance from a position information acquisition device such as LiDAR to the measurement target, the reflection intensity, and three-dimensional coordinates. The point cloud data is stored, for example, in a point cloud data storage unit 70 (see FIG. 8).

[0024] 2, when performing annotation processing, first, the cluster generating unit 11 generates a plurality of clusters by grouping point cloud data corresponding to three-dimensional position information of a measurement target (step S1). For example, the cluster generating unit 11 generates clusters such that points corresponding to the same structure are included in the same cluster.

[0025] FIG. 3 is a diagram for explaining an example of an operation for generating a plurality of clusters, and is a diagram for explaining an example of graph cut segmentation. First, as shown in the left diagram of FIG. 3, a graph is created based on the neighborhood relationships of each point on the point cloud data. In FIG. 3, differences in shape features of each point, including points 21, 22, and 23 of the point cloud data, are expressed by differences in shading. Then, as shown in the right diagram of FIG. 3, multiple clusters 31, 32, and 33 are generated by cutting the edges of the graph (graph cut) so that points with similar values ​​of shape features are grouped together.

[0026] In the example shown in Fig. 3, a plurality of adjacent points having similar shape feature values ​​to point 21 form cluster 31. A plurality of adjacent points having similar shape feature values ​​to point 22 form cluster 32. A plurality of adjacent points having similar shape feature values ​​to point 23 form cluster 33. Note that the cluster generation method shown in Fig. 3 is just an example, and in this embodiment, a plurality of clusters may be generated using a method other than the method shown in Fig. 3.

[0027] Next, the presentation order determination unit 12 shown in Fig. 1 determines the presentation order of the multiple clusters based on the similarities of the multiple clusters generated by the cluster generation unit 11 (step S2 in Fig. 2). Specifically, the presentation order determination unit 12 determines the presentation order of the multiple clusters based on at least one of the spatial similarity and the appearance similarity of the multiple clusters. In other words, the presentation order determination unit 12 determines the presentation order of the multiple clusters such that clusters that are similar in appearance and space are grouped together in order.

[0028] 4A to 4D are diagrams for explaining an example of an operation for determining the presentation order of clusters. In FIG. 4A to FIG. 4D, symbols C1 to C13 indicate a plurality of clusters. Note that in FIG. 4A to FIG. 4D, for the sake of simplicity, the plurality of clusters C1 to C13 are shown as circles, triangles, and rectangles.

[0029] First, as shown in FIG. 4A, the presentation order determination unit 12 creates a graph by connecting each of the clusters C1 to C13 with edges E1 to E19 based on the spatial similarity of the clusters C1 to C13. For example, the presentation order determination unit 12 may connect clusters that are spatially adjacent to each other with edges. In this case, the presentation order determination unit 12 may use a K-nearest neighbor search to connect clusters that are adjacent to each other with edges. In addition, a threshold may be set for the distance between the centers of gravity of the clusters, and clusters whose threshold is equal to or less than a predetermined value may be connected with edges. In addition, a threshold may be set for the shortest distance between the clusters, and clusters whose threshold is equal to or less than a predetermined value may be connected with edges.

[0030] Next, as shown in FIG. 4B, the presentation order determination unit 12 sets weights for the edges E1 to E19 connecting the clusters based on the appearance similarity of the clusters C1 to C13. Specifically, the presentation order determination unit 12 sets the weights of the edges connecting the clusters similar in appearance to be small. As shown in FIG. 4, the clusters C1 to C3 are squares, and the cluster C6 is a circle, and these shapes are different, so the weights of the edges E2, E3, and E5 connecting the clusters C1 to C3 and the cluster C6 are set to "large" (in FIG. 4B, the larger the weight, the thicker the line. In other words, the thicker the line, the greater the difference). Note that the clusters C4 and C5 are squares, and the cluster C6 is a circle, and these shapes are different but similar, so the weights of the edges E7 and E9 are set to "medium."

[0031] Moreover, the clusters C1 to C3 are rectangular and have similar shapes and sizes, so the weights of the edges E1 and E4 connecting the clusters C1 to C3 are set to "small." The clusters C3 and C4 are rectangular and have similar shapes but different sizes, so the weight of the edge E6 connecting the clusters C3 and C4 is set to "medium." The clusters C4 and C5 are rectangular and have similar shapes and sizes, so the weight of the edge E8 connecting the clusters C4 and C5 is set to "small." The weights of the edges E10 to E19 connecting the clusters C7 to C13 can also be set using a similar method. The weights of the edges E1 to E19 connecting the clusters C1 to C13 may be set by performing error calculations based on, for example, Principal Component Analysis (PCA) features or normal distribution.

[0032] Next, the presentation order determination unit 12 determines the presentation order of each of the clusters C1 to C13 based on the set weight of each of the edges E1 to E19.

[0033] Specifically, first, the presentation order determination unit 12 creates a minimum spanning tree of multiple clusters C1 to C13 based on the weights of the set edges E1 to E19, as shown in Fig. 4C. In the example shown in Fig. 4C, the minimum spanning tree is created by giving priority to edges with small weights. Although the weight of edge E10 is set to "large," edge E10 is left because it is necessary for joining clusters C5 and C7.

[0034] Thereafter, the presentation order determination unit 12 performs a depth-first search on the minimum spanning tree to determine the presentation order of each of the clusters C1 to C13, as shown in Fig. 4D. In the example shown in Fig. 4D, the presentation order of the cluster C1 is set to be first, followed by the clusters C2, C3, ..., C13.

[0035] At this time, the presentation order determination unit 12 may perform a depth-first search on the minimum spanning tree with a priority on reducing the spatial gaps of the clusters C1 to C13. In other words, by selecting the branch priority order and the root node so as to reduce the spatial gaps of the presentation clusters, the clusters can be presented to the user in an appropriate order.

[0036] To explain this in more detail using Figure 5, in the left diagram of Figure 5, cluster 35 is ranked fifth, so the spatial gap between the fourth and fifth clusters is large. In contrast, in the right diagram of Figure 5, cluster 35 is ranked third, so the spatial gap between the third cluster 35 and the fourth cluster can be made smaller, and the spatial gap of the entire minimum spanning tree can be made smaller.

[0037] 1 presents the clusters C1 to C13 in order based on the presentation order determined by the presentation order determination unit 12 (step S3 in FIG. 2). For example, the cluster presentation unit 13 presents the clusters C1 to C13 to the user in order by displaying each of the clusters C1 to C13 in order on a separately provided display unit 50 (see FIG. 8).

[0038] The label assignment unit 14 assigns a predetermined label to each of the clusters C1 to C13 presented in order by the cluster presentation unit 13 (step S4). Specifically, the label assignment unit 14 assigns a label to each of the clusters C1 to C13 in response to a predetermined input. For example, the label assignment unit 14 assigns a label to each of the clusters C1 to C13 in response to an input result from a separately provided input unit 60 (see FIG. 8).

[0039] Then, if the assignment of labels to the clusters C1 to C13 is not completed, that is, if the annotation is insufficient (step S5: No), the operations of steps S3 to S5 are repeated. That is, in this embodiment, the process of the cluster presenting unit 13 sequentially presenting each of the multiple clusters C1 to C13 in order and the process of the label assignment unit 14 assigning a label to each of the clusters C1 to C13 in response to a predetermined input are repeated, whereby annotation processing can be performed on the point cloud data.

[0040] Specifically, the cluster presenting unit 13 first displays the cluster C1 on the display unit and presents the cluster C1 to the user. After that, the user inputs a label for the cluster C1 using the input unit. The labeling unit 14 assigns a label to the cluster C1 according to the user's input. Next, the cluster presenting unit 13 displays the cluster C2 on the display unit and presents the cluster C2 to the user. After that, the user inputs a label for the cluster C2 using the input unit. The labeling unit 14 assigns a label to the cluster C2 according to the user's input. Next, the cluster presenting unit 13 displays the cluster C3 on the display unit and presents the cluster C3 to the user. After that, the user inputs a label for the cluster C3 using the input unit. The labeling unit 14 assigns a label to the cluster C3 according to the user's input. Thereafter, the same process is performed for the clusters C4 to C13, so that the annotation process can be performed. Then, when the assignment of labels to the clusters C1 to C13 is completed, that is, when it is determined that the annotation is sufficient (step S5: Yes), the annotation process ends.

[0041] In the flowchart shown in Fig. 2, an example has been described in which the presentation order determination unit 12 determines the presentation order for all clusters, and then the cluster presentation unit 13 presents the multiple clusters to the user in order. However, in this embodiment, the presentation order determination unit 12 may determine the presentation order for multiple clusters one by one. In this case, the presentation order determination unit 12 determines the cluster to be presented next, the cluster presentation unit 13 presents the cluster, and the label assignment unit 14 assigns a label to the cluster. This process is performed for each cluster. That is, in the flowchart of Fig. 2, after No in step S5, the process may return to step S2.

[0042] In the above description, the presentation order determination unit 12 determines the presentation order of the clusters in the steps shown in FIG. 4A to FIG. 4D. However, in this embodiment, if similar clusters are presented in a similar order, it is not necessary to perform all the steps shown in FIG. 4A to FIG. 4D. For example, as shown in FIG. 4A, the presentation order determination unit 12 connects the clusters C1 to C13 with the sides E1 to E19 based on the spatial similarity of the clusters C1 to C13. Then, the presentation order of the clusters C1 to C13 may be determined by tracing the sides E1 to E19 in an arbitrary order. In this case, the spatially similar clusters are presented in order.

[0043] In this embodiment, the user does not necessarily need to input a label for a cluster every time in the label assignment process in step S4 in Fig. 2. For example, if a currently presented cluster is sufficiently similar to a previously presented cluster, the label assignment unit 14 may automatically assign the same label as the previously presented cluster to the currently presented cluster.

[0044] Next, the processing of steps S3 to S5 will be specifically described with reference to Fig. 7A to Fig. 7D. Fig. 7A to Fig. 7D are diagrams for explaining an example of annotation processing, and are diagrams for explaining a case where annotation processing is performed on the point cloud data shown in Fig. 6. It is assumed that the processing of steps S1 to S2 has been performed in advance on the point cloud data shown in Fig. 6. That is, it is assumed that the point cloud data shown in Fig. 6 has been grouped to generate a plurality of clusters, and the presentation order of the plurality of clusters has been determined based on the similarity of the generated plurality of clusters.

[0045] First, as shown in FIG. 7A, the cluster presentation unit 13 displays the cluster 51 to be presented first, which is determined by the presentation order determination unit 12, on the display unit 50. The user inputs a label for the cluster 51 displayed on the display unit 50. Since the cluster 51 is classified as a "wall", the user inputs a label corresponding to "wall". The label assignment unit 14 assigns a label corresponding to "wall" to the cluster C1.

[0046] Next, the cluster presentation unit 13 displays the cluster 52 to be presented second, which is determined by the presentation order determination unit 12, on the display unit 50, as shown in FIG. 7B. The cluster 52 is similar to the cluster 51 in terms of space and appearance. The user inputs a label for the cluster 52 displayed on the display unit 50. Since the cluster 52 is classified as a "wall", the user inputs a label corresponding to "wall". The label assignment unit 14 assigns a label corresponding to "wall" to the cluster C52.

[0047] Next, the cluster presentation unit 13 displays cluster 53 to be presented third, which is determined by the presentation order determination unit 12, on the display unit 50, as shown in FIG. 7C. Cluster 53 is similar to cluster 52 in terms of space and appearance. The user inputs a label for cluster 53 displayed on the display unit 50. Since cluster 53 is classified as a "wall", the user inputs a label corresponding to "wall". The label assignment unit 14 assigns a label corresponding to "wall" to cluster C53.

[0048] Next, the cluster presentation unit 13 displays the fourth cluster 54 to be presented, which is determined by the presentation order determination unit 12, on the display unit 50, as shown in FIG. 7D. The cluster 54 is similar in spatial and appearance to the clusters 51 to 53. The user inputs a label for the cluster 54 displayed on the display unit 50. Since the cluster 54 is classified as a "wall", the user inputs a label corresponding to the "wall". The label assignment unit 14 assigns the label corresponding to the "wall" to the cluster 54. Thereafter, by repeating the same operation, annotation processing can be performed on the point cloud data.

[0049] In the above example, since the clusters 51 to 54 are similar to each other, the labeling unit 14 may automatically assign the same label ("wall") as the previously presented cluster to the currently presented cluster. In this case, the user need not input a label.

[0050] 7A to 7D show an example in which clusters 51 to 54 are presented to the user in order. However, in this embodiment, the cluster presentation unit 13 may present clusters that are similar to each other together. For example, when presenting cluster 51, the cluster presentation unit 13 may display clusters 52 to 54 that are spatially and externally similar to cluster 51 together.

[0051] 7A to 7D, an example has been described in which clusters 51 to 54 are added to display unit 50 in order, but in this embodiment, the method of displaying clusters is not limited to adding them in order. For example, in FIG. 7B, cluster 51 may be hidden when cluster 52 is displayed on display unit 50.

[0052] 7A to 7D, the clusters 51 to 54 are displayed in order on the blank display unit 50, but in this embodiment, the clusters may be displayed with emphasis. That is, the original point cloud data 40 (see FIG. 6) may be displayed in the background, and the clusters 51 to 54 to which labels are to be added may be displayed with emphasis in red or with a thick frame on the point cloud data 40 in the background.

[0053] As described in the background art, in order to automatically identify a specific object using point cloud data of a measurement target, it is necessary to create teacher data by assigning a specific label to the point cloud data in advance. The annotation work of assigning a specific label to the point cloud data is performed, for example, by a user assigning a specific label to each point of the point cloud data. However, when the user performs the annotation work while checking all the clusters displayed on the display unit on the screen, there is a problem that the annotation work becomes complicated because the point cloud data of the measurement target is huge (i.e., the point cloud data includes a very large number of points).

[0054] For example, if multiple clusters are generated by grouping point cloud data and then displayed together on a display unit, the user's task of labeling becomes cumbersome. In particular, when the number of clusters increases, the visibility of the operation screen decreases significantly. In addition, when annotating a part of the point cloud data by a user specifying the range with a mouse, the user needs to specify the range for each of the huge amount of point cloud data, which makes the task cumbersome.

[0055] In contrast, in the present embodiment, point cloud data is grouped to generate multiple clusters, the presentation order of the multiple clusters is determined based on the similarity of the multiple clusters, and the multiple clusters are presented to the user in order based on the determined presentation order. Therefore, the number of clusters displayed on the display unit can be reduced, thereby improving visibility. In addition, since point cloud data is grouped to automatically generate multiple clusters, the user does not need to perform operations such as range specification. Therefore, operability can be improved.

[0056] Therefore, the present disclosure makes it possible to provide an annotation device, an annotation method, and a non-transitory computer-readable medium that can improve annotation workability.

[0057] In this embodiment, the processing of each component can also be realized by causing a CPU (Central Processing Unit) to execute a computer program.

[0058] In other words, this may be achieved by having a computer execute an annotation processing program that includes the steps of: grouping point cloud data corresponding to three-dimensional position information of the measurement target to generate multiple clusters; determining an order of presentation of the multiple clusters based on the similarity of the multiple clusters generated; presenting the multiple clusters in sequence based on the determined presentation order; and assigning a predetermined label to each of the clusters presented in sequence.

[0059] Fig. 8 is a block diagram for explaining an example of a hardware configuration including an annotation device according to this embodiment. As shown in Fig. 8, the annotation device 1 according to this embodiment can be configured using a CPU (101) and a memory 102. That is, the annotation device 1 can be configured by executing the above-mentioned annotation processing program on the CPU (101). A display unit 50, an input unit 60, a point cloud data storage unit 70, and an annotation data storage unit 80 are connected to the annotation device 1.

[0060] The display unit 50 is configured using a liquid crystal display, an organic EL (electro-luminescence) display, or the like. A plurality of clusters output from the cluster presenting unit 13 (see FIG. 1) are displayed on the display unit 50. At this time, the plurality of clusters are displayed on the display unit 50 in order based on the presentation order determined by the presentation order determining unit 12. This allows the user to visually recognize each cluster in a predetermined order.

[0061] The input unit 60 is a device operated by a user to input label information of multiple clusters, and can be configured using a keyboard, a mouse, etc. The label information input by the input unit 60 is supplied to the label assignment unit 14 of the annotation device 1. The label assignment unit 14 assigns a label to each cluster in accordance with the label information supplied from the input unit 60.

[0062] The point cloud data storage unit 70 stores point cloud data, which is three-dimensional position information of a measurement target. The point cloud data is acquired in advance using a three-dimensional distance sensor such as a LIDAR. When the annotation device 1 performs annotation processing, the point cloud data storage unit 70 supplies the point cloud data, which is the target of the annotation processing, to the annotation device 1.

[0063] The annotation data storage unit 80 stores data after annotation processing. Specifically, the annotation data storage unit 80 stores data on each cluster and data on the labels assigned to each cluster in association with each other. The annotation data stored in the annotation data storage unit 80 can be used as teacher data for machine learning.

[0064] 8 shows a case where the point cloud data storage unit 70 and the annotation data storage unit 80 are configured separately, but the point cloud data storage unit 70 and the annotation data storage unit 80 may be configured using the same storage unit (storage unit). Also, a storage unit for storing the point cloud data and annotation data may be built into the annotation device 1. For example, the annotation device 1 may be configured as a personal computer.

[0065] In addition, in this embodiment, the annotation device 1 may be configured as an application server. In this case, a point cloud data storage unit 70, an annotation data storage unit 80, a display unit 50, and an input unit 60 are provided on the client side. When performing annotation processing, a user on the client side transmits point cloud data stored in the point cloud data storage unit 70 to the annotation device 1. The annotation device 1 performs a process of grouping the point cloud data transmitted from the user to generate multiple clusters, and a process of determining the presentation order of the multiple clusters based on the similarity of the generated multiple clusters.

[0066] Thereafter, the annotation device 1 displays each cluster in order on the user's display unit 50 based on the determined presentation order. The user inputs label information for each cluster displayed on the display unit 50 using the input unit 60. The label information input using the input unit 60 is transmitted to the annotation device 1. The annotation device 1 assigns the label information to each cluster. The annotation device 1 performs annotation processing by repeating a process of presenting each of the multiple clusters to the user in order and a process of assigning the label input by the user to the cluster. As a result, annotation data is generated. The generated annotation data is transmitted from the annotation device 1 to the annotation data storage unit 80 of the user.

[0067] When the annotation device 1 is configured as an application server, a plurality of users can access the annotation device 1 to perform annotation processing.

[0068] In the above embodiment, the program can be stored and supplied to the computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (specifically, flexible disks, magnetic tapes, and hard disk drives), magneto-optical recording media (specifically, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (specifically, mask ROMs, PROMs (Programmable ROMs), and EPROMs (Erasable PROMs)), flash ROMs, and RAMs (Random Access Memory). The program may be supplied to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can supply the program to the computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

[0069] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0070] 1 Annotation device 11 Cluster Generation Section 12 Presentation order determining section 13 Cluster Presentation 14 Labeling section 21, 22, 23 points 31, 32, 33 Clusters 50 Display 51, 52, 53, 54 Clusters 60 Input section 70 Point cloud data storage section 80 Annotation Data Storage 101 CPU 102 Memory

Claims

1. a cluster generation unit that groups point cloud data corresponding to three-dimensional position information of a measurement target to generate a plurality of clusters; a presentation order determination unit that determines a presentation order of the plurality of clusters based on similarities between the plurality of clusters; a cluster presentation unit that presents the plurality of clusters in order based on the determined presentation order; a labeling unit that assigns a predetermined label to each of the clusters presented in the order; the presentation order determination unit determines a presentation order of the plurality of clusters based on at least one of spatial similarity and appearance similarity of the plurality of clusters, and determines that the spatial similarity is higher when the spatial positional relationship of the plurality of clusters is closer, and determines that the appearance similarity is higher when the appearances of the plurality of clusters are more similar. Annotation device.

2. The presentation order determination unit is creating a graph by connecting each of the plurality of clusters with an edge based on the spatial similarity of the plurality of clusters; assigning a weight to each edge connecting each of the clusters based on the appearance similarity of the clusters; determining a presentation order for each of the clusters based on the set edge weights; The annotation device according to claim 1 .

3. The presentation order determination unit is creating a minimum spanning tree of the plurality of clusters based on the set edge weights; performing a depth-first search on the minimum spanning tree to determine an order of presentation of each of the clusters; The annotation device according to claim 2 .

4. The annotation device according to claim 3 , wherein the presentation order determination unit performs a depth-first search on the minimum spanning tree, giving priority to reducing a spatial gap between the clusters.

5. the cluster presentation unit sequentially presents each of the plurality of clusters in order based on the presentation order determined by the presentation order determination unit; the labeling unit assigns a label to each of the successively presented clusters in response to a predetermined input; The annotation device according to any one of claims 1 to 4.

6. generating a plurality of clusters by grouping point cloud data corresponding to three-dimensional position information of the measurement target; determining a presentation order of the plurality of clusters based on the similarities of the plurality of clusters; presenting the plurality of clusters in sequence based on the determined presentation order; assigning a predetermined label to each of the clusters presented in the order; determining a presentation order of the plurality of clusters based on at least one of spatial similarity and appearance similarity of the plurality of clusters, and determining that the closer the spatial positional relationship of the plurality of clusters is, the higher the spatial similarity is, and that the more similar the appearances of the plurality of clusters are, the higher the appearance similarity is; Annotation method.

7. A process of grouping point cloud data corresponding to three-dimensional position information of a measurement target to generate a plurality of clusters; A process of determining a presentation order of the plurality of clusters based on similarities between the plurality of clusters; presenting the plurality of clusters in order based on the determined presentation order; and a process of assigning a predetermined label to each of the clusters presented in the order, In the process of determining the presentation order of the plurality of clusters, the presentation order of the plurality of clusters is determined based on at least one of spatial similarity and appearance similarity of the plurality of clusters, and the spatial similarity is determined to be higher as the spatial positional relationship of the plurality of clusters is closer, and the appearance similarity is determined to be higher as the appearances of the plurality of clusters are more similar. A program for causing a computer to execute annotation processing.

Images