Information processing device, information processing method and information processing program

Abstract

To provide an information processing device, an information processing method and an information processing program capable of acquiring a representative of points of a point group.SOLUTION: An information processing device includes: an acquisition part for acquiring point group information; a division part for dividing a three-dimensional space including a plurality of points recorded in the point group information acquired by the acquisition part in a voxel manner; an impartment part for imparting, to each point, a machine point label showing which machine point among a plurality of machine points each of the plurality of machine points in each voxel corresponds to and is acquired each of a plurality of voxels divided by the division part; a calculation part for specifying a point to which the most same machine point labels are imparted among a plurality of points in the voxel to calculate an average value of feature quantities of the point; and a replacement part for replacing all of the plurality of points in each voxel with one point of the average value calculated by the calculation part, and imparting the machine point label specified in calculating one point to the one point.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] There is LiDAR measurement, which generates point cloud information by emitting laser light from a measurement point and receiving the light reflected by an object at the measurement point. In the system of Patent Document 1, the point cloud obtained by LiDAR measurement is represented in a spherical coordinate system. The system converts the spherical coordinate system representing the point cloud into a Cartesian coordinate system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-533601 Summary of the Invention [Problem to be solved by the invention]

[0004] The system described in Patent Document 1 converts the coordinate system of a point cloud, but does not mention the use of coordinate values ​​represented in the coordinate system. In addition, LiDAR measurements acquire a point cloud represented by a large number of points, but in some cases it is desirable to perform calculations using a smaller number of points that make up the point cloud.

[0005] The present disclosure provides an information processing device, an information processing method, and an information processing program capable of acquiring a representative of points included in a point cloud. [Means for solving the problem]

[0006] An information processing device of one embodiment includes an acquisition unit that acquires point cloud information by outputting light or sound waves from each of a plurality of instrument points and obtaining light or sound waves reflected from an object at the instrument points; a division unit that divides a three-dimensional space including a plurality of points recorded in the point cloud information acquired by the acquisition unit into voxels; an assignment unit that assigns an instrument point label to each of the plurality of points in each of the plurality of voxels divided by the division unit, indicating which of the plurality of instrument points the point corresponded to when it was acquired; a calculation unit that identifies the point among the plurality of points in the voxel that is assigned the most identical instrument point label and calculates the average value of the features of the points; and a replacement unit that replaces all of the plurality of points in each voxel with one point of the average value calculated by the calculation unit, and assigns the one point with the instrument point label identified when calculating the one point. Effect of the Invention

[0007] According to one aspect, a representative of the points in the point cloud may be obtained. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an information processing device according to an embodiment. [Diagram 2] 1A and 1B are diagrams for explaining an example of processing for a point cloud, in which (A) is a diagram when divided into voxels, and (B) is a diagram after replacement with an average value. [Diagram 3] FIG. 1 is a block diagram illustrating an information processing device according to an embodiment. [Figure 4] FIG. 11 is a first diagram for explaining a method for setting a first feature amount. [Diagram 5] FIG. 2 is a second diagram for explaining the technique for setting the first feature amount. [Figure 6] FIG. 11 is a first diagram for explaining a method for setting a second feature amount. [Figure 7] FIG. 2 is a second diagram for explaining the method for setting the second feature amount. [Figure 8]1 is a flowchart illustrating an information processing method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment will be described below.

[0010] [Overview of information processing device 100] First, an overview of an information processing device 100 according to an embodiment will be described. FIG. 1 is a diagram illustrating an information processing device 100 according to an embodiment. 2A and 2B are diagrams for explaining an example of processing for a point cloud, in which (A) is a diagram when divided into 400 voxels, and (B) is a diagram after replacement with an average value.

[0011] The information processing device 100 may be configured as an identifying device or the like that identifies a plurality of points (representative points) from among the points included in the point cloud information. The information processing device 100 may also be configured as a replacing device or the like that replaces some points (a plurality of points) from among the points included in the point cloud information with one point (representative point). The information processing device 100 may also be configured as a setting device or the like that sets, for each point, a feature amount used when identifying a representative point. The information processing device 100 is not limited to the device of the above example, and may be configured as various devices or the like. The information processing device 100 may be a computer such as a server, a desktop, a laptop, a tablet, or a smartphone.

[0012] The information processing device 100 obtains point cloud information by outputting light or sound waves from each of a plurality of instrument points and obtaining the light or sound waves reflected from an object at the instrument points. In LiDAR measurement, when light (e.g., laser light) emitted from a light emitting / receiving device is reflected by an object, the reflected light (reflected light) is repeatedly received by the light emitting / receiving device to generate point cloud information (point cloud information) consisting of a plurality of points 300. Note that, like LiDAR measurement, it is also possible to obtain point cloud information by using sound waves instead of light. Note that LiDAR measurement and measurement that obtains point cloud information using sound waves may be collectively referred to as "point cloud measurement." In this embodiment, the positions where the light emitting and receiving devices are arranged are defined as instrument points 200, and LiDAR measurements are performed at each of the multiple instrument points 200 to generate point cloud information (the same applies to sound waves). In this case, the point cloud information may be generated by simultaneously performing point cloud measurements at each of the multiple instrument points 200. Alternatively, the point cloud information may be generated by performing point cloud measurements at each of the multiple instrument points 200 in order, by sequentially performing point cloud measurements at any of the multiple instrument points 200. The number of instrument points 200 is not limited to two as illustrated in FIG. 1, but may be three or more.

[0013] The point cloud information may include, for example, information on the position (e.g., coordinates, etc.) of the point 300 measured at each instrument point 200 based on that instrument point 200, and the distance from that instrument point 200. Therefore, each of the multiple points 300 included in the point cloud information includes a three-dimensional positional relationship.

[0014] The information processing device 100 divides a three-dimensional positional relationship (three-dimensional space) including a plurality of points 300 into voxels 400. As an example, as shown in FIG. 2(A), the information processing device 100 divides a three-dimensional positional relationship (three-dimensional space) for a plurality of points 300 into a space (voxels 400) such as a regular cube. The number of voxels 400 is not limited to four as illustrated in FIG. 2, and may be two, three, five or more. Each of the plurality of voxels 400 (for example, a regular cube) includes a plurality of points 300 obtained by point cloud measurement. The information processing device 100 assigns an instrument point label to each point 300 indicating which instrument point 200 each of the plurality of points 300 in each voxel 400 was obtained at. As an example of this case, information indicating the relationship between the instrument point 200 and the point 300 obtained at the instrument point 200 is recorded in the point cloud information. Therefore, based on the point cloud information, the information processing device 100 assigns an instrument point label to each of the multiple points 300. In the example shown in Fig. 2, the instrument point label is represented by a point 300A that indicates a point as "plain" and a point 300B that indicates a point as "shaded".

[0015] The information processing device 100 identifies the point 300 to which the most identical instrument point labels are assigned among the multiple points 300 in the voxel 400. The information processing device 100 calculates the average value of the feature amounts of the points 300 to which the most identical instrument point labels are assigned for each voxel 400. An example of the feature amount may be the coordinates of the point 300. The information processing device 100 replaces all of the multiple points 300 in each voxel 400 with the above-mentioned average value point 310 (see FIG. 2(B)). The information processing device 100 assigns the most identical instrument point label in the voxel 400 to the replaced average value point 310.

[0016] [Details of the information processing device 100] Next, the information processing device 100 according to an embodiment will be described in detail. FIG. 3 is a block diagram illustrating the information processing device 100 according to an embodiment.

[0017] The information processing device 100 includes, for example, a communication unit 131, a storage unit 132, a display unit 133, and a control unit. The communication unit 131, the storage unit 132, and the display unit 133 may be an embodiment of an output unit. The control unit includes, for example, an acquisition unit 111, a division unit 112, an assignment unit 113, a calculation unit 114, a replacement unit 115, an output control unit 116, a first identification unit 117, a second identification unit 118, a first setting unit 119, and a second setting unit 120. The control unit 110 may be configured by, for example, an arithmetic processing device of the information processing device 100. The control unit 110 (e.g., a calculation processing unit, etc.) may realize the functions of each unit (e.g., the acquisition unit 111, the division unit 112, the assignment unit 113, the calculation unit 114, the replacement unit 115, the output control unit 116, the first determination unit 117, the second determination unit 118, the first setting unit 119, the second setting unit 120, etc.) by, for example, appropriately reading and executing various programs, etc. stored in the memory unit 132, etc.

[0018] The communication unit 131 is, for example, a communication interface capable of transmitting and receiving various information to and from a device (external device (not shown)) outside the information processing device 100. The external device may be, for example, a server, a user terminal, etc. The user terminal is, for example, a terminal used by a user of the information processing device 100, and may be a desktop, a laptop, a tablet, a smartphone, etc.

[0019] The storage unit 132 may store, for example, various information and programs. Examples of the storage unit 132 may be a memory, a solid state drive, a hard disk drive, etc. Note that the storage unit 132 may be, for example, a storage area and a server on a cloud.

[0020] The display unit 133 is, for example, a display capable of displaying various characters, symbols, images, and the like.

[0021] The acquisition unit 111 acquires point cloud information by point cloud measurement in which light or sound waves are output from each of a plurality of instrument points 200 and the light or sound waves reflected from an object are received at the instrument points 200. The point cloud information may be stored, for example, in an external memory (not shown) and a server (not shown). The acquisition unit 111 may acquire the point cloud information from an external device such as a server via the communication unit 131, for example. Alternatively, for example, when an external memory in which the point cloud information is stored is connected to an interface of the information processing device 100, the acquisition unit 111 may acquire the point cloud information from the external memory.

[0022] The point cloud information is information on a point cloud composed of a plurality of points 300 acquired for each light emitting and receiving device (instrument point 200). The point cloud information may include, for example, information on the position (coordinates) of each point 300 based on the instrument point 200 and the distance from the instrument point 200 to each point 300. The point cloud information may also include information indicating the relationship between the instrument point 200 and the points 300 acquired at that instrument point 200 (which point 300 was acquired by which instrument point 200).

[0023] The dividing unit 112 divides a three-dimensional space including the multiple points 300 recorded in the point cloud information acquired by the acquiring unit 111 into voxels 400. The dividing unit 112 divides the three-dimensional space (relationship between three-dimensional coordinate positions) corresponding to the coordinates (three-dimensional coordinates) recorded in each of the multiple points 300 into voxels 400. As a specific example, the dividing unit 112 divides into voxels 400 using multiple unit spaces such as a regular cube, a rectangular parallelepiped, a cone, a cylinder, and a polyhedron. Each of the multiple voxels 400 (e.g., a regular cube) includes therein the multiple points 300 obtained by the point cloud measurement.

[0024] The assigning unit 113 assigns, to each of the multiple voxels 400 divided by the dividing unit 112, an instrument point label indicating which of the multiple instrument points 200 the multiple points 300 in each voxel 400 correspond to when the points 300 were acquired. The assigning unit 113 assigns, to each of the multiple points 300 in each voxel 400, an instrument point label indicating which instrument point 200 the points 300 in each voxel 400 were acquired at. For example, when information indicating the relationship between the instrument point 200 and the point 300 acquired at the instrument point 200 is recorded in the point cloud information, the assigning unit 113 assigns an instrument point label to each of the multiple points 300 based on the point cloud information. The instrument point label may be of various types as long as it can identify the instrument point 200, for example.

[0025] The calculation unit 114 identifies the point 300 to which the largest number of identical instrument point labels are assigned among the multiple points 300 in the voxel 400, and calculates the average value of the feature amounts of the points 300. For each voxel 400, the calculation unit 114 calculates the average value of the feature amounts of the points to which the largest number of identical instrument point labels are assigned. The average value may be, for example, an arithmetic mean or a geometric mean, or may be a centroid value or a median value (the concept of "average value" in this disclosure includes a centroid value and a median value). An example of the feature amount may be the coordinates of the point. A method for setting feature points for each point will be described later.

[0026] In the example shown in Figure 2(A), in the upper left voxel 400A, there are two points 300A indicated by "solid color" labeled with instrument point A, and one point 300B indicated by "shaded" labeled with instrument point B. In voxel 400A, there are the most points 300A indicated by the instrument point A label, so the average value of the feature quantities of each of the two points 300A is calculated.

[0027] The replacement unit 115 replaces all of the multiple points in each voxel 400 with one point 310 of the average value calculated by the calculation unit 114, and assigns the one point 310 with the instrument point label specified when calculating the one point 310. For each voxel 400, the replacement unit 115 replaces all of the multiple points 300 in that voxel 400 with one point 310 of the average value calculated in that voxel 400 as described above. The replacement unit 115 assigns the most common identical instrument point label in the voxel 400 to the replaced one point 310 of the average value. In other words, the replacement unit 115 assigns to the replaced one point 310 of the average value the instrument point label of the point used when calculating the average value.

[0028] In the example shown in Fig. 2(B), in the upper left voxel 400A, the average value of the feature amounts of the two points 300A is calculated by the calculation unit 114 described above, and the voxel is replaced with the point (point 310 (310A)) that is the average value. Point 310A is also labeled as instrument point A (point 310A is shown as "blank" in the example shown in Fig. 2(B)). Similarly, for voxel 400B shown in the lower left of FIG. 2(B), once the average value of the features of each of the three points 300B (see FIG. 2(A)) has been calculated, the voxel is replaced with a point (point 310 (310B)) that is the average value, and is labeled as instrument point B (point 310B is shown with a diagonal line in the example of FIG. 2(B)).

[0029] The output control unit 116 may control the output unit to output the voxel 400, the point 310 of the average value in the voxel 400, and the instrument point label assigned to the average value (point 310). The voxel 400, the point 310 of the average value in the voxel 400, and the instrument point label assigned to the average value (point 310) may be collectively referred to as the "average value." The output unit may be, for example, the communication unit 131, the storage unit 132, the display unit 133, etc. That is, the output control unit 116 may control the communication unit 131 to transmit information on the average value to an external device (not shown), for example. The external device may be a server, a user terminal, or the like. The output control unit 116 may, for example, control the storage unit 132 to store information on the average value. The output control unit 116 may, for example, control the display unit 133 to display the average value.

[0030] (Feature setting function) Next, the function of setting the above-mentioned feature amount will be described. There are two methods for setting features, each of which is explained below.

[0031] (First feature setting method) First, the method for setting the first feature amount will be described. FIG. 4 is a first diagram for explaining a method for setting the first feature amount. FIG. 5 is a second diagram for explaining the method of setting the first feature amount.

[0032] The first identifying unit 117 identifies which of the multiple instrument points 200 the points acquired correspond to, based on the point cloud information acquired by the acquiring unit 111. The point cloud information may include, for example, information on the position (coordinates) of each point based on the instrument point 200 and the distance from the instrument point 200 to each point. The point cloud information may also include information indicating the relationship between the instrument point 200 and the points acquired at the instrument point 200 (which point was acquired by which instrument point 200). Therefore, the first identifying unit 117 may identify which instrument point 200 acquired the point cloud (each of the multiple points) included in the point cloud information, based on the point cloud information.

[0033] In the example shown in Fig. 4, there are two instrument points 200, namely, instrument point 200A shown as "plain" and instrument point 200B shown as "shaded". In addition, for the points 300 (300A, 300B) acquired at each of the multiple instrument points 200A, 200B, point 300A acquired at instrument point 200A is shown as "plain", and point 300B acquired at instrument point 200B is shown as "shaded". Note that the number of instrument points 200 and points 300 is not limited to the example shown in Fig. 4, and may be any other number.

[0034] The first setting unit 119 acquires the coordinate values ​​(angle θ and distance r) of the spherical coordinate system of the point 300 corresponding to the instrument point 200 identified by the first identification unit 117, based on each of the multiple instrument points 200, sets the coordinate values ​​(angle θ and distance r) as the feature amount of the point 300, and provides the feature amount to the calculation unit 114. The coordinates (coordinate values) indicating the position of the point group (each of the multiple points) recorded in the point group information may be coordinates (coordinate values) expressed in a spherical coordinate system based on the instrument point 200 that acquired the point 300. The first setting unit 119 acquires the coordinates (coordinate values ​​of the spherical coordinate system) of each point 300 based on the point group information. The first setting unit 119 may set the coordinates (coordinate values ​​of the spherical coordinate system) of the point 300 acquired as described above as the feature amount of the point 300.

[0035] In the example shown in Figure 4, the spherical coordinate system is a coordinate system based on each of the multiple instrument points 200A, 200B, and is expressed by the angle θ based on a specific axis S, and the distance r between each of the instrument points 200A, 200B and its corresponding points 300A, 300B.

[0036] Although the above description using FIG. 4 is a two-dimensional description, it may be expanded to three dimensions as exemplified in FIG. That is, as shown in an example in Figure 5, the coordinate values ​​of the three-dimensional spherical coordinates (angles θ, φ and distance r) are calculated for each instrument point 200A, 200B based on a specific axis S, and the coordinate values ​​(angles θ, φ and distance r) calculated with respect to the specific axis S may be set as the feature values ​​of that point 300.

[0037] (Second feature setting method) Next, a method for setting the second feature amount will be described. FIG. 6 is a first diagram for explaining the method for setting the second feature amount. FIG. 7 is a second diagram for explaining the method for setting the second feature amount.

[0038] The second identification unit 118 identifies one of the multiple points in the data after replacement by the replacement unit 115 as the query 320, and also identifies each of the multiple points adjacent to the query 320 as a neighboring point 330. The query 320 may be, for example, any one of the multiple points. The neighboring point 330 may be, for example, a point in the vicinity of the query 320 (for example, within a predetermined distance from the query 320). Therefore, the neighboring point 330 is not limited to being adjacent to the query 320, and may be a point at a position relatively distant from the query 320 (at a position in the vicinity of the query 320 described above).

[0039] The second setting unit 120 sets the instrument point label assigned by the replacement unit 115 for each of the multiple neighboring points 330. and applies the coordinate values ​​of the instrument point 200 in a spherical coordinate system based on the query 320 as a feature value of the neighboring point 330. The second identification unit 118 identifies the instrument point 200 that has acquired each of the multiple neighboring points 330 based on the point cloud information. In addition, by using a known calculation, the coordinate values ​​of the spherical coordinate system and the coordinate values ​​of the Cartesian coordinate system (XYZ coordinate system) can be converted into each other. For example, when the coordinates of the point group (each of the multiple points) included in the point group information are recorded in the spherical coordinate system, the second setting unit 120 may convert them into the Cartesian coordinate system. The second identification unit 118 calculates the difference between the coordinate values ​​(XYZ coordinates) in the Cartesian coordinate system of the instrument point 200 from which each of the multiple neighboring points 330 was obtained and the coordinate values ​​(XYZ coordinates) in the Cartesian coordinate system of the query 320, for each of the multiple neighboring points 330. The second identification unit 118 further converts the difference (coordinate values) obtained as described above into a spherical coordinate system to obtain a value (coordinate value). The value becomes the position (coordinate value) of the instrument point 200 in the spherical coordinate system for each of the multiple neighboring points 330, with the query 320 as the reference. The second identification unit 118 sets the value (coordinate value) obtained as described above as the feature amount of the neighboring point 330 (point 310).

[0040] In the example shown in Fig. 6, there are two instrument points 200, namely instrument point 200A shown as "plain" and instrument point 200B shown as "shaded". In addition, with regard to the points acquired at each of the multiple instrument points 200, the point acquired at instrument point 200A (neighboring point 330A) is shown as "plain", and the point acquired at instrument point 200B (neighboring point 330B) is shown as "shaded". Note that the number of instrument points 200 and points (neighboring points 330) is not limited to the case shown in Fig. 6 as an example, and may be other numbers.

[0041] As described above, when the second identification unit 118 regards the coordinate values ​​of the instrument point 200 in the spherical coordinate system based on the query 320 as the feature of the nearby point 330 (point 310), it takes the difference in the Cartesian coordinate system and converts the difference into the spherical coordinate system, and as a result, the angle θ of the instrument point 200 based on a specific axis S in the query 320 and the distance r between the query 320 and the instrument point 200 become the feature values ​​(θ, r) of the nearby point 330 (point 310). In the example shown in Fig. 6, the neighboring point 330 (330A) (neighboring point 1, 2) is the point 300 (310A) corresponding to the instrument point 200A, and the feature amount is represented by (θA, rA). Here, θA indicates the angle of the instrument point 200A with respect to the query 320 based on the predetermined axis S, and rA indicates the distance between the query 320 and the instrument point 200A. Similarly, in the example shown in Fig. 6, the neighboring point 330 (330B) (neighboring points 3 to 5) is the point 310 (310B) corresponding to the instrument point 200B, and the feature amount is represented by (θB, rB). Here, θB indicates the angle of the instrument point 200B relative to the query 320 based on the predetermined axis S, and rB indicates the distance between the query 320 and the instrument point 200B.

[0042] The contents described above using Fig. 6 are two-dimensional descriptions, but may be expanded to three dimensions as exemplified in Fig. 7. That is, Fig. 7 may also be expressed three-dimensionally, taking into consideration instrument points 200A, 200B and neighboring point 330, as in Fig. 6. Although neighboring point 330 is not shown in Fig. 7, neighboring point 330 (330A) corresponds to neighboring points 1 and 2 showing the characteristic amounts in Fig. 7, and neighboring point 330 (330B) corresponds to neighboring points 3 to 5 showing the characteristic amounts in Fig. 7. That is, as shown in an example in Fig. 7, the neighboring point 330 (330A) (not shown in Fig. 7) (neighboring points 1 and 2) is the point 300 (310A) corresponding to the instrument point 200A, and the feature amount is expressed as (θA, φA, rA). Here, θA and φA indicate the angle of the instrument point 200A relative to the query 320 based on the predetermined axis S, and rA indicates the distance between the query 320 and the instrument point 200A. Similarly, in the example shown in Fig. 7, the neighboring point 330 (330B) (not shown in Fig. 7) (neighboring points 3 to 5) is the point 310 (310B) corresponding to the instrument point 200B, and the feature amount is expressed by (θB, φB, rB). Here, θB and φB indicate the angle of the instrument point 200B relative to the query 320 based on the predetermined axis S, and rB indicates the distance between the query 320 and the instrument point 200B.

[0043] Note that a normalization process may be performed on each of the feature amounts (two feature amounts) in each of the first and second feature amount setting methods described above.

[0044] Here, normalization may be a process of converting large data values ​​into data within a certain range. To explain normalization, for example, normalization may be a process of converting data values ​​between 0 and 100 into data values ​​between 0 and 1. The definition of the values ​​is as follows: That is, let the N values ​​to be normalized be X = {xn|n = 0, ..., N}, and the nth value be x n In this case, "x n "The maximum value that x can take," n The minimum value that x can take is max , x min Here, x min , x max , x n x min ≦x n ≦x max Let X'={x'n|n=0,…,N} be the N values ​​after normalization of X, and let the nth normalized value be x' n In this case, the "normalized maximum value" and the "normalized minimum value" are x' max , x' min Here, x' is defined asmin ≦x' n ≦x' max x max , x min , x' max , x' min can be set to any value as long as each constraint is satisfied. The definition of normalization is: x' n =((x n -x min ) / d)×(x' max -x' min )+x' min Here, the denominator is the range d=x max -x min It is.

[0045] In the formula defining the above normalization, when applying it to each feature (two feature) in each of the setting methods described above, the target and range d to be normalized are as follows (1) and (2).

[0046] (1) For the spherical coordinates of the instrument point coordinate system (first feature setting method), the object of normalization is the radius of the spherical coordinates, and the range d of the radius for each instrument point is used for normalization. In other words, as a first specific example, the object to be normalized is the radius of the spherical coordinate system, and 0 is set to the minimum value x min The maximum radius value that the sensor at instrument point 200 can take is the maximum value x max may be used for normalization as As a second specific example, the objects to be normalized are two angles and a radius of the spherical coordinates, and the range d of each value may be as follows (i) or (ii). (i) The two angles may each be normalized to a range d that the angles can take (for example, 2π). (ii) Set the radius to 0 and set the minimum value x min The maximum radius value that the sensor at instrument point 200 can take is the maximum value x max Normalization may be performed as follows.

[0047] (2) For the spherical coordinates of the instrument point position from the query (the second feature setting method), the object of normalization is the radius of the transformed spherical coordinates, and the minimum value x min The radius of the closest instrument point from the query, the maximum value x max is used as the radius vector of the instrument point farthest from the query for normalization. That is, as a specific example, the objects to be normalized are two angles and a radius of the transformed spherical coordinates, and the range d of each value may be as follows (i) or (ii). (i) The two angles may each be normalized to a range d that the angles can take (for example, 2π). (ii) For the radius, the minimum value x min Let x be the radius of the closest instrument point from the query, and the maximum value x max Normalization may be performed by taking the radius vector of the instrument point farthest from the query.

[0048] [Information processing method] Next, an information processing method according to an embodiment will be described. FIG. 8 is a flowchart illustrating an information processing method according to an embodiment.

[0049] In step ST101, the acquisition unit 111 acquires point cloud information by outputting light or sound waves from each of the multiple instrument points 200 and obtaining the light or sound waves reflected from an object at the instrument point 200.

[0050] In step ST102, the dividing unit 112 divides a three-dimensional space including a plurality of points recorded in the point cloud information acquired in step ST101 into voxels 400.

[0051] In step ST103, the assignment unit 113 assigns to each of the multiple voxels 400 divided in step ST102 an instrument point label indicating which of the multiple instrument points 200 the points in each voxel 400 correspond to when the points were acquired.

[0052] In step ST104, the control unit 110 sets a feature amount for each of the multiple points. The first identification unit 117 identifies which of the multiple instrument points 200 the point 300 corresponds to, based on the point cloud information acquired in step ST101. The first setting unit 119 acquires the coordinate values ​​of a spherical coordinate system of the point 300 corresponding to the instrument point 200 identified by the first identification unit 117, based on each of the multiple instrument points 200, sets the coordinate values ​​as the feature amount of the point 300, and provides the feature amount to step ST105 described later.

[0053] In step ST105, the calculation unit 114 identifies the point to which the same instrument point label is most frequently assigned among the multiple points in the voxel 400 in step ST103, and calculates the average value of the feature amounts of the points. The calculation unit 114 may use, as the feature amount, a feature amount set using one of the setting methods of the first example and the second example of step ST104.

[0054] In step ST106, the replacement unit 115 replaces all of the multiple points 300 in each voxel 400 with one point 310 that is the average value calculated in step ST105, and assigns to the one point 310 the instrument point label identified when calculating the one point 310.

[0055] In step ST107, the second identification unit 118 identifies one of the multiple points 310 in the data after replacement in step ST106 as a query 320, and identifies each of the multiple points 310 adjacent to the query 320 as a neighboring point 330. The second setting unit 120 identifies, for each of the multiple neighboring points 330, an instrument point 200 corresponding to the neighboring point 330 based on the instrument point label assigned in step ST106, and applies the coordinate value of the instrument point 200 in the spherical coordinate system based on the query 320 as a feature value of the point 310 (330) as the neighboring point 330.

[0056] After step ST107, the output control unit 116 may control the output unit to output the voxel 400, one point 310 of the average value within the voxel 400, the instrument point label assigned to the average value (point 310), and the feature amount.

[0057] [Functions and circuits] Next, the functions and circuits of the above-mentioned information processing device 100 will be described. Each unit of the information processing device 100 may be realized as a function of a computer arithmetic processing device or the like. That is, the acquisition unit 111, division unit 112, assignment unit 113, calculation unit 114, replacement unit 115, output control unit 116, first determination unit 117, second determination unit 118, first setting unit 119, and second setting unit 120 (control unit 110) of the information processing device 100 may be realized as an acquisition function, division function, assignment function, calculation function, replacement function, output control function, first determination function, second determination function, first setting function, and second setting function (control function) by the computer arithmetic processing device or the like. The information processing program can cause a computer to realize each of the above-mentioned functions. The information processing program may be recorded in a non-transitory computer-readable recording medium, such as a memory, a solid-state drive, a hard disk drive, or an optical disk. The recording medium may be referred to as a non-transitory computer-readable medium, for example. As described above, each unit of the information processing device 100 may be realized by an arithmetic processing device of a computer or the like. The arithmetic processing device or the like is configured by, for example, an integrated circuit or the like. Therefore, each unit of the information processing device 100 may be realized as a circuit constituting the arithmetic processing device or the like. That is, the acquisition unit 111, the division unit 112, the assignment unit 113, the calculation unit 114, the replacement unit 115, the output control unit 116, the first determination unit 117, the second determination unit 118, the first setting unit 119, and the second setting unit 120 (control unit 110) of the information processing device 100 may be realized as an acquisition circuit, a division circuit, an assignment circuit, a calculation circuit, a replacement circuit, an output control circuit, a first determination circuit, a second determination circuit, a first setting circuit, and a second setting circuit (control circuits) constituting the arithmetic processing device of a computer or the like. The communication unit 131, the storage unit 132, and the display unit 133 (output unit) of the information processing device 100 may be realized as a communication function, a storage function, and a display function (output function) including the functions of an arithmetic processing device, etc. The communication unit 131, the storage unit 132, and the display unit 133 (output unit) of the information processing device 100 may be realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured, for example, by an integrated circuit, etc. The communication unit 131, the storage unit 132, and the display unit 133 (output unit) of the information processing device 100 may be realized as a communication device, a storage device, and a display device (output device) by being configured, for example, by being configured by a plurality of devices.

[0058] The information processing device 100 can combine one or any two or more of the above-mentioned units. In this disclosure, the term "information" is used, but the term "information" can be replaced with "data" and the term "data" can be replaced with "information."

[0059] [Aspects and Effects of the Present Embodiment] Next, one aspect of this embodiment and the effects of each aspect will be described. Note that each aspect described below is an example at the time of filing, and this embodiment is not limited to the aspects described below. In other words, this embodiment is not limited to each aspect described below, and may be realized by appropriately combining each of the above-mentioned parts. In addition, a lower aspect may be able to cite any of the higher aspects. The effects described below are merely examples, and the effects of each aspect are not limited to those described below. Each aspect may, for example, have at least one of the effects described below.

[0060] (Aspect 1) An information processing device of one embodiment includes an acquisition unit that acquires point cloud information by outputting light or sound waves from each of a plurality of instrument points and obtaining light or sound waves reflected from an object at the instrument points; a division unit that divides a three-dimensional space including a plurality of points recorded in the point cloud information acquired by the acquisition unit into voxels; an assignment unit that assigns an instrument point label to each of the plurality of points in each of the plurality of voxels divided by the division unit, indicating which of the plurality of instrument points the point corresponded to when it was acquired; a calculation unit that identifies the point among the plurality of points in the voxel that is assigned the most identical instrument point label and calculates the average value of the features of the points; and a replacement unit that replaces all of the plurality of points in each voxel with one point of the average value calculated by the calculation unit, and assigns the one point with the instrument point label identified when calculating the one point. The information processing device can obtain a representative of the points included in the point cloud, that is, one average value within each voxel. As a result, when performing various calculations such as deep learning and machine learning, the information processing device can perform the calculations using one average value in each voxel (multiple average values ​​for the entire point cloud), which can reduce the processing load of the calculations and shorten the calculation time compared to when performing the calculations using all the points that make up the point cloud. In other words, the information processing device can relatively reduce the calculation cost.

[0061] In addition, the information processing device becomes able to make estimations about point clouds based on point cloud information in accordance with the above-mentioned learning contents, based on a trained model that has learned the point clouds and various estimation contents related to the point clouds, such as teacher data and labels, the average value replaced by the replacement unit, and the label assigned to the average value. In this case, various estimation contents may be used, for example, noise and objects contained in the point cloud may be estimated. A specific example of the processing contents may be as follows.

[0062] That is, the information processing device (first estimation unit) may estimate noise contained in a point cloud based on point cloud information based on a trained model (first trained model) that has learned the point cloud and the noise of the point cloud, the average value replaced by the replacement unit, and the label assigned to the average value. Furthermore, the information processing device (removal unit) may remove the noise points (average values) estimated by the first estimation unit described above from the average values ​​(multiple average values ​​because there are multiple voxels) replaced by the replacement unit. In addition, the information processing device (second estimation unit) may estimate objects included in the point cloud based on a trained model (second trained model) that has learned about the point cloud and objects included in the point cloud (for example, various objects including buildings, etc.), and the average value from which noise has been removed by the above-mentioned removal unit (multiple average values ​​because there are multiple voxels).

[0063] Alternatively, the information processing device (third estimation unit) may estimate objects included in a point cloud based on point cloud information based on a trained model (third trained model) that has learned the point cloud and the objects included in the point cloud, the average value replaced by the replacement unit, and the label assigned to the average value.

[0064] (Aspect 2) An information processing device of one embodiment may include a first identification unit that identifies which of a plurality of instrument points the acquired point corresponds to based on point cloud information acquired by the acquisition unit, and a first setting unit that acquires coordinate values ​​in a spherical coordinate system of a point corresponding to the instrument point identified by the first identification unit based on each of the plurality of instrument points, sets the coordinate values ​​as features of the point, and provides the features to a calculation unit. This allows the information processing device to obtain the feature values ​​of each of the multiple points included in the point cloud information. Since the feature values ​​are coordinate values, the information processing device can use the coordinate values ​​(feature values) obtained by the point cloud measurement to perform various calculations, such as estimation calculations using the above-mentioned learning (deep learning, machine learning, etc.).

[0065] (Aspect 3) An information processing device of one embodiment includes a second identification unit that identifies one of a plurality of points in the data after replacement by the replacement unit as a query, and each of a plurality of points adjacent to the query as neighboring points, and a second setting unit that identifies, for each of the plurality of neighboring points, an instrument point corresponding to the neighboring point based on the instrument point label assigned by the replacement unit, and applies the coordinate values ​​of the instrument point in a spherical coordinate system based on the query as features of the point as the neighboring point. This allows the information processing device to obtain the feature values ​​of each of the multiple points included in the point cloud information. Since the feature values ​​are coordinate values, the information processing device can perform various calculations, such as the estimation calculation using the learning described above, by using the coordinate values ​​(feature values) obtained by the point cloud measurement.

[0066] (Aspect 4) In one embodiment of the information processing method, a computer executes the following steps: an acquisition step in which a computer acquires point cloud information by outputting light or sound waves from each of a plurality of instrument points and obtaining light or sound waves reflected from an object at the instrument point; a division step in which a three-dimensional space including the plurality of points recorded in the point cloud information acquired by the acquisition step is divided into voxels; an assignment step in which, in each of the plurality of voxels divided by the division step, an instrument point label is assigned to each of the plurality of points in each voxel indicating which of the plurality of instrument points the voxel corresponded to when the point was acquired; a calculation step in which the point that is most frequently assigned the same instrument point label among the plurality of points in the voxel is identified and the average value of the features of the points is calculated; and a replacement step in which the computer replaces all of the plurality of points in each voxel with one point with the average value calculated by the calculation step, and assigns the one point with the instrument point label identified when calculating the one point. As a result, the information processing method can achieve the same effects as the information processing device according to the above-described aspect.

[0067] (Aspect 5) An information processing program of one embodiment provides a computer with the following functions: an acquisition function for acquiring point cloud information by outputting light or sound waves from each of a plurality of instrument points and obtaining light or sound waves reflected from an object at the instrument point; a division function for dividing a three-dimensional space including a plurality of points recorded in the point cloud information acquired by the acquisition function into voxels; an assignment function for assigning an instrument point label to each of the plurality of points in each of the plurality of voxels divided by the division function, indicating which of the plurality of instrument points the point corresponded to when it was acquired; a calculation function for identifying the point among the plurality of points in the voxel that is assigned the most identical instrument point label and calculating the average value of the features of that point; and a replacement function for replacing all of the plurality of points in each voxel with one point of the average value calculated by the calculation function, and assigning the one point with the instrument point label identified when calculating the one point. As a result, the information processing program can achieve the same effects as the information processing device according to the above-described aspect. [Explanation of symbols]

[0068] 100 Information processing device 110 Control section 111 Acquisition Department 112 Division 113 Granting Department 114 Calculation section 115 Substitution part 116 Output control section 117 First Specific Part 118 Second Specific Part 119 First Setting Section 120 Second Setting Section 131 Communications Department 132 Storage section 133 Display section 200 instrument stations 300(300A,300B) points (point cloud) 310(310A,310B) Average point 320 queries 330(330A,330B) Nearby points 400 Voxels

Claims

1. an acquisition unit that acquires point cloud information by outputting light or sound waves from each of a plurality of instrument points and acquiring the light or sound waves reflected from an object at the instrument points; a division unit that divides a three-dimensional space including a plurality of points recorded in the point cloud information acquired by the acquisition unit into voxels; an assigning unit that assigns, to each of the plurality of voxels divided by the dividing unit, an instrument point label indicating which of the plurality of instrument points the point in each of the plurality of voxels corresponds to when the point was acquired; A calculation unit that identifies a point that is assigned the most identical instrument point labels among a plurality of points in the voxel, and calculates an average value of feature amounts of the identified points; a replacement unit that replaces all of the multiple points in each of the voxels with one point of the average value calculated by the calculation unit, and assigns an instrument point label to the one point that was specified when calculating the one point; An information processing device comprising:

2. A first identification unit that identifies a point corresponding to one of the plurality of instrument points based on the point cloud information acquired by the acquisition unit; a first setting unit that acquires coordinate values ​​of a spherical coordinate system of a point corresponding to the instrument point identified by the first identification unit based on each of the plurality of instrument points, sets the coordinate values ​​as feature values ​​of the point, and provides the feature values ​​to a calculation unit; The information processing device according to claim 1 .

3. a second identification unit that identifies one of a plurality of points in the data after the replacement by the replacement unit as a query, and identifies each of a plurality of points adjacent to the query as a neighboring point; a second setting unit that identifies an instrument point corresponding to each of the plurality of neighboring points based on the instrument point label assigned by the replacement unit, and applies the coordinate values ​​of the instrument point in a spherical coordinate system based on the query as feature values ​​of the neighboring point; The information processing device according to claim 1 or 2, comprising:

4. The computer an acquisition step of acquiring point cloud information by outputting light or sound waves from each of a plurality of instrument points and acquiring the light or sound waves reflected from an object at the instrument points; a division step of dividing a three-dimensional space including a plurality of points recorded in the point cloud information acquired by the acquisition step into voxels; an assignment step of assigning an instrument point label to each of the plurality of points in each of the plurality of voxels divided by the division step, the instrument point label indicating which of the plurality of instrument points the point corresponds to when the point was acquired; A calculation step of identifying a point to which the most identical instrument point labels are assigned among a plurality of points in the voxel, and calculating an average value of the feature amounts of the identified points; a replacement step of replacing all of the points in each of the voxels with one point of the average value calculated by the calculation step, and assigning the instrument point label specified when calculating the one point to the one point; An information processing method for performing the above.

5. On the computer, An acquisition function for acquiring point cloud information by outputting light or sound waves from each of a plurality of instrument points and acquiring the light or sound waves reflected from an object at the instrument points; A division function that divides a three-dimensional space including a plurality of points recorded in the point cloud information acquired by the acquisition function into voxels; an assignment function for assigning an instrument point label to each of a plurality of points in each of a plurality of voxels divided by the division function, the instrument point label indicating which of a plurality of instrument points the point corresponds to when the point was acquired; A calculation function for identifying the point to which the most identical instrument point labels are assigned among the multiple points in the voxel, and calculating the average value of the feature amounts of the identified points; A replacement function that replaces all of the multiple points in each of the voxels with one point of the average value calculated by the calculation function, and assigns the instrument point label specified when calculating the one point to the one point; An information processing program that realizes this.

Images