Information processing device, control method, program, and storage medium

Abstract

To provide an information processing device capable of preferably estimating a reflection characteristic.SOLUTION: A controller 43 of a reflection characteristic estimation device 4 acquires point group data D1 measured at a plurality of irradiation positions by a lidar 2 for performing measurement by receiving reflection light of irradiated light. Then, the controller 43 generates integrated point group data D3 obtained by associating the irradiation positions with information on reflection intensity at each measurement point. Then, the controller 43 estimates a reflection characteristic representing a relation between an irradiation angle and the reflection intensity in each cluster of measurement points shown by the integrated point group data D3.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a technique for estimating reflection characteristics. [Background technology]

[0002] Conventionally, there have been known techniques for detecting features and the like present around roads. For example, Patent Document 1 discloses a technique for performing image segmentation using semantic segmentation based on an image captured by a camera and reflection characteristics included in point cloud data output by a lidar. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-156973 Summary of the Invention [Problem to be solved by the invention]

[0004] The reflection characteristics of objects around the road vary, and there are retroreflective materials that have a uniform reflection intensity regardless of the irradiation angle, and materials that have a low reflection intensity in a specific direction, etc. Therefore, when accurately estimating the reflection characteristics of an object, it is necessary to take the irradiation direction into consideration.

[0005] The present invention has been made to solve the above-mentioned problems, and a main object of the present invention is to provide an information processing device that can suitably estimate reflection characteristics. [Means for solving the problem]

[0006] The claimed invention is an acquisition means for acquiring point cloud data measured at a plurality of irradiation positions by a measurement device that performs measurement by irradiating light and receiving reflected light; an integration processing means for generating integrated point cloud data by integrating the point cloud data into a common coordinate system, in which information about the irradiation position and reflection intensity is associated with each measurement point; a clustering processing means for performing clustering based on the distances between the measurement points constituting the integrated point cloud data, so that neighboring measurement points are grouped into the same cluster; a reflection characteristic estimation means for estimating a reflection characteristic representing a relationship between an irradiation angle and a reflection intensity for each cluster of measurement points represented by the integrated point cloud data generated by the clustering; a calculation means for calculating a reflection intensity for each of the measurement points obtained when the viewpoint is set as a virtual irradiation position of the measurement device, based on the reflection characteristics estimated by the reflection characteristic estimation means and viewpoint information that specifies a viewpoint; The information processing device has the following.

[0007] The claimed invention also includes: Acquire point cloud data measured at a plurality of irradiation positions using a measurement device that performs measurements by receiving reflected light of irradiated light; generating integrated point cloud data by integrating the point cloud data into a common coordinate system, in which information about the irradiation position and reflection intensity is associated with each measurement point; clustering the adjacent measurement points into the same cluster based on the distances between the measurement points that make up the integrated point cloud data; For each cluster of measurement points represented by the integrated point cloud data generated by the clustering, a reflection characteristic representing the relationship between the irradiation angle and the reflection intensity is estimated. death, calculating a reflection intensity for each of the measurement points obtained when the viewpoint is set as a virtual irradiation position of the measurement device, based on the estimated reflection characteristics and viewpoint information that specifies a viewpoint; It is a control method.

[0008] The claimed invention also includes: Acquire point cloud data measured at a plurality of irradiation positions using a measurement device that performs measurements by receiving reflected light of irradiated light; generating integrated point cloud data by integrating the point cloud data into a common coordinate system, in which information about the irradiation position and reflection intensity is associated with each measurement point; clustering the adjacent measurement points into the same cluster based on the distances between the measurement points that make up the integrated point cloud data; For each cluster of measurement points represented by the integrated point cloud data generated by the clustering, a reflection characteristic representing the relationship between the irradiation angle and the reflection intensity is estimated. death, Based on the estimated reflection characteristics and viewpoint information specifying a viewpoint, the reflection intensity for each measurement point obtained when the viewpoint is set as a virtual irradiation position of the measurement device is calculated. It is a program that causes a computer to execute the process. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows a schematic configuration of a reflection characteristic estimation system. [Figure 2] FIG. 2 is a block diagram showing a functional configuration of the reflection characteristic estimation device. [Figure 3] FIG. 2 shows a functional block diagram of a controller of the reflection characteristic estimation device. [Figure 4] (A) A diagram showing each measurement point of the point cloud data measured when the lidar was located at position P1. (B) A diagram showing each measurement point of the point cloud data measured when the lidar was located at position P2. (C) A diagram showing each measurement point of the integrated point cloud data. [Figure 5] (A) shows the data structure of each measurement point shown in Figure 4(A). (B) shows the data structure of each measurement point shown in Figure 4(B). (C) shows the data structure of each measurement point shown in Figure 4(C). [Figure 6] 10A is a diagram showing the relationship between measurement points obtained for an object 63 and the lidar irradiation position, and FIG. 10B is a diagram showing the relationship between measurement points obtained for an object 64 and the lidar irradiation position. [Figure 7] 4 is a flowchart illustrating an example of a processing procedure executed by a controller in the first embodiment. [Figure 8] FIG. 10 is a functional block diagram of a controller according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] According to a preferred embodiment of the present invention, an information processing device includes: an acquisition unit that acquires point cloud data measured at a plurality of irradiation positions by a measurement device that performs measurements by receiving reflected light of irradiated light; an integration processing unit that generates integrated point cloud data by integrating the point cloud data, in which information related to the irradiation position and reflection intensity is associated with each measurement point; and a reflection characteristic estimation unit that estimates reflection characteristics that represent the relationship between the irradiation angle and reflection intensity for each cluster of measurement points indicated by the integrated point cloud data. With this aspect, the information processing device can preferably estimate the reflection characteristics of an object represented by a cluster of measurement points based on data obtained by integrating the point cloud data measured by the measurement device.

[0011] In one aspect of the information processing device, the information processing device further includes a physical property estimation means for estimating a physical property corresponding to each of the clusters based on the reflection property estimated by the reflection property estimation means. With this aspect, the information processing device can preferably estimate the physical property corresponding to each cluster.

[0012] In another aspect of the information processing device, the physical property estimation means estimates a category related to physical properties to which each of the clusters belongs, based on the reflection properties estimated by the reflection property estimation means and known physical property information representing reflection properties for each category of different physical properties. With this aspect, the information processing device can suitably estimate a category related to physical properties corresponding to each cluster.

[0013] In another aspect of the information processing device, the information processing device further includes a clustering processing unit that performs clustering to group nearby measurement points into the same cluster based on the distances between the measurement points that make up the integrated point cloud data. With this aspect, the information processing device can preferably form a cluster for each object.

[0014] In another aspect of the information processing device, the clustering processing means performs the clustering based on the distance and the reflection intensity. With this aspect, the information processing device can more accurately generate clusters for each object by taking the reflection intensity into consideration.

[0015] In another aspect of the information processing device, the information processing device further includes a physical property estimation means for estimating physical properties corresponding to each of the clusters based on the reflection properties estimated by the reflection property estimation means, and the clustering processing means divides the clusters based on the estimation results of the physical property estimation means and the reflection intensities. With this aspect, when a plurality of objects with different physical properties are clustered into one cluster due to their proximity to each other, the information processing device can suitably divide the cluster into each object.

[0016] In another aspect of the information processing device, the reflection characteristic estimating means calculates the irradiation angle for each of the measurement points based on the irradiation position and the measurement position indicated by the measurement point, and estimates the reflection characteristic based on the calculated irradiation angle and the reflection intensity. With this aspect, the information processing device can preferably estimate the reflection characteristic taking the irradiation angle into consideration.

[0017] In another aspect of the information processing device, the integration processing means generates the integrated point cloud data by converting the point cloud data measured at a plurality of irradiation positions into a common coordinate system based on movement information of the measurement device when measuring the point cloud data. This aspect allows the information processing device to preferably generate the integrated point cloud data.

[0018] According to another preferred embodiment of the present invention, there is provided a control method executed by a computer, which acquires point cloud data measured at a plurality of irradiation positions by a measurement device that performs measurements by irradiating light and receiving reflected light, generates integrated point cloud data by integrating the point cloud data in which information about the irradiation position and reflection intensity is associated with each measurement point, and estimates reflection characteristics representing the relationship between irradiation angle and reflection intensity for each cluster of measurement points represented by the integrated point cloud data. By executing this control method, the computer can suitably estimate the reflection characteristics of an object represented by the cluster of measurement points.

[0019] According to another preferred embodiment of the present invention, a program causes a computer to acquire point cloud data measured at multiple irradiation positions by a measurement device that performs measurements by irradiating light and receiving reflected light, generate integrated point cloud data by integrating the point cloud data in which information about the irradiation position and reflection intensity is associated with each measurement point, and estimate reflection characteristics that represent the relationship between irradiation angle and reflection intensity for each cluster of measurement points indicated by the integrated point cloud data. By executing this program, the computer can preferably estimate the reflection characteristics of an object represented by the cluster of measurement points. Preferably, the program is stored in a storage medium. [Example]

[0020] Preferred embodiments of the present invention will now be described with reference to the drawings.

[0021] <First Example> (1) System Overview Fig. 1 is a schematic diagram of a reflection characteristic estimation system according to Example 1. The reflection characteristic estimation system shown in Fig. 1 is a system that estimates the reflection characteristics of features present around a road based on measurement data generated by a measurement vehicle traveling on the road, and mainly includes a measurement vehicle and a reflection characteristic estimation device 4.

[0022] The measurement vehicle mainly includes an on-board device 1, a Lidar (Light Detection and Ranging or Laser Illuminated Detection and Ranging) 2, and a sensor unit 3.

[0023] The LIDAR (range sensor) 2 emits a pulsed laser beam over a predetermined angular range in the horizontal and vertical directions to discretely measure the distance to an object in the external world and generate point cloud information consisting of measurement points representing the object's three-dimensional position and reflection intensity. In this case, the LIDAR 2 has an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives the reflected (scattered) light of the irradiated laser light reflected by the object, and an output unit that outputs scan data based on the light receiving signal output by the light receiving unit. The scan data is point cloud data and is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the distance to the object in that irradiation direction, which is determined based on the above-mentioned light receiving signal. Hereinafter, each point constituting the point cloud data will also be referred to as a "measurement point." The LIDAR 2 may be installed at any position on the measurement vehicle, or multiple LIDARs 2 may be installed on the measurement vehicle. Date and time data indicating the measurement date and time (generation date and time) is added to the point cloud data.

[0024] The sensor unit 3 is a group of sensors that generate movement information that indicates the time-series position and orientation (direction of travel) of the LIDAR 2. The sensor unit 3 includes sensors such as a GPS receiver, an acceleration sensor, a gyro sensor, and an IMU (Inertial Measurement Unit). The GPS receiver may generate highly accurate position information that indicates the absolute position of the measurement vehicle (e.g., a three-dimensional position of latitude, longitude, and altitude) based on an RTK positioning method (i.e., an interferometric positioning method). The sensor unit 3 may also be a sensor provided in the LIDAR 2 so as to directly detect the position and orientation of the LIDAR 2. Date and time data indicating the date and time of generation is added to the output data of the sensor unit 3.

[0025] The vehicle-mounted device 1 is electrically connected to the LIDAR 2 and the sensor unit 3 via a wired or wireless connection, and stores data generated by the LIDAR 2 and the sensor unit 3. The vehicle-mounted device 1 is also capable of data communication with the reflection property estimation device 4, and transmits measurement data "Im" based on output data from the LIDAR 2 and the sensor unit 3 to the reflection property estimation device 4 at a predetermined timing. Specifically, the vehicle-mounted device 1 transmits the measurement data Im, which includes point cloud data measured by the LIDAR 2 at multiple irradiation positions and movement information of the LIDAR 2 based on the output of the sensor unit 3, to the reflection property estimation device 4. The vehicle-mounted device 1 may be configured as a drive recorder together with the sensor unit 3.

[0026] The vehicle-mounted device 1 may perform processing to appropriately correct the data output from the LIDAR 2 and the sensor unit 3. For example, the vehicle-mounted device 1 may detect a difference between the reference times in the LIDAR 2 and the sensor unit 3, and correct at least one of the date and time data included in the point cloud data and the movement information based on the time difference so that the date and time data included in the point cloud data and the movement information are synchronized.

[0027] The reflection characteristic estimation device 4 receives measurement data Im including point cloud data and movement information of the lidar 2 from the vehicle-mounted device 1, and performs processing such as estimating the reflection characteristics of the features measured by the lidar 2 based on the received measurement data Im.

[0028] The configuration of the reflection characteristic estimation system shown in FIG. 1 is an example, and various modifications may be made to the configuration shown in FIG. 1. For example, instead of acquiring the measurement data Im through data communication with the on-vehicle device 1, the reflection characteristic estimation device 4 may acquire the measurement data Im by reading the measurement data Im stored in a storage medium by the on-vehicle device 1 from the storage medium. In this case, the storage medium is electrically connected to the on-vehicle device 1 during measurement by the measurement vehicle, so that the on-vehicle device 1 writes the measurement data Im. After measurement by the measurement vehicle, the storage medium is electrically connected to the reflection characteristic estimation device 4, so that the measurement data Im is read by the reflection characteristic estimation device 4. Alternatively, the lidar 2 and the sensor unit 3 may each store log data generated in a storage medium, and the storage medium may be read by the reflection characteristic estimation device 4 to supply the measurement data Im to the reflection characteristic estimation device 4. In this case, the reflection characteristic estimation system does not need to include the on-vehicle device 1. The reflection characteristic estimation device 4 may be composed of multiple devices. In this case, the reflection characteristic estimation device 4 includes a plurality of devices that execute pre-assigned processes and exchange necessary data between the devices.

[0029] (2) Configuration of the reflection characteristics estimation device 2 is a block diagram showing the functional configuration of the reflection characteristic estimation device 4. The reflection characteristic estimation device 4 has an interface 41, a memory 42, and a controller 43. These elements are connected to each other via a bus line.

[0030] The interface 41 performs interface operations related to the exchange of data between the reflection characteristic estimation device 4 and an external device. In this embodiment, the interface 41 receives the measurement data Im generated by the vehicle-mounted device 1. The interface 41 may be a wireless interface such as a network adapter for wireless communication with the vehicle-mounted device 1, or may be a hardware interface for reading the measurement data Im from a storage medium or the like that stores the measurement data Im. The interface 41 may also perform interface operations with various peripheral devices such as an input device, a display device, and a sound output device.

[0031] The memory 42 is configured by various types of volatile and non-volatile memory, such as a random access memory (RAM), a read only memory (ROM), a hard disk drive, a flash memory, etc. The memory 42 stores a program for the controller 43 to execute predetermined processes. The program executed by the controller 43 may be stored in a storage medium other than the memory 42.

[0032] The memory 42 also stores point cloud data D1, movement information D2, integrated point cloud data D3, cluster information D4, reflection property information D5, known physical property information D6, and physical property estimation information D7.

[0033] The point cloud data D1 is point cloud data sequentially measured by the vehicle-mounted device 1 while changing the irradiation position. The movement information D2 is movement information that indicates the position of the lidar 2 at each measurement time of the point cloud data D1. The point cloud data D1 and the movement information D2 are accumulated based on the measurement data Im received from the vehicle-mounted device 1.

[0034] The integrated point cloud data D3 is point cloud data obtained by integrating (integrating) the point cloud data D1 using a common coordinate system. As will be described later, information about the irradiation position of the lidar 2 (i.e., the measurement position where the measurement light is emitted) is added to each measurement point of the integrated point cloud data D3.

[0035] The cluster information D4 is information for identifying each cluster (group) obtained by performing a clustering process on the integrated point cloud data represented by the integrated point cloud data D3. The cluster information D4 may be a cluster identification label added to each measurement point of the integrated point cloud data D3. In this case, the cluster information D4 is incorporated into the integrated point cloud data D3.

[0036] The reflection property information D5 is information representing the reflection property estimated for each cluster by the reflection property estimation device 4. As will be described later, the reflection property information D5 is information representing the relationship between the irradiation angle and the reflection intensity. The known physical property information D6 is information relating to the reflection property measured in advance for each object. The physical property estimation information D7 is information relating to the estimation result of the physical property for each cluster estimated by the reflection property estimation device 4 based on the known physical property information D6.

[0037] At least one of the point cloud data D1, movement information D2, integrated point cloud data D3, cluster information D4, reflection property information D5, known physical property information D6, and physical property estimation information D7 may be stored in a storage device external to the reflection property estimation device 4, such as a hard disk connected to the reflection property estimation device 4 via the interface 41. The storage device may be a server device that communicates with the reflection property estimation device 4. The storage device may also be composed of multiple devices.

[0038] The controller 43 includes one or more processors, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit), and controls the entire reflection property estimation device 4. In this case, the controller 43 performs processing related to reflection property estimation by executing programs stored in the memory 42 or the like. For example, the controller 43 stores the point cloud data and movement information of the lidar 2 included in the measurement data Im received from the vehicle-mounted device 1 via the interface 41 as point cloud data D1 and movement information D2, respectively, in the memory 42. The controller 43 also generates integrated point cloud data D3, cluster information D4, reflection property information D5, and physical property estimation information D7 based on the point cloud data D1, movement information D2, known physical property information D6, and stores these in the memory 42. The controller 43 functions as an "acquisition means," an "integration processing means," a "clustering processing means," a "reflection property estimation means," a "physical property estimation means," a computer that executes programs, and the like.

[0039] The processes executed by the controller 43 are not limited to being realized by software programs, but may be realized by any combination of hardware, firmware, and software. The processes executed by the controller 43 may also be realized by a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcomputer. In this case, the program executed by the controller 43 in this embodiment may be realized using this integrated circuit. Thus, the controller 43 may be realized by various hardware. Furthermore, the functions of the controller 43 may be realized by the cooperation of multiple computers, for example, using cloud computing technology.

[0040] (3) Estimation of reflectance and physical properties Next, we will explain the estimation process of reflection characteristics and physical properties executed by the reflection characteristic estimation device 4. In summary, the reflection characteristic estimation device 4 generates integrated point cloud data D3 including information on the illumination position of the lidar 2 based on the point cloud data D1 and movement information D2, and estimates the reflection characteristics and corresponding physical properties taking into account the illumination angle for each cluster configured by the integrated point cloud data D3. In this way, the reflection characteristic estimation device 4 accurately estimates the reflection characteristics and physical properties taking into account differences in reflection intensity according to the illumination angle.

[0041] (3-1) Functional Blocks Fig. 3 is a block diagram showing the functional configuration of the controller 43 of the reflection characteristic estimation device 4 in this embodiment. As shown in Fig. 3, the controller 43 functionally includes an integration processing unit 51, a clustering processing unit 52, a reflection characteristic estimation unit 53, and a physical property estimation unit 54. Note that in Fig. 3, blocks where data is exchanged are connected by solid lines, but the combination of blocks where data is exchanged is not limited to that shown in Fig. 3. The same applies to other functional block diagrams described later.

[0042] The integration processing unit 51 generates integrated point cloud data D3 including information on the irradiation position for each measurement point, based on the point cloud data D1 and movement information D2 stored in the memory 42 on the basis of the measurement data Im supplied from the vehicle-mounted device 1. The clustering processing unit 52 performs clustering on the integrated point cloud data D3, and generates cluster information D4 representing clusters (groups) of each measurement point that constitutes the integrated point cloud data D3.

[0043] The reflection property estimation unit 53 estimates the reflection property for each cluster of the integrated point cloud data D3 specified based on the cluster information D4, taking into account the irradiation angle, and generates reflection property information D5 representing the reflection property for each cluster. The physical property estimation unit 54 estimates the physical properties of the object corresponding to each cluster based on the reflection property information D5 and known physical property information D6, and generates the estimated physical property results as physical property estimation information D7.

[0044] (3-2) Details of the integration processing unit The integration processing unit 51 generates integrated point cloud data D3 including information on the irradiation position for each measurement point based on the point cloud data D1 and the movement information D2. Hereinafter, the method for generating the integrated point cloud data D3 by the integration processing unit 51 and the details of the data structure of the integrated point cloud data D3 will be described with reference to Figures 4(A) to (C) and 5(A) to (C).

[0045] FIG. 4(A) is a diagram showing each measurement point of the point cloud data measured when the lidar 2 is located at position "P1" in a coordinate system with the illumination position of the lidar 2 as the origin (here, O(0,0,0)). FIG. 4(B) is a diagram showing each measurement point of the point cloud data measured when the lidar 2 is located at position "P2" transitioned from position P1 in a coordinate system with the illumination position of the lidar 2 as the origin (here, O(0,0,0)). FIG. 4(C) is a diagram showing each measurement point of the integrated point cloud data D3 obtained by integrating each point cloud data shown in FIGS. 4(A) and 4(B). Here, each measurement point shown in FIGS. 4(A) to 4(C) is shaded based on the measured reflection intensity, with darker shading indicating lower reflection intensity.

[0046] In addition, Figure 5(A) shows the data structure of each measurement point shown in Figure 4(A), Figure 5(B) shows the data structure of each measurement point shown in Figure 4(B), and Figure 5(C) shows the data structure of each measurement point shown in Figure 4(C).

[0047] As shown in Figures 4(A) and 4(B), each piece of point cloud data before being integrated is represented by a coordinate system (also referred to as a "lidar coordinate system") based on the lidar 2 at the time of measurement, with the irradiation position as the origin (here, O(0,0,0)). Also, as shown in Figures 5(A) and 5(B), each measurement point of each piece of point cloud data before being integrated includes three-dimensional coordinate values ​​(X coordinate value, Y coordinate value, Z coordinate value) with the irradiation position as the origin, and reflection intensity. Note that the point cloud data shown in Figure 4(A) includes measurement points A to C, and the point cloud data shown in Figure 4(B) includes measurement points D to F.

[0048] 4(C), the integrated point cloud data D3 is data that represents measurement points measured at different irradiation positions in a common coordinate system (also referred to as an "integrated coordinate system"). For example, the integrated point cloud data D3 uses a coordinate system (e.g., a coordinate system based on latitude, longitude, and altitude) that represents the position of the lidar 2 in the movement information D2 as the integrated coordinate system, and becomes data that uniformly represents measurement points measured at different irradiation positions. In this case, the integration processing unit 51 converts the coordinate values ​​of each lidar coordinate system of the measurement points of each point cloud data before integration into coordinate values ​​of the integrated coordinate system using information on the position and orientation of the lidar 2 at the time of measurement that is indicated by the movement information D2.

[0049] 5(C), the integration processing unit 51 generates integrated point cloud data D3 in which each measurement point is associated with a three-dimensional coordinate value in the integration coordinate system, a reflection intensity, and an irradiation position. In this case, for each measurement point, the integration processing unit 51 associates the reflection intensity included in the measurement point before integration and the irradiation position of the LIDAR 2 used for the conversion to the integration coordinate system with the three-dimensional coordinate value in the integration coordinate system converted from the LIDAR coordinate system. As a result, the integration processing unit 51 generates integrated point cloud data D3 including information on the irradiation position, and causes the reflection characteristic estimation unit 53 to preferably perform a process of estimating reflection characteristics that takes into account the irradiation angle based on the integrated point cloud data D3.

[0050] (3-3) Details of the clustering processing unit The clustering processing unit 52 performs clustering to group nearby measurement points into the same cluster based on the distances between the measurement points that make up the integrated point cloud data D3, and generates cluster information D4 that identifies the cluster (group) of each measurement point. As a result, the clustering processing unit 52 appropriately sets a cluster for each object in the integrated point cloud data D3.

[0051] For example, in the integrated point cloud data D3 shown in Fig. 4(C), the clustering processing unit 52 regards the group of measurement points within the dashed frame 61 as a first cluster, and the group of measurement points including measurement points A to F within the dashed frame 62 as a second cluster. Then, the clustering processing unit 52 generates cluster information D4 indicating the cluster to which each measurement point belongs.

[0052] Here, a specific aspect of clustering will be described.

[0053] In the first clustering mode, the clustering processing unit 52 performs clustering based on the three-dimensional coordinate values ​​of each measurement point constituting the integrated point cloud data D3, such that measurement points that are close to each other in distance (distance in the integrated coordinate system) are grouped into the same cluster. In this case, the clustering processing unit 52 may perform clustering by employing any clustering method (hierarchical cluster analysis, non-hierarchical cluster analysis).

[0054] In a second clustering mode, the clustering processing unit 52 performs clustering based on both the three-dimensional coordinate values ​​and the reflection intensity of each measurement point constituting the integrated point cloud data D3. In this case, the clustering processing unit 52 performs clustering based on, for example, the distance between measurement points in a four-dimensional coordinate system obtained by adding a coordinate axis of reflection intensity to an integrated coordinate system, which is a three-dimensional coordinate system. In another example, after performing provisional clustering based on the first mode, the clustering processing unit 52 further performs grouping based on reflection intensity for each provisional cluster. Then, the clustering processing unit 52 sets each set group as a final cluster.

[0055] After performing the initial clustering, the clustering processing unit 52 may correct the clustering using the physical property estimation information D7 generated by the physical property estimation unit 54 based on the results of the clustering. For example, for each cluster based on the results of the initial clustering, the clustering processing unit 52 compares the reflection intensity of each measurement point belonging to the target cluster with the estimation result of the physical property estimation unit 54 for the target cluster. Then, if there is a cluster of measurement points whose physical properties (reflection characteristics) differ from those estimated by the physical property estimation unit 54, the clustering processing unit 52 regards the cluster of measurement points as a separate cluster and divides the target cluster. Thereafter, for the divided clusters, the reflection characteristics estimation unit 53 and the physical properties estimation unit 54 perform estimation of the reflection characteristics.

[0056] In this way, the clustering processing unit 52 may regard points whose reflection characteristics are clearly different from those of nearby measurement points as belonging to a different cluster. This makes it possible to suitably separate, for example, tree branches that cover signs from the sign cluster and perform estimation of reflection characteristics and physical properties.

[0057] (3-4) Details of the reflection characteristics estimation unit The reflection characteristic estimation unit 53 estimates the reflection characteristic for each cluster of the integrated point cloud data D3 identified based on the cluster information D4. In this case, the reflection characteristic estimation unit 53 first calculates pairs of irradiation positions and reflection intensities for all measurement points in the integrated point cloud data D3. Then, the reflection characteristic estimation unit 53 tallies the calculated pairs of irradiation positions and reflection intensities for each cluster, and generates reflection characteristic information D5 based on the tallied results for each cluster.

[0058] Here, a specific example of a method for estimating the irradiation angle of each measurement point will be described. For example, the reflection characteristic estimation unit 53 calculates an approximation plane using three or more measurement points for a cluster to which the target measurement point belongs. Then, the reflection characteristic estimation unit 53 calculates, as the irradiation angle, the angle formed by the calculated approximation plane and a vector connecting the three-dimensional coordinate value indicated by the target measurement point and the irradiation position. In another example, the reflection characteristic estimation unit 53 may calculate a normal vector for the target measurement point, and calculate, as the irradiation angle, the angle formed by the calculated normal vector and a vector connecting the three-dimensional coordinate position indicated by the target measurement point and the irradiation position.

[0059] The reflection characteristic estimation unit 53 may quantize the irradiation angle using a predetermined quantization number and calculate the reflection intensity for each quantized irradiation angle. In this case, if there are multiple measurement points with the same irradiation angle within the same cluster, the reflection characteristic estimation unit 53 calculates a representative value (average, median, etc.) of the reflection intensity at these multiple measurement points. The reflection characteristic estimation unit 53 can preferably generate reflection characteristic information D5 representing the relationship between the irradiation angle and the reflection intensity for each cluster. The reflection characteristic information D5 may be graph information representing the relationship between the irradiation angle and the reflection intensity for each cluster. The reflection characteristic estimation unit 53 may also generate reflection characteristic information D5 by interpolating reflection intensity values ​​for which no irradiation angle exists using any interpolation process. The reflection characteristic estimation unit 53 is not limited to the above quantization and may also calculate, as the reflection characteristic information D5, a function that approximately represents the relationship between the irradiation angle and the reflection intensity. In this case, any function approximation method may be used.

[0060] (3-5) Details of the physical property estimation section The physical property estimation unit 54 estimates the physical properties of the object corresponding to each cluster based on the reflection property information D5 and the known physical property information D6, and generates the estimated physical property results as physical property estimation information D7.

[0061] Here, the known physical property information D6 is, for example, information on reflection characteristics that indicates the relationship between the irradiation angle and reflection intensity for each category of object present around the road (signboards, trees, vehicles, etc.) or for each category of material of the object (for example, retroreflective material, wood, water, iron, other metals, etc.). In another example, the known physical property information D6 may be information on reflection characteristics that indicates the relationship between the irradiation angle and reflection intensity for each category of different physical properties (for example, first physical property, second physical property, etc.). In this way, the known physical property information D6 is information on reflection characteristics for each category of object, material, physical property, etc. (also referred to as "existing physical property category").

[0062] The physical property estimation unit 54 then estimates an existing physical property category corresponding to each cluster based on, for example, a comparison between the reflection characteristic for each existing physical property category registered in the known physical property information D6 and the reflection characteristic for each cluster indicated by the reflection property information D5. The physical property estimation unit 54 then generates information associating the existing physical property category estimated for each cluster with the corresponding cluster as physical property estimation information D7.

[0063] Here, an example of a method for estimating an existing physical property category will be described. For example, the physical property estimation unit 54 calculates the similarity of the reflection characteristics of each cluster with each existing physical property category registered in the known physical property information D6. This similarity calculation method may be any similarity calculation method using a cross-correlation function, distance in a feature space, or the like. Then, the physical property estimation unit 54 determines the existing physical property category with the highest similarity for each cluster. Then, the physical property estimation unit 54 regards the determined existing physical property category as representing the physical property of the object represented by the target cluster, and generates physical property estimation information D7 that associates the determined existing physical property category with the target cluster.

[0064] Fig. 6(A) is a diagram showing the relationship between the measurement points obtained for an object 63 and the irradiation position of the LIDAR 2. Fig. 6(B) is a diagram showing the relationship between the measurement points obtained for an object 64 and the irradiation position of the LIDAR 2. In Figs. 6(A) and 6(B), the higher the reflection intensity, the lighter the measurement point is displayed.

[0065] 6(A), in the case of object 63, measurement points are generated that exhibit a substantially uniform reflection intensity regardless of the irradiation position and irradiation angle. Therefore, in this case, for example, the cluster corresponding to object 63 has the highest similarity in reflection characteristics to the existing physical property category corresponding to a retroreflective material or a sign made from such a material. Therefore, in this case, the physical property estimation unit 54 generates physical property estimation information D7 indicating that the cluster corresponding to object 63 belongs to the existing physical property category corresponding to a retroreflective material or a sign made from such a material.

[0066] 6(B), in the case of object 64, the reflection intensity is highest from the front direction, and the reflection intensity decreases as the irradiation angle deviates from the front direction of object 64. Therefore, in this case, for example, the cluster corresponding to object 63 has the highest similarity in reflection characteristics to the existing physical property category corresponding to a material with weak reflection in the side direction or an object made from that material. Therefore, in this case, the physical property estimation unit 54 generates physical property estimation information D7 indicating that the cluster corresponding to object 64 belongs to the existing physical property category corresponding to a material with weak reflection in the side direction or an object made from that material.

[0067] In this way, the physical property estimation unit 54 can estimate the reflection characteristics for each cluster, taking into account the irradiation angle, thereby making it possible to perform highly accurate physical property estimation.

[0068] (4) Processing Flow Fig. 7 is an example of a flowchart executed by the reflection characteristic estimation device 4 in the first embodiment. The reflection characteristic estimation device 4 executes the processing of the flowchart shown in Fig. 7 for each predetermined road section, for example.

[0069] The reflection characteristic estimation device 4 acquires point cloud data D1 and movement information D2 (step S11). In this case, the reflection characteristic estimation device 4 receives measurement data Im from the vehicle-mounted device 1, for example, and acquires point cloud data D1 and movement information D2 based on the received measurement data Im.

[0070] Next, based on the point cloud data D1 and the movement information D2, the reflection property estimation device 4 generates integrated point cloud data D3 including information on the irradiation position of the lidar 2 (step S12). In this case, the integration processing unit 51 of the reflection property estimation device 4 converts the point cloud data D1 expressed in the lidar coordinate system into an integrated coordinate system based on the movement information D2 and generates integrated point cloud data D3.

[0071] Then, the reflection characteristic estimation device 4 performs clustering of the measurement points of the integrated point cloud data D3 (step S13). In this case, the clustering processing unit 52 of the reflection characteristic estimation device 4 performs clustering such that measurement points that are close to each other in the integrated coordinate system or a predetermined feature space are grouped into the same cluster.

[0072] Next, the reflection characteristic estimation device 4 estimates reflection characteristics that represent the relationship between the irradiation angle and the reflection intensity for each cluster identified in step S13 (step S14). In this case, the reflection characteristic estimation unit 53 of the reflection characteristic estimation device 4 estimates the reflection characteristic for each cluster by aggregating, for each cluster, the pairs of irradiation angle and reflection intensity calculated for each measurement point of the integrated point cloud data D3.

[0073] Next, the reflection property estimation device 4 estimates physical properties for each cluster based on the reflection properties estimated in step S14 (step S15). In this case, the physical property estimation unit 54 of the reflection property estimation device 4 estimates the existing physical property category that best fits each cluster based on the comparison result between the known physical property information D6 and the reflection properties for each cluster estimated in step S14.

[0074] As described above, the controller 43 of the reflection property estimation device 4 in the first embodiment acquires point cloud data D1 measured at multiple irradiation positions by the LIDAR 2, which performs measurements by irradiating light and receiving reflected light. The controller 43 then generates integrated point cloud data D3 that associates information related to the irradiation position and reflection intensity for each measurement point. The controller 43 then estimates reflection properties that represent the relationship between the irradiation angle and reflection intensity for each cluster of measurement points indicated by the integrated point cloud data D3. This allows the reflection property estimation device 4 to suitably estimate the reflection properties of an object measured by the LIDAR 2, taking the irradiation angle into consideration.

[0075] <Second Example> In the second embodiment, in addition to the processing in the first embodiment, the reflection characteristic estimation device 4 calculates the physical property estimation information D7, and then simulates the reflection intensity at each measurement point of the integrated point cloud data D3 when the irradiation position is virtually changed.

[0076] 8 is an example of a functional block diagram of a reflection characteristic estimation device 4 in the second embodiment. In the second embodiment, the controller 43 of the reflection characteristic estimation device 4 functionally includes an integration processing unit 51, a clustering processing unit 52, a reflection characteristic estimation unit 53, a physical property estimation unit 54, and a reflection characteristic recalculation unit 55. Hereinafter, the same components as those in the first embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0077] Based on viewpoint information "D8," which is information specifying the irradiation position (herein also referred to as "viewpoint") of the virtual LIDAR 2, the reflection characteristic recalculator 55 calculates the reflection intensity at each measurement point obtained when the viewpoint is set as the irradiation position. In this case, the reflection characteristic recalculator 55 may set a measurable range based on the maximum measuring distance and field of view of the LIDAR 2, and calculate the reflection intensity for each measurement point present within the measurable range. Then, the reflection characteristic recalculator 55 outputs information indicating the reflection intensity calculated for each measurement point as reflection characteristic output information "D9."

[0078] In this case, viewpoint information D8 is information specified by an input signal generated by an input device such as a mouse, keyboard, or voice input device connected to interface 41. In this case, reflection property recalculation unit 55 may display a GUI (Graphical User Interface) representing a space or the like for which a viewpoint is to be specified on a display or the like connected by interface 41, and may receive an input specifying the viewpoint.

[0079] Here, a method for calculating the reflection intensity of each measurement point will be described. First, the reflection property recalculation unit 55 uses a geometric method to calculate the irradiation angle of the target measurement point when the specified viewpoint is the irradiation position. The reflection property recalculation unit 55 also references the physical property estimation information D7 to recognize the physical properties of the cluster to which the measurement point for which the reflection intensity is to be recalculated belongs (i.e., the reflection property corresponding to the irradiation angle). The reflection property recalculation unit 55 then recognizes the reflection intensity corresponding to the calculated irradiation angle based on the reflection property indicated in the physical property estimation information D7 of the target cluster. The reflection property recalculation unit 55 may also recognize the reflection property for each irradiation angle of the target cluster by further referencing known physical property information D6 linked to an existing physical property category in the physical property estimation information D7.

[0080] Thereafter, the reflection characteristic recalculation unit 55 may, for example, display the calculated reflection characteristic output information D9 on a display connected to the interface 41, or may transmit it to a terminal device that performs a predetermined display based on the reflection characteristic output information D9.

[0081] In this way, according to the second embodiment, it is possible to suitably simulate the reflection intensity at the measurement point obtained for the specified viewpoint.

[0082] In each of the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a controller or the like that is a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)).

[0083] Although the present invention has been described above with reference to the examples, the present invention is not limited to the above examples. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art in accordance with the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and other documents are incorporated herein by reference. [Explanation of symbols]

[0084] 1 On-vehicle device 2 Rider 3 Sensor Unit 4 Reflection characteristics estimation device 41 Interface 42 memory 43 Controller

Claims

1. an acquisition means for acquiring point cloud data measured at a plurality of irradiation positions by a measurement device that performs measurement by irradiating light and receiving reflected light; an integration processing means for generating integrated point cloud data by integrating the point cloud data into a common coordinate system, in which information about the irradiation position and reflection intensity is associated with each measurement point; a clustering processing means for performing clustering based on the distances between the measurement points constituting the integrated point cloud data, so that neighboring measurement points are grouped into the same cluster; a reflection characteristic estimation means for estimating a reflection characteristic representing a relationship between an irradiation angle and a reflection intensity for each cluster of measurement points represented by the integrated point cloud data generated by the clustering; a calculation means for calculating a reflection intensity for each of the measurement points obtained when the viewpoint is set as a virtual irradiation position of the measurement device, based on the reflection characteristics estimated by the reflection characteristic estimation means and viewpoint information that specifies a viewpoint; An information processing device having the above.

2. The information processing apparatus according to claim 1 , further comprising: physical property estimation means for estimating physical properties corresponding to each of the clusters based on the reflection properties estimated by the reflection property estimation means.

3. 3. The information processing device according to claim 2, wherein the physical property estimation means estimates a category related to physical properties to which each of the clusters belongs based on the reflection properties estimated by the reflection property estimation means and known physical property information representing reflection properties for each category having different physical properties.

4. 4. The information processing apparatus according to claim 1, wherein the clustering processing means performs the clustering based on the distance and the reflection intensity.

5. The information processing device according to any one of claims 1 to 4, wherein the clustering processing means divides the clusters based on the reflection intensity and the estimation results of the physical properties corresponding to each of the clusters based on the reflection characteristics estimated by the reflection characteristic estimation means.

6. The information processing device according to any one of claims 1 to 5, wherein the reflection characteristic estimation means calculates the irradiation angle for each of the measurement points based on the irradiation position and the measurement position indicated by the measurement point, and estimates the reflection characteristic based on the calculated irradiation angle and the reflection intensity.

7. The information processing device according to any one of claims 1 to 6, wherein the integrated processing means generates the integrated point cloud data by converting the point cloud data measured at multiple irradiation positions into the common coordinate system based on movement information of the measurement device when measuring the point cloud data.

8. A computer-implemented control method comprising: Acquire point cloud data measured at a plurality of irradiation positions using a measurement device that performs measurements by receiving reflected light of irradiated light; generating integrated point cloud data by integrating the point cloud data into a common coordinate system, in which information about the irradiation position and reflection intensity is associated with each measurement point; clustering the adjacent measurement points into the same cluster based on the distances between the measurement points that make up the integrated point cloud data; estimating a reflection characteristic representing a relationship between an irradiation angle and a reflection intensity for each cluster of measurement points represented by the integrated point cloud data generated by the clustering; calculating a reflection intensity for each of the measurement points obtained when the viewpoint is set as a virtual irradiation position of the measurement device, based on the estimated reflection characteristics and viewpoint information that specifies a viewpoint; Control method.

9. Acquire point cloud data measured at a plurality of irradiation positions using a measurement device that performs measurements by receiving reflected light of irradiated light; generating integrated point cloud data by integrating the point cloud data into a common coordinate system, in which information about the irradiation position and reflection intensity is associated with each measurement point; clustering the adjacent measurement points into the same cluster based on the distances between the measurement points that make up the integrated point cloud data; estimating a reflection characteristic representing a relationship between an irradiation angle and a reflection intensity for each cluster of measurement points represented by the integrated point cloud data generated by the clustering; A program that causes a computer to execute a process of calculating the reflection intensity for each measurement point obtained when the viewpoint is set as a virtual irradiation position of the measurement device, based on the estimated reflection characteristics and viewpoint information that specifies the viewpoint.

10. A storage medium storing the program according to claim 9.

Images