Information processing device, method, program, and storage medium

Abstract

To provide an information processing device capable of suitably performing collation between map data and measurement data.SOLUTION: An information processing device includes point cloud map data acquisition means, scan data acquisition means, scan data processing means, and collation means. The point cloud map data acquisition means acquires point cloud map data that is point cloud data whose information on reflection intensity is equal to or larger than a threshold value. The scan data acquisition means acquires scan data that is point cloud data measured by receiving reflected light of emitted light. The scan data processing means generates, from the scan data, high reflection intensity scan data whose reflection intensity is equal to or larger than a predetermined threshold value. The collation means performs collation between the point cloud map data and the high reflection intensity scan data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle attitude estimation technique. [Background technology]

[0002] Conventionally, there have been known techniques for estimating a vehicle's own position based on measurement data from a measurement unit such as a radar or a camera. For example, Patent Document 1 discloses a technique for estimating a vehicle's own position by matching the output of a measurement sensor with position information of a feature registered in advance on a map. Furthermore, Non-Patent Document 1 discloses a position estimation method that uses ICP (Iterative Closest Point) matching, combining two different ICP algorithms, a point-to-point algorithm and a point-to-plane algorithm. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-257742 [Non-patent literature]

[0004] [Non-Patent Document 1] S. Takai, H. Date, S. Kanai, Y. Niina, K. Oda, and T. Ikeda: Accurate registration of MMS point clouds of urban areas using trajectory, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5 / W2, pp. 277-282. 2013. Summary of the Invention [Problem to be solved by the invention]

[0005] In an environment where there are not many objects with distinctive 3D structures in the location where measurement data is acquired, there is little distinctive shape information in the acquired measurement data that is necessary for stable matching, making it difficult to match map data with measurement data with high accuracy. As a result, there is a problem in that errors in position and orientation occur in the process of estimating the position and orientation of the vehicle, etc.

[0006] The present disclosure has been made to solve the above-mentioned problems, and a main object of the present disclosure is to provide an information processing device that can suitably execute matching between map data and measurement data. [Means for solving the problem]

[0007] The claimed invention is a point cloud map data acquisition means for acquiring high reflection intensity map data, which is point cloud data representing points on a reference surface where the reflection intensity is equal to or greater than a threshold, and three-dimensional space map data, which is point cloud data of features existing in three-dimensional space; a scan data acquisition means for acquiring scan data, which is point cloud data measured by receiving reflected light of emitted light; a scan data processing means for generating, from the scan data, data representing points on a reference surface where the reflection intensity is equal to or greater than a predetermined threshold, as high reflection intensity scan data, and generating, as three-dimensional space scan data, the scan data or data obtained by applying a filter to the scan data; a comparison means for performing a first comparison between the three-dimensional space map data and the three-dimensional space scan data, and a second comparison between the high reflection intensity map data and the high reflection intensity scan data; The information processing device has the following.

[0008] The claimed invention also includes: 1. A computer-implemented method comprising: Acquire high-reflection-intensity map data, which is point cloud data representing points on a reference surface where the reflection intensity is equal to or greater than a threshold, and three-dimensional spatial map data, which is point cloud data of features existing in three-dimensional space; The system acquires scan data, which is point cloud data measured by receiving the reflected light of the emitted light, From the scan data, data representing points on a reference surface where the reflection intensity is equal to or greater than a predetermined threshold is generated as high reflection intensity scan data, and the scan data or data obtained by applying a filter to the scan data is generated as three-dimensional space scan data; This is a control method that performs a first comparison between the three-dimensional space map data and the three-dimensional space scan data, and a second comparison between the high reflection intensity map data and the high reflection intensity scan data.

[0009] The claimed invention also includes: Acquire high-reflection-intensity map data, which is point cloud data representing points on a reference surface where the reflection intensity is equal to or greater than a threshold, and three-dimensional spatial map data, which is point cloud data of features existing in three-dimensional space; The system acquires scan data, which is point cloud data measured by receiving the reflected light of the emitted light, From the scan data, data representing points on a reference surface where the reflection intensity is equal to or greater than a predetermined threshold is generated as high reflection intensity scan data, and the scan data or data obtained by applying a filter to the scan data is generated as three-dimensional space scan data; The program causes a computer to execute a process of first matching the three-dimensional space map data with the three-dimensional space scan data, and second matching the high reflection intensity map data with the high reflection intensity scan data. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic configuration diagram of a driving assistance system. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the vehicle-mounted device. [Figure 3]FIG. 2 is a functional block diagram of a control unit related to position estimation; [Figure 4] 10 is an example of a flowchart illustrating a procedure of a position estimation process. [Figure 5] 10 is an example of a flowchart showing a procedure of Combined-ICP processing. [Figure 6] 10 is an example of a flowchart showing a procedure of an ICP loop process. [Figure 7] FIG. 1 is a diagram illustrating an outline of scan data processing. [Figure 8] 10 is an example of a histogram used to set a reflection intensity threshold. [Figure 9] FIG. 2 is a diagram illustrating an outline of map data processing. [Figure 10] FIG. 1 is a perspective view showing points of point cloud map data used in a simulation plotted in a three-dimensional space. [Figure 11] FIG. 10 is a diagram showing a matching result in a comparative example. [Figure 12] FIG. 10 is a diagram showing a matching result obtained by the matching method according to the present embodiment. [Figure 13] FIG. 10 is a diagram showing the results of the second matching individually. [Figure 14] 10 is a graph showing the relationship between the number of iterations of ICP loop processing and squared error when the method of the embodiment and the method of the comparative example are executed. DETAILED DESCRIPTION OF THE INVENTION

[0011] According to a preferred embodiment of the present invention, an information processing device includes a point cloud map data acquisition unit that acquires point cloud map data, which is point cloud data with a reflection intensity equal to or greater than a threshold; a scan data acquisition unit that acquires scan data, which is point cloud data measured by receiving reflected light of emitted light; a scan data processing unit that generates high-reflection-intensity scan data, which has a reflection intensity equal to or greater than a predetermined threshold, from the scan data; and a comparison unit that compares the point cloud map data with the high-reflection-intensity scan data. The "threshold" for the "high-reflection-intensity scan data with a reflection intensity equal to or greater than the predetermined threshold" and the "threshold" for the "point cloud map data, which is point cloud data with a reflection intensity equal to or greater than the threshold," are set to optimal values ​​based on the reflection intensity characteristics of a measurement device, such as a lidar, used to acquire the point cloud map data and the reflection intensity characteristics of a measurement device, such as a lidar, used to create the point cloud map data, respectively. This aspect enables the information processing device to use scan data with high reflection intensity to effectively compare map data with scan data, even in an environment where there are not many objects with three-dimensional structures.

[0012] In one aspect of the information processing device, the point cloud map data acquisition means acquires, as the point cloud map data, high reflection intensity map data that is point cloud data including points on a reference surface where the reflection intensity is equal to or greater than the threshold, the scan data processing means generates, from the scan data, data including points on the reference surface where the reflection intensity is equal to or greater than the threshold as the high reflection intensity scan data, and the matching means matches the high reflection intensity map data with the high reflection intensity scan data. This aspect allows the information processing device to preferably perform matching using high reflection portions of the reference surface as a reference.

[0013] In another aspect of the information processing device, the matching means performs the matching based on a point-to-point algorithm. This aspect makes it possible to perform matching with high accuracy using a highly reflective portion of a reference surface as a reference.

[0014] In another aspect of the information processing device, the point cloud map data acquisition means acquires the high-reflection-intensity map data and 3D spatial map data, which is point cloud data of features existing in 3D space. The scan data processing means generates 3D spatial scan data from the scan data or data obtained by applying a filter to the scan data. The matching means performs a first matching of the 3D spatial map data with the 3D spatial scan data and a second matching of the high-reflection-intensity map data with the high-reflection-intensity scan data. This aspect allows the information processing device to preferably perform matching using scan data other than that of a reference plane. In a preferred example, the matching means performs the first matching based on a point-to-plane algorithm and the second matching based on a point-to-point algorithm.

[0015] In another aspect of the information processing device, the matching means weights the first matching and the second matching based on the number of points of the 3D spatial scan data to be matched in the first matching and the number of points of the high reflection intensity scan data to be matched in the second matching. With this aspect, the information processing device can obtain a matching result that appropriately combines the first matching and the second matching by weighting according to the number of points to be processed.

[0016] In another aspect of the information processing device, the reference surface includes a road surface, which allows the information processing device to suitably compare map data with scan data using road surface paint as a reference, even in an environment where there are not many objects with three-dimensional structures.

[0017] In another aspect of the information processing device, the scan data acquisition unit acquires the scan data measured by an external sensor that receives reflected light of emitted light. With this aspect, the information processing device can suitably acquire scan data including information on reflection intensity.

[0018] In another aspect of the information processing device, the matching unit calculates, through the matching, parameters for converting the coordinate system of the scan data into the coordinate system of the point cloud map data. By performing the above-mentioned matching with high accuracy, the information processing device can calculate, with high accuracy, parameters for converting the coordinate system of the scan data into the coordinate system of the point cloud map data.

[0019] In another aspect of the information processing device, the scan data processing means sets the threshold value based on a histogram of the reflection intensity of the scan data, allowing the information processing device to adaptively calculate the threshold value used to generate high reflection intensity scan data.

[0020] According to another preferred embodiment of the present invention, there is provided a control method executed by a computer, comprising: acquiring point cloud map data, which is point cloud data having a reflection intensity equal to or greater than a threshold; acquiring scan data, which is point cloud data measured by receiving reflected light of emitted light; generating high-reflection-intensity scan data from the scan data, which has a reflection intensity equal to or greater than a predetermined threshold; and comparing the point cloud map data with the high-reflection-intensity scan data. By executing this control method, an information processing device can preferably compare the map data with the scan data.

[0021] According to another preferred embodiment of the present invention, a program causes a computer to execute a process of acquiring point cloud map data, which is point cloud data having a reflection intensity equal to or greater than a threshold, acquiring scan data, which is point cloud data measured by receiving reflected light of emitted light, generating high reflection intensity scan data from the scan data, which has a reflection intensity equal to or greater than a predetermined threshold, and comparing the point cloud map data with the high reflection intensity scan data. By executing this program, the computer can preferably compare the map data with the scan data. Preferably, the program is stored in a storage medium. [Example]

[0022] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. For convenience, in this specification, any symbol followed by "^" will be represented as "A^" (where "A" is any character).

[0023] (1) Overview of the driving assistance system 1 shows a schematic configuration of a driving assistance system according to this embodiment. The driving assistance system includes an on-board device 1 that moves together with a vehicle, which is a moving body, and a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3.

[0024] The vehicle-mounted device 1 is electrically connected to external sensors such as a LIDAR 3 and internal sensors (not shown), and estimates a reference position (including attitude) in the driving assistance system based on the outputs of these sensors. The reference position may be the center position of the vehicle on which the vehicle-mounted device 1 is mounted, or the position of the LIDAR 3 used for position estimation. Based on the position estimation result, the vehicle-mounted device 1 performs driving assistance such as automatic driving control of the vehicle so that the vehicle travels along a route to a set destination. The vehicle-mounted device 1 stores point cloud map data 10, which is map data of a point cloud indicating the three-dimensional position and reflection intensity for each point measured on the surface of roads and features around the roads. The vehicle-mounted device 1 then compares this point cloud map data 10 with scan data, which is the measurement result by the LIDAR 3, to estimate the position.

[0025] The LIDAR 3 emits a pulsed laser beam over a predetermined angular range in the horizontal and vertical directions to discretely measure the distance to an object in the external world and generate scan data representing a three-dimensional point cloud that indicates the position of the object. In this case, the LIDAR 3 has an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data based on the light receiving signal output by the light receiving unit. The scan data is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light identified based on the above-mentioned light receiving signal. The scan data obtained by one scan corresponds to one frame of scan data.

[0026] The installation position of the LIDAR 3 is not limited to the position shown in Fig. 1, but may be any installation position that includes at least the road surface in its illumination range. Furthermore, multiple LIDARs 3 may be installed on a vehicle. The vehicle-mounted device 1 is an example of an "information processing device," and the LIDAR 3 is an example of an "external sensor."

[0027] (2) In-vehicle device configuration 2 is a block diagram showing the functional configuration of the vehicle-mounted device 1. The vehicle-mounted device 1 mainly includes an interface 11, a storage unit 12, a communication unit 13, an input unit 14, a control unit 15, and an information output unit 16. These elements are interconnected via a bus line.

[0028] The interface 11 functions as a hardware interface and supplies data output by the sensor group 2 to the control unit 15. The interface 11 also supplies signals related to vehicle driving control generated by the control unit 15 to an electronic control unit (ECU) of the vehicle.

[0029] The sensor group 2 is composed of various external and internal sensors required for position estimation. The sensor group 2 includes a lidar 3, a GPS receiver 4, a gyro sensor 5, and a vehicle speed sensor 6. Instead of or in addition to the gyro sensor 3, the sensor group 2 may include an inertial measurement unit (IMU) that measures the acceleration and angular velocity of the measurement vehicle in three axial directions. The sensor group 2 may also include external sensors other than the lidar 3 (for example, a camera).

[0030] The storage unit 12 stores programs executed by the control unit 15 and information required for the control unit 15 to execute predetermined processes. The storage unit 12 is configured with, for example, volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. In this embodiment, the storage unit 12 stores point cloud map data 10.

[0031] The point cloud map data 10 may be updated periodically. In this case, for example, the control unit 15 receives map data related to the area to which the vehicle's position belongs from a server device that manages map data via the communication unit 13, and reflects the received map data in the point cloud map data 10. The point cloud map data 10 may be stored in one or more server devices that can communicate with the in-vehicle device 1, instead of being stored in the memory unit 12. In this case, for example, the control unit 15 communicates with the server device via the communication unit 13, thereby acquiring point cloud map data around the vehicle's position that is required for processing at the required timing. The point cloud map data 10 may also be data generated by the in-vehicle device 1 using SLAM (Simultaneous Localization and Mapping).

[0032] The input unit 14 is a user interface such as a button, touch panel, remote controller, or voice input device for user operation, and accepts inputs to specify a destination for route search, inputs to specify whether autonomous driving is on or off, etc. The information output unit 16 is, for example, a display, a speaker, or the like that outputs information based on the control of the control unit 15.

[0033] The control unit 15 includes one or more processors, such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and controls the entire in-vehicle device 1 by executing programs stored in the storage unit 12, etc. The control unit 15 is an example of a "point cloud map data acquisition means," a "scan data acquisition means," a "scan data processing means," a "comparison means," and a "computer" that executes the programs. Note that at least some of these means are not limited to being realized by software programs, but may be realized by any combination of hardware, firmware, and software. Furthermore, at least some of these components may be realized by a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller.

[0034] (3) Location estimation Next, the position estimation performed by the control unit 15 of the vehicle-mounted device 1 by matching (collating) the point cloud map data 10 with the scan data obtained by the LIDAR 3 will be described.

[0035] (3-1) Overview In summary, the control unit 15 extracts scan data of three-dimensional space and scan data of road surfaces where the reflection intensity is equal to or greater than a predetermined threshold from the scan data output by the LIDAR 3, and matches each of these data with prepared map data. This allows the control unit 15 to stably perform matching for position estimation based on the scan data of road surface paint, even in an environment where no features with distinctive three-dimensional structures exist.

[0036] Hereinafter, the scan data of the three-dimensional space will be referred to as "3D space scan data Ds1," and the scan data of the road surface with a reflection intensity equal to or greater than a predetermined threshold will be referred to as "high reflection intensity scan data Ds2." Furthermore, the map data of the three-dimensional space extracted from the point cloud map data 10 to be used for matching with the 3D space scan data Ds1 will be referred to as "3D space map data Dm1," and the map data of the highly reflective portions of the road surface (i.e., road surface paint) extracted from the point cloud map data 10 to be used for matching with the high reflection intensity scan data Ds2 will be referred to as "high reflection intensity map data Dm2."

[0037] The control unit 15 also performs the above-described matching using ICP. ICP algorithms include a Point-to-Point (P2P) algorithm, which uses the degree of matching between points as an evaluation index, and a Point-to-Plane (P2PL) algorithm, which uses an evaluation index based on normal information. In this embodiment, the control unit 15 performs the above-described matching based on a "Combined ICP" algorithm, which combines the P2P and P2PL algorithms. Specifically, the control unit 15 uses a P2PL evaluation index when matching the 3D space scan data Ds1 with the 3D space map data Dm1 (also referred to as "first matching"), and a P2P evaluation index when matching the high reflection intensity scan data Ds2 with the high reflection intensity map data Dm2 (also referred to as "second matching"). Hereinafter, the data to be matched will be referred to as the "target," and the data to be matched will be referred to as the "source."

[0038] (3-2) Block diagram 3 is an example of a functional block diagram of the control unit 15 related to position estimation. Functionally, the control unit 15 includes a map data processing unit 20, a normal vector calculation unit 21, a first matching unit 22, a scan data processing unit 23, selection units 24 and 25, a second matching unit 26, an integrated simultaneous equations construction unit 27, a singular value decomposition unit 28, a conversion unit 29, a 3D space squared error calculation unit 30, a high reflection intensity squared error calculation unit 31, an adaptive weighting calculation unit 32, a weighting addition unit 33, a convergence determination unit 34, a transformation matrix update unit 35, and a switching unit 36.

[0039] The map data processing unit 20 generates 3D spatial map data Dm1 and high reflection intensity map data Dm2 required for matching in position estimation based on the point cloud map data 10. How the 3D spatial map data Dm1 and high reflection intensity map data Dm2 are generated will be described in detail in the section "(3-3) Data Processing."

[0040] The normal vector calculation unit 21 calculates the normal vectors required for the first matching for each point indicated by the 3D spatial map data Dm1. Note that the P2PL algorithm executed in the first matching is a method that uses normal information of the target, and in the first matching, the 3D spatial map data Dm1 corresponds to the target, and the 3D spatial scan data Ds1 corresponds to the source.

[0041] The first association unit 22 associates the 3D space map data Dm1 with the 3D space scan data Ds1. In this case, the first association unit 22 searches for the closest point (nearest point) in the 3D space map data Dm1 for each point included in the 3D space scan data Ds1. Then, the first association unit 22 supplies the 3D space map data Dm1 and the 3D space scan data Ds1 in which the points are associated with each other, as well as information on the normal vectors, to the integrated simultaneous equations constructor 27.

[0042] The scan data processing unit 23 performs predetermined processing on the scan data generated by the LIDAR 3 to generate 3D space scan data Ds1 and high reflection intensity scan data Ds2. In this case, the scan data processing unit 23 performs processing to generate the 3D space scan data Ds1 and high reflection intensity scan data Ds2 from the most recent frame of scan data obtained, for example, in accordance with the period (frame period) at which the LIDAR 3 performs scanning. The method for generating the 3D space scan data Ds1 and high reflection intensity scan data Ds2 will be described later.

[0043] Here, the scan data generated by the LIDAR 3 is in a coordinate system based on the LIDAR 3. Therefore, the scan data processing unit 23 may provisionally convert the scan data into the map coordinate system adopted in the point cloud map data 10 based on, for example, a vehicle position provisionally estimated based on output signals from the sensor group 2 other than the LIDAR 3 and information such as the installation position of the LIDAR 3 stored in advance in the storage unit 12. In this case, the scan data processing unit 23 generates 3D spatial scan data Ds1 and high reflection intensity scan data Ds2 in the map coordinate system.

[0044] The selection unit 24 selects either the 3D space scan data Ds1 generated by the scan data processing unit 23 or the 3D space scan data Ds1 provisionally converted by the conversion unit 29 in the previous loop processing of the current scan data frame. Then, the selection unit 24 supplies the selected 3D space scan data Ds1 to the first association unit 22.

[0045] Specifically, when the loop count "i" of the ICP loop processing (also referred to as "ICP loop processing") targeting the frame of scan data to be processed is the first time (i=1), the selector 24 supplies the 3D space scan data Ds1 supplied from the scan data processor 23 to the first association unit 22. On the other hand, when the ICP loop processing is the second or subsequent time (i>1) and the termination condition of the ICP loop processing is not satisfied, the selector 24 supplies the 3D space scan data Ds1 converted by the converter 29 to the first association unit 22. In this case, the termination condition of the ICP loop processing may be that the ICP loop processing has been executed a predetermined number of times, or that the ICP loop processing has converged. The termination condition of the ICP loop processing is determined by a convergence determination unit 34, which will be described later.

[0046] The selector 25 selects either the high reflection intensity scan data Ds2 generated by the scan data processor 23 or the high reflection intensity scan data Ds2 provisionally converted by the converter 29 in the previous loop processing for the current scan data frame. The selector 25 then supplies the selected high reflection intensity scan data Ds2 to the second association unit 26. Specifically, when the number i of ICP loop processing is the first (i=1), the selector 25 supplies the high reflection intensity scan data Ds2 supplied from the scan data processor 23 to the second association unit 26. On the other hand, when the ICP loop processing is the second or subsequent iteration (i>1) and the termination condition for the ICP loop processing is not satisfied, the selector 25 supplies the high reflection intensity scan data Ds2 converted by the converter 29 to the second association unit 26.

[0047] The second association unit 26 associates the high reflection intensity map data Dm2 supplied from the map data processing unit 20 with the high reflection intensity scan data Ds2 supplied from the selection unit 25. In this case, the second association unit 26 searches for the closest point (nearest point) in the high reflection intensity map data Dm2 for each point included in the high reflection intensity scan data Ds2. The second association unit 26 then supplies the high reflection intensity map data Dm2 and the high reflection intensity scan data Ds2, which have been associated for each point, to the integrated simultaneous equations construction unit 27.

[0048] The integrated simultaneous equations constructing unit 27 constructs simultaneous equations required for the first matching and the second matching. The integrated simultaneous equations constructing unit 27 includes a first constructing unit 27X and a second constructing unit 27Y. The first constructing unit 27X constructs simultaneous equations related to the first matching (also referred to as the "first simultaneous equations") based on information supplied from the first association unit 22. The second constructing unit 27Y constructs simultaneous equations related to the second matching (also referred to as the "second simultaneous equations") based on information supplied from the second association unit 26. The integrated simultaneous equations constructing unit 27 then constructs simultaneous equations (also referred to as the "integrated simultaneous equations") by integrating the first simultaneous equations and the second simultaneous equations, and supplies the constructed integrated simultaneous equations to the singular value decomposition unit 28.

[0049] The singular value decomposition unit 28 calculates a solution to the integrated simultaneous equations supplied from the integrated simultaneous equations constructor 27 by singular value decomposition. The solution to the integrated simultaneous equations includes parameters of the transformation matrices to be applied to the 3D spatial scan data Ds1 and the high-reflection intensity scan data Ds2, respectively. This transformation matrix includes a rotation matrix "R" and a translation vector "T," hereinafter also referred to as "[RT]." Note that the transformation matrix calculated by the singular value decomposition unit 28 in each loop is a temporary transformation matrix. The transformation matrix calculated in one loop is referred to as the "temporary transformation matrix," and the rotation matrix and translation vector are referred to as "R_tmp" and "T_tmp," respectively.

[0050] The conversion unit 29 converts the 3D space scan data Ds1 and the high reflection intensity scan data Ds2 based on the temporary conversion matrix calculated by the singular value decomposition unit 28. The conversion unit 29 has a first conversion unit 29X that converts the 3D space scan data Ds1 and a second conversion unit 29Y that converts the high reflection intensity scan data Ds2. The first conversion unit 29X supplies the converted 3D space scan data Ds1 to the selection unit 24 and the 3D space squared error calculation unit 30, respectively, and the second conversion unit 29Y supplies the converted high reflection intensity scan data Ds2 to the selection unit 25 and the high reflection intensity squared error calculation unit 31.

[0051] The 3D space squared error calculation unit 30 calculates the squared error ("3D space squared error E") between the 3D space scan data Ds1 converted by the first conversion unit 29X and the 3D space map data Dm1. PT_PL The high reflection intensity squared error calculation unit 31 calculates the squared error (also called "high reflection intensity squared error E") between the high reflection intensity scan data Ds2 converted by the second conversion unit 29Y and the high reflection intensity map data Dm2. PT_PT ") is calculated.

[0052] The adaptive weighting calculation unit 32 calculates the 3D spatial squared error E PT_PL and high reflection intensity square error E PT_PT For example, the adaptive weight calculation unit 32 determines the weights based on the ratio of the number of processing points handled in the first matching and the second matching for the frame to be processed. For example, the number of processing points in the first matching (i.e., the number of points included in the 3D spatial scan data Ds1) is set to "N1", the number of processing points in the second matching (i.e., the number of points included in the high reflection intensity scan data Ds2) is set to "N2", and the 3D spatial squared error E PT_PL The weight for the high reflection intensity square error E PT_PT Assuming that the weight for is "w2", the adaptive weight calculation unit 32 calculates the weight w1 and the weight w2 for each frame based on the following equations. w1=N1 / (N1+N2) w2=1-(N1 / (N1+N2))

[0053] The weighting addition unit 33 uses the weights w1 and w2 calculated by the adaptive weighting calculation unit 32 to calculate a total error "E _total " is calculated. E _total =w1·E PT_PL +w2·E PT_PT

[0054] The convergence determination unit 34 determines whether the ICP loop processing has ended or not by determining whether the ICP loop processing has ended or not. Specifically, the convergence determination unit 34 determines whether the ICP loop processing has ended or not when the ICP loop processing has been executed a predetermined number of times in the current frame, or when the total error E _total and the total error E in the previous loop _total If the difference between the convergence determination unit 34 and the ICP loop processing is within the predetermined difference, the convergence determination unit 34 determines that the termination condition for the ICP loop processing is satisfied. Then, the convergence determination unit 34 supplies the determination result to the switching unit 36.

[0055] The transformation matrix update unit 35 calculates a transformation matrix that represents the cumulative rotation and translation in the current frame based on the provisional transformation matrix calculated by the singular value decomposition unit 28 for each loop of the ICP loop processing.

[0056] When the switching unit 36 ​​detects the end of the ICP loop processing based on the determination result supplied from the convergence determination unit 34, it outputs the transformation matrix supplied from the transformation matrix update unit 35 as the transformation matrix to be calculated. Then, based on the transformation matrix output by the switching unit 36, the vehicle-mounted device 1 recognizes the position (including the attitude) of the LIDAR 3 in the map coordinate system. Note that, if installation information indicating the relative position (including the attitude) of the LIDAR 3 with respect to the vehicle is stored in the storage unit 12 or the like, the vehicle-mounted device 1 may estimate the position of the vehicle based on the installation information and the above-mentioned transformation matrix.

[0057] (3-3) Processing flow 4 is an example of a flowchart showing the procedure of the position estimation process in this embodiment. The control unit 15 of the vehicle-mounted device 1 repeatedly executes the process of the flowchart shown in FIG. 4, for example, in accordance with the frame period of the scan data.

[0058] First, the control unit 15 reads the point cloud map data 10 (step S11). As a result, the control unit 15 acquires the 3D spatial map data Dm1 and the high reflection intensity map data Dm2. In this case, the control unit 15 may refer to the point cloud map data 10 that includes the 3D spatial map data Dm1 and the high reflection intensity map data Dm2 in an identifiable manner, or may generate the 3D spatial map data Dm1 and the high reflection intensity map data Dm2 from the point cloud map data 10 using a method described later in section "(3-5) Map data processing."

[0059] Next, the control unit 15 calculates the normal vector of each point of the 3D spatial map data Dm1 (step S12). Furthermore, the control unit 15 sets parameters (ICP convergence parameters) used for determining convergence of the ICP loop processing (step S13). Here, the control unit 15 sets the maximum loop count "Max_iteration" and the convergence determination threshold "epsilon" as the ICP convergence parameters. The processing of steps S11 to S13 corresponds to the initialization processing of the system.

[0060] Next, the control unit 15 receives input of scan data at a frame cycle (step S14). In this case, the control unit 15 acquires the scan data generated by the LIDAR 3 on a frame-by-frame basis and executes the processes of steps S15 to S17. Then, the control unit 15 generates 3D space scan data Ds1 and high reflection intensity scan data Ds2 from the scan data input in step S14 (step S15).

[0061] Next, the control unit 15 executes Combined-ICP processing (step S16). Details of the Combined-ICP processing will be specifically described with reference to the flowchart in Fig. 5. Then, the control unit 15 outputs information related to the estimated position and attitude obtained by the Combined-ICP processing in step S16 (step S17). The information related to the estimated position and attitude is supplied to a processing unit that performs driving assistance such as automatic driving and route guidance, for example.

[0062] FIG. 5 is an example of a flowchart showing the procedure of the Combined-ICP process in step S16.

[0063] First, the control unit 15 sets the 3D space scan data Ds1 (step S21). The control unit 15 also sets the high reflection intensity scan data Ds2 (step S22). Then, the control unit 15 initializes variables used in the ICP loop processing (ICP loop processing variables) (step S23). Specifically, the control unit 15 sets the rotation matrix R and translation vector, which are elements of the transformation matrix to be calculated, to "R=I" and "T=0", respectively, and sets the loop number counter "N_iter" to "0".

[0064] Then, the control unit 15 executes the ICP loop process (step S24). Details of the ICP loop process will be described with reference to FIG.

[0065] Then, the control unit 15 outputs the transformation matrix [RT] at the time of ICP convergence (step S25). Then, based on the rotation matrix R output in step S25, the control unit 15 calculates the rotation angles "φ," "θ," and "ψ" around the three axes that represent the attitude of the LIDAR 3 in the map coordinate system adopted in the point cloud map data 10 (step S26). Furthermore, the control unit 15 extracts the translation vector "T = (tx, ty, tz)" as the relative position of the LIDAR 3 (step S27). This identifies the relative positional relationship between the origin of the map coordinate system and the origin of the LIDAR coordinate system that the LIDAR 3 uses as its reference.

[0066] FIG. 6 is an example of a flowchart showing the procedure of the ICP loop process executed in step S24.

[0067] First, the control unit 15 associates the 3D space scan data Ds1 with the 3D space map data Dm1 (step S31). In this case, the control unit 15 searches for the nearest point in the 3D space map data Dm1 for each point included in the 3D space scan data Ds1 using an arbitrary search algorithm such as KD-Tree. Next, the control unit 15 associates the high reflection intensity scan data Ds2 with the high reflection intensity map data Dm2 (step S32). In this case, the control unit 15 searches for the nearest point in the high reflection intensity map data Dm2 for each point included in the high reflection intensity scan data Ds2 using an arbitrary search algorithm such as KD-Tree.

[0068] Next, the control unit 15 constructs a first simultaneous equation "Ax-b" which is a simultaneous equation for the 3D spatial data (step S33). Here, "x" is a vector of parameters related to the transformation matrix, "b" is a vector whose components are the inner product of the difference between points associated with each other in the 3D spatial map data Dm1 and the 3D spatial scan data Ds1 and the corresponding normal vector, and "A" is a matrix calculated from the 3D spatial map data Dm1, the 3D spatial scan data Ds1, and the normal vector. Details of the first simultaneous equation (i.e., simultaneous equations based on the P2PL algorithm) are disclosed in, for example, Non-Patent Document 1.

[0069] The control unit 15 also constructs a second simultaneous equation "A^ix = b^i" for the high reflection intensity data (step S34). Here, "i" indicates the index of the point in the high reflection intensity scan data Ds2, the vector "b^i" is the difference vector between the points in the high reflection intensity scan data Ds2 and the high reflection intensity map data Dm2 for the point with index i, and "A^i" is a matrix based on the position coordinates of the point with index i in the high reflection intensity scan data Ds2. Details of the second simultaneous equation (i.e., the simultaneous equation based on the P2P algorithm) are disclosed in, for example, Non-Patent Document 1.

[0070] Then, the control unit 15 constructs an integrated simultaneous equation by integrating the first simultaneous equation and the second simultaneous equation (step S35). Note that construction of a simultaneous equation by integrating the simultaneous equations based on the P2PL algorithm and the simultaneous equations based on the P2P algorithm is disclosed in, for example, Non-Patent Document 1.

[0071] Then, the control unit 15 calculates a solution "Xopt=(α, β, γ, tx, ty, tz)" of the integrated simultaneous equations constructed in step S35 using singular value decomposition or the like (step S36). Then, the control unit 15 calculates a temporary rotation matrix "R_tmp" using the solution of the parameters related to the rotation around the three axes (α, β, γ) (step S37). Furthermore, the control unit 15 sets the solution of the parameters related to the translation around the three axes (tx, ty, tz) to a temporary translation vector "T_tmp" (step S38). Then, the control unit 15 generates transformed scan data by transforming the input scan data (i.e., the 3D space scan data Ds1 and the high-reflection intensity scan data Ds2) using the temporary transformation matrix [R_tmp T_tmp] obtained based on steps S37 and S38 (step S39).

[0072] Then, the control unit 15 updates the transformation matrix [RT] based on the temporary transformation matrix [R_tmp T_tmp] calculated in the current loop (step S40). Specifically, the control unit 15 sets the updated R to the product of the pre-update R and R_tmp, and the updated T to the sum of the product of the pre-update T and R_tmp and T_tmp.

[0073] Next, the control unit 15 calculates a 3D spatial square error "E" corresponding to the square error between each point of the converted scan data and the corresponding point of the map data. PT_PL " and high reflection intensity square error "E PT_PT " (step S41). Then, the control unit 15 calculates the 3D spatial square error E PT_PL and high reflection intensity square error E PT_PT The total error E is the weighted sum of _total is calculated (step S42).

[0074] Next, the control unit 15 determines whether the loop number counter N_iter is 0 or not (step S43). If the loop number counter N_iter is 0 (step S43; Yes), that is, if this is the first loop of the ICP loop processing, the control unit 15 sets the difference "diff_E_total" between the total error in the current loop and the total error in the previous loop to infinity (step S44). On the other hand, if the loop number counter N_iter is not 0 (step S43; Yes), that is, if this is the second or subsequent loop of the ICP loop processing, the control unit 15 sets the total error E _total and the total error in the previous loop "Prev_E _total The control unit 15 calculates the difference between the total error E _total the total error in the previous loop, Prev_E _total (Step S46). Then, the control unit 15 increments the loop number counter N_iter (Step S47).

[0075] Then, the control unit 15 determines whether the termination condition of the ICP loop processing is satisfied (step S48). Specifically, the control unit 15 determines whether the difference diff_E_total is greater than the convergence determination threshold epsilon and the loop number counter N_iter is less than the maximum loop number Max_iteration (step S48). If the difference diff_E_total is greater than the convergence determination threshold epsilon and the loop number counter N_iter is less than the maximum loop number Max_iteration (step S48; Yes), the control unit 15 determines that the termination condition of the ICP loop processing is not satisfied. Therefore, in this case, the control unit 15 returns the process to step S31. On the other hand, if the difference diff_E_total is equal to or less than the convergence determination threshold epsilon or the loop number counter N_iter is equal to or greater than the maximum loop number Max_iteration (step S48; No), the control unit 15 determines that the termination condition of the ICP loop processing is satisfied. Therefore, in this case, the control unit 15 terminates the ICP loop processing.

[0076] (3-4) Scan data processing Next, the scan data processing, which is executed by the scan data processing unit 23 in FIG. 3 and corresponds to step S15 in FIG. 4, will be described in detail.

[0077] Fig. 7 is a diagram showing an overview of scan data processing. Fig. 7 shows the flow of processing for generating 3D spatial scan data Ds1 and high reflection intensity scan data Ds2 from scan data generated by the LIDAR 3. Each point in the scan data generated by the LIDAR 3 includes coordinate values ​​for three axes (X, Y, Z) in the LIDAR coordinate system and reflection intensity.

[0078] First, the generation of the 3D space scan data Ds1 used in the first matching will be described. As shown in Fig. 7, the scan data processing unit 23 applies thinning processing (i.e., downsampling) such as a voxel grid filter to the scan data so that the scan data has the required grid size, thereby generating the 3D space scan data Ds1. Furthermore, if unnecessary, the reflection intensity information attached to the scan data may be deleted from the 3D space scan data Ds1, or may be left as is if it does not pose a problem in processing.

[0079] Next, the generation of the high reflection intensity scan data Ds2 used in the second matching will be described. As shown in Fig. 7, the scan data processing unit 23 generates the high reflection intensity scan data Ds2 by performing road surface extraction processing, high reflection intensity extraction processing, and thinning processing such as voxel grid filtering on the scan data. The details will be described below.

[0080] The scan data processing unit 23 first performs a road surface extraction process on the scan data, which is a process for extracting a road surface. For example, the scan data processing unit 23 performs the road surface extraction process as follows: (a) Smoothing and normal calculation using MLS (Moving Least Squares), (b) Normal filtering, (c) Creating a histogram of each point based on height. (d) Height-based filtering In "(b) filtering in the normal direction," the scan data processing unit 23 performs, for example, a process of retaining only points for which the normal calculated in (a) is close to vertical (i.e., within a predetermined angle difference from the normal). In "(c) creating a histogram of each point based on height," the scan data processing unit 23 creates a histogram based on the height direction component of the scan data (e.g., Z coordinate value). In "(d) filtering process based on height," the scan data processing unit 23 extracts each point corresponding to the height of the road surface based on the histogram generated in (c). In this case, the scan data processing unit 23 extracts, for example, points belonging to the height with the highest frequency (and the height closest to it) in the histogram as points on the road surface.

[0081] Furthermore, the scan data processing unit 23 performs a high reflection intensity extraction process, which is a process of extracting points with relatively high reflection intensity, on the points extracted from the scan data by the road surface extraction process (also called "road surface candidate points"). For example, the scan data processing unit 23 performs the high reflection intensity extraction process as follows: (e) Creating a histogram based on the reflection intensity of the road surface candidate points. (f) Selection of the reflectance intensity threshold based on the histogram distribution (g) Extraction of points with a reflection intensity value greater than the reflection intensity threshold Ith (h) Noise removal processing such as SOR (Statistical Outlier Removal), The following steps are carried out in sequence.

[0082] FIG. 8 shows an example of a histogram generated in "(e) Creation of a histogram based on the reflection intensity of road surface candidate points." In FIG. 8, the scan data processing unit 23 generates a histogram of road surface candidate points based on the reflection intensity. Here, the histogram shown in FIG. 8 has a peak reflection intensity "Ia." In this case, the scan data processing unit 23 determines the reflection intensity threshold "Ith" determined in (f) based on the peak reflection intensity Ia. For example, the scan data processing unit 23 determines a reflection intensity that is greater than the peak reflection intensity Ia by a predetermined value or a predetermined rate as the reflection intensity threshold Ith. In this case, the predetermined value or rate is determined in advance depending on, for example, the type of LIDAR 3 used and is stored in the storage unit 12 so that it can be referenced by the scan data processing unit 23. After setting the reflection intensity threshold Ith, the scan data processing unit 23 extracts road surface candidate points having a reflection intensity value greater than the reflection intensity threshold Ith as road surface paint candidate points based on the above-mentioned (g). Thereafter, the scan data processing unit 23 further performs noise removal processing such as SOR on the extracted road paint candidate points based on (h).

[0083] Next, the scan data processing unit 23 thins out the points extracted by the high reflection intensity extraction process using a voxel grid filter or the like as needed, and deletes reflection intensity information as needed, thereby generating high reflection intensity scan data Ds2 consisting of points on the road surface that have high reflection intensity.

[0084] Scan data processing may need to be performed in real time depending on the system to which it is applied. For example, this applies when applied to real-time vehicle position and attitude estimation. However, this does not apply when real-time performance is not required, such as when generating maps using offline SLAM (Simultaneous Localization and Mapping).

[0085] (3-5) Map data processing Next, the map data processing, which is the processing for generating the 3D spatial map data Dm1 and the high reflection intensity map data Dm2 executed by the map data processing unit 20 in FIG. 3, will be described in detail.

[0086] Fig. 9 is a diagram showing an overview of map data processing. Fig. 9 shows the flow of processing for generating 3D spatial map data Dm1 and high reflection intensity map data Dm2 from point cloud map data 10. Each point constituting the point cloud map data 10 includes coordinate values ​​of three axes (X, Y, Z) in the lidar coordinate system and reflection intensity.

[0087] First, the generation of the 3D spatial map data Dm1 used in the first matching will be described. As shown in Fig. 9, the map data processing unit 20 applies thinning processing such as a voxel grid filter to the scan data so that the scan data has the required grid size, thereby generating the 3D spatial map data Dm1. Furthermore, if unnecessary, the reflection intensity information attached to the scan data may be deleted from the 3D spatial map data Dm1, or may be left in place if it does not pose a problem in processing.

[0088] Next, the generation of the high reflection intensity map data Dm2 used in the second matching will be described. As shown in Fig. 9, the scan data processing unit 23 generates the high reflection intensity map data Dm2 by performing road surface extraction processing, high reflection intensity extraction processing, and thinning processing such as voxel grid filtering on the point cloud map data 10. The details will be described below.

[0089] The scan data processing unit 23 first performs a road surface extraction process, which is a process of extracting road surfaces, on the point cloud map data 10. For example, the map data processing unit 20 performs the road surface extraction process as follows: (i) Smoothing and normal calculation using MLS, (j) Normal filtering, (k) Clustering (l) Extraction of clusters corresponding to road surfaces The above steps are performed sequentially. Here, (i) and (j) are the same processes as (a) and (b) in "(3-4) Scan Data Processing." In "(k) Clustering," the map data processing unit 20 executes an arbitrary clustering method based on at least height, thereby generating multiple clusters that are groups of points that are at least similar in height. Then, in "(l) Extraction of Clusters Corresponding to Road Surfaces," the map data processing unit 20 performs a process of deleting clusters with a number of points less than a predetermined threshold, by regarding them as noise (including, for example, the flat surface of a median strip).

[0090] Furthermore, the map data processing unit 20 performs a high reflection intensity extraction process, which is a process of extracting points with relatively high reflection intensity, on the road surface candidate points extracted by the road surface extraction process from the point cloud map data 10. For example, the scan data processing unit 23 performs the high reflection intensity extraction process as follows: (m) Creation of a histogram based on the reflection intensity of road surface candidate points; (n) Selection of reflectance intensity threshold based on histogram distribution (o) Extraction of points with a reflection intensity value greater than the reflection intensity threshold Ith (m) to (o) are the same processes as (e) to (g) in "(3-4) Scan data processing", for example.

[0091] Next, the map data processing unit 20 performs thinning processing using a voxel grid filter or the like as needed on the points extracted by the high reflection intensity extraction processing, and deletes reflection intensity information as needed, thereby generating high reflection intensity map data Dm2.

[0092] Note that the map data processing by the map data processing unit 20 is not necessarily limited to being performed in real time. Alternatively, the vehicle-mounted device 1 may store in advance in the storage unit 12 or the like the 3D spatial map data Dm1 and the high reflection intensity map data Dm2 that have been generated in advance by offline processing so that the control unit 15 can read them in step S11 of FIG.

[0093] (4) Technical effects Next, the technical effects of this embodiment will be described.

[0094] A first effect is the utilization of road surface paint. In this embodiment, the vehicle-mounted device 1 performs matching based on ICP by utilizing both three-dimensional spatial data and high-reflection intensity data indicating road surface paint. On the other hand, in general ICP, matching is performed using only object shape information (three-dimensional coordinate values ​​only), so even if road surface paint with a distinctive shape exists on the road plane, it is not utilized. In consideration of the above, in this embodiment, the vehicle-mounted device 1 prepares 3D spatial map data Dm1 and high-reflection intensity map data Dm2 and performs matching for each of the three-dimensional space and road surface paint, thereby making it possible to suitably execute ICP utilizing road surface paint with a distinctive shape on the road plane.

[0095] A second effect is matching of road paint that is not dependent on the "magnitude" of the reflection intensity value. Generally, the reflection intensity in scan data varies depending on the distance, angle of incidence, etc., even for the same measurement target. Therefore, to use the reflection intensity value itself, highly accurate calibration is required so that it is independent of the distance and angle of incidence. Taking the above into consideration, in this embodiment, the vehicle-mounted device 1 does not use the reflection intensity value itself, but instead extracts road paint points with high reflection intensity through threshold processing using a reflection intensity threshold. This allows the vehicle-mounted device 1 to perform matching processing using the three-dimensional coordinate values ​​of the shape with high reflection intensity, without using the reflection intensity value itself.

[0096] A third effect is the realization of highly accurate matching by using the P2P algorithm for matching road surface paint (second matching). It is known that the P2PL algorithm is capable of matching with higher accuracy than the P2P algorithm for general three-dimensional space. However, for reasons described below, the P2PL algorithm is considered inappropriate for matching road surface paint on a flat road surface. Taking the above into consideration, in this embodiment, the vehicle-mounted device 1 executes a combined ICP algorithm, which performs first matching, which is matching of three-dimensional space, using the evaluation index of the P2PL algorithm, and performs second matching, which is matching of road surface paint, using the evaluation index of the P2P algorithm. This enables the vehicle-mounted device 1 to perform matching with high accuracy.

[0097] Here, we explain why the P2PL algorithm is inappropriate for matching road paint on a road plane. When the corresponding points to be matched are on the road plane, the vector connecting those points is perpendicular to the normal to the road plane, so the P2PL evaluation index is zero and misalignment cannot be detected. Specifically, if the scan data is pushed to a height that nearly matches the road plane in the ICP loop processing (in this case, the remaining part is horizontal matching), the vectors of corresponding points between the source high-reflection-intensity scan data Ds2 and the target high-reflection-intensity map data Dm2 will be oriented nearly parallel to the road surface. Here, the PTPL algorithm minimizes an evaluation function equivalent to the sum of the dot product of the vector between these corresponding points and the normal to the point in the target high-reflection-intensity map data Dm2, resulting in an evaluation function that is nearly zero. Therefore, after the plane's Z direction is nearly matched, the PTPL algorithm cannot detect misalignment in the horizontal directions (X and Y directions). Taking the above into consideration, the vehicle-mounted device 1 performs the second matching, which is matching of road surface paint, using the evaluation index of the P2P algorithm.

[0098] Next, the results of a simulation based on the method of this embodiment will be described.

[0099] FIG. 10 is a perspective view showing the points of the point cloud map data 10 used in this simulation plotted in three-dimensional space. In FIG. 10, points with higher reflection intensity are displayed in white. As shown in FIG. 10, the reflection intensity of road paint such as white lines is relatively high. Note that this simulation uses point cloud map data 10 and scan data from which points 70 cm or higher above the ground have been removed. This assumes a situation where there are few structures in the surrounding area, with only guardrails or the like present.

[0100] FIG. 11 shows the matching results of a comparative example for comparison with the method according to this embodiment. In this comparative example, the second matching is not performed, and the transformation matrix is ​​calculated based on the first matching, which matches the 3D spatial map data Dm1 with the 3D spatial scan data Ds1 using the P2PL algorithm. In FIG. 11, points of the 3D spatial scan data Ds1 transformed based on the matching results are shown in white, and points of the point cloud map data 10 are shown in gray (light white). In this case, misalignment (vertical misalignment and slight counterclockwise rotational misalignment) is observed within the range indicated by the circle 70.

[0101] Fig. 12 is a diagram showing the matching results obtained by the matching method according to this embodiment (i.e., by the Combined ICP algorithm). In Fig. 12, points in the scan data converted based on the matching results are shown in white, and points in the point cloud map data 10 are shown in gray (light white). As shown in Fig. 12, in this case, there is almost no deviation between the scan data converted by the matching method according to this embodiment and the point cloud map data 10, and it can be seen that a good matching result is obtained.

[0102] 13 shows the results of matching high reflection intensity map data Dm2 and high reflection intensity scan data Ds2 using the P2P algorithm when the matching method according to this embodiment is executed. In FIG. 13, points in the high reflection intensity scan data Ds2 converted based on the matching results are shown in white, and points in the high reflection intensity map data Dm2 are shown in gray (light white). As shown in FIG. 13, there is almost no deviation between the high reflection intensity scan data Ds2 converted based on the matching method according to this embodiment and the high reflection intensity map data Dm2, and it can be seen that good matching results are obtained.

[0103] FIG. 14 is a graph showing the relationship between the number of iterations of the ICP loop process and the squared error when the method of the embodiment (i.e., the Combined ICP algorithm) and the method of the comparative example (i.e., the P2PL algorithm) are executed. As shown in FIG. 14, as the number of iterations of the ICP loop process increases, the method of the embodiment produces a smaller error than the method of the comparative example. Furthermore, when comparing the error per point (measurement point) when the method of the comparative example is executed with the error when the method of the embodiment is executed, the error in the former is 0.12 m (number of points: 6706), and the error in the latter is 0.10 m (number of points: 6711). As such, the method of the embodiment can utilize the shape of the road paint as a feature during matching, and therefore can perform matching even when there are few surrounding structures in the space. As a result, it is believed that better results were obtained compared to the method of the comparative example, which uses only the P2PL algorithm.

[0104] (5) Variations The following describes preferred modifications of the above-described embodiment. The following modifications may be applied to these embodiments in combination.

[0105] (First Modification) The control unit 15 may set different grid sizes per point of the data used in the first matching and the second matching (i.e., the grid sizes determined by the Voxel Grid Filer). This allows for adjustment of the balance between processing time and accuracy. For example, by setting the grid sizes of the high reflection intensity scan data Ds2 and the high reflection intensity map data Dm2 used in the second matching smaller than the grid sizes of the 3D space scan data Ds1 and the 3D space map data Dm1 used in the first matching, matching results that emphasize the second matching can be obtained.

[0106] (Second Modification) The control unit 15 may correct the reflection intensity included in the scan data output by the LIDAR 3 according to the measurement direction and / or the measurement distance.

[0107] In this case, for example, the control unit 15 stores in advance in the storage unit 12 a lookup table or the like that indicates an appropriate correction amount for each measurement direction and measurement distance. Then, by referring to the lookup table or the like, the control unit 15 corrects the reflection intensity of each point in the scan data output by the LIDAR 3 according to the corresponding measurement direction and / or measurement distance. This allows the control unit 15 to perform matching based on this embodiment based on scan data that includes more accurate reflection intensity information.

[0108] (Third Modification) In the second matching based on the high reflection intensity scan data Ds2 and the high reflection intensity map data Dm2, the control unit 15 may perform matching using other features with high reflection intensity as a reference in addition to road surface paint.

[0109] In this case, the high reflection intensity scan data Ds2 and the high reflection intensity map data Dm2 include not only points on the road paint, but also points on signs such as directional signs, guardrails, and the surfaces of other features painted with high reflection intensity (including concrete walls at road boundaries and walls inside tunnels). In this case, in the process of generating the high reflection intensity scan data Ds2 and the high reflection intensity map data Dm2, the control unit 15 executes the process of extracting high reflection intensity points described above in addition to the process of extracting road paint points shown in FIGS. 7 and 9. In this case, as the latter process, the control unit 15 executes, for example, a process of extracting a reference surface on which target high reflection intensity points exist (road surface extraction process for road paint), and a high reflection intensity extraction process, thereby extracting target high reflection intensity feature points from the scan data and point cloud map data 10.

[0110] According to this modification, the control unit 15 can suitably improve the matching accuracy by using points with high reflection intensity (high reflection intensity data) including road surface paint. In this modification, the road surface and the surface of an object painted with high reflection intensity are examples of the "reference surface."

[0111] (Fourth Modification) Instead of adaptively determining the reflection intensity threshold used in the map data processing unit 20 and the scan data processing unit 23 according to the histogram, the control unit 15 may set it to a predetermined value stored in advance in the storage unit 12. In this case, the predetermined value is determined in advance to be a value that is higher than the reflection intensity of the road surface other than road paint and lower than the reflection intensity of the road paint, and is stored in the storage unit 12.

[0112] (Fifth Modification) Instead of executing the Combined ICP algorithm, which combines the first and second matching, the control unit 15 may execute only the second matching. In this case, the control unit 15 calculates a transformation matrix by matching the high reflection intensity scan data Ds2 with the high reflection intensity map data Dm2 based on the P2P algorithm. Even in this case, the control unit 15 can effectively match the map data with the scan data even in an environment where there are not many objects with three-dimensional structures.

[0113] (Sixth Modification) The configuration of the driving assistance system shown in Fig. 1 is one example, and the configuration of a driving assistance system to which the present invention can be applied is not limited to the configuration shown in Fig. 1. For example, instead of having an on-board device 1, the driving assistance system may have an electronic control unit of the vehicle that executes the processing of the block diagram shown in Fig. 3. In this case, point cloud map data 10 is stored in, for example, a storage unit in the vehicle or a server device that communicates data with the vehicle, and the electronic control unit of the vehicle refers to this point cloud map data 10 to execute processing such as matching with scan data output by the LIDAR 3.

[0114] As described above, the information processing device according to this embodiment includes a point cloud map data acquisition means, a scan data acquisition means, a scan data processing means, and a matching means. The point cloud map data acquisition means acquires point cloud map data, which is point cloud data having a reflection intensity equal to or greater than a threshold. The scan data acquisition means acquires scan data, which is point cloud data measured by receiving reflected light of emitted light. The scan data processing means generates high reflection intensity scan data, which has a reflection intensity equal to or greater than a predetermined threshold, from the scan data. The matching means matches the point cloud map data with the high reflection intensity scan data. This allows the information processing device to suitably match map data with scan data even in an environment where there are not many objects with three-dimensional structures.

[0115] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and other documents are incorporated herein by reference. [Explanation of symbols]

[0116] 1 On-vehicle device 2 Sensor group 3 Rider 4 GPS receivers 5 Gyro sensor 6 Vehicle speed sensor 10 Point cloud map data

Claims

1. a point cloud map data acquisition means for acquiring high reflection intensity map data, which is point cloud data representing points on a reference surface where the reflection intensity is equal to or greater than a threshold, and three-dimensional space map data, which is point cloud data of features existing in three-dimensional space; a scan data acquisition means for acquiring scan data, which is point cloud data measured by receiving reflected light of emitted light; a scan data processing means for generating, from the scan data, data representing points on a reference surface where the reflection intensity is equal to or greater than a predetermined threshold, as high reflection intensity scan data, and generating, as three-dimensional space scan data, the scan data or data obtained by applying a filter to the scan data; a comparison means for performing a first comparison between the three-dimensional space map data and the three-dimensional space scan data, and a second comparison between the high reflection intensity map data and the high reflection intensity scan data; An information processing device having the above.

2. The information processing apparatus according to claim 1 , wherein the matching means performs the second matching based on a point-to-point algorithm.

3. The information processing apparatus according to claim 1 , wherein the matching means performs the first matching based on a point-to-plane algorithm and the second matching based on a point-to-point algorithm.

4. 4. The information processing device according to claim 1, wherein the matching means weights the first matching and the second matching based on the number of points of the three-dimensional spatial scan data to be matched in the first matching and the number of points of the high reflection intensity scan data to be matched in the second matching.

5. The information processing device according to claim 1, wherein the reference surface includes a road surface.

6. 6. The information processing apparatus according to claim 1, wherein the scan data acquisition means acquires the scan data measured by an external sensor that receives reflected light of emitted light.

7. 7. The information processing apparatus according to claim 1, wherein the comparison means calculates, through the comparison, parameters for converting a coordinate system of the scan data into a coordinate system of the high reflection intensity map data.

8. 8. The information processing apparatus according to claim 1, wherein the scan data processing means sets the threshold value based on a histogram of reflection intensity of the scan data.

9. 1. A computer-implemented method comprising: Acquire high-reflection-intensity map data, which is point cloud data representing points on a reference surface where the reflection intensity is equal to or greater than a threshold, and three-dimensional space map data, which is point cloud data of features existing in three-dimensional space; The system acquires scan data, which is point cloud data measured by receiving the reflected light of the emitted light, From the scan data, data representing points on a reference surface where the reflection intensity is equal to or greater than a predetermined threshold is generated as high reflection intensity scan data, and the scan data or data obtained by applying a filter to the scan data is generated as three-dimensional space scan data; A method for performing a first matching between the three-dimensional space map data and the three-dimensional space scan data, and a second matching between the high reflectivity map data and the high reflectivity scan data.

10. Acquire high-reflection-intensity map data, which is point cloud data representing points on a reference surface where the reflection intensity is equal to or greater than a threshold, and three-dimensional space map data, which is point cloud data of features existing in three-dimensional space; The system acquires scan data, which is point cloud data measured by receiving the reflected light of the emitted light, From the scan data, data representing points on a reference surface where the reflection intensity is equal to or greater than a predetermined threshold is generated as high reflection intensity scan data, and the scan data or data obtained by applying a filter to the scan data is generated as three-dimensional space scan data; A program that causes a computer to execute a process of first matching the three-dimensional space map data with the three-dimensional space scan data, and second matching the high reflection intensity map data with the high reflection intensity scan data.

11. A storage medium storing the program according to claim 10.

Images