Information processing device, control method, program, and storage medium

Abstract

To provide an information processing device capable of suitably performing downsampling.SOLUTION: A controller 13 of an on-vehicle device 1 acquires point group data output by a lidar 2. Then, the controller 13 generates processing point group data by downsampling the point group data. Then, the controller 13 matches the processing point group data with voxel data VD representing the position of an object for each voxel which is a unit area, thereby associating measurement points and each voxel constituting the processing point group data. Then, the controller 13 changes the size of the downsampling to be performed next based on the number of corresponding measurement points Nc, which is the number of measurement points associated with voxels among the measurement points of the processing point group data.SELECTED DRAWING: Figure 12

Description

Technical Field

[0001] The present invention relates to downsampling of data used for position estimation.

Background Art

[0002] Conventionally, a technique for estimating the self-position of a vehicle is known by comparing (matching) shape data of surrounding objects measured using a measuring device such as a laser scanner with map information in which the shapes of the surrounding objects are stored in advance. For example, Patent Document 1 discloses an autonomous mobile system that determines whether a detected object in a voxel obtained by dividing space according to a predetermined rule is a stationary object or a moving object, and performs matching between map information and measurement data for voxels in which stationary objects exist. Further, Patent Document 2 discloses a scan matching method for estimating the position of the host vehicle by comparing voxel data including the average vector and covariance matrix of stationary objects for each voxel with point cloud data output by a lidar.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] When estimating the vehicle position by collating voxel data with the point cloud data output by the lidar as in Patent Document 2, a downsampling process, which is a thinning process, is performed by averaging the lidar point cloud data in a space divided into a predetermined size. By performing this downsampling, it is possible to make the density of the point cloud data uniform, with a dense area near the vehicle and a sparse area far from the vehicle, and to reduce the total number of point cloud data. Thus, both the improvement of the self-position estimation accuracy and the suppression of the calculation time are achieved. However, in a space with few objects around the vehicle, the number of point cloud data detected by the lidar is small. Therefore, due to the downsampling process, the number of point cloud data used for calculation may be significantly reduced, resulting in problems such as a decrease in the accuracy and robustness of self-position estimation.

[0005] The present invention has been made to solve the above problems, and its main object is to provide an information processing device capable of suitably executing downsampling.

Means for Solving the Problems

[0006] The invention according to the claims is an acquisition means for acquiring point cloud data output by a measurement device, a downsampling processing means for generating processed point cloud data obtained by performing downsampling on the point cloud data, and an association means for associating each of the measurement points constituting the processed point cloud data with each of the voxels by collating the processed point cloud data with voxel data representing the position of an object for each voxel, which is a unit area. The downsampling processing means is an information processing device that changes the size of the next downsampling based on the number of corresponding measurement points, which is the number of measurement points associated with the voxels among the measurement points.

[0007] Also, the invention according to the claims is a control method executed by a computer, acquiring point cloud data output by a measurement device, Processed point cloud data is generated by downsampling the aforementioned point cloud data. By comparing the aforementioned processed point cloud data with voxel data representing the position of an object in each voxel, which is a unit area, an association is made between the measurement points constituting the processed point cloud data and each of the voxels. This control method changes the size of the next downsampling to be performed based on the number of corresponding measurement points, which is the number of measurement points among the measurement points that are associated with the voxel.

[0008] Furthermore, the invention described in the claims is, The point cloud data output by the measuring device is acquired, Processed point cloud data is generated by downsampling the aforementioned point cloud data. By comparing the aforementioned processed point cloud data with voxel data representing the position of an object in each voxel, which is a unit area, an association is made between the measurement points constituting the processed point cloud data and each of the voxels. This program causes a computer to perform a process to change the size of the downsampling, which is the next step in the process, based on the number of corresponding measurement points, which is the number of measurement points among the aforementioned measurement points that are associated with the voxels. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of the driver assistance system. [Figure 2] This is a block diagram showing the functional configuration of an in-vehicle device. [Figure 3] An example of the functional blocks of the downsampling processing unit 14 is shown. [Figure 4] This figure shows the results of the first execution, where downsampling was performed with different downsampling sizes. [Figure 5] This figure shows the results of a second execution, where downsampling was performed using different downsampling sizes. [Figure 6] This figure shows the results of a third execution, where downsampling was performed using different downsampling sizes. [Figure 7] This shows an overhead view of the area surrounding the vehicle on which the in-vehicle device is installed. [Figure 8] This diagram shows the vehicle's position, which the vehicle position estimation unit should estimate, represented in two-dimensional Cartesian coordinates. [Figure 9] An example of a general data structure for voxel data is shown. [Figure 10] This is an example of a functional block in the vehicle position estimation unit. [Figure 11] This shows the positional relationship between voxels containing voxel data and measurement points indicating the location near those voxels on a 2D plane in the world coordinate system. [Figure 12] This is an example of a flowchart illustrating the procedure for estimating the vehicle's own position. [Figure 13] The results of downsampling using a cubic grid size of 0.5m per side and self-position estimation using NDT processing are shown. [Figure 14] The results of downsampling using a cubic grid size of 0.5m per side and self-position estimation using NDT processing are shown. [Figure 15] The results of downsampling using a cubic grid size of 2.0m per side and self-position estimation using NDT processing are shown. [Figure 16] The results of downsampling using a cubic grid size of 2.0m per side and self-position estimation using NDT processing are shown. [Figure 17] Figure 12 shows the results of self-position estimation using downsampling and NDT processing according to the processing procedure shown. [Figure 18] Figure 12 shows the results of self-position estimation using downsampling and NDT processing according to the processing procedure shown. [Figure 19] The results of estimating the vehicle's position using downsampling and NDT processing, following the processing procedure shown in Figure 12, when occlusion occurs due to other vehicles, are shown. [Figure 20]Under the same conditions as in Figure 19, the results are shown when the downsampling size is determined so that the number of measurement points after downsampling falls between 600 and 800. [Figure 21] Based on Modification 1, the downsampling size in the direction of travel is fixed at 0.5m, and the downsampling sizes in the lateral and vertical directions are adaptively changed to perform self-position estimation using NDT processing. [Figure 22] Based on Modification 1, the downsampling size in the direction of travel is fixed at 0.5m, and the downsampling sizes in the lateral and vertical directions are adaptively changed to perform self-position estimation using NDT processing. [Modes for carrying out the invention]

[0010] According to a preferred embodiment of the present invention, the information processing device includes: acquisition means for acquiring point cloud data output by a measuring device; downsampling processing means for generating processed point cloud data by downsampling the point cloud data; and matching means for matching the measured points constituting the processed point cloud data with each of the voxels by comparing the processed point cloud data with voxel data representing the position of an object for each voxel which is a unit area. The downsampling processing means changes the size of the downsampling to be performed next based on the number of corresponding measured points, which is the number of measured points among the measured points that are associated with the voxels. According to this embodiment, the information processing device can adaptively change the size of the downsampling based on the number of corresponding measured points, which is the number of measured points in the processed point cloud data that are associated with voxels in which voxel data exists, and suitably optimize the number of corresponding measured points.

[0011] In one embodiment of the above-described information processing device, the downsampling processing means fixes the size in the first direction of spatial division during downsampling, and changes the size in directions other than the first direction based on the corresponding number of measurement points. In a preferred example, the first direction is the direction of travel of the measuring device. In this embodiment, the information processing device can suitably ensure resolution by fixing the downsampling size in directions where resolution of the processed point cloud data is required.

[0012] In another embodiment of the information processing device described above, the downsampling processing means changes the size based on a comparison between the number of corresponding measurement points and the target range of the number of corresponding measurement points. In this embodiment, the information processing device can accurately change the size of the downsampling.

[0013] In another embodiment of the information processing device described above, the downsampling processing means increases the size by a predetermined percentage or value if the number of corresponding measurement points is greater than the upper limit of the target range, decreases the size by a predetermined percentage or value if the number of corresponding measurement points is less than the lower limit of the target range, and maintains the size if the number of corresponding measurement points is within the target range. In this embodiment, the information processing device can accurately change the downsampling size so that the number of corresponding measurement points is maintained within the target range.

[0014] In another embodiment of the information processing device described above, the downsampling processing means determines the reference size of the downsampling to be performed next based on the number of corresponding measurement points, and determines the size for each direction based on the reference size and the ratio to the reference size set for each direction in which the space is divided in the downsampling. In this embodiment, the information processing device can suitably determine the size of the downsampling for each direction based on a ratio corresponding to the accuracy and importance of each direction.

[0015] In another embodiment of the above-described information processing device, the information processing device further includes a position estimation means that performs position estimation of a moving object equipped with the measuring device based on the result of comparing the voxel data of the voxels associated by the matching means with the measurement points associated with the voxels. In this embodiment, the information processing device can suitably achieve both accuracy in position estimation and processing time by using processed point cloud data generated by adaptively changing the downsampling size for position estimation.

[0016] In another embodiment of the information processing apparatus described above, the downsampling processing means changes the size based on the processing time of the downsampling processing means and the position estimation means. In a preferred example, the downsampling processing means may change the size based on a comparison of the processing time with a target range for said processing time. In this embodiment, the information processing apparatus can suitably set the downsampling size so that the processing time of the downsampling processing means and the position estimation means is appropriate.

[0017] According to another preferred embodiment of the present invention, a control method executed by an information processing device involves acquiring point cloud data output by a measuring device, generating processed point cloud data by downsampling the point cloud data, matching the processed point cloud data with voxel data representing the position of an object for each voxel which is a unit area, thereby associating the measurement points constituting the processed point cloud data with each of the voxels, and changing the size of the downsampling to be executed next based on the number of corresponding measurement points, which is the number of measurement points that are associated with the voxels among the measurement points. By executing this control method, the information processing device can adaptively change the size of the downsampling based on the number of corresponding measurement points and suitably optimize the number of corresponding measurement points.

[0018] According to yet another preferred embodiment of the present invention, the program acquires point cloud data output by a measuring device, generates processed point cloud data by downsampling the point cloud data, and associates the processed point cloud data with each of the voxels by comparing the processed point cloud data with voxel data representing the position of an object for each voxel which is a unit area. Based on the number of corresponding measurement points, which is the number of measurement points that are associated with the voxels, the program causes the computer to execute a process to change the size of the downsampling that is to be executed next. By executing this program, the computer can adaptively change the size of the downsampling based on the number of corresponding measurement points and suitably optimize the number of corresponding measurement points. Preferably, the above program is stored in a storage medium. [Examples]

[0019] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. Note that any character with "^" or "-" above it will be referred to as "A^" or "A" in this specification for convenience. - This is represented as (where "A" is any letter).

[0020] (1) Overview of the driver assistance system Figure 1 shows a schematic configuration of the driver assistance system according to this embodiment. The driver assistance system includes an in-vehicle unit 1 that moves with the vehicle, a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 2, a gyro sensor 3, a vehicle speed sensor 4, and a GPS receiver 5.

[0021] The on-board unit 1 is electrically connected to the lidar 2, gyro sensor 3, vehicle speed sensor 4, and GPS receiver 5, and based on their outputs, it estimates the position of the vehicle on which the on-board unit 1 is installed (also called "vehicle position"). Then, based on the estimated vehicle position, the on-board unit 1 performs automatic driving control of the vehicle so that it travels along a set route to a destination. The on-board unit 1 stores a map database (DB: Database) 10 that contains voxel data "VD". Voxel data VD is data that records position information of stationary structures for each voxel, which represents a cube (normal grid) that is the smallest unit of three-dimensional space. Voxel data VD includes data that represents the measured point cloud data of stationary structures within each voxel according to a normal distribution, and is used for scan matching using NDT (Normal Distributions Transform), as described later. The on-board unit 1 estimates at least the position and yaw angle of the vehicle on the plane using NDT scan matching. The onboard unit 1 may further estimate the vehicle's height, pitch angle, and roll angle. Unless otherwise specified, the vehicle's position includes attitude angles such as the yaw angle of the vehicle being estimated.

[0022] The lidar 2 discretely measures the distance to an object in the external environment by emitting a pulsed laser within a predetermined angular range in the horizontal and vertical directions, and generates three-dimensional point cloud data indicating the position of the object. In this case, the lidar 2 has an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data (points that constitute the point cloud data, hereafter referred to as "measurement points") based on the received signal output by the light receiving unit. The measurement points are generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light identified based on the received signal described above. Generally, the accuracy of the lidar's distance measurement is higher the closer the distance to the object, and lower the accuracy the farther the distance. The lidar 2, gyro sensor 3, vehicle speed sensor 4, and GPS receiver 5 each supply output data to the in-vehicle unit 1.

[0023] The in-vehicle unit 1 is an example of an "information processing device" in the present invention, and the lidar 2 is an example of a "measurement device" in the present invention. The driving assistance system may have an inertial measurement unit (IMU) that measures the acceleration and angular velocity of the vehicle in three axes, instead of or in addition to the gyro sensor 3.

[0024] (2) In-vehicle unit configuration Figure 2 is a block diagram showing an example of the hardware configuration of the in-vehicle unit 1. The in-vehicle unit 1 mainly consists of an interface 11, a memory 12, and a controller 13. Each of these elements is interconnected via a bus line.

[0025] Interface 11 performs interface operations related to the exchange of data between the in-vehicle unit 1 and external devices. In this embodiment, interface 11 acquires output data from sensors such as the lidar 2, gyro sensor 3, vehicle speed sensor 4, and GPS receiver 5, and supplies it to the controller 13. Interface 11 also supplies signals related to vehicle driving control generated by the controller 13 to the vehicle's electronic control unit (ECU). Interface 11 may be a wireless interface such as a network adapter for wireless communication, or it may be a hardware interface for connecting to external devices via cables, etc. Interface 11 may also perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0026] Memory 12 is composed of various volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. Memory 12 stores programs for the controller 13 to execute predetermined processes. Note that the programs executed by the controller 13 may be stored in storage media other than memory 12.

[0027] Furthermore, memory 12 stores target range information 9 and map DB 10, which includes voxel data VD.

[0028] Target range information 9 is information used to set the downsampling size when downsampling is performed on point cloud data obtained when lidar 2 performs one scan cycle. Specifically, target range information 9 represents the target range of the number of measurement points (also called "corresponding measurement points Nc") in the downsampled point cloud data that are associated with the voxel data VD in the NDT matching performed at each processing time based on the scan cycle of lidar 2. Hereafter, the target range of corresponding measurement points Nc indicated by target range information 9 will be referred to as "target range R Nc It is also called "".

[0029] Furthermore, at least one of the target range information 9 or the map DB 10 may be stored in an external storage device of the in-vehicle unit 1, such as a hard disk connected to the in-vehicle unit 1 via the interface 11. The storage device may be a server device that communicates with the in-vehicle unit 1. The storage device may also consist of multiple devices. The map DB 10 may also be updated periodically. In this case, for example, the controller 13 receives partial map information relating to the area to which the vehicle's position belongs from the server device that manages the map information via the interface 11 and reflects it in the map DB 10.

[0030] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire in-vehicle unit 1. In this case, the controller 13 performs processing related to vehicle position estimation by executing a program stored in memory 12, etc.

[0031] Furthermore, the controller 13 functionally includes a downsampling processing unit 14 and a vehicle position estimation unit 15. The controller 13 functions as an "acquisition means," a "downsampling processing means," a "correspondence means," and a computer or the like that executes the program.

[0032] The downsampling processing unit 14 generates corrected point cloud data (also called "processed point cloud data") that reduces the number of measurement points by downsampling the point cloud data output from the lidar 2. In this downsampling, the downsampling processing unit 14 generates processed point cloud data with a reduced number of measurement points by averaging the point cloud data within a space divided into predetermined sizes. In this case, the downsampling processing unit 14 sets the number of corresponding measurement points Nc in the NDT matching at each processing time to be within the target range R. Nc The predetermined size mentioned above in downsampling is adaptively set so that it fits within the specified limits.

[0033] The vehicle position estimation unit 15 estimates the vehicle's position by performing scan matching based on NDT (NDT scan matching) using the processed point cloud data generated by the downsampling processing unit 14 and the voxel data VD corresponding to the voxel to which the point cloud data belongs.

[0034] (3) Setting the downsampling size Next, the details of the downsampling processing unit 14's processing will be explained. In general terms, at each processing time, the downsampling processing unit 14 processes the corresponding number of measurement points Nc and the target range R indicated by the target range information 9. Nc The system compares the results after downsampling and sets the downsampling size for the next processing time according to the result of the comparison. This allows the downsampling processing unit 14 to determine when the number of corresponding measurement points Nc falls within the target range R. Nc The size of the downsampling is adaptively determined to maintain the internal values.

[0035] Figure 3 shows an example of the functional blocks of the downsampling processing unit 14. Functionally, the downsampling processing unit 14 includes a downsampling block 16, a corresponding measurement point acquisition block 17, a comparison block 18, and a downsampling size setting block 19. In Figure 3, blocks that exchange data are connected by solid lines, but the combination of blocks that exchange data is not limited to Figure 3. The same applies to the diagrams of other functional blocks described later.

[0036] The downsampling block 16 generates processed point cloud data for each processing time by downsampling the point cloud data of one scan cycle generated by the lidar 2 at each processing time. In this case, the downsampling block 16 divides the space into grids based on the set downsampling size and calculates representative points of the measurement points of the point cloud data of the lidar 2 contained within each grid. In this case, the space is, for example, a three-dimensional coordinate system space with the direction of travel, height, and lateral direction (i.e., the direction perpendicular to the direction of travel and height) of the in-vehicle device 1 as axes, and a grid is formed by dividing it at equal intervals along each of these directions. The downsampling size is the initial size stored in memory 12, etc., or the size set immediately before by the downsampling size setting block 19. The downsampling block 16 supplies the generated processed point cloud data to the vehicle position estimation unit 15.

[0037] The downsampling size may differ for the direction of travel, height, and lateral movement of the in-vehicle unit 1. Furthermore, any Voxel grid filter method may be used for processing the downsampling block 16.

[0038] The corresponding measurement point number acquisition block 17 acquires the corresponding measurement point number Nc from the host vehicle position estimation unit 15. In this case, the host vehicle position estimation unit 15 performs NDT matching for each processing time on the processed point cloud data supplied from the downsampling block 16, and outputs the corresponding measurement point number Nc for each processing time to the corresponding measurement point number acquisition block 17. The corresponding measurement point number acquisition block 17 supplies the corresponding measurement point number Nc to the comparison block 18.

[0039] The comparison block 18 performs a comparison of the magnitude between the target range R indicated by the target range information 9 and the corresponding measurement point number Nc for each processing time, and supplies the comparison result to the downsampling size setting block 19. In this case, the comparison block 18, for example, Nc determines whether the corresponding measurement point number Nc is greater than the upper limit of the target range R, or the corresponding measurement point number Nc is within the target range R, or Nc the corresponding measurement point number Nc is less than the lower limit of the target range R, and supplies the determination result to the downsampling size setting block 19. For example, when the target range R Nc is 600 to 800, the comparison block 18 performs the above determination with the upper limit of the target range R being 800 and the lower limit of the target range R being 600. Note that the target range R is not limited to 600 to 800, and is determined according to the hardware used, the performance of the processor, and the processing time to be completed (that is, the time interval of the next processing time). Nc Nc Nc Nc Nc

[0040] The downsampling size setting block (19) sets the size of the downsampling to be applied at the next processing time based on the comparison result of the comparison block 18. In this case, when the corresponding measurement point number Nc is greater than the upper limit of the target range R, Nc the downsampling size setting block 19 increases the downsampling size, and when the corresponding measurement point number Nc is within the target range R NcIf it is below the lower limit, the downsampling size is reduced. On the other hand, the downsampling size setting block 19 sets the corresponding number of measurement points Nc to the target range R. Nc If it falls within the specified range, maintain (i.e., do not change) the downsampling size.

[0041] Furthermore, Figure 3 illustrates a specific embodiment of a downsampling size setting block 19 having a gain setting subblock 191 and a multiplication subblock 192.

[0042] The gain setting subblock 191 sets the gain to be multiplied by the current downsampling size in the multiplication subblock 192, based on the comparison result in the comparison block 18. In this case, the gain setting subblock 191 sets the corresponding number of measurement points Nc to the target range R. Nc If the number exceeds the upper limit, increase the gain to more than 1, and the corresponding number of measurement points Nc is within the target range R. Nc If the value is less than the lower limit, the gain should be set to less than 1, and the corresponding measurement point Nc should be within the target range R. Nc If it is within the specified range, the gain is set to 1. In Figure 3, the gain is set as follows as an example. The number of corresponding measurement points Nc is within the target range R. Nc If it is greater than the upper limit ⇒ Gain "1.1", The number of corresponding measurement points Nc is within the target range R. Nc Less than the lower limit ⇒ Gain "1 / 1.1", The number of corresponding measurement points Nc is within the target range R. Nc Within the range ⇒ Gain "1.0"

[0043] The multiplication subblock 192 outputs a value obtained by multiplying the current downsampling size by the gain set by the gain setting subblock 191, as the downsampling size to be used at the next processing time.

[0044] With this configuration, the downsampling size setting block 19 sets the number of corresponding measurement points Nc to the target range R. NcIf the number of points exceeds the upper limit, the downsampling size can be increased to reduce the number of measurement points in the processed point cloud data. Similarly, the downsampling size setting block 19 sets the number of corresponding measurement points Nc to the target range R. Nc If the number of points is less than the lower limit, the downsampling size can be reduced to increase the number of measurement points in the processed point cloud data.

[0045] Note that the gain values ​​set in the gain setting subblock 191 are examples only, and should be set to appropriate values ​​based on experimental results, etc. Furthermore, the downsampling size setting block 19 may perform addition or subtraction of a predetermined value instead of multiplying the gain. In this case, the downsampling size setting block 19 sets the corresponding number of measurement points Nc to the target range R. Nc If the number exceeds the upper limit, the downsampling size is increased by a predetermined positive number, and the corresponding number of measurement points Nc is within the target range R. Nc If the value is less than the lower limit, the downsampling size is reduced by a predetermined positive value. Note that the above gain and predetermined value may be set to different values ​​for each direction (direction of travel, lateral direction, and height direction).

[0046] Here, we will provide a supplementary explanation regarding the validity of the configuration of the downsampling processing unit 14 shown in Figure 3.

[0047] Generally, while there is a relationship where a larger downsampling size results in fewer data points (i.e., fewer measurement points in the processed point cloud data), the relationship between the downsampling size and the number of data points is not linear. Therefore, it is not appropriate to determine the downsampling size using linear feedback control. Taking the above into consideration, in this embodiment, the target range R is expressed as a range for the target number of corresponding measurement points Nc. Nc By using this method, a dead zone is established, and the number of data points and the target range R are also determined. NcThe downsampling processing unit 14 is configured to gradually reach the target level of the corresponding measurement points Nc by setting a constant gain regardless of the difference amount.

[0048] Next, we will explain a specific example of downsampling. For the sake of simplicity, we will use voxel data VD corresponding to four voxels and the point cloud data belonging to those voxels as examples.

[0049] Figures 4(A) to 4(C) show the results of the first execution of downsampling with different downsampling sizes. In Figure 4(A), the downsampling size is set to 1x the voxel size, in Figure 4(B), the downsampling size is set to 2 / 3x the voxel size, and in Figure 4(C), the downsampling size is set to 1 / 2x the voxel size. Figures 4(A) to 4(C) clearly show the voxel data VD represented by a normal distribution, the point cloud data before downsampling, and the processed point cloud data after downsampling.

[0050] The downsampling block 16 generates a grid by dividing the space corresponding to the four target voxels according to the downsampling size set in each case of Figures 4(A) to 4(C). The downsampling block 16 then calculates the average of the positions indicated by each measurement point of the lider 2 for each grid, and generates the calculated average position as the measurement point of the processing point cloud data corresponding to the target grid.

[0051] Here, we consider the case where the target number of measurement points after downsampling (i.e., the number of corresponding measurement points Nc) is 4. In other words, we assume that the target number of points is 4 in order to achieve both sufficient position estimation accuracy and to keep the calculation within a predetermined time. In the examples of Figures 4(A) to 4(C), the measurement points before downsampling are relatively densely packed. Therefore, in the examples of Figures 4(B) and 4(C), where the downsampling size is relatively small, the number of measurement points after downsampling becomes excessive. That is, the matching computation load becomes large, and there is a possibility that the calculation will not be completed within the predetermined processing time. On the other hand, in the example of Figure 4(A), where the downsampling size is the largest, the number of measurement points after downsampling is 4. Therefore, in the examples of Figures 4(A) to 4(C), the number of measurement points after downsampling can be optimized by making the downsampling size the same as the voxel size.

[0052] Figures 5(A) to 5(C) show the results of a second execution where downsampling was performed with different downsampling sizes. In Figure 5(A), the downsampling size is set to 1x the voxel size, in Figure 5(B), the downsampling size is set to 2 / 3x the voxel size, and in Figure 5(C), the downsampling size is set to 1 / 2x the voxel size. Next, we consider the case where the target number of measurement points after downsampling (i.e., the number of corresponding measurement points Nc) is 4.

[0053] In this case, the voxel data and the measurement points before downsampling are relatively sparse. Therefore, in the examples of Figures 5(A) and 5(B), where the downsampling size is relatively large, the number of measurement points after downsampling is insufficient. In other words, there is a possibility that the accuracy of position estimation will deteriorate because there are fewer matching targets. In particular, in the case of Figure 5(A), it can be seen that the direction of rotation is not determined when matching voxel data and measurement points. On the other hand, in the example of Figure 5(C), where the downsampling size is smallest, the number of measurement points after downsampling is 4. Even when there is only one voxel data, it can be seen that matching is possible because the 4 measurement points surround that voxel data. Therefore, in the examples of Figures 5(A) to 5(C), the number of measurement points after downsampling can be optimized by setting the downsampling size to 1 / 2 the voxel size.

[0054] Figures 6(A) to 6(C) show the results of a third execution where downsampling was performed with different downsampling sizes. In Figure 6(A), the downsampling size is set to 1x the voxel size, in Figure 6(B), it is set to 2 / 3x the voxel size, and in Figure 6(C), it is set to 1 / 2x the voxel size. Next, we consider the case where the target number of measurement points after downsampling (i.e., the number of corresponding measurement points Nc) is 4.

[0055] In this case, in the example of Figure 6(A), where the downsampling size is largest, the number of measurement points after downsampling is too small, while in the example of Figure 6(C), where the downsampling size is smallest, the number of measurement points after downsampling is too large. On the other hand, in the example of Figure 6(B), where the downsampling size is intermediate, the number of measurement points after downsampling becomes 4. Therefore, in the examples of Figures 6(A) to 6(C), the number of measurement points after downsampling can be optimized by setting the downsampling size to 2 / 3 of the voxel size.

[0056] Thus, the appropriate downsampling size varies depending on the obtained point cloud data. Taking the above into consideration, in this embodiment, by adaptively changing the downsampling size, the number of corresponding measurement points Nc is brought within the target range R. Nc Maintain this within a predetermined time frame, keep the calculation time within the required time, and maintain the accuracy of the vehicle's position estimation.

[0057] Here, the effects of adaptively changing the downsampling size will be explained with reference to Figures 7(A) to 7(C).

[0058] Figures 7(A) to 7(C) show an overhead view of the area around the vehicle on which the in-vehicle unit 1 is mounted, when the downsampling size is set to 1 times the voxel size. In Figures 7(A) to 7(C), voxels containing voxel data VD are indicated by rectangular frames, and the positions of the measurement points of the lidar 2 obtained by scanning one cycle are indicated by dots. Here, Figure 7(A) shows an example where there are many structures around the lidar 2, Figure 7(B) shows an example where there are few structures around the lidar 2, and Figure 7(C) shows an example where the light beam of the lidar is blocked by another vehicle near the vehicle (called occlusion). Note that voxels corresponding to the surface positions of structures have corresponding voxel data VD.

[0059] As shown in Figures 7(A) to 7(C), the number of measurement points in the point cloud data acquired by Rider 2 depends not only on Rider 2's measurement capabilities and field of view, but also on the environment surrounding the vehicle. For example, in densely populated areas (see Figure 7(A)), the number of measurement points in the point cloud data is large, while in sparsely populated areas (see Figure 7(B)), the number of measurement points in the point cloud data is small. Furthermore, if occlusion occurs due to other vehicles (see Figure 7(C)), the number of corresponding measurement points Nc available for NDT scan matching decreases. In the example in Figure 7(A), the number of corresponding measurement points Nc available for NDT scan matching is sufficient, resulting in high position estimation accuracy by NDT scan matching. However, in the examples in Figures 7(B) and 7(C), the number of corresponding measurement points Nc is small, resulting in low position estimation accuracy by NDT scan matching. Therefore, it is necessary to reduce the downsampling size to increase the number of measurement points Nc. Furthermore, if the number of surrounding structures increases even more than in Figure 7(A), and the number of measurement points Nc increases, there is a possibility that the calculation may not be completed within the specified time. In that case, it will be necessary to set a larger downsampling size.

[0060] Taking the above into consideration, the in-vehicle device 1 according to this embodiment adaptively changes the downsampling size even when the environment around the vehicle changes, thereby reducing the number of corresponding measurement points Nc to the target range R Nc Adjust the settings to maintain the position within the specified range. This allows for optimal maintenance of the position estimation accuracy of NDT scan matching.

[0061] (4) NDT scan matching-based position estimation Next, we will explain the position estimation based on NDT scan matching performed by the vehicle position estimation unit 15.

[0062] Figure 8 shows the vehicle position to be estimated by the vehicle position estimation unit 15, represented in two-dimensional Cartesian coordinates. As shown in Figure 8, the vehicle position on a plane defined on the two-dimensional Cartesian coordinate system is represented by the coordinates "(x, y)" and the vehicle's orientation (yaw angle) "ψ". Here, the yaw angle ψ is defined as the angle between the vehicle's direction of travel and the x-axis. The coordinates (x, y) are, for example, an absolute position corresponding to a combination of latitude and longitude, or world coordinates indicating a position with a predetermined point as the origin. The vehicle position estimation unit 15 then performs vehicle position estimation using these x, y, and ψ as estimation parameters. In addition to x, y, and ψ, the vehicle position estimation unit 15 may also perform vehicle position estimation by further estimating at least one of the vehicle's height position, pitch angle, and roll angle in a three-dimensional Cartesian coordinate system as estimation parameters.

[0063] Next, we will explain the voxel data VD used for NDT scan matching. The voxel data VD includes data representing the measured point cloud data of stationary structures within each voxel according to a normal distribution.

[0064] Figure 9 shows an example of a schematic data structure for voxel data VD. Voxel data VD contains parameter information for representing the point cloud within the voxels using a normal distribution. In this embodiment, as shown in Figure 9, it includes voxel ID, voxel coordinates, mean vector, and covariance matrix.

[0065] "Voxel coordinates" represent the absolute three-dimensional coordinates of a reference position, such as the center position of each voxel. Each voxel is a cube that divides space into a grid, and its shape and size are predetermined; therefore, the space of each voxel can be identified using voxel coordinates. Voxel coordinates may also be used as voxel IDs.

[0066] The "mean vector" and "covariance matrix" represent the mean vector and covariance matrix, which correspond to the parameters when representing the point cloud within the target voxel using a normal distribution. Note that the coordinates of any point "i" within any voxel "n" are X n (i) = [x n(i), y n (i), z n (i)] T Defined as such, the number of points in voxel n is "N n If we do this, then the mean vector "μ" in voxel n n " and the covariance matrix "V n These are expressed by equations (1) and (2) below, respectively.

[0067]

number

[0068]

number

[0069] Next, we will explain the overview of NDT scan matching using voxel data VD.

[0070] Scan matching using NDT for a vehicle involves estimation parameters that include the amount of movement within the road plane (here, x and y coordinates) and the orientation of the vehicle. P=[t x t y t ψ ] T This will lead to the estimation of "t x " indicates the amount of movement in the x direction, and "t y " indicates the amount of movement in the y direction, and "t ψ " indicates the yaw angle.

[0071] Also, the coordinates of the point cloud data output by LIDA2 are X L (j)=[x(j), y(j), z(j)] T Therefore, X L (j) Average value "L' n This can be expressed by the following equation (3).

[0072]

number

[0073]

number

[0074]

number

[0075] The vehicle position estimation unit 15 then calculates an overall evaluation function value (also called a "score value") "E(k)" for all voxels subject to matching, as shown by the following equation (6).

[0076]

number

[0077]

number

[0078] Figure 10 shows an example of the functional blocks of the vehicle position estimation unit 15. As shown in Figure 10, the vehicle position estimation unit 15 includes a dead reckoning block 21, a position prediction block 22, a coordinate transformation block 23, a point cloud data mapping block 24, and a position correction block 25.

[0079] The dead reckoning block 21 uses the vehicle's speed and angular velocity based on the outputs of the gyro sensor 3, vehicle speed sensor 4, and GPS receiver 5 to determine the distance traveled and the change in direction from the previous time. The position prediction block 22 calculates the estimated vehicle position X at time k-1 calculated in the previous measurement update step. ^ Adding the calculated distance traveled and change in direction to (k-1), we predict the vehicle's position X at time k. - Calculate (k).

[0080] The coordinate transformation block 23 transforms the downsampled processed point cloud data output from the downsampling processing unit 14 into the world coordinate system, which is the same coordinate system as the map DB 10. In this case, the coordinate transformation block 23 performs a coordinate transformation on the processed point cloud data at time k, for example, based on the predicted vehicle position output by the position prediction block 22 at time k. Alternatively, instead of performing the above coordinate transformation on the downsampled processed point cloud data, the above coordinate transformation may be performed on the point cloud data before downsampling. In this case, the downsampling processing unit 14 generates processed point cloud data in the world coordinate system by downsampling the point cloud data in the world coordinate system after the coordinate transformation. Processes for transforming point cloud data in a coordinate system based on a lidar installed on a vehicle into a vehicle coordinate system, and processes for transforming from the vehicle coordinate system into a world coordinate system, etc., are disclosed, for example, in International Publication WO2019 / 188745.

[0081] The point cloud data mapping block 24 maps the processed point cloud data to the voxels by comparing the processed point cloud data in the world coordinate system output by the coordinate transformation block 23 with the voxel data VD, which is also represented in the same world coordinate system. The position correction block 25 calculates an individual evaluation function value based on equation (5) for each voxel that has been mapped to the processed point cloud data, and calculates the estimated parameter P that maximizes the score value E(k) based on equation (6). Then, based on equation (7), the position correction block 25 calculates the predicted vehicle position X output by the position prediction block 22. - By applying the estimated parameter P obtained at time k to (k), the estimated vehicle position X can be obtained. ^ Calculate (k).

[0082] Here, we will provide a supplementary explanation using a simple example regarding the specific procedure for mapping measurement points to voxel data VD.

[0083] Figure 11 shows the positional relationship between voxels "Vo1" to "Vo6" containing voxel data VD on a two-dimensional xy plane in the world coordinate system, and measurement points 61 to 65 indicating the locations near these voxels. For the sake of explanation, it is assumed that the z-coordinates in the world coordinate system of the center positions of voxels Vo1 to Vo6 and the z-coordinates in the world coordinate system of measurement points 61 to 65 are the same.

[0084] First, the coordinate transformation block 23 converts the point cloud data, including measurement points 61-65, into the world coordinate system. Then, the point cloud data mapping block 24 performs rounding operations on the fractional parts of the measurement points 61-65 in the world coordinate system. In the example in Figure 11, since the size of each voxel, which is a cube, is 1m, the point cloud data mapping block 24 rounds the decimal parts of the x, y, and z coordinates of each measurement point 61-65 to the nearest whole number.

[0085] Next, the point cloud data mapping block 24 determines the voxel corresponding to each measurement point 61 to 65 by comparing the voxel data VD corresponding to voxels Vo1 to Vo6 with the coordinates of each measurement point 61 to 65. In the example in Figure 11, the (x,y) coordinates of measurement point 61 are (2,1) due to the rounding described above, so the point cloud data mapping block 24 associates measurement point 61 with voxel Vo1. Similarly, the (x,y) coordinates of measurement points 62 and 63 are (3,2) due to the rounding described above, so the point cloud data mapping block 24 associates measurement points 62 and 63 with voxel Vo5. Also, the (x,y) coordinates of measurement point 64 are (2,3) due to the rounding described above, so the point cloud data mapping block 24 associates measurement point 64 with voxel Vo6. On the other hand, the (x,y) coordinates of measurement point 65 become (4,1) due to the rounding described above, so the point cloud data mapping block 24 determines that there is no voxel data VD corresponding to measurement point 65. Subsequently, the position correction block 25 estimates the estimated parameter P using the measurement points and voxel data VD mapped by the point cloud data mapping block 24. In this example, although the number of measurement points after downsampling is 5, the number of corresponding measurement points Nc is 4. As mentioned above, the number of corresponding measurement points Nc decreases when there are few structures around the road and the space is sparse, or when occlusion occurs because other vehicles are present near the vehicle.

[0086] (5) Processing flow Figure 12 is an example flowchart showing the procedure for vehicle position estimation performed by the controller 13 of the in-vehicle unit 1. The controller 13 starts the process shown in the flowchart of Figure 12 when it becomes necessary to perform vehicle position estimation, such as when the power is turned on.

[0087] First, the controller 13 sets the downsampling size to an initial value (step S11). Here, as an example, the downsampling size is set to 1m in the direction of travel, lateral direction, and height direction. Next, the controller 13 determines the predicted vehicle position based on the positioning results of the GPS receiver 5 (step S12).

[0088] The controller 13 then determines whether or not it was able to acquire point cloud data from the rider 2 (step S13). If the controller 13 is unable to acquire point cloud data from the rider 2 (step S13; No), it continues to perform the determination in step S13. During the period when point cloud data cannot be acquired, the controller 13 continues to determine the predicted vehicle position based on the positioning results of the GPS receiver 5. Note that the controller 13 may determine the predicted vehicle position not only based on the positioning results of the GPS receiver 5, but also based on the output of any sensor other than point cloud data.

[0089] Then, if the controller 13 has acquired point cloud data from the lidar 2 (step S13; Yes), it performs downsampling on the point cloud data for one period scanned by the lidar 2 at the current processing time, using the currently set downsampling size (step S14). In this case, the set downsampling size corresponds to the initial value determined in step S11, or the updated value determined in the immediately preceding steps S19 to S22.

[0090] The controller 13 then uses the vehicle's speed and angular velocity, based on the outputs of the gyro sensor 3 and the vehicle speed sensor 4, to determine the distance traveled and the change in orientation from the previous time. Based on this, the position prediction block 22 calculates the predicted vehicle position (which may include attitude angles) at the current processing time from the estimated vehicle position (which may include attitude angles such as yaw angle) obtained one time step ago (the most recent processing time) (step S15). If the estimated vehicle position one time step ago has not been calculated (i.e., step S23 has not been executed), the controller 13 may consider the predicted vehicle position determined in step S12 as the estimated vehicle position and proceed with the processing in step S15.

[0091] Next, the controller 13 performs NDT matching (step S16). In this case, the controller 13 performs processes such as converting the machined point cloud data to world coordinate system data, and associating the converted machined point cloud data with the voxels where the voxel data VD exists. The controller 13 may also perform a process to calculate the estimated vehicle position (including attitude angles such as yaw angle) at the current processing time based on the NDT matching.

[0092] Furthermore, the controller 13 calculates the number of corresponding measurement points Nc based on the result of the correspondence between the machining point cloud data and the voxel data VD in the NDT matching process (step S17). Then, the controller 13 sets the number of corresponding measurement points Nc to the target range R indicated by the target range information 9. Nc Compare with (Step S18). Here, as an example, target range R Nc We will explain assuming that the value is between 600 and 800.

[0093] Then, the number of corresponding measurement points Nc is within the target range R. Nc If the value is less than 600, which corresponds to the lower limit, the controller 13 changes the downsampling size setting to 1 / 1.1 (step S19). Note that "1 / 1.1" is just an example, and the controller 13 may change the setting by any multiplier less than 1 or by subtracting a value greater than 0. Furthermore, if a lower limit for the downsampling size is set, the controller 13 restricts the change to the downsampling size setting so that it does not fall below that lower limit (step S20).

[0094] Furthermore, the number of corresponding measurement points Nc is within the target range R. NcIf the value is greater than 800, which corresponds to the upper limit, the controller 13 changes the downsampling size setting to 1.1 times (step S21). Note that "1.1" is just an example, and the controller 13 may change the above setting by adding any multiplier greater than 1 or a value greater than 0. Furthermore, if an upper limit is set for the downsampling size, the controller 13 restricts the change in the downsampling size setting so as not to exceed that upper limit (step S22). Note that the controller 13 considers the corresponding number of measurement points Nc to be within the target range R Nc If it falls into this category, the downsampling size setting is maintained.

[0095] Then, based on the NDT matching results, the controller 13 calculates the estimated vehicle position (including attitude angles such as yaw angle) at the current processing time (step S23). The controller 13 then determines whether or not to terminate the vehicle position estimation process (step S24). If the controller 13 determines that the vehicle position estimation process should be terminated (step S24; Yes), it terminates the process in the flowchart. On the other hand, if the controller 13 decides to continue the vehicle position estimation process (step S24; No), it returns to step S13 and estimates the vehicle position at the next processing time.

[0096] (6) Variation The following describes suitable modifications of the above-described embodiments. These modifications may be applied in combination to these embodiments.

[0097] (Variation 1) The in-vehicle device 1 may fix the downsampling size in the direction with relatively poor position estimation accuracy (also called the "low-accuracy direction") among the coordinate axes of the three-dimensional coordinate system that constitutes the space in which downsampling is performed, and adaptively change the downsampling size in the other directions based on the above embodiment. The low-accuracy direction is, for example, the direction of travel. The low-accuracy direction may also be the lateral direction or the height direction.

[0098] In this case, for example, the low-accuracy direction is empirically identified before the vehicle position estimation process is executed, depending on the specifications of the lidar 2 or driver assistance system used, and a fixed value for the downsampling size in the low-accuracy direction is stored in memory 12. The in-vehicle machine 1 then sets the downsampling size in the low-accuracy direction to the fixed value stored in memory 12, and changes the downsampling size in other directions in steps S19 to S22 of the flowchart in Figure 12. By doing so, the in-vehicle machine 1 can ensure the resolution of the processed point cloud data in the low-accuracy direction and reliably suppress the decrease in position estimation accuracy in the low-accuracy direction.

[0099] (Modification 2) The in-vehicle device 1 may set the downsampling size in the low-precision direction to be less than one times the downsampling size in the other directions.

[0100] For example, the in-vehicle machine 1 determines the downsampling size for each direction based on a size determined based on the number of corresponding measurement points Nc (also called the "reference size") and a ratio predetermined for each direction according to accuracy, importance, etc. For example, the in-vehicle machine 1 sets the downsampling size in the direction of travel to half the reference size determined in step S11 or steps S19 to S22 of the flowchart in Figure 12. On the other hand, the in-vehicle machine 1 sets the downsampling size in the lateral direction to twice the reference size and the downsampling size in the height direction to be the same as the reference size (equal size). In this way, the in-vehicle machine 1 can always increase the relative resolution of the processed point cloud data with respect to other directions in the low-accuracy direction.

[0101] (Variation 3) The in-vehicle device 1 may measure the processing time (length of processing time) of the vehicle position estimation process (including downsampling) that is executed at each processing time, and set the downsampling size according to the measured processing time.

[0102] For example, if the execution cycle of the vehicle position estimation process is 83.3 ms, the in-vehicle unit 1 reduces the downsampling size by a predetermined value or a predetermined percentage if the processing time is less than 30 ms. On the other hand, if the processing time is longer than 70 ms, the in-vehicle unit 1 increases the downsampling size by a predetermined value or a predetermined percentage. Furthermore, if the processing time is between 30 ms and 70 ms, the in-vehicle unit 1 maintains the downsampling size setting. Note that the in-vehicle unit 1 may change the downsampling size setting based on this modified example, for example, after the execution of step S23 in Figure 12, or immediately before or after steps S18 to S22.

[0103] According to this modification, the in-vehicle device 1 can suitably set the downsampling size so that the processing time falls within an appropriate range with respect to the execution cycle.

[0104] (Modification 4) The configuration of the driver assistance system shown in Figure 1 is an example, and the configuration of a driver assistance system to which the present invention can be applied is not limited to the configuration shown in Figure 1. For example, instead of having an in-vehicle unit 1, the driver assistance system may have the vehicle's electronic control unit perform the processing of the downsampling processing unit 14 and the vehicle position estimation unit 15 of the in-vehicle unit 1. In this case, the map DB 10 is stored, for example, in a storage unit in the vehicle or in a server device that communicates data with the vehicle, and the vehicle's electronic control unit refers to this map DB 10 to perform downsampling and vehicle position estimation based on NDT scan matching, etc.

[0105] (Variation 5) The voxel data VD is not limited to a data structure that includes a mean vector and a covariance matrix, as shown in Figure 9. For example, the voxel data VD may include the point cloud data measured by the measurement and maintenance vehicle used to calculate the mean vector and covariance matrix.

[0106] (7) Discussion based on experimental results Next, we will discuss the experimental results related to the above-described embodiments (including modified examples). The applicant drove a vehicle equipped with two lidars at the front and two at the rear, each with a horizontal field of view of 60 degrees and an operating frequency of 12 Hz (period of 83.3 ms), along a certain road. Position estimation based on NDT was performed by matching pre-created voxel data (ND map) for the road with point cloud data from the lidars obtained during the drive. In addition, for accuracy evaluation, RTK-GPS positioning results were used as the ground truth position data.

[0107] Figures 13(A) to 13(F) and 14(A) to 14(E) show the results of downsampling to a cubic grid size of 0.5m on each side and self-position estimation using NDT processing. Here, for the RTK-GPS positioning results, Figure 13(A) shows the error in the direction of travel, Figure 13(B) shows the error in the lateral direction, Figure 13(C) shows the error in the height direction, and Figure 13(D) shows the error in the yaw angle. Figure 13(E) shows the number of measurement points in the point cloud data before downsampling, and Figure 13(F) shows the size of the downsampling. Furthermore, Figure 14(A) shows the number of measurement points in the processed point cloud data after downsampling, Figure 14(B) shows the number of corresponding measurement points Nc, Figure 14(C) shows the ratio of the number of corresponding measurement points Nc to the number of measurement points in the processed point cloud data (also called the "correspondence rate"), Figure 14(D) shows the score value E(k), and Figure 14(E) shows the processing time. In Figure 14(E), the period of the lidar (83.3 ms) is clearly indicated by a horizontal line.

[0108] Here, as shown in Figure 14(E), there is a period in which the processing time exceeds the lidar period of 83.3 ms. During this period, the number of measurement points before downsampling (shown in Figure 13(E)), the number of measurement points after downsampling (shown in Figure 14(A)), and the number of corresponding measurement points Nc (shown in Figure 14(B)) are all large. Thus, it can be seen that there is a positive correlation between the number of point cloud data and the processing time.

[0109] Furthermore, as shown in Figure 13(A), there is a momentary increase in the error in the direction of travel around 170s. It can be inferred that the estimated position shifted because the area where the vehicle was traveling at that time had few features in the direction of travel, and the processing time exceeded the period of the rider, resulting in insufficient processing time.

[0110] Thus, processing needs to be completed in time with the lidar's cycle, and if the downsampling size is fixed, this problem may not be solved.

[0111] Figures 15(A) to 15(F) and 16(A) to 16(E) show the results of downsampling to a cubic grid size of 2.0m on each side and estimating the vehicle's position using NDT processing. Here, for the RTK-GPS positioning results, Figure 15(A) shows the error in the direction of travel, Figure 15(B) shows the error in the lateral direction, Figure 15(C) shows the error in the height direction, and Figure 15(D) shows the error in the yaw angle. Figure 15(E) shows the number of measurement points in the point cloud data before downsampling, and Figure 15(F) shows the downsampling size. In addition, Figure 16(A) shows the number of measurement points in the processed point cloud data after downsampling, Figure 16(B) shows the number of corresponding measurement points Nc, Figure 16(C) shows the correspondence rate, Figure 16(D) shows the score value E(k), and Figure 16(E) shows the processing time. In Figure 16(E), the period of the lidar (83.3 ms) is clearly indicated by a horizontal line.

[0112] In this case, compared to the examples in Figures 13(A) to 13(F) and Figures 14(A) to 14(E), increasing the downsampling size reduces the number of corresponding measurement points Nc shown in Figure 16(B), and also reduces the processing time shown in Figure 16(E). As a result, the processing time remains within the lidar period of 83.3 ms in all cases. On the other hand, the error in the direction of travel shown in Figure 15(A) has increased (especially at 250s to 280s). This is presumed to be due to a decrease in the resolution of the processed point cloud data and a reduction in the number of measurement points after downsampling. Note that if the number of measurement points in the processed point cloud data is too small, there may be insufficient measurement points to match in the NDT process, which can reduce stability.

[0113] Thus, even when the downsampling size is set to a larger, fixed size, it is not possible to achieve both the maintenance of vehicle position estimation accuracy and the reduction of processing time.

[0114] Figures 17(A) to 17(F) and 18(A) to 18(E) show the results of vehicle position estimation by downsampling and NDT processing according to the processing procedure shown in Figure 12 for this embodiment. Here, for the RTK-GPS positioning result, Figure 17(A) shows the error in the direction of travel, Figure 17(B) shows the error in the lateral direction, Figure 17(C) shows the error in the height direction, and Figure 17(D) shows the error in the yaw angle. Figure 17(E) shows the number of measurement points in the point cloud data before downsampling, and Figure 17(F) shows the size of the downsampling. In addition, Figure 18(A) shows the number of measurement points in the processed point cloud data after downsampling, Figure 18(B) shows the number of corresponding measurement points Nc, Figure 18(C) shows the correspondence rate, Figure 18(D) shows the score value E(k), and Figure 18(E) shows the processing time. In Figure 18(E), the period of the lidar (83.3 ms) is clearly indicated by a horizontal line.

[0115] In this case, as shown in Figure 17(F), the downsampling size changes dynamically. As a result, the processing time shown in Figure 18(E) is always below the lidar period of 83.3 ms, and the error in the direction of travel shown in Figure 17(A) is also suitably suppressed.

[0116] Thus, by adaptively changing the downsampling size based on this embodiment, it is possible to achieve both the maintenance of vehicle position estimation accuracy and the reduction of processing time.

[0117] Figures 19(A) to 19(G) show the results of self-position estimation by downsampling and NDT processing according to the processing procedure shown in Figure 12 for this embodiment, when occlusion by another vehicle occurs. Figures 20(A) to 20(G) show the results under the same conditions as Figures 19(A) to 19(G), but instead of the corresponding number of measurement points Nc, the downsampling size is determined so that the number of measurement points after downsampling falls between 600 and 800. Here, Figures 19(A) and 20(A) show the number of measurement points before downsampling, Figures 19(B) and 20(B) show the downsampling size, Figures 19(C) and 20(C) show the number of measurement points after downsampling, Figures 19(D) and 20(D) show the number of corresponding measurement points Nc, Figures 19(E) and 20(E) show the correspondence rate, Figures 19(F) and 20(F) show the score value E(k), and Figures 19(G) and 20(G) show the processing time.

[0118] In these examples, occlusion by other vehicles occurs for a while after 180s. Furthermore, as shown in the examples in Figures 20(A) to 20(G), if the downsampling size is controlled based on the number of measurement points before they are mapped to the voxel data VD, the number of corresponding measurement points Nc becomes insufficient when occlusion occurs. As a result, as shown in Figure 20(D), the number of corresponding measurement points Nc decreases during the period when occlusion occurs. On the other hand, when the downsampling size is controlled according to this embodiment, as shown in Figure 19(D), the fluctuation in the number of corresponding measurement points Nc is small even during the period when occlusion occurs. Therefore, the stability of position estimation in the examples in Figures 20(A) to 20(G) is reduced compared to the examples in Figures 19(A) to 19(G). From this example, it can be seen that if there are many measurement points that cannot be mapped to voxel data based on stationary structures due to occlusion, it is not advisable to control the measurement points after downsampling to fall within the target range.

[0119] Figures 21(A) to 21(F) and 22(A) to 22(E) show the results of self-position estimation by NDT processing, based on Modification 1, where the downsampling size in the direction of travel is fixed at 0.5m, and the downsampling sizes in the lateral and vertical directions are adaptively changed. Specifically, for the RTK-GPS positioning results, Figure 21(A) shows the error in the direction of travel, Figure 21(B) shows the error in the lateral direction, Figure 21(C) shows the error in the vertical direction, and Figure 21(D) shows the error in the yaw angle. Figure 21(E) shows the number of measurement points in the point cloud data before downsampling, and Figure 21(F) shows the downsampling size. In addition, Figure 22(A) shows the number of measurement points in the processed point cloud data after downsampling, Figure 22(B) shows the number of corresponding measurement points Nc, Figure 22(C) shows the correspondence rate, Figure 22(D) shows the score value E(k), and Figure 22(E) shows the processing time. In Figure 22(E), the period of the lidar (83.3 ms) is clearly indicated by a horizontal line.

[0120] In this example, considering that there are many guardrails, shoulders, and median strips on both sides of the road, and that the lateral error is better than the error in the direction of travel, the direction of travel is considered the low-accuracy direction, and the downsampling size in the direction of travel is fixed at 0.5m. As a result of fixing the downsampling size in the direction of travel to 0.5m, the downsampling sizes in the lateral and vertical directions are larger than when the downsampling size is adaptively changed, including the direction of travel (see Figure 17(F)), as shown in Figure 21(F). On the other hand, in the modified example 1 as well, by controlling the corresponding number of measurement points Nc shown in Figure 22(B) to be maintained at 600-800, the processing time shown in Figure 22(E) is also suitably within the lidar cycle.

[0121] As a result, when Modification 1 is performed, the lateral error shown in Figure 21(B) remains almost unchanged, while the error in the direction of travel (especially around 170s) shown in Figure 21(A) is reduced. Thus, even when controlling the downsampling size based on Modification 1, it is possible to suitably achieve both maintaining the accuracy of self-position estimation and suppressing processing time.

[0122] As described above, the controller 13 of the in-vehicle device 1 in this embodiment acquires point cloud data output by the lidar 2. The controller 13 then generates processed point cloud data by downsampling the point cloud data. The controller 13 then matches the processed point cloud data with the voxel data VD, which represents the position of an object in each voxel, which is a unit area, to each of the measurement points that make up the processed point cloud data. The controller 13 then changes the size of the downsampling to be performed next based on the number of corresponding measurement points Nc, which is the number of measurement points in the processed point cloud data that are associated with voxels. As a result, the in-vehicle device 1 can adaptively determine the size of the downsampling and suitably achieve both the accuracy of self-localization and processing time.

[0123] In the embodiments described above, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer, such as a controller. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs (Random Access Memory)).

[0124] The present invention has been described above with reference to the examples, but the present invention is not limited to the above examples. Various modifications to the structure and details of the present invention can be made that can be understood by a person skilled in the art within the scope of the present invention. That is, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical idea. Furthermore, each disclosure of the above-mentioned patent documents, etc., that has been cited is incorporated herein by reference. [Explanation of Symbols]

[0125] 1 On-vehicle device 2 Riders 3. Gyroscope 4. Vehicle speed sensor 5 GPS receivers 10 Map Database

Claims

1. An acquisition means for acquiring point cloud data output by a measuring device, A downsampling processing means for generating processed point cloud data by downsampling the aforementioned point cloud data, The system includes a matching means that performs correspondence between the measurement points constituting the processed point cloud data and each of the voxels by comparing the processed point cloud data with voxel data representing the position of an object in each voxel, which is a unit area. The downsampling processing means is an information processing device that changes the size of the next downsampling to be performed based on the number of corresponding measurement points, which is the number of measurement points among the measurement points that are associated with the voxel.

2. The information processing apparatus according to claim 1, wherein the downsampling processing means fixes the size in a first direction among the directions for dividing space in the downsampling, and changes the size in directions other than the first direction based on the corresponding number of measurement points.

3. The information processing apparatus according to claim 2, wherein the first direction is the direction of travel of the measuring device.

4. The information processing apparatus according to any one of claims 1 to 3, wherein the downsampling processing means changes the size based on a comparison between the corresponding number of measurement points and the target range of the corresponding number of measurement points.

5. The downsampling processing means is If the number of corresponding measurement points exceeds the upper limit of the target range, the size is increased by a predetermined percentage or a predetermined value. If the number of corresponding measurement points is less than the lower limit of the target range, the size is reduced by a predetermined percentage or a predetermined value. If the number of corresponding measurement points is within the target range, the size is maintained. The information processing apparatus according to claim 4.

6. The information processing apparatus according to any one of claims 1 to 5, wherein the downsampling processing means determines a reference size for the next downsampling to be performed based on the corresponding number of measurement points, and determines the size for each direction based on the reference size and a ratio set for each direction in which the space is divided in the downsampling.

7. The information processing apparatus according to any one of claims 1 to 6, further comprising a position estimation means for estimating the position of a moving body equipped with the measuring device based on the result of comparing the voxel data of the voxel that has been associated by the association means with the measurement point associated with the voxel.

8. The information processing apparatus according to claim 7, wherein the downsampling processing means changes the size based on the processing time of the downsampling processing means and the position estimation means.

9. The information processing apparatus according to claim 8, wherein the downsampling processing means changes the size based on a comparison of the processing time with a target range for the processing time.

10. A control method performed by a computer, The point cloud data output by the measuring device is acquired, Processed point cloud data is generated by downsampling the aforementioned point cloud data. By comparing the aforementioned processed point cloud data with voxel data representing the position of an object in each voxel, which is a unit area, an association is made between the measurement points constituting the processed point cloud data and each of the voxels. A control method for changing the size of the next downsampling to be performed, based on the number of corresponding measurement points, which is the number of measurement points among the measurement points that are associated with the voxel.

11. The point cloud data output by the measuring device is acquired, Processed point cloud data is generated by downsampling the aforementioned point cloud data. By comparing the aforementioned processed point cloud data with voxel data representing the position of an object in each voxel, which is a unit area, an association is made between the measurement points constituting the processed point cloud data and each of the voxels. A program that causes a computer to perform a process to change the size of the downsampling that is to be executed next, based on the number of corresponding measurement points, which is the number of measurement points among the measurement points that are associated with the voxel.

12. A storage medium storing the program described in claim 11.

Images