Information processor, control method, program and storage medium

Abstract

To provide an information processor that can execute downsampling in a suitable manner.SOLUTION: A controller 13 of an on-vehicle machine 1 acquires point group data to be output by a lidar 2 mounted on a moving object, or a vehicle. Then, the controller 13 acquires a position estimation performance prediction value that is prediction information regarding performance of self-position estimation associated with an area to which the position of the vehicle belongs on map data. Then, the controller 13 executes downsampling for the point group data based on the position estimation performance prediction value.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to downsampling of data used for position estimation. [Background technology]

[0002] Conventionally, a technology is known for estimating the vehicle's own position by collating (matching) shape data of surrounding objects measured by a measuring device such as a laser scanner with map information in which the shapes of surrounding objects are stored in advance. For example, Patent Document 1 discloses an autonomous mobile system that determines whether a detected object in a voxel obtained by dividing a space according to a predetermined rule is a stationary object or a moving object, and matches map information with measurement data for voxels in which stationary objects exist. In addition, Patent Document 2 discloses a scan matching method that estimates the vehicle's own position by collating voxel data including the mean vector and covariance matrix of stationary objects for each voxel with point cloud data output by a lidar. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication WO2013 / 076829 [Patent Document 2] International Publication WO2018 / 221453 Summary of the Invention [Problem to be solved by the invention]

[0004] As in Patent Document 2, when estimating the vehicle position by matching voxel data with point cloud data output by a lidar, a downsampling process is performed, which is a process of thinning out the lidar point cloud data by averaging it in a space divided into a predetermined size. By performing this downsampling, the density of the point cloud data, which is dense in areas close to the vehicle and sparse in areas far from the vehicle, can be made uniform, and the total number of point cloud data can be reduced, thereby improving the accuracy of the self-position estimation and reducing the calculation time. However, in a space with few objects around the vehicle, the number of point cloud data detected by the lidar is small, so the downsampling process may significantly reduce the number of point cloud data used for calculation, resulting in a problem of reduced accuracy and robustness of the self-position estimation.

[0005] The present invention has been made in order to solve the above-mentioned problems, and has as its main object to provide an information processing device capable of suitably performing downsampling. [Means for solving the problem]

[0006] The claimed invention is A point cloud data acquisition means for acquiring point cloud data output by a measuring device provided on the moving object; a prediction information acquisition means for acquiring prediction information relating to a performance of self-location estimation, the prediction information being associated with the position of the moving object in map data; a downsampling unit that performs downsampling on the point cloud data based on the prediction information; The information processing device has the following features.

[0007] The claimed invention also includes: A computer-implemented control method, comprising: Acquire point cloud data output by a measuring device installed on the moving object, Obtaining predicted information on the performance of self-location estimation associated with the position of the moving object in map data; performing downsampling on the point cloud data based on the prediction information; It is a control method.

[0008] The claimed invention also includes: Acquire point cloud data output by a measuring device installed on the moving object, Obtaining predicted information on the performance of self-location estimation associated with the position of the moving object in map data; The program causes a computer to execute a process of performing downsampling on the point cloud data based on the prediction information. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic configuration diagram of a driving assistance system. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of an in-vehicle device. [Diagram 3] 4 shows an example of a functional block of a downsampling processing unit. [Figure 4] 13 shows an example of a data structure of location estimation performance prediction information. [Diagram 5] FIG. 11 is a diagram showing a first execution result in which downsampling is performed using different downsampling sizes. [Figure 6] An overhead view of the vehicle and its surroundings on which the onboard device is mounted is shown. [Figure 7] 2 is a diagram showing a vehicle position to be estimated by a vehicle position estimation unit in two-dimensional orthogonal coordinates. [Figure 8] 2 shows an example of a schematic data structure of voxel data. [Figure 9] 4 is an example of a functional block of a vehicle position estimation unit. [Figure 10] This shows the positional relationship between voxels in which voxel data exists and measurement points indicating positions near these voxels on a two-dimensional plane in the world coordinate system. [Figure 11] 4 is an example of a flowchart showing a procedure of a process relating to vehicle position estimation. [Figure 12]This shows an aerial view of a road, clearly showing slice lines (grid lines) set at equal intervals in the direction of travel and horizontally when points on different roads are targeted. [Figure 13] FIG. 13 is a diagram clearly illustrating three slice lines extracted from the slice lines shown in FIG. 12(A) in the horizontal direction and three slice lines extracted from the slice lines in the traveling direction. [Figure 14] 14 is a diagram showing voxels for which voxel data exists on a cut surface cut by a slice line shown in FIG. 13. FIG. [Figure 15] FIG. 13 is a diagram clearly illustrating three slice lines each extracted in the lateral direction and the traveling direction from among the slice lines shown in FIG. 12(B). [Figure 16] 16 is a diagram showing voxels for which voxel data exists on a cut surface cut by a slice line shown in FIG. 15. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] According to a preferred embodiment of the present invention, an information processing device includes a point cloud data acquisition means for acquiring point cloud data output by a measuring device provided on a moving object, a prediction information acquisition means for acquiring prediction information related to a performance of a self-location estimation that is associated with a position of the moving object in map data, and a downsampling means for performing downsampling on the point cloud data based on the prediction information. According to this aspect, the information processing device can preferably perform downsampling based on the prediction information related to the performance of the self-location estimation according to the position of the moving object by referring to the map data.

[0011] In one aspect of the information processing device, the map data is associated with the prediction information for each area, and the prediction information acquisition means acquires the prediction information corresponding to the area to which the position of the mobile object belongs from the map data. With this aspect, the information processing device can acquire appropriate prediction information from the prediction information of the position estimation performance provided for each area and reflect it in the downsampling process.

[0012] In another aspect of the information processing device, the map data is associated with prediction information for each direction in which a space is divided in the downsampling, the prediction information acquisition means acquires the prediction information for each direction from the map data, and the downsampling means determines a downsampling size for each direction based on the prediction information for each direction. With this aspect, the information processing device can appropriately determine a downsampling size for each direction in which a space is divided in the downsampling.

[0013] In another aspect of the information processing device, the downsampling means reduces the size of the downsampling in the corresponding direction as the prediction information for each direction becomes lower. With this aspect, the information processing device can ensure the resolution of the point cloud data in the direction in which the position estimation accuracy is predicted to be low, and can suitably suppress a decrease in the position estimation accuracy.

[0014] In another aspect of the information processing device, the downsampling means determines a direction in which the downsampling size is fixed and a direction in which the downsampling size is variable for each processing time based on the prediction information for each direction. With this aspect, the information processing device can prioritize position estimation accuracy while maintaining the resolution of the point cloud data for the direction in which the downsampling size is fixed.

[0015] In another aspect of the information processing device, the downsampling means determines an initial value of a downsampling size that is variable for each processing time based on the prediction information for each direction. With this aspect, the information processing device can appropriately set the initial value of the downsampling size.

[0016] In another aspect of the information processing device, the information processing device further includes a matching means for matching measurement points constituting the processed point cloud data with each of the voxels by comparing the processed point cloud data generated by the downsampling means from the point cloud data with voxel data representing the position of an object for each voxel which is a unit area, and a downsampling size determination means for determining the size of the downsampling to be performed next based on the number of corresponding measurement points, which is the number of measurement points among the measurement points that are associated with the voxel. With this aspect, the information processing device can suitably change the downsampling size in a direction in which the downsampling size is variable, so as to achieve both position estimation accuracy and overall processing speed.

[0017] In another aspect of the information processing device, the information processing device further includes a position estimation unit that estimates the position of the moving object based on a comparison result between the processed point cloud data generated from the point cloud data by the downsampling unit and voxel data representing the position of the object for each voxel, which is a unit area. In this aspect, the information processing device performs downsampling with an appropriately set size by referring to map data, and uses the processed point cloud data generated by the downsampling for position estimation, thereby preferably achieving both accuracy of position estimation and processing time.

[0018] According to another preferred embodiment of the present invention, there is provided a control method executed by a computer, which includes acquiring point cloud data output by a measuring device provided on a moving object, acquiring prediction information on a performance of a self-location estimation that is associated with a position of the moving object in map data, and performing downsampling on the point cloud data based on the prediction information. By executing this control method, an information processing device can suitably perform downsampling based on the prediction information on a performance of a self-location estimation according to a position of the moving object.

[0019] According to yet another preferred embodiment of the present invention, a program causes a computer to execute a process of acquiring point cloud data output by a measuring device provided on a moving object, acquiring prediction information on the performance of self-location estimation associated with the position of the moving object in map data, and performing downsampling on the point cloud data based on the prediction information. By executing this program, the computer can preferably perform downsampling based on the prediction information on the performance of self-location estimation according to the position of the moving object. Preferably, the program is stored in a storage medium. EXAMPLES

[0020] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In this specification, for the sake of convenience, any symbol followed by "^" or "-" is referred to as "A^" or "A - ” (where “A” is any letter).

[0021] (1) Overview of the driving assistance system 1 is a schematic configuration of a driving assistance system according to the present embodiment. The driving assistance system includes an on-board device 1 that moves together with a vehicle, which is a moving body, a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 2, a gyro sensor 3, a vehicle speed sensor 4, and a GPS receiver 5.

[0022] The vehicle-mounted device 1 is electrically connected to the lidar 2, the gyro sensor 3, the vehicle speed sensor 4, and the GPS receiver 5, and estimates the position of the vehicle in which the vehicle-mounted device 1 is installed (also referred to as "own vehicle position") based on the outputs of these sensors. Then, based on the estimation result of the own vehicle position, the vehicle-mounted device 1 performs automatic driving control of the vehicle so that the vehicle travels along a route to a set destination. The vehicle-mounted device 1 stores a map database (DB: DataBase) 10 including voxel data "VD". The voxel data VD is data that records position information of a stationary structure for each voxel that indicates a cube (regular lattice) that is the smallest unit of a three-dimensional space. The voxel data VD includes data that expresses measured point cloud data of a stationary structure in each voxel using a normal distribution, and is used for scan matching using NDT (Normal Distributions Transform) as described later. The vehicle-mounted device 1 estimates at least the vehicle's position on a plane and yaw angle by NDT scan matching. The vehicle-mounted device 1 may further estimate the vehicle height position, pitch angle, and roll angle. Unless otherwise specified, the vehicle position is assumed to include the vehicle's attitude angle, such as the yaw angle, to be estimated.

[0023] The lidar 2 measures the distance to an object in the outside world discretely by emitting a pulsed laser in a predetermined angular range in the horizontal and vertical directions, and generates three-dimensional point cloud data indicating the position of the object. In this case, the lidar 2 has an irradiation unit that irradiates a laser beam while changing the irradiation direction, a light receiving unit that receives the reflected light (scattered light) of the irradiated laser beam, and an output unit that outputs scan data (points that constitute the point cloud data, hereinafter referred to as "measurement points") based on the light receiving signal output by the light receiving unit. The measurement points are generated based on the irradiation direction corresponding to the laser beam received by the light receiving unit and the response delay time of the laser beam specified based on the above-mentioned light receiving signal. In general, the closer the distance to the object, the higher the accuracy of the distance measurement value of the lidar, and the farther the distance, the lower the accuracy. The lidar 2, the gyro sensor 3, the vehicle speed sensor 4, and the GPS receiver 5 each supply output data to the vehicle-mounted device 1.

[0024] The vehicle-mounted device 1 is an example of an "information processing device" in the present invention, and the lidar 2 is an example of a "measurement device" in the present invention. Note that the driving assistance system may have an inertial measurement unit (IMU) that measures the acceleration and angular velocity of the measurement vehicle in three axial directions instead of or in addition to the gyro sensor 3.

[0025] (2) Vehicle-mounted device configuration 2 is a block diagram showing an example of a hardware configuration of the vehicle-mounted device 1. The vehicle-mounted device 1 mainly includes an interface 11, a memory 12, and a controller 13. These elements are connected to each other via a bus line.

[0026] The interface 11 performs an interface operation related to the transmission and reception of data between the in-vehicle device 1 and an external device. In this embodiment, the interface 11 acquires output data from sensors such as the lidar 2, the gyro sensor 3, the vehicle speed sensor 4, and the GPS receiver 5, and supplies the data to the controller 13. The interface 11 also supplies a signal related to the vehicle driving control generated by the controller 13 to an electronic control unit (ECU) of the vehicle. The interface 11 may be a wireless interface such as a network adapter for wireless communication, or may be a hardware interface for connecting to an external device via a cable or the like. The interface 11 may perform an interface operation with various peripheral devices such as an input device, a display device, and a sound output device.

[0027] The memory 12 is composed of various types of volatile and non-volatile memories such as a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk drive, a flash memory, etc. The memory 12 stores a program for the controller 13 to execute a predetermined process. The program executed by the controller 13 may be stored in a storage medium other than the memory 12.

[0028] The memory 12 also stores a map DB 10 including voxel data VD and position estimation performance prediction information PD.

[0029] The position estimation performance prediction information PD is information indicating a predicted value of position estimation performance by NDT scan matching using the voxel data VD (also called "predicted position estimation performance value"). In this case, for example, the position estimation performance prediction information PD is information in which a predicted position estimation performance value is associated with each area partitioned according to a predetermined rule, and details of the data structure will be described later with reference to FIG. 4. The predicted position estimation performance value is an example of "prediction information".

[0030] The map DB 10 may be stored in a storage device external to the vehicle-mounted device 1, such as a hard disk connected to the vehicle-mounted device 1 via the interface 11. The storage device may be a server device that communicates with the vehicle-mounted device 1. The storage device may be composed of a plurality of devices. The map DB 10 may be updated periodically. In this case, for example, the controller 13 receives partial map information related to the area to which the vehicle position belongs from a server device that manages map information via the interface 11, and reflects the partial map information in the map DB 10.

[0031] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire vehicle-mounted device 1. In this case, the controller 13 executes a program stored in the memory 12 or the like to perform processing related to vehicle position estimation.

[0032] Moreover, functionally, the controller 13 has a downsampling processing unit 14 and a vehicle position estimation unit 15. The controller 13 functions as a "point cloud data acquisition means", a "prediction information acquisition means", a "downsampling size determination means", a "downsampling means", a "position estimation means", a computer that executes a program, and the like.

[0033] The downsampling processing unit 14 generates point cloud data (also called "processed point cloud data") that has been corrected to reduce the number of measurement points by downsampling the point cloud data output from the LIDAR 2. In this downsampling, the downsampling processing unit 14 averages the point cloud data within a space divided into a predetermined size to generate processed point cloud data with a reduced number of measurement points. In this case, the downsampling processing unit 14 determines the downsampling size by referring to the position estimation performance prediction information PD.

[0034] The vehicle position estimation unit 15 estimates the vehicle position by performing NDT-based scan matching (NDT scan matching) based on the processed point cloud data generated by the downsampling processing unit 14 and the voxel data VD corresponding to the voxel to which the point cloud data belongs.

[0035] (3) Setting the downsampling size Next, a detailed description will be given of the process of the downsampling processor 14. In summary, the downsampling processor 14 refers to the position estimation performance prediction information PD and determines the downsampling size based on the position estimation performance prediction value corresponding to the area corresponding to the current position. As a result, the downsampling processor 14 appropriately sets the downsampling size so that the position estimation accuracy of NDT scan matching does not decrease.

[0036] Fig. 3 shows an example of functional blocks of the downsampling processing unit 14. Functionally, the downsampling processing unit 14 has a downsampling size setting block 16 and a downsampling block 17. Note that in Fig. 3, blocks between which data is exchanged are connected by solid lines, but the combination of blocks between which data is exchanged is not limited to that shown in Fig. 3. The same applies to other functional block diagrams described later.

[0037] The downsampling size setting block 16 refers to the position estimation performance prediction information PD and acquires a position estimation performance prediction value in the area where the vehicle-mounted device 1 exists based on the predicted vehicle position at the current processing time supplied from the vehicle position estimation unit 15. Then, the downsampling size setting block 16 determines a downsampling size based on the position estimation performance prediction value. Detailed processing of the downsampling size setting block 16 will be described later.

[0038] The downsampling block 17 generates processed point cloud data for each processing time by downsampling the point cloud data of one cycle of scanning generated by the LIDAR 2 for each processing time. In this case, the downsampling block 17 divides the space into grids based on the downsampling size set by the downsampling size setting block 16, and calculates representative points of the measurement points of the point cloud data of the LIDAR 2 included in each grid. In this case, the above space is, for example, a space of a three-dimensional coordinate system with the travel direction, height direction, and lateral direction (i.e., directions perpendicular to the travel direction and height direction) of the vehicle-mounted device 1 as axes, and a grid is formed by dividing the space at equal intervals along each of these directions. The processing of the downsampling block 17 may use any Voxel grid filter method. The downsampling block 17 supplies the generated processed point cloud data to the vehicle position estimation unit 15.

[0039] Fig. 4 shows an example of the data structure of the location estimation performance prediction information PD. The location estimation performance prediction information PD shown in Fig. 4 is a table having the items "area number", "center position", and "location estimation performance predicted value".

[0040] An "area number" indicates an identification number of an area to which the corresponding location estimation performance predicted value is applied. Here, each area to which an area number is assigned may be an area obtained by dividing a world coordinate system into a grid based on latitude and longitude, or an area obtained by dividing a world coordinate system into regions based on the identity of the location estimation performance predicted value. In yet another example, an area may be an area divided by roads or road sections. In this case, the area number may be an identification number (such as a link ID) assigned to each road or road section.

[0041] The "center position" indicates a pair of latitude and longitude of the center position of the corresponding area. Note that the location estimation performance prediction information PD may be provided with various items for storing information specifying the location of the area indicated by each area number, instead of or in addition to the "center position".

[0042] The "position estimation performance predicted value" indicates the position estimation performance predicted value for each element (also called "element to be estimated") that is the subject of estimation in estimating the position of the vehicle. Here, the elements to be estimated are the traveling direction, lateral direction, height direction, and orientation, and the position estimation performance predicted value for each of these elements to be estimated is included in the position estimation performance predicted information PD for each area. The position estimation performance predicted value has a value range of 0 to 1, with 1 being the best value. Here, as shown in FIG. 4, the magnitude relationship of the position estimation performance predicted value for each element to be estimated differs depending on the area. Note that, for an example of a method for determining each position estimation performance predicted value to be recorded in the position estimation performance predicted information PD, see "(6) Method for generating location estimation performance prediction information " section.

[0043] Then, the downsampling size setting block 16 determines the downsampling size by referring to the position estimation performance prediction information PD having a data structure as shown in Fig. 4. In this case, the downsampling size setting block 16 first recognizes the area number of the area in which the on-board device 1 is located based on the predicted vehicle position at the current processing time. In the example of Fig. 4, the downsampling size setting block 16 recognizes the area number whose center position is closest to the predicted vehicle position as the area number of the area in which the on-board device 1 is located.

[0044] Then, the downsampling size setting block 16 determines the sizes of the downsampling in the travel direction, horizontal direction, and height direction based on the position estimation performance predicted values ​​in the travel direction, horizontal direction, and height direction corresponding to the recognized area number. Here, specific examples (first to third setting examples) of setting the downsampling size from the position estimation performance predicted value will be described.

[0045] In the first setting example, the downsampling size setting block 16 sets the downsampling size to a smaller value in a direction with a lower position estimation performance prediction value. In this way, the downsampling size setting block 16 preferably ensures the resolution of the point cloud data in a direction with a lower position estimation performance, and preferably suppresses a decrease in the position estimation accuracy in the same direction.

[0046] In this case, for example, the downsampling size setting block 16 may determine the downsampling size corresponding to each direction to a size proportional to the position estimation performance prediction value in the corresponding direction. For example, in area number A-1, the downsampling size in each direction is determined so that the ratio of the downsampling sizes in the traveling direction, horizontal direction, and height direction is "0.6:0.8:0.8". In this case, for example, the downsampling size setting block 16 may determine the downsampling size in each direction by multiplying the corresponding position estimation performance prediction value to a downsampling reference size previously stored in the memory 12. In another example, the downsampling size setting block 16 may store in the memory 12 an equation or a lookup table indicating the correspondence between the position estimation performance prediction value and the downsampling size to be set, and determine the downsampling size in each direction from the position estimation performance prediction value in each direction based on the correspondence. According to the first setting example, for example, when the position estimation performance prediction value in the traveling direction is low, the downsampling size in the traveling direction is reduced, and the accuracy of the traveling direction can be appropriately maintained.

[0047] In the second setting example, the downsampling size setting block 16 fixes the downsampling size in the direction in which the position estimation performance prediction value is smaller than the threshold value, and adaptively determines the downsampling size in the other directions for each processing time. The above-mentioned threshold value is pre-stored in, for example, the memory 12. In this way, the vehicle-mounted device 1 can ensure the resolution of the processed point cloud data in the low-precision direction in which the position estimation accuracy is low, and reliably suppress the deterioration of the position estimation accuracy in the low-precision direction.

[0048] Here, a method of adaptively changing the downsampling size in the second setting example will be described. In this case, the downsampling size setting block 16 determines the downsampling size to be used at the next processing time at each processing time so that the number of measurement points (also called the "number of corresponding measurement points Nc") associated with the voxel data VD in the point cloud data after downsampling falls within a predetermined target range. For example, when the number of corresponding measurement points Nc is greater than the upper limit of the target range at each processing time, the downsampling size setting block 16 increases the downsampling size at the next processing time by a predetermined rate (for example, 1.1) or a predetermined value. In this way, the downsampling size setting block 16 suitably changes the downsampling size so that the number of corresponding measurement points Nc at the next processing time is likely to decrease. On the other hand, when the number of corresponding measurement points Nc is less than the lower limit of the target range, the downsampling size setting block 16 decreases the downsampling size at the next processing time by a predetermined rate or a predetermined value. In this way, the downsampling size setting block 16 suitably changes the downsampling size so that the number of corresponding measurement points Nc at the next processing time is likely to increase. On the other hand, if the number of corresponding measurement points Nc is within the target range at each processing time, the downsampling size setting block 16 maintains the downsampling size.

[0049] Generally, the larger the downsampling size, the smaller the number of data (i.e., the number of measurement points in the processing point cloud data), but the relationship between the downsampling size and the number of data is not linear. Therefore, it is not appropriate to determine the downsampling size by linear feedback control. Taking the above into consideration, in this embodiment, the target number of corresponding measurement points Nc is expressed by a range to provide a dead zone, and the downsampling size is changed by a certain ratio or value regardless of the difference between the number of data and the target range. This allows the number of corresponding measurement points Nc to gradually reach the target level.

[0050] According to the second setting example in which the downsampling size is adaptively changed as described above, the downsampling size setting block 16 can change the downsampling size so as to maintain the resolution of the processing point cloud data in the low-precision direction while also achieving a balance between processing time and position estimation accuracy in other directions.

[0051] In the third setting example, the downsampling size setting block 16 determines the initial value of the downsampling size in each direction based on the first setting example, and performs the adaptive change of the downsampling size in each direction described in the second setting example. This allows the downsampling size setting block 16 to appropriately set the initial value of the downsampling size in each direction, and to quickly converge the downsampling size in each direction to an appropriate size.

[0052] Next, a specific example of downsampling will be described.

[0053] 5(A) to 5(C) are diagrams showing the first execution results of downsampling with different downsampling sizes. The downsampling size in FIG. 5(A) is set to 1x the voxel size, the downsampling size in FIG. 5(B) is set to 2 / 3x the voxel size, and the downsampling size in FIG. 5(C) is set to 1 / 2x the voxel size. In FIG. 5(A) to 5(C), voxel data VD expressed by normal distribution, point cloud data before downsampling, and processed point cloud data after downsampling are clearly shown. Here, for the sake of simplicity, the voxel data VD corresponding to four voxels and the point cloud data belonging to the voxels are the subject of the description.

[0054] The downsampling block 17 generates grids by dividing the space corresponding to the four target voxels according to the downsampling size set in each case of Fig. 5(A) to Fig. 5(C). The downsampling block 17 then calculates the average of the positions indicated by the measurement points of the lidar 2 for each grid, and generates the calculated average position as the measurement point of the processing point cloud data corresponding to the target grid. In this case, as shown in Fig. 5(A) to Fig. 5(C), the smaller the downsampling size, the more the number of measurement points of the processing point cloud data after downsampling, and the higher the resolution.

[0055] Next, the effect of changing the downsampling size will be described with reference to FIGS. 6(A) and 6(B).

[0056] 6(A) and 6(B) show an overhead view of the periphery of a vehicle equipped with an on-board device 1 when the downsampling size is set to one time the voxel size. In FIG. 6(A) and FIG. 6(B), voxels in which voxel data VD exists are indicated by rectangular frames, and the positions of the measurement points of the lidar 2 obtained by one period of scanning are indicated by dots. Here, FIG. 6(A) shows an example of an area in which the predicted position estimation performance value is large, and FIG. 6(B) shows an example of an area in which the predicted position estimation performance value is small. Note that the voxels corresponding to the surface positions of the structures have corresponding voxel data VD.

[0057] As shown in Fig. 6(A), in areas with a large predicted position estimation performance, there are many structures with voxel data VD in the surrounding area, and the number of measurement points of the point cloud data available for NDT scan matching is also large. On the other hand, as shown in Fig. 6(B), in areas with a small predicted position estimation performance, there are few structures with voxel data VD in the surrounding area, and the number of measurement points of the point cloud data available for NDT scan matching is also small.

[0058] In this way, the number of measurement points of the point cloud data available for NDT scan matching changes as the environment around the vehicle changes. As shown in FIG. 6B, if the downsampling size is large when the number of measurement points of the point cloud data available for NDT scan matching is small, the number of measurement points of the processed point cloud data to be matched with the voxel data VD becomes too small, and the position estimation accuracy obtained by NDT scan matching decreases. Therefore, it is necessary to reduce the downsampling size and increase the number of measurement points. On the other hand, if there are more surrounding structures than in FIG. 6A and the number of measurement points increases, there is a possibility that the calculation will not be completed within a predetermined time. In that case, it is necessary to set the downsampling size large and reduce the number of measurement points. In consideration of the above, the vehicle-mounted device 1 according to this embodiment adaptively changes the downsampling size based on the position estimation performance prediction value corresponding to the traveling position, thereby suitably maintaining the position estimation accuracy of NDT scan matching.

[0059] (4) Position Estimation Based on NDT Scan Matching Next, the position estimation based on NDT scan matching executed by the vehicle position estimating unit 15 will be described.

[0060] FIG. 7 is a diagram showing the vehicle position to be estimated by the vehicle position estimation unit 15 in two-dimensional orthogonal coordinates. As shown in FIG. 7, the vehicle position on a plane defined on the two-dimensional orthogonal coordinates of xy is represented by coordinates "(x, y)" and the vehicle's orientation (yaw angle) "ψ". Here, the yaw angle ψ is defined as the angle between the vehicle's traveling direction and the x-axis. The coordinates (x, y) are, for example, absolute positions corresponding to a combination of latitude and longitude, or world coordinates indicating a position with a predetermined point as the origin. The vehicle position estimation unit 15 performs vehicle position estimation using these x, y, and ψ as estimation parameters. Note that the vehicle position estimation unit 15 may perform vehicle position estimation in which at least one of the vehicle's height position, pitch angle, and roll angle in a three-dimensional orthogonal coordinate system is further estimated as an estimation parameter in addition to x, y, and ψ.

[0061] Next, the voxel data VD used in the NDT scan matching will be described. The voxel data VD includes data in which measured point cloud data of a stationary structure in each voxel is expressed by a normal distribution.

[0062] Fig. 8 shows an example of a schematic data structure of the voxel data VD. The voxel data VD includes parameter information when expressing a point group in a voxel by a normal distribution, and in this embodiment, as shown in Fig. 8, includes a voxel ID, a voxel coordinate, a mean vector, and a covariance matrix.

[0063] "Voxel coordinates" indicate the absolute three-dimensional coordinates of a reference position such as the center position of each voxel. Each voxel is a cube that divides space into a grid, and since its shape and size are determined in advance, it is possible to identify the space of each voxel by its voxel coordinates. The voxel coordinates may be used as a voxel ID.

[0064] The "mean vector" and "covariance matrix" refer to the mean vector and covariance matrix that correspond to the parameters when expressing the point group in the target voxel as a normal distribution. Note that the coordinates of an arbitrary point "i" in an arbitrary voxel "n" are X n (i)=[x n (i), y n (i), z n (i)] T and the number of points in voxel n is defined as "N n ", then the mean vector at voxel n is "μ n " and the covariance matrix "V n " are expressed by the following formulas (1) and (2), respectively.

[0065]

number

[0066]

number

[0067] Next, an overview of NDT scan matching using the voxel data VD will be described.

[0068] Scan matching using NDT assuming a vehicle estimates parameters based on the amount of movement within the road plane (here, xy coordinates) and the vehicle direction. P=[t x , t y , t ψ ] T Here, "t x " indicates the amount of movement in the x direction, and "t y " indicates the amount of movement in the y direction, and "t ψ " indicates the yaw angle.

[0069] In addition, the coordinates of the point cloud data output by LIDAR 2 are X L (j) = [x(j), y(j), z(j)] T Then, X L The average value of (j) "L´ n " is expressed by the following equation (3).

[0070]

number

[0071]

number

[0072]

number

[0073] Then, the vehicle position estimation unit 15 calculates a comprehensive evaluation function value (also called a "score value") "E(k)" for all voxels that are targets of matching, as shown in the following equation (6).

[0074]

number

[0075]

number

[0076] Fig. 9 is an example of a functional block of the vehicle position estimation unit 15. As shown in Fig. 9, the vehicle position estimation unit 15 has a dead reckoning block 21, a position prediction block 22, a coordinate conversion block 23, a point cloud data association block 24, and a position correction block 25.

[0077] The dead reckoning block 21 uses the vehicle's moving speed and angular velocity based on the outputs of the gyro sensor 3, the vehicle speed sensor 4, and the GPS receiver 5 to calculate the moving distance and the change in direction from the previous time. The position prediction block 22 calculates the estimated vehicle position X at time k-1 calculated in the immediately preceding measurement update step. ^ Add the calculated travel distance and direction change to (k-1) to obtain the predicted vehicle position X at time k. - The position prediction block 22 calculates the predicted vehicle position X - (k) is supplied to the downsampling processing unit 14, the coordinate conversion block 23 and the position correction block 25.

[0078] The coordinate conversion block 23 converts the processed point cloud data after downsampling output from the downsampling processing unit 14 into a world coordinate system, which is the same coordinate system as the map DB 10. In this case, the coordinate conversion block 23 performs coordinate conversion of the processed point cloud data at time k based on the predicted vehicle position output by the position prediction block 22 at time k, for example. Note that, instead of performing the above-mentioned coordinate conversion on the processed point cloud data after downsampling, the above-mentioned coordinate conversion may be performed on the point cloud data before downsampling. In this case, the downsampling processing unit 14 generates processed point cloud data in a world coordinate system by downsampling the point cloud data in the world coordinate system after the coordinate conversion. Note that the process of converting the point cloud data in a coordinate system based on a lidar installed on a vehicle into a vehicle coordinate system, and the process of converting from the vehicle coordinate system to the world coordinate system are disclosed, for example, in International Publication WO2019 / 188745.

[0079] The point cloud data matching block 24 matches the processed point cloud data in the world coordinate system output by the coordinate transformation block 23 with the voxel data VD expressed in the same world coordinate system, thereby matching the processed point cloud data with the voxels. The position correction block 25 calculates an individual evaluation function value based on equation (5) for each voxel that has been matched with the processed point cloud data, and calculates an estimated parameter P that maximizes the score value E(k) based on equation (6). The position correction block 25 then calculates the predicted vehicle position X output by the position prediction block 22 based on equation (7). - By applying the estimation parameter P obtained at time k to the estimated vehicle position X ^ Calculate (k).

[0080] Here, a specific procedure for associating the measurement points with the voxel data VD will be further explained using a simple example.

[0081] 10 shows the positional relationship between voxels "Vo1" to "Vo6" in which voxel data VD exists on a two-dimensional xy plane in the world coordinate system, and measurement points 61 to 65 indicating positions near these voxels. For ease of explanation, it is assumed here that the z coordinates of the centers of the voxels Vo1 to Vo6 in the world coordinate system are the same as the z coordinates of the measurement points 61 to 65 in the world coordinate system.

[0082] First, the coordinate conversion block 23 converts the point cloud data including the measurement points 61 to 65 into the world coordinate system. After that, the point cloud data association block 24 performs a process of rounding off the fractions of the measurement points 61 to 65 in the world coordinate system. In the example of Fig. 10, since the size of each voxel, which is a cube, is 1m, the point cloud data association block 24 rounds off the decimal points of the x, y, and z coordinates of each of the measurement points 61 to 65.

[0083] Next, the point cloud data matching block 24 checks the voxel data VD corresponding to the voxels Vo1 to Vo6 against the coordinates of each of the measurement points 61 to 65 to determine the voxels corresponding to each of the measurement points 61 to 65. In the example of FIG. 10, the (x, y) coordinates of the measurement point 61 become (2, 1) due to the above-mentioned rounding, so the point cloud data matching block 24 matches the measurement point 61 with the voxel Vo1. Similarly, the (x, y) coordinates of the measurement points 62 and 63 become (3, 2) due to the above-mentioned rounding, so the point cloud data matching block 24 matches the measurement points 62 and 63 with the voxel Vo5. In addition, the (x, y) coordinates of the measurement point 64 become (2, 3) due to the above-mentioned rounding, so the point cloud data matching block 24 matches the measurement point 64 with the voxel Vo6. On the other hand, since the (x, y) coordinates of the measurement point 65 become (4, 1) due to the above-mentioned rounding, the point cloud data matching block 24 determines that there is no voxel data VD corresponding to the measurement point 65. Thereafter, the position correction block 25 estimates the estimation parameters P using the measurement point and the voxel data VD associated by the point cloud data matching block 24. Note that in this example, the number of measurement points after downsampling is five, but the number of corresponding measurement points Nc is four. The number of corresponding measurement points Nc is smaller when there are few structures around the road and the space is sparse, and also when a phenomenon occurs in which another vehicle is present near the vehicle and the lidar's light beam is blocked (called occlusion).

[0084] (5) Processing flow Fig. 11 is an example of a flowchart showing a procedure of a process related to vehicle position estimation executed by the controller 13 of the vehicle-mounted device 1. The controller 13 starts the process of the flowchart in Fig. 11 when it becomes necessary to perform vehicle position estimation, for example, when the power is turned on.

[0085] First, the controller 13 determines a predicted vehicle position based on the positioning result of the GPS receiver 5 (step S10). Then, the controller 13 judges whether or not the point cloud data of the LIDAR 2 has been acquired (step S11). Then, if the controller 13 cannot acquire the point cloud data of the LIDAR 2 (step S11; No), the controller 13 continues to perform the judgment of step S11. Furthermore, during a period in which the controller 13 cannot acquire the point cloud data, the controller 13 continues to determine a predicted vehicle position based on the positioning result of the GPS receiver 5. Note that the controller 13 may determine a predicted vehicle position based on the output of any sensor other than the point cloud data, not limited to the positioning result of only the GPS receiver 5.

[0086] Then, when the point cloud data of the LIDAR 2 can be acquired (step S11; Yes), the controller 13 obtains the movement distance and the change in orientation from the previous time using the movement speed and the angular velocity of the vehicle based on the output of the gyro sensor 3 and the vehicle speed sensor 4, etc. As a result, the position prediction block 22 calculates the predicted vehicle position (which may include the attitude angle) at the current processing time from the estimated vehicle position (which may include the attitude angle such as the yaw angle) obtained one time ago (the immediately previous processing time) (step S12). Note that, when the estimated vehicle position one time ago has not been calculated, the controller 13 may perform the processing of step S12 by regarding the predicted vehicle position determined in step S10 as the estimated vehicle position.

[0087] Next, the controller 13 refers to the location estimation performance prediction information PD and determines the current area from the predicted vehicle position at the current processing time (step S13). For example, if the location estimation performance prediction information PD has the data structure shown in Fig. 4, the controller 13 identifies the corresponding area number based on the acquired predicted vehicle position and the center position of each area recorded in the location estimation performance prediction information PD.

[0088] Then, the controller 13 acquires the position estimation performance predicted value for the current area determined in step S13 from the position estimation performance predicted information PD (step S14). In this case, if the position estimation performance predicted value for each direction is recorded in the position estimation performance predicted information PD, the controller 13 acquires the position estimation performance predicted value for each direction.

[0089] Next, the controller 13 determines the downsampling size based on the position estimation performance prediction value acquired in step S14 (step S15). In this case, for example, the controller 13 determines the downsampling size based on the position estimation performance prediction value acquired in step S14 (step S15). Setting the downsampling size The downsampling size in each direction is determined according to any one of the first to third setting examples described in the section ". Note that controller 13 may execute the processes of steps S13 to S15 at predetermined time intervals, instead of every time (at each processing time). In other words, controller 13 may set the cycle for executing the process for setting the downsampling size to be longer than the execution cycle of vehicle position estimation by a predetermined multiple.

[0090] Then, the controller 13 performs downsampling on the point cloud data for one period of scanning by the LIDAR 2 at the current processing time, based on the downsampling size determined in step S15 (step S16).

[0091] Next, the controller 13 executes an NDT matching process (step S17). In this case, the controller 13 performs a process of converting the processing point cloud data into data in a world coordinate system, and a process of associating the processing point cloud data converted into the world coordinate system with voxels in which the voxel data VD exists. Then, the controller 13 calculates an estimated vehicle position (including attitude angles such as a yaw angle) at the current processing time based on the result of the NDT matching process (step S18).

[0092] Then, the controller 13 judges whether or not the vehicle position estimation process should be ended (step S19). If the controller 13 judges that the vehicle position estimation process should be ended (step S19; Yes), the controller 13 ends the process of the flowchart. On the other hand, if the controller 13 judges that the vehicle position estimation process should be continued (step S19; No), the controller 13 returns the process to step S11 and estimates the vehicle position at the next processing time.

[0093] (6) Method for generating location estimation performance prediction information Next, a method for generating the position estimation performance prediction information PD will be described. When performing self-position estimation by NDT scan matching, accurate self-position estimation is possible in a place where there are many structures around the road, but the accuracy may decrease when there are few features. In addition, the accuracy of the self-position estimation in each of the traveling direction and the lateral direction depends on the arrangement of the features. Therefore, it is possible to predict the position estimation performance in each direction depending on the arrangement of the voxels in which the voxel data VD stored in the map DB 10 exists. The vehicle-mounted device 1 stores the position estimation performance prediction information PD including the position estimation performance prediction value calculated by the method described below as a part of the map DB 10.

[0094] The following describes an example of a method for calculating a predicted position estimation performance value at each position from the voxel data VD included in the map DB 10. This process may be executed by the vehicle-mounted device 1, or may be executed in advance by a device other than the vehicle-mounted device 1 as part of a process for generating map data.

[0095] In general, voxel data VD within a specified distance range from a target point (also called a "target point") for which a position estimation performance prediction value is to be calculated is extracted from the map DB10, the target range is sliced ​​in the travel direction and the lateral direction, and the height difference of the voxels on the sliced ​​lines where the voxel data VD exists from the target point is calculated. Then, the above-mentioned difference for each slice is integrated, and the position estimation performance in the travel direction and the lateral direction is predicted based on the integration result.

[0096] 12(A) and 12(B) show an overhead view of a road clearly showing slice lines (grid lines) arranged at equal intervals in the travel direction and the lateral direction when points on different roads are set as target points. In FIG. 12(A) and FIG. 12(B), the target points are represented by vehicles. Here, in the example of FIG. 12(A), there is no feature object that generates voxel data VD in the lateral direction of the target point, and a sparse space spreads, while there are feature objects that generate voxel data VD in front of and behind the travel direction. In addition, in the road shown in FIG. 12(B), there are no structures in the vicinity, but guardrails are installed. Therefore, it is presumed that, in the target point shown in FIG. 12(B), although there is no object that can maintain the estimation accuracy in the travel direction, the estimation accuracy can be maintained in the lateral direction by the guardrail.

[0097] Fig. 13 is a diagram showing three slice lines "ha1" to "hc1" and "va1" to "vc1" extracted in the horizontal direction and the traveling direction from among the slice lines shown in Fig. 12(A). Fig. 14(A) to Fig. 14(F) are diagrams showing voxels in which voxel data VD exists on a cut surface cut by the slice lines shown in Fig. 13.

[0098] In this example, since there are features in front and behind the moving direction from which voxel data VD is generated, the integral value in the moving direction is large, and since the space is sparse in the horizontal direction, the integral value in the horizontal direction is small. For example, in the moving direction slice line va1, the voxel height difference in the forward direction is "+3", and the voxel height difference in the backward direction is "+6". In addition, in the moving direction slice line vb1, the voxel height difference in the forward direction is "+2", and the voxel height difference in the backward direction is "0". In addition, in the moving direction slice line vc1, the voxel height difference in the forward direction is "+7", and the voxel height difference in the backward direction is "+2". On the other hand, in the horizontal slice lines ha1 to hc1, the voxel height difference in both the left and right directions is "0". Then, the integral value of the voxel height difference of each slice line in the traveling direction shown in Fig. 12(A) and the integral value of the voxel height difference of each slice line in the horizontal direction shown in Fig. 12(A) are calculated, and the higher the integral value, the higher the position estimation performance prediction value is set. Therefore, in the example of Fig. 13, the position estimation performance prediction value in the traveling direction is set to a value larger than the position estimation performance prediction value in the horizontal direction according to the integral value. Note that, as for the position estimation performance prediction value in the height direction, since voxels of tall structures are included and the distribution in the height direction is wide, a value larger than the position estimation performance prediction value in the horizontal direction and approximately the same as the position estimation performance prediction value in the traveling direction is set.

[0099] Fig. 15 is a diagram showing three slice lines "ha2" to "hc2" and three slice lines "va2" to "vc2" in the horizontal direction and three slice lines "va2" to "vc2" in the forward direction, respectively, extracted from the slice lines shown in Fig. 13(B). Fig. 16(A) to Fig. 16(F) are diagrams showing voxels in which voxel data VD exists on a cut surface cut by the slice lines shown in Fig. 15.

[0100] In this example, the space is sparse in the traveling direction, so the integral value in the traveling direction is small, and the integral value in the horizontal direction is medium due to the presence of the guardrail. For example, in the traveling direction slice lines va2 to vc2, the voxel height difference in the backward direction is "0" in both the forward and right directions. On the other hand, in the horizontal direction slice lines ha2 to hc2, the voxel height difference in both the left and right directions is "1". In this way, in each horizontal slice line, although the individual voxel height difference is small, the horizontal integral value becomes a certain size due to the continuity of the guardrail. Therefore, in the example of FIG. 15, the position estimation performance prediction value in the traveling direction is set to a value smaller than the position estimation performance prediction value in the horizontal direction. Note that, for the position estimation performance prediction value in the height direction, since the voxels are the road surface and the guardrail part and the distribution in the height direction is narrow, a value between the position estimation performance prediction value in the traveling direction and the position estimation performance prediction value in the horizontal direction is set.

[0101] As described above, the predicted value of the position estimation performance for each point can be suitably calculated by the simulation based on the voxel data VD stored in the map DB 10. The calculation result of the predicted value of the position estimation performance is stored in the map DB 10 as the predicted information PD of the position estimation performance, so that the vehicle-mounted device 1 can suitably perform the downsampling size adjustment described in the above-mentioned embodiment.

[0102] As described above, the controller 13 of the vehicle-mounted device 1 according to this embodiment acquires point cloud data output by the LIDAR 2 provided on a vehicle, which is a moving body. Then, the controller 13 acquires a position estimation performance prediction value, which is prediction information related to the performance of self-position estimation, associated in the map data with the area to which the vehicle's position belongs. Then, the controller 13 performs downsampling on the point cloud data based on the position estimation performance prediction value. With this embodiment, the vehicle-mounted device 1 can suitably perform downsampling in consideration of the self-position estimation accuracy.

[0103] (7) Variations Preferred modifications of the above-described embodiment will now be described. The following modifications may be combined and applied to these embodiments.

[0104] (Variation 1) The configuration of the driving assistance system shown in Fig. 1 is an example, and the configuration of the driving assistance system to which the present invention can be applied is not limited to the configuration shown in Fig. 1. For example, instead of having the on-board device 1, the driving assistance system may have an electronic control device of the vehicle execute the processing of the downsampling processing unit 14 and the vehicle position estimation unit 15 of the on-board device 1. In this case, the map DB 10 is stored in, for example, a storage unit in the vehicle or a server device that performs data communication with the vehicle, and the electronic control device of the vehicle executes the vehicle position estimation based on downsampling and NDT scan matching by referring to this map DB 10.

[0105] (Variation 2) The voxel data VD is not limited to a data structure including a mean vector and a covariance matrix as shown in Fig. 8. For example, the voxel data VD may include point cloud data measured by a measurement maintenance vehicle and used to calculate the mean vector and the covariance matrix.

[0106] (Variation 3) Instead of determining the size of downsampling based on the location estimation performance prediction information PD, the vehicle-mounted device 1 may determine whether downsampling is necessary. For example, the vehicle-mounted device 1 refers to the location estimation performance prediction information PD and performs downsampling when the location estimation performance prediction values ​​in all directions corresponding to the current area are equal to or greater than a threshold, and does not perform downsampling when the location estimation performance prediction value in any direction is less than the threshold. Even in this example, downsampling can be suitably performed based on the location estimation performance prediction information PD.

[0107] (Variation 4) In the position estimation performance prediction information PD shown in FIG. 4, a position estimation performance predicted value is associated with each estimation target element (travel direction, lateral direction, height direction, and orientation). Alternatively, in the position estimation performance prediction information PD, one position estimation performance predicted value may be associated with each area regardless of the estimation target element. In this case, for example, the vehicle-mounted device 1 refers to the position estimation performance prediction information PD and determines a common downsampling size in the travel direction, lateral direction, and height direction based on the position estimation performance predicted value corresponding to the current area. In another example, when the ratio of the downsampling size in the travel direction, lateral direction, and height direction is predetermined, the vehicle-mounted device 1 may determine the downsampling size in each direction based on the position estimation performance predicted value corresponding to the current area and the ratio.

[0108] In the above-described embodiment, the program can be stored using various types of non-transitory computer readable media and supplied to a controller or the like that is a computer. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).

[0109] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In other words, the present invention naturally includes various modifications and corrections that a person skilled in the art could make in accordance with the entire disclosure, including the claims, and the technical ideas. In addition, the disclosures of the above cited patent documents and the like are incorporated herein by reference. [Explanation of symbols]

[0110] 1 On-vehicle device 2. Rider 3 Gyro sensor 4 Vehicle speed sensor 5 GPS receiver 10 Map DB

Claims

[Claim 1] A point cloud data acquisition means for acquiring point cloud data output by a measuring device provided on the moving object; a prediction information acquisition means for acquiring prediction information relating to a performance of self-location estimation, the prediction information being associated with the position of the moving object in map data; a downsampling unit that performs downsampling on the point cloud data based on the prediction information; An information processing device having the above configuration.

Images