3D map generation method and 3D map generation device

Abstract

To provide a three-dimensional map generation method and the like which generate a three-dimensional map highly accurately.SOLUTION: A three-dimensional map generation method comprises: a node image generation step (S11) of generating a plurality of YX node images expressing the rectangular ground surface including a road and Z node images expressing the altitude from a point cloud acquired by a LiDAR 11 mounted on a vehicle 20; and a map generation step (S13) of generating an XY surface map and a Z surface map by reducing a positional error on the plurality of generated node images. The map generation step (S13) includes a loop closing step (S13a) of reducing the positional error on the XY surface map by performing loop closing being an example of re-visiting processing of reducing a cumulative error. The loop closing step (S13a) determines whether or not an altitude difference is less than a first threshold for the two XY node images including the same spots and performs loop closing when the altitude difference is less than the first threshold (S21).SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a 3D map generation method and device, and more particularly to a method for generating a 3D map from a point cloud. [Background technology]

[0002] In recent years, maps have been generated by applying SLAM (Simultaneous Localization and Mapping) to point clouds, which represent 3D range images acquired from LiDAR (Light Detection and Ranging) mounted on vehicles. In map generation using SLAM, loop closure is performed, which significantly reduces cumulative errors by observing the same point along a circular path (loop) and adding this data to simultaneous equations.

[0003] Therefore, various loop closure techniques have been proposed (see Non-Patent Document 1). Non-Patent Document 1 focuses on detecting loop closure events (events for loop closure; also simply called "loop closure") in map data depending on the vehicle's position, and extracting several features to be registered as environmental signatures. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] M. Vlaminck, H. Luong and W. Philips, “Have I Seen This Place Before? A Fast and Robust Loop Detection and Correction Method for 3D Lidar SLAM” Sensors, vol. 23, no. 19, 2019. Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the technology in Non-Patent Document 1, it is difficult to generate a map with high accuracy without false positives because (1) there is a lack of point clouds, (2) traffic flow affects point distribution, and (3) the true relationship between vehicle positions will be incorrect, especially when vehicles arrive at the same location at different speeds.

[0006] Therefore, an object of the present disclosure is to provide a method for generating a 3D map with high accuracy. [Means for solving the problem]

[0007] In order to achieve the above object, a 3D map generation method according to one embodiment of the present disclosure includes a step of: generating node images, which are pairs of an XY node image representing a rectangular ground surface including a driving route and a Z node image representing an elevation of a point indicated by each pixel constituting the XY node image, from a point cloud acquired by an object detection device for map creation that is mounted on a vehicle and detects an environment including a driving route; Multiple The method includes a node image generation step of generating a node image, and a map generation step in which a map generation unit of the three-dimensional map generation device generates a three-dimensional map including an XY plane map and a Z plane map by reducing positional errors in the generated plurality of node images, and the map generation step includes a revisiting processing unit of the three-dimensional map generation device revisiting the same points in a graph showing connections of the plurality of node images. identification do Processing included The method includes a revisiting step for reducing a position error in the XY plane map by performing a revisiting process for reducing an accumulated error, and in the revisiting step, for two of the XY node images including the same location, two of the corresponding Z node images are comparison By doing so, it is determined whether the elevation difference is less than the first threshold. This includes processing , The aforementioned Judgment of Based on the result, the revisit process is carried out.

[0008] In addition, in order to achieve the above-mentioned object, a 3D map generation device according to one embodiment of the present disclosure includes a node image generation unit that is mounted on a vehicle and generates, from a point cloud acquired by an object detection device for map creation that detects the environment including the driving path, multiple node images that are pairs of an XY node image representing a rectangular ground surface including the driving path and a Z node image representing the elevation of a point indicated by each pixel constituting the XY node image, and a map generation unit that generates a 3D map including an XY plane map and a Z plane map by reducing positional errors in the multiple generated node images, wherein the map generation unit includes a revisit processing unit that reduces positional errors in the XY plane map by performing a revisit processing that reduces cumulative errors by observing the same location in a graph showing the connection of the multiple node images, and the revisit processing unit determines whether the elevation difference between two XY node images that include the same location is less than a first threshold by referring to the two corresponding Z node images, and performs the revisit processing depending on the determination result. [Effects of the Invention]

[0009] The present disclosure provides a method for generating a three-dimensional map with high accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of a 3D map generating device according to an embodiment. [Figure 2] FIG. 2 is a flowchart showing the operation procedure of the 3D map generating device according to the embodiment. [Figure 3] FIG. 3 is a schematic diagram showing an outline of the characteristic processing shown in the flowchart of FIG. [Figure 4] FIG. 4 is a diagram showing an example of the exterior of a vehicle used to generate a 3D map and a point cloud acquired from a LiDAR mounted on the vehicle. [Figure 5] FIG. 5 is a flowchart showing the details of step S13a in FIG. [Figure 6]FIG. 6 is a diagram illustrating the details of S12 in FIG. [Figure 7] FIG. 7 is a diagram illustrating the details of S13a in FIG. [Figure 8] FIG. 8 is a diagram illustrating the results of an experiment conducted with the 3D map generating device according to this embodiment at a location including an intersection area consisting of a bridge-underpass multi-level intersection in Kanazawa City. [Figure 9A] FIG. 9A is a diagram showing the location of another experiment in which the 3D map generating device according to this embodiment was tested on a highway with a long underground tunnel including a multi-layer course. [Figure 9B] FIG. 9B is a diagram showing an example of the experimental results shown in FIG. 9A (an example of a correlation matrix between two maps). [Figure 9C] FIG. 9C is a diagram showing another example of the experimental results shown in FIG. 9A (an example of map-combined detection using two maps). DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. Numerical values, experimental locations, components such as sensors, arrangement and connection of components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, each drawing is not necessarily an exact illustration. In each drawing, substantially identical components are assigned the same reference numerals, and duplicated explanations are omitted or simplified.

[0012] 1 is a block diagram showing the configuration of a 3D map generation device 10 according to an embodiment. The 3D map generation device 10 is a device that generates a 3D map using graph-based SLAM, and includes a node image generation unit 13 that is connected to a map creation object detection device (here, a LiDAR 11) and a GPS 12 mounted on a vehicle traveling on a road, and a map generation unit 14.

[0013] The node image generation unit 13 acquires point clouds and positioning data from the LiDAR 11 and GPS 12, which are examples of object detection devices for map creation, respectively. From the acquired point clouds and positioning data, the unit generates multiple node images along the vehicle's travel route, each of which is a pair of an XY node image representing a rectangular ground surface including the travel route, such as a road, and a Z node image representing the elevation of a point indicated by each pixel constituting the XY node image. The generated multiple node images are not necessarily rectangles of the same geographical size, depending on the vehicle's travel route. Therefore, the node image generation unit 13 has a sub-image division unit 13a that divides each of the multiple XY node images into multiple sub-images corresponding to rectangles of the same geographical size (rectangular regions with fixed widths and fixed heights). Note that a node image is also simply referred to as a node. The travel route is not limited to roads but also includes places where vehicles can travel, such as construction sites and farmland.

[0014] The map generation unit 14 generates a 3D map including an XY plane map and a Z plane map by reducing positional errors in the node images generated by the node image generation unit 13. To this end, the map generation unit 14 has a revisit processing unit 14a that reduces positional errors in the XY plane map by performing a revisit processing that reduces accumulated errors by observing the same location in a graph showing the connections between multiple node images generated by the node image generation unit 13, and a SLAM unit 14d that applies SLAM to the 3D map that has been subjected to the revisit processing and outputs the resulting final 3D map 15. The revisit processing unit 14a determines whether the elevation difference between two XY node images containing the same location is less than a first threshold by referring to the two corresponding Z node images, and performs the revisit processing depending on the determination result. To this end, as a specific revisiting process, the revisiting processing unit 14a has a loop closing unit 14b that reduces positional errors on the XY plane map by performing loop closure to reduce cumulative errors by having the vehicle travel around a loop in the graph showing the connection of multiple node images generated by the node image generating unit 13 and observing the same points, and a map combining unit 14c that reduces positional errors on the XY plane map by performing map combining to reduce cumulative errors by observing the same points in the graph showing the connection of multiple node images generated by the node image generating unit 13.

[0015] The loop closing unit 14b determines whether the elevation difference between two XY node images containing the same location is less than a first threshold by referring to the two corresponding Z node images, and performs loop closing if the elevation difference is less than the first threshold (i.e., if the first condition is satisfied). The elevation difference between the XY node images is, for example, the average elevation indicated by each pixel constituting the corresponding Z node image. Furthermore, in accordance with the pre-setting, the loop closing unit 14b performs loop closing if, in addition to the first condition, the two XY node images containing the same location share a number of sub-images equal to or greater than a second threshold (i.e., if the second condition is satisfied). Furthermore, in accordance with the pre-setting, the loop closing unit 14b performs loop closing if, in addition to the first and second conditions, the number of shared pixels indicating the road surface in the number of shared sub-images equal to or greater than the second threshold is equal to or greater than a third threshold (i.e., if the third condition is satisfied). Furthermore, in addition to the first to third conditions, the loop closing unit 14b performs loop closing according to the prior settings when two XY node images containing the same location are separated by a fourth threshold or more in the number of nodes (they are discontinuous; not continuous) (i.e., when the fourth condition is satisfied). Note that "according to the prior settings" means that it depends on the settings made by the user in the 3D map generating device 10 (storage of information such as flags), and the user can arbitrarily select whether or not to execute the corresponding process.

[0016] The SLAM unit 14d performs SLAM on the 3D map that has been revisited by the revisit processing unit 14a, thereby suppressing relative position errors in each layer and errors in the geographical positions between the two maps in the absolute coordinate system, and generating a high-precision 3D map or a combined high-precision 3D map.

[0017] The node image generation unit 13 and the map generation unit 14 are functional blocks that execute data processing, and can be realized by a computer device that includes a non-volatile memory that stores programs, a volatile memory that temporarily stores data, a processor that executes programs, an input / output interface that performs input / output with peripheral devices, etc.

[0018] Fig. 2 is a flowchart showing the operation procedure (i.e., the 3D map generating method) of the 3D map generating device 10 according to the embodiment. Fig. 3 is a schematic diagram showing an overview of the characteristic processing shown in the flowchart of Fig. 2.

[0019] First, the node image generation unit 13 acquires a point cloud and positioning data from the LiDAR 11 and the GPS 12, respectively (S10 in FIG. 2). Fig. 4 is a diagram showing an external view ((a) in FIG. 4) of the vehicle 20 used to generate the 3D map, and an example of a point cloud acquired from the LiDAR 11 mounted on the vehicle 20 ((b) in FIG. 4).

[0020] Next, the node image generation unit 13 generates a plurality of node images, which are pairs of an XY node image representing a rectangular ground surface including a driving route such as a road and a Z node image representing the elevation of a point indicated by each pixel constituting the XY node image, along the vehicle's driving route from the point cloud and positioning data acquired from the LiDAR 11 and the GPS 12 (see S11 in FIG. 2; (a) in FIG. 3). The generated plurality of node images are not necessarily rectangular in size, depending on the vehicle's driving route.

[0021] More specifically, the node image generation unit 13 generates, from the point cloud acquired from the LiDAR 11, LiDAR frames of intensity images (brightness images) representing the ground surface including roads and other driving paths (more specifically, cut at a height of 0.3 m) and LiDAR frames of elevation images indicating the elevation of pixels constituting the intensity images. The node image generation unit 13 then generates a plurality of XY node images by concatenating the LiDAR frames of the intensity images in a predetermined rectangular area, a Z node image by concatenating the LiDAR frames of the elevation images corresponding to the intensity images, and a pair of the XY node image and the Z node image as a node image along the vehicle's driving path. At this time, the node image generation unit 13 assigns, to each of the plurality of node images, an identifier corresponding to the geographical position of the upper left corner (minimum X coordinate, maximum Y coordinate) of the rectangular area indicated by the corresponding XY node image for the XY node image, and an identifier indicating the average elevation value indicated by each pixel of the corresponding Z node image for the Z node image. The method of assigning an identifier is not limited to this method. For example, an identifier corresponding to the geographical position of the bottom right corner (maximum X coordinate, minimum Y coordinate) may be assigned to an XY node image.

[0022] Next, the sub-image dividing unit 13a divides each of the multiple XY node images generated by the node image generating unit 13 into multiple sub-images corresponding to rectangles of the same geographical size (rectangular regions having a fixed width and a fixed height) (S12 in FIG. 2). More specifically, as shown in (b) to (c) in FIG. 3, the sub-image dividing unit 13a first performs housing (housing) on each XY node image generated by the node image generating unit 13 by enclosing it with a rectangle that is an integer multiple of the sub-image, then divides (cuts) the housing rectangle into an integer number of rectangular sub-images of fixed size (for example, images with a width and height of 512 pixels (corresponding to a geographical length of 64 m)), and further assigns an identifier corresponding to the geographical position of the upper left corner to each divided sub-image (assigning an identifier based on the average elevation value indicated by non-zero pixels to the corresponding sub-image in the Z plane).

[0023] Next, the map generation unit 14 generates a three-dimensional map including an XY plane map and a Z plane map by linking the XY node images and linking the Z node images while reducing positional errors in the node images generated by the node image generation unit 13 (S13 in Figure 2).

[0024] At this time, the revisit processing unit 14a of the map generation unit 14 reduces positional errors in the XY plane map by performing loop closure using the loop closure unit 14b, and combines two or more maps using the map combination unit 14c (S13a in FIG. 2). FIG. 5 is a flowchart showing the details of step S13a in FIG. 2 (particularly, loop closure). Referring to FIG. 5, the loop closure unit 14b first identifies two XY node images that include the same location (the same position where the vehicle 20 has completed a loop and returned) as candidates for loop closure (S20 in FIG. 5). Then, as shown in (d) in FIG. 3, the loop closure unit 14b determines whether the elevation difference between the two candidate XY node images is less than a first threshold (for example, 2.5 m (Road Structure Threshold)) (i.e., whether the first condition is satisfied) by referring to the two corresponding Z node images (S21 in FIG. 5).

[0025] If the elevation difference is less than the first threshold (Yes in S21 of FIG. 5), the loop closing unit 14b then determines whether the two candidate XY node images share a second threshold (e.g., four (Road Area Threshold)) or more sub-images (i.e., satisfy the second condition) as shown in (d) of FIG. 3 (S22 of FIG. 5).

[0026] If the number of shared sub-images is equal to or greater than the second threshold (Yes in S22 of FIG. 5), the loop closing unit 14b then determines whether the number of shared pixels among the pixels indicating the road surface in the sub-images shared by the two candidate XY node images, the number of which is equal to or greater than the second threshold, is equal to or greater than a third threshold (e.g., 10,000 (Road Texture Threshold)) (i.e., the third condition is satisfied) (S23 of FIG. 5), as shown in (d) of FIG. 3.

[0027] If the number of shared pixels is equal to or greater than the third threshold (Yes in S23 of FIG. 5), the loop closing unit 14b then determines whether the distance (nodes) between the two candidate XY node images is equal to or greater than the fourth threshold (e.g., 2), that is, whether the number of nodes between the two candidate XY node images is equal to or greater than the fourth threshold (e.g., 2) (i.e., whether they are non-sequential; satisfy the fourth condition) (S24 of FIG. 5).

[0028] If the two XY node images are not consecutive (Yes in S24 in FIG. 5), the loop closing unit 14b performs loop closing on the two candidate XY node images (S25 in FIG. 5). That is, for two candidate XY node images that include the same location (the same position where the vehicle 20 has completed a loop and returned), the data is added to simultaneous equations so that the same location becomes the same geographical position, thereby reducing the accumulated error.

[0029] If a negative determination is made in any of steps S21 to S24 in FIG. 5 (No in any of S21 to S24 in FIG. 5), the loop is not closed (the process ends).

[0030] Referring again to FIG. 2 , finally, the map generation unit 14 applies SLAM by the SLAM unit 14d to the 3D map in which the position error has been reduced by the loop closing unit 14b to compensate for the relative position error, and outputs the 3D map with the compensated relative position error as the final 3D map 15 to an output device such as an external storage device or a display (S14 in FIG. 2 ). More specifically, as shown in FIG. 3 , the map generation unit 14 detects loop closures between multiple nodes at the same Z position (elevation / layer) based on the matrix representing the road structure by the loop closing unit 14b, and performs processing to ensure that the multiple nodes share one important road segment in the absolute coordinate system based on the matrix representing the road region and one road texture in the image region based on the matrix representing the road texture. Therefore, the map generation unit 14 compensates for the relative position error between the nodes for which the loop closure has been performed by the loop closing unit 14b by the SLAM unit 14d, and generates an accurate 3D map 15.

[0031] Although all of the determinations in steps S21 to S24 are performed in the flowchart of Fig. 5, the determinations in steps S22 to S24 are optional and may be selectively performed according to a preset setting. Furthermore, similar to the loop closing unit 14b, the map combining unit 14c of the map generating unit 14 performs map combining on two maps (or node images) for which the conditions in steps S21 to S24 in Fig. 5 are satisfied (see Fig. 3).

[0032] Figure 6 is a diagram showing an example of an XY node image to explain the details of S12 (division of a node image into sub-images) in Figure 2. Figure 6(a) shows an example of an XY node image representing a certain length of road, and Figure 6(b) shows an example of nine sub-images into which the XY node image is divided. Figure 6(c) shows an example of a Z node image corresponding to Figure 6(a), and Figure 6(d) shows an example of nine sub-images into which the Z node image is divided, corresponding to Figure 6(b).

[0033] In (a) of FIG. 6, the X-axis and Y-axis of an absolute coordinate system (ACS) are shown on an XY node image, an example of one LiDAR frame and the position of the vehicle 20 is shown, the road edge is shown, and the position C of the upper left corner of the XY node image is used as an identifier of the XY node image. TL is shown.

[0034] In Figure 6(b), the position CH of the upper left corner of the housed XY node image is shown. TL is shown, and the position coordinates of the upper left corner of each divided sub-image are shown as an identifier for each divided sub-image.

[0035] Position CH of the upper left corner of the housed XY node image TL is the position of the top left corner of the XY node image C TL It is calculated using the following formula 1 (with specific numerical examples):

[0036] CH TL (x)=floor(C TL (x) / (Res*w)) = floor(-5092 / (0.125*512))=-80 CH TL (y)=ceil(C TL (y) / (Res*h)) = ceil(-40487 / (0.125*512))=-632 ·· (Formula 1)

[0037] where Res is the pixel resolution (0.125m x 0.125m), w and h are the width and height of the sub-image (512 pixels; 64m), respectively, and floor and ceil are the rounding down and rounding up functions, respectively.

[0038] The identifier (XY-ID) of each sub-image shown in (b) of FIG. 6 is the position CH of the upper left corner of the housed XY node image. TLand the relative position in the housed XY node image. For example, the identifier (-79, -632) of the sub-image 1 at the center (1,0) in the top row in FIG. 6(b) is calculated by the following formula 2:

[0039] ID1(x)=CHTL(x)+1=-80+1=-79 ID1(y)=CHTL(y)+0=-632+1=-632 ··· (Formula 2)

[0040] In this way, all XY nodal images are divided into identifiable sub-images of the same size, each with an identifier (XY-ID), which means that all images in the XY plane are represented in a unified format in the absolute coordinate system (ACS).

[0041] For the Z node image, as shown in (c) and (d) of Figure 6, after the division of the XY node image into sub-images is completed, the Z node image is divided into sub-images using a similar procedure, and for each sub-image, the average elevation value indicated by the non-zero pixels is assigned as an identifier (Z-ID), as shown in the following equation 3.

[0042]

number

[0043] where ID(z) denotes the sub-image identifier (Z-ID), and Z N (u,v) indicates the elevation value of the pixel at coordinates (u,v) that constitutes the sub-image, w indicates the length (width) of the sub-image in the x-axis direction, h indicates the length (height) of the sub-image in the y-axis direction, and UV indicates the number of non-zero pixels.

[0044] Note that for sub-images with no pixels (i.e., black), the average elevation value for the Z node image may be assigned as an identifier, since in all cases, black images are a collection of zero pixels in the intensity image and therefore do not affect the creation of the road structure matrix.

[0045] 7 is a diagram illustrating the details of step S13a in FIG. 2 (especially loop closure). Here, three XY node images (Node3, Node4, Node7) are extracted from the consecutive XY node images (Node1, Node2, Node3, ...) generated by the node image generation unit 13, and the details of loop closure are explained. That is, (a) to (c) of FIG. 7 show examples of the third XY node image (Node3), the fourth XY node image (Node4), and the seventh XY node image (Node7), respectively, and (d) to (f) of FIG. 7 show, by numerical values indicated in squares specified by the X and Y coordinates, the number of shared sub-images ("road area"), the number of shared pixels ("road texture"), and the elevation difference in the Z plane of the shared sub-images ("road structure") between two XY node images (X coordinates and Y coordinates) selected and combined from the three XY node images.

[0046] 7(a) to 7(c), the third XY node image (Node3), the fourth XY node image (Node4), and the seventh XY node image (Node7) are composed of eight, six, and eight sub-images, respectively. As can be seen from the sub-image identifiers, the third XY node image (Node3) and the fourth XY node image (Node4) share two sub-images (identifiers (-672)(926), (-672)(925)), and the third XY node image (Node3) and the seventh XY node image (Node7) share four sub-images (identifiers (-669)(926), (-670)(926), (-669)(925), (-670)(925)).

[0047] As indicated by the "4" in the two squares in the "road area" of Figure 7(d), the third XY node image (Node3; N3) and the seventh XY node image (Node7; N7) share four sub-images (identifiers (-669)(926), (-670)(926), (-669)(925), (-670)(925)), satisfying the second condition of sharing a number of sub-images equal to or greater than the second threshold (here, four (road area threshold)).

[0048] Also, in Figure 7 (e) "Road Texture" two squares "40*10 3 As shown in ", the third XY node image (Node3; N3) and the seventh XY node image (Node7; N7) share 40,000 pixels that represent the surface of the road in the sub-image, and satisfy the third condition of sharing a number of pixels greater than or equal to the third threshold (here, 40,000 (road texture threshold)).

[0049] Furthermore, as can be seen from the fact that the elevation difference in the Z plane of the shared sub-images is not shown in (f) "Road Structure" of Figure 7, the third XY node image (Node3; N3) and the seventh XY node image (Node7; N7) satisfy the first condition that the elevation difference (Z-ID difference) of the shared sub-images is less than the first threshold (here, 2.5 m (road structure threshold)). This indicates that these two nodes are at the same Z level and have the same road structure matrix.

[0050] Furthermore, the third XY node image (Node3; N3) and the seventh XY node image (Node7; N7) satisfy the fourth condition that the number of nodes is separated by a fourth threshold (here, 2) or more (i.e., they are non-sequential).

[0051] Based on the above situation, the loop closing unit 14b determines that the third XY node image (Node3;N3) and the seventh XY node image (Node7;N7) satisfy the first condition (the elevation difference is less than the first threshold), the second condition (the number of shared sub-images is equal to or greater than the second threshold), the third condition (the number of shared pixels is equal to or greater than the third threshold), and the fourth condition (the number of nodes is separated by equal to or greater than the fourth threshold), and performs a loop closing calculation on the same location in the third XY node image (Node3;N3) and the seventh XY node image (Node7;N7), thereby reducing positional errors in the XY plane map. This allows for robust loop closure detection and highly accurate map generation.

[0052] FIG. 8 is a diagram illustrating the results of an experiment using the 3D map generating device 10 according to this embodiment, conducted at a location in Kanazawa City that includes an intersection area consisting of a bridge-underpass grade-separated intersection. More specifically, (a) of FIG. 8 shows an XY correlation matrix and an XY decorrelation matrix in the XY plane that represent the intersection area, as well as a node string (Node 1-Node 5) with top-left corners as identifiers. (b) of FIG. 8 shows loop closure at the outbound node (Node 7) and the inbound node (Node 11) on the bridge layer (XY plane at the same elevation). (c) and (d) of FIG. 8 show two nodes (Node 2 and Node 4) that include the intersection area on the underground layer and the bridge layer, as well as a virtual intersection diagram (the diagram shown between (c) and (d)).

[0053] In the experiment, vehicle 20 traveled on a surface road and then on a bridge road in both directions. Node identifiers are represented in the upper left corner of Figure 8(a), which shows the structure of a real-world bridge. The relationships between nodes are represented by the XY correlation matrix and the decorrelation matrix, as shown in the lower left corner of Figure 8(a). The diagonal elements of the XY correlation matrix indicate a continuous linkage pattern between nodes. Due to the distribution pattern of the LiDAR laser beam, two consecutive nodes share some pixels at the end-start link (connection point). Loop closure occurs primarily in the bridge layer, indicated by the off-diagonal elements in the correlation matrix. It can be seen that the number of shared pixels is significantly greater than the number of consecutive pixels. Figure 8(b) illustrates loop closure in the bridge region by showing two nodes with corresponding subimage identifiers. The two nodes share three subimages, primarily the image with identifier (-662, 957). However, the number of shared pixels is sufficient to indicate the possibility of loop closure.

[0054] The bridge layer is clearly distinguished by the XY decorrelation matrix shown in FIG. 8(a). This XY decorrelation matrix is symmetric, which typically indicates the presence of multi-level structures in the map. Therefore, the bridge-underpass node is clearly determined by identifying non-zero entries in the XY decorrelation matrix. FIGS. 8(c) and 8(d) show two nodes containing intersection areas in the two layers. Based on the number of shared pixels and the difference in elevation values, the sub-image showing the exact intersection area is efficiently identified. Furthermore, the XY decorrelation matrix shows another peak, highlighting the dramatic change in road context in the Z direction at the upward node from the overpass layer to the bridge layer. This demonstrates the robustness of the 3D map generation device 10 according to this embodiment, which provides many functions for automatically recognizing and analyzing road structures in a map and extracting many detailed information (characteristic information).

[0055] 9A shows the location of another experiment conducted on a long expressway with underground tunnels including a multi-layered course, using the 3D map generating device 10 according to this embodiment. More specifically, the image shows the trajectories (dark and light colored dots) and 126 nodes within the Yamate Tunnel and Ohashi Junction scanned during two separate runs.

[0056] In this experiment, we used a location containing the world's second-longest tunnel (Yamate Tunnel; 18 km), as shown in FIG. 9A, to collect map data. This tunnel consists of two disconnected tunnel tubes, each with two lanes in one direction of travel. This environment is highly likely to disrupt GPS satellite signals, causing relative and global position errors in the map. The tunnel ends at a bridge junction with four pile-driving loops extending from 35 m underground to 35 m above ground. Because each tunnel tube ends with two upward loops at the bridge junction, as shown in FIG. 9A, to allow for one direction of travel, map data was collected for each tunnel tube in two phases (with a time difference of approximately 3 hours). Because the collected data in the two phases are unrelated, two maps must be generated separately. This allows us to check the scalability of the 3D map generation device 10 according to this embodiment for detecting map merging events. Furthermore, to distinguish the node correspondence between the two maps, one starting point was located in an area that included the open sky area just before the tunnel entrance, and the second starting point was located closer to the entrance.

[0057] The technique disclosed herein, which divides the real world into fixed-size sub-images using an XY plane identifier, represents the geometric existence of the road surface and is independent of the vehicle's position, allowing global detection by the 3D map generator 10. Therefore, the 3D map generator 10 is scalable, and inter-map merging events can be directly detected regardless of the vehicle's trajectory or sensor calibration parameters, i.e., the sub-images generated by different 3D map generators. That is, each divided sub-image in the XY plane has a fixed size and is assigned an identifier (XY-ID) corresponding to a specific geographic location indicated by the sub-image, and each divided sub-image in the Z plane has a fixed size and is assigned an identifier (Z-ID) corresponding to the elevation value indicated by the sub-image. This enables the generation of 3D maps and merging between maps that are independent of the vehicle's position.

[0058] Figure 9B shows an example of the experimental results shown in Figure 9A (correlation matrix between two maps). More specifically, Figure 9B (a) shows the correlation matrix between nodes in two maps (MapA, MapB) based on the IDs of shared subimages. The correlation matrix is divided into four partitions, symmetrically showing local loop closure events in each map and map merge events between the two maps. Each partition can be decomposed into two sections: Yamate Tunnel and Ohashi Junction. In the tunnel section, the correlation matrix shows the connection of consecutive subimages at consecutive nodes in each map. Each node shares some subimages with the previous and next nodes according to the vehicle trajectory. For example, the node shown in Figure 9B (b) shares the last two subimages with the next node. Meanwhile, the correlation matrix of the "map(AB)" partition also shows the pattern of consecutive relationships in the tunnel section. This demonstrates the robustness of the 3D map generating device 10 according to the present embodiment, which detects a map combination event between two maps by the map combination unit 14c, by scanning the course twice.

[0059] The bridge junction is a multi-layered road structure, and mapA and mapB each contain two upward loop scan runs that must be locally distinguished. The problem of detecting map merge events becomes even more challenging because a total of four loops must be processed. Therefore, the two lower loops of the two maps must be correctly combined, just like the upper loop. In the bridge section (Figure 9B (a)), loop closure events and map merge events are indicated by four partitions. Because there is no intersection between the nodes of the two loops in the real world, the loop closure events indicated by the partitions of mapA and mapB are false (non-existent). Therefore, the XY decorrelation matrix is calculated based on the Z-plane map and a first threshold β for the elevation difference, as shown in Figure 9B (b). The XY decorrelation matrix clearly indicates the number of corresponding images between nodes whose elevation difference exceeds the XY decorrelation matrix β. Furthermore, the presence of these numbers indicates XY decorrelation between the nodes of each map. The mapAB partition shows XY decorrelation between the maps of the bridge section, indicating nodes at the same Z level (elevation). Therefore, a correlation matrix based on the number of shared pixels can be safely calculated, as shown in Figure 9B(c). This correlation matrix indicates that there are no loop closure events between the nodes of each map, and map merge events are continuously represented between the maps of the bridge section. In other words, the two scan runs of the bridge junction can be obtained by simulation calculation using two springs with the same length and strength. That is, the springs are stretched into lines by the XY-ID correlation matrix and the XY decorrelation matrix, and the two lines are joined (i.e., by the shared pixel correlation matrix) to form a single line. As mentioned above, in a bridge junction consisting of a two-turn three-dimensional spiral road, no loop closure or map merge events occur between the upper and lower loops at the twisted position. Instead, the map combining unit 14c assumes that both ends of the three-dimensional spiral road in mapA and mapB are stretched and transformed into a simple linear two-dimensional road, and then executes a map combining event at each point on these roads.

[0060] FIG. 9C shows another example of the experimental results shown in FIG. 9A (an example of map combination detection by the map combination unit 14c using two maps). More specifically, FIG. 9C (a) shows a sub-image of a tunnel node using two maps. In such a case, large deviations in various patterns may be observed due to GPS satellite signal interference. FIG. 9C (b) shows two upward loops of a bridge junction in mapA separated based on the XY decorrelation matrix. FIG. 9C (c) shows two maps of the entrance, i.e., the first and second layers, correctly combined based on the matrix shown in FIG. 9B. In such a highway environment, driving the vehicle 20 multiple times to combine map data is necessary to increase map density and improve position accuracy in the global coordinate system. The 3D map generating device 10 according to this embodiment checks the shared texture (number of pixels), the shared area size (number of sub-images), and the true intersection of the XY plane in the multi-layer road environment (decorrelation matrix) to clearly detect true map join events / loop closure events, thereby enabling the generation of highly accurate 3D maps.

[0061] In this way, the 3D map generation device 10 according to this embodiment can automatically detect a multi-layer environment and detect loop closures in each detected layer. Furthermore, the 3D map generation device 10 technically makes it possible to combine multiple maps regardless of the road structure by using the map combining unit 14c.

[0062] As described above, the 3D map generation method according to this embodiment is a method for generating a 3D map, and includes a node image generation step (S11) for generating a plurality of node images, each of which is a pair of an XY node image representing a rectangular ground surface including a driving route such as a road and a Z node image representing the elevation of a point indicated by each pixel constituting the XY node image, from a point cloud acquired by a mapping object detection device (LiDAR 11) mounted on a vehicle 20 and detecting an environment including a driving route such as a road; and a step for generating a 3D map including an XY plane map and a Z plane map by reducing positional errors in the plurality of generated node images. The map generation step (S13) includes, as an example of a revisit processing step, a loop closure step (S13a) that reduces positional errors in the XY plane map by having the vehicle 20 travel around a loop in a graph showing the connection of multiple node images and observe the same location to perform loop closure, thereby reducing cumulative errors. In the loop closure step (S13a), for two XY node images that include the same location, by referring to the two corresponding Z node images, it is determined whether the elevation difference is less than a first threshold, and if the elevation difference is less than the first threshold, the loop is closed (S21).

[0063] As a result, loop closure is performed on two node images as an example of a revisit process, taking into account the difference in elevation, and a highly accurate 3D map is generated through robust loop closure. That is, the 3D map generation device 10 robustly detects a multilevel environment in the map data, separates layers in each map data, and performs loop closure between maps in each layer. Therefore, by inputting such correct relationships to the SLAM unit 14d, the SLAM unit 14d suppresses relative position errors in each layer and errors in the geographic positions between two maps in the absolute coordinate system, enabling the generation of a highly accurate combined map.

[0064] Furthermore, the multiple XY node images generated in the node image generation step (S11) include multiple XY node images corresponding to rectangles of different geographical sizes, and the node image generation step (S11) includes a sub-image division step (S12) that divides each of the multiple XY node images into multiple sub-images corresponding to rectangles of the same geographical size, and in the loop closure step (S13a), loop closure is performed when two XY node images containing the same location share a number of sub-images equal to or greater than a second threshold (S22) and the elevation difference is less than a first threshold.

[0065] This allows the real world to be divided into sub-images of fixed size before loop closure is performed, so that loop closure events can be detected and loop closure can be performed stably and reliably without relying on node images of various sizes.

[0066] The elevation difference of an XY node image is, for example, the average value of the elevations indicated by the pixels that make up the corresponding Z node image.

[0067] Furthermore, in the loop closing step (S13a), if the number of shared pixels representing the road surface in the shared sub-images equal to or greater than a second threshold is equal to or greater than a third threshold, loop closing is performed (S23). As a result, loop closing is performed only when two XY node images share a certain number of pixels or more, which suppresses fluctuation factors dependent on the geographical environment and enables robust loop closing.

[0068] Furthermore, in the loop closure step (S13a), if two XY node images containing the same location are separated by a fourth threshold or more in the number of nodes, loop closure is performed (S24). As a result, loop closure is performed on two discontinuous, separated XY node images, and the effects of checking elevation difference, sub-image commonality, and pixel commonality are exerted, ensuring robust loop closure.

[0069] Furthermore, as another example of the revisiting step, the 3D map generating device 10 performs a map merging step (S13a) in which positional errors in the XY plane map are reduced by observing the same location in a graph showing the connections of multiple node images to perform map merging, which reduces cumulative errors. In the map merging step (S13a), for two XY node images containing the same location, the device determines whether the elevation difference is less than a first threshold by referencing the two corresponding Z node images, and performs map merging if the elevation difference is less than the first threshold. This allows multiple maps to be merged taking the elevation difference into consideration, achieving map merging with high accuracy.

[0070] In the map generation step, the relative position error may be reduced by applying SLAM (Simultaneous Localization and Mapping) to the generated 3D map, and a 3D map with reduced relative position error may be output. This reduces the relative position error in each layer and the error in the geographic position between two maps in the absolute coordinate system, making it possible to generate a highly accurate combined map.

[0071] Furthermore, the 3D map generation device 10 according to this embodiment is a device for generating a 3D map, and is mounted on a vehicle 20. The 3D map generation device 10 includes a node image generation unit 13 that generates a plurality of node images, each of which is a pair of an XY node image representing a rectangular ground surface including a driving route such as a road and a Z node image representing the elevation of a point indicated by each pixel constituting the XY node image, from a point cloud acquired by a mapping object detection device (LiDAR 11) for detecting an environment including a driving route such as a road, and a node image generation unit 13 that generates a plurality of node images, each of which is a pair of an XY node image representing a rectangular ground surface including a driving route such as a road and a Z node image representing the elevation of a point indicated by each pixel constituting the XY node image, by reducing positional errors in the plurality of generated node images. The map generation unit 14 includes a map generation unit 14 that generates a three-dimensional map, and the map generation unit 14 includes, as an example of a revisit processing unit 14a, a loop closure unit 14b that reduces positional errors in the XY plane map by performing loop closure to reduce cumulative errors by having the vehicle 20 travel around a loop in a graph showing the connection of a plurality of node images and observing the same location, and the loop closure unit 14b determines whether the elevation difference is less than a first threshold value by referring to two corresponding Z node images for two XY node images that include the same location, and performs loop closure if the elevation difference is less than the first threshold value.

[0072] This allows loop closure for the two node images to be performed taking into account the elevation difference, resulting in a highly accurate 3D map generated through robust loop closure. In other words, the 3D map generation method robustly detects a multilevel environment in the map data, separates layers in each map data, and performs loop closure between maps in each layer. Therefore, by inputting this correct relationship to the SLAM unit 14d, the SLAM unit 14d suppresses relative position errors in each layer and errors in the geographic positions between the two maps in the absolute coordinate system, generating a highly accurate combined map.

[0073] The present disclosure may also be realized by a program for causing a computer to execute the steps included in the above-described 3D map generation method, and a computer-readable non-transitory recording medium such as a CD-ROM on which the program is recorded.

[0074] The 3D map generation method and 3D map generation device 10 of the present disclosure have been described above based on the embodiments, but the present disclosure is not limited to these embodiments. As long as they do not deviate from the gist of the present disclosure, various modifications that would occur to those skilled in the art to the present embodiments and other forms constructed by combining some of the components of the embodiments are also included within the scope of the present disclosure.

[0075] For example, in the embodiment, the 3D map generation device 10 uses the LiDAR 11 as an object detection device for map creation, but this is not limited thereto and a visible camera or an infrared camera may also be used. The 3D map generation device 10 uses the LiDAR 11 and the GPS 12 mounted on the vehicle 20, but this combination is not limited thereto. The vehicle 20 may also be equipped with a camera that captures images of the environment including the driving route such as roads, and the 3D map generation device 10 may generate a LiDAR frame of an intensity image by complementarily using images acquired from the camera with respect to the point cloud acquired from the LiDAR 11. [Industrial Applicability]

[0076] The 3D map generation device according to the present disclosure can be used as a 3D map generation device that generates a 3D map with high positional accuracy from data obtained by a vehicle equipped with LiDAR, for example, as a device that generates a 3D map to be used in an autonomous vehicle. [Explanation of symbols]

[0077] 10. 3D map generator 11 LiDAR 12 GPS 13 Node Image Generation Unit 13a Sub-image division unit 14 Map generation section 14a Revisit Processing Unit 14b Loop Closure 14c Map junction 14d SLAM section 15 3D Maps 20 vehicles

Claims

1. 1. A method for generating a three-dimensional map, comprising: a node image generation step in which a node image generation unit of the three-dimensional map generation device generates, from a point cloud acquired by an object detection device for map creation that is mounted on a vehicle and detects the environment including the driving route, a plurality of node images, each of which is a pair of an XY node image representing a rectangular ground surface including the driving route and a Z node image representing the elevation of a point indicated by each pixel constituting the XY node image; a map generation step in which a map generation unit of the three-dimensional map generation device generates a three-dimensional map including an XY plane map and a Z plane map by reducing positional errors in the generated plurality of node images; the map generation step includes a revisiting step in which a revisiting processing unit of the three-dimensional map generation device performs a revisiting process for reducing cumulative errors, the revisiting process including a process for identifying the same location in a graph showing connections between the plurality of node images, thereby reducing positional errors in the XY plane map; The revisiting step includes a process of determining whether or not the elevation difference between the two XY node images including the same location is less than a first threshold value by comparing the two corresponding Z node images, and performing the revisiting process based on the result of the determination. A method for generating a 3D map.

2. the revisiting step includes a loop closing step of performing loop closing to reduce cumulative errors by identifying the same point on a loop in a graph showing connections of the plurality of node images, thereby reducing positional errors in the XY plane map; The loop closing step includes a process of determining whether or not the elevation difference is less than a first threshold value by comparing the two corresponding Z node images for the two XY node images including the same location, and performing the loop closing when the elevation difference is less than the first threshold value.

2. The method of claim 1.

3. The plurality of XY node images generated in the node image generating step include a plurality of XY node images corresponding to rectangles of different geographical sizes, the node image generation step includes a sub-image division step of dividing each of the plurality of XY node images into a plurality of sub-images corresponding to rectangles of the same geographical size; In the loop closing step, the loop is closed when the two XY node images including the same location share a number of sub-images equal to or greater than a second threshold and the elevation difference is less than the first threshold.

3. The method of claim 2.

4. The elevation difference of the XY node image is the difference between the average elevations indicated by the pixels constituting the corresponding Z node image.

4. The method of claim 3.

5. In the loop closing step, the loop is closed when the number of shared pixels indicating the surface of the travel path in the shared sub-images equal to or greater than the second threshold is equal to or greater than a third threshold.

5. The method for generating a three-dimensional map according to claim 3 or 4.

6. In the loop closing step, when the number of nodes between the two XY node images including the same location is different by a fourth threshold or more, the loop is closed.

5. The method for generating a three-dimensional map according to claim 3 or 4.

7. the revisiting step includes a map combining step of reducing a position error in the XY plane map by performing map combining to reduce an accumulated error, the map combining including a process of identifying the same location in a graph showing connections of the plurality of node images; In the map combining step, for the two XY node images including the same location, the two corresponding Z node images are compared to determine whether or not the elevation difference is less than a first threshold, and if the elevation difference is less than the first threshold, the map combining is performed.

2. The method of claim 1.

8. A method for outputting a three-dimensional map in which relative position errors are reduced using the three-dimensional map generation method according to any one of claims 1 to 7, comprising: The reducing of the relative position error includes applying SLAM (Simultaneous Localization and Mapping) to the three-dimensional map generated in the map generation step. method.

9. An apparatus for generating a three-dimensional map, comprising: a node image generation unit that generates, from a point cloud acquired by an object detection device for map creation that is mounted on a vehicle and detects the environment including the driving route, a plurality of node images that are pairs of an XY node image that represents a rectangular ground surface including the driving route and a Z node image that represents the elevation of a point indicated by each pixel that constitutes the XY node image; a map generation unit that generates a three-dimensional map including an XY plane map and a Z plane map by reducing positional errors in the generated node images, the map generation unit includes a revisit processing unit that reduces positional errors in the XY plane map by performing a revisit processing that reduces cumulative errors, the revisit processing including a process of identifying the same location in a graph that indicates connections between the plurality of node images; the revisit processing unit includes a process of determining whether or not an elevation difference is less than a first threshold value by comparing the two corresponding Z node images for the two XY node images including the same location, and performs the revisit processing according to the result of the determination. 3D map generator.

10. A program for causing a computer to execute the steps included in the three-dimensional map generating method according to any one of claims 1 to 7.

11. A program for causing a computer to execute the steps included in the method according to claim 8.

Images