Map generation device, map generation method, and computer program for map generation

Abstract

The present application relates to a map generation device, a map generation method, and a computer program. The map generation device has: an intersection detection section (12) that detects an intersection and a plurality of roads connected thereto from an aerial image; a lane detection section (13) that detects, for each road connected to the intersection, an entry lane included in the road that enters the intersection and an exit lane that exits the intersection; a sign structure detection section (14) that detects, for each road, a road sign indicating a direction in which a vehicle can travel or a road structure indicating a direction in which a vehicle can travel, which is shown on the road, from the aerial image; and a lane network generation section (15) that generates, for each road, a lane network indicating a connection relationship between lanes of the intersection in a manner in which an entry lane of the road is connected to an exit lane of another road in the plurality of roads from which a vehicle can travel from the entry lane of the road, based on a road sign or a structure of the road that is indicated on the entry lane of the road.

Description

TECHNICAL FIELD

[0001] The present application relates to a map generation device, a map generation method, and a map generation computer program that generate a map from an image. BACKGROUND

[0002] A technique of generating a map containing information utilized for an automated driving vehicle from an image representing a road has been proposed (see Japanese Patent Application Publication No. 2017-54346 and Japanese Patent Application Publication No. 2020-38365). The road data generation device disclosed in Japanese Patent Application Publication No. 2017-54346 extracts a road surface high flat area at a ground road height, and extracts a center line in a remaining area of the road surface high flat area excluding an area having a width wider than a pre-set upper limit width of a road as a road area seed (a starting point). Further, the road data generation device generates a road area from the road area seed using height information obtained from a DSM (Digital Surface Model) and spectral information obtained from an aerial photograph image, and performs region growing. In addition, the travel track data generation device in a crossroad disclosed in Japanese Patent Application Publication No. 2020-38365 generates travel track data that is data capable of determining a travel track in a crossroad for automated driving, using image data obtained by photographing a crossroad, recognizing road marks (road signs) in the crossroad in accordance with a recognition result. SUMMARY

[0003] As one of the information contained in a map and utilized for an automated driving vehicle, there is information representing a connection relationship between lanes for reference to set a travel planned track of a vehicle controlled by automated driving. Therefore, in order to generate a map, in particular, with respect to a crossroad, it is required to acquire information representing a connection relationship between lanes in which a vehicle can travel among roads connected to the crossroad.

[0004] Therefore, an object of the present application is to provide a map generation device capable of automatically extracting a connection relationship between lanes in which a vehicle can travel among roads connected to a crossroad based on an aerial view image.

[0005] As one of the technical solutions of the present application, a map generation device is provided. The map generation device includes: an intersection detection unit configured to detect an intersection and a plurality of roads connected to the intersection from an aerial image; a lane detection unit configured to detect, for each of the plurality of roads connected to the intersection, an entry lane included in the road that enters the intersection and an exit lane included in the road that exits the intersection; a sign structure detection unit configured to detect, for each of the plurality of roads, a road sign indicating a direction in which a vehicle can travel or a road structure indicating the direction in which the vehicle can travel, which is shown in the road, from the aerial image; and a lane network generation unit configured to generate, for each of the plurality of roads, a lane network indicating a connection relationship between lanes at the intersection, based on the road sign indicated on the entry lane of the road or the structure of the road, in a manner in which the entry lane of the road is connected to exit lanes of other roads of the plurality of roads in which a vehicle can travel from the entry lane of the road.

[0006] In the map generation device, preferably, the sign structure detection unit detects an arrow indicating a direction in which a vehicle can travel as the road sign, and the lane network generation unit generates, for each of the plurality of roads, the lane network in a manner in which the entry lane of the road is connected only to exit lanes of other roads of the plurality of roads, other than the road, which are located in a direction indicated by the arrow indicated on the entry lane of the road.

[0007] Alternatively, in the map generation device, preferably, the sign structure detection unit detects a bypass directly connecting an entry lane of any one of the plurality of roads connected to the intersection and an exit lane of other roads of the plurality of roads, other than the any one of the plurality of roads, as the road structure, and the lane network generation unit generates the lane network in a manner in which the entry lane connected by the bypass is connected to the exit lane.

[0008] As another technical solution of the present application, a map generation method is provided. The map generation method includes: detecting an intersection and a plurality of roads connected to the intersection from an aerial image; detecting, for each of the plurality of roads connected to the intersection, an entry lane included in the road that enters the intersection and an exit lane included in the road that exits the intersection; detecting, for each of the plurality of roads, a road sign indicating a direction in which a vehicle can travel or a road structure indicating the direction in which the vehicle can travel, which is shown in the road, from the aerial image; and generating, for each of the plurality of roads, a lane network indicating a connection relationship between lanes at the intersection, based on the road sign indicated on the entry lane of the road or the structure of the road, in a manner in which the entry lane of the road is connected to exit lanes of other roads of the plurality of roads in which a vehicle can travel from the entry lane of the road.

[0009] As still another aspect of the present application, a computer program for map generation is provided. The computer program for map generation includes commands for causing a computer to execute the following processing: detecting a crossroad and a plurality of roads connected to the crossroad from a bird's-eye image; detecting, for each of the plurality of roads connected to the crossroad, an entry lane into the crossroad and an exit lane from the crossroad included in the road; detecting, for each of the plurality of roads, a road marking indicating a direction in which a vehicle can travel or a road structure indicating a direction in which a vehicle can travel, shown in the road, from the bird's-eye image; and generating, for each of the plurality of roads, a lane network indicating a connection relationship of lanes to each other at the crossroad in a manner that an entry lane of the road is connected to an exit lane of another road of the plurality of roads in which a vehicle can travel from the entry lane of the road, based on the road marking indicated on the entry lane of the road or the structure of the road.

[0010] The map generation device according to the present application achieves the following effect: it is possible to automatically extract a connection relationship between lanes in which a vehicle can travel among roads connected to a crossroad based on a bird's-eye image. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 Fig. 1 is a hardware configuration diagram of a map generation device according to an embodiment.

[0012] Figure 2 Fig. 2 is a functional block diagram of a processor of the map generation device according to an embodiment.

[0013] Figure 3A Fig. 3 is an explanatory diagram of an outline of crossroad region detection.

[0014] Figure 3B Fig. 4 is an explanatory diagram of an outline of crossroad region detection.

[0015] Figure 3C Fig. 5 is an explanatory diagram of an outline of crossroad region detection.

[0016] Figure 4 Fig. 6 is an explanatory diagram of an outline of reference point detection of a crossroad based on skeletonization or thinning processing.

[0017] Figure 5 Fig. 7 is a diagram indicating an example of determining a connection relationship of an entry lane and an exit lane based on a travel direction indication marker.

[0018] Figure 6 Fig. 8 is a workflow diagram of a map generation process.

[0019] Figure 7is a diagram indicating an example of a connection relationship between an entry lane and an exit lane determined based on a road structure. DETAILED DESCRIPTION

[0020] Hereinafter, a map generation device, a map generation method used in the map generation device, and a map generation computer program will be described with reference to the drawings. The map generation device detects a crossroad region including a crossroad and a plurality of roads connected to the crossroad from an aerial image indicating roads. In addition, the map generation device detects, for each of the plurality of roads connected to the crossroad, an entry lane into the crossroad and an exit lane from the crossroad. Furthermore, the map generation device generates, for each of the plurality of roads connected to the crossroad, a lane network indicating a connection relationship between lanes at the crossroad in such a manner that an entry lane of the road is connected to an exit lane of another road through which a vehicle can travel from the entry lane of the road. At this time, the map generation device detects, for each of the plurality of roads connected to the crossroad, a road sign indicating a direction in which a vehicle can travel, shown on the entry lane of the road, from the aerial image, and generates the lane network in such a manner that the entry lane of the road is connected to the exit lane of another road through which a vehicle can travel from the entry lane of the road, in accordance with the direction indicated by the road sign.

[0021] In addition, in each of the embodiments and modified examples described below, the aerial image indicating roads, which is the object of the map generation processing, can be, for example, an image obtained by photographing a ground surface from a vertical direction above and in which each road sign indicated on a road can be recognized, such as an image indicating a high-resolution satellite photograph or an aerial photograph. In addition, in the following description, the aerial image which is the object of the map generation processing will be simply referred to as an image.

[0022] Figure 1 is a hardware configuration diagram of the map generation device to which the embodiment relates. As shown in Figure 1 , the map generation device 1 has a communication interface 2, an input device 3, a display device 4, a memory 5, a storage medium access device 6, and a processor 7.

[0023] The communication interface 2 has a communication interface and its control circuit for connecting with a communication network in compliance with a communication standard such as Ethernet (registered trademark). The communication interface 2 receives various information or data from other devices (not shown) connected via the communication network and delivers them to the processor 7. The data received by the communication interface 2 can also include an image representing a road, which is the object of the map generation processing, and information representing a geographical range shown in the image (e.g., the longitude and latitude of a predetermined position (e.g., the upper left end or the center) of the region shown in the image, the actual spatial size of the region in the horizontal and vertical directions, and the orientation). In addition, the communication interface 2 can also output a road map, which is an example of the map information obtained as a result of the execution of the map generation processing, to other devices via the communication network from the processor 7.

[0024] The input device 3 has, for example, a pointing device such as a keyboard and a mouse. Also, the input device 3 generates an operation signal corresponding to an operation performed by a user, such as an operation of selecting an image which is the object of the map generation processing, an operation of instructing the start of the execution of the map generation processing, or an operation of causing the display device 4 to display a generated road map, and outputs the operation signal to the processor 7.

[0025] The display device 4 has, for example, a liquid crystal display or an organic EL display. Also, the display device 4 displays display data received from the processor 7, such as data representing candidates of an image for which the map generation processing is performed or a generated road map or a part thereof.

[0026] In addition, the input device 3 and the display device 4 can also be an integrated device like a touch panel display.

[0027] The memory 5 is an example of a storage unit, and is, for example, a read-and-write semiconductor memory and a read-only semiconductor memory. Also, the memory 5 stores, for example, a computer program for the map generation processing performed by the processor 7, various data used in the map generation processing, such as a parameter set which specifies a recognizer used in the map generation processing, and various data generated in the course of the execution of the map generation processing. Further, the memory 5 can also store an image which is the object of the map generation processing and information representing a geographical range shown in the image. Further, the memory 5 can also store a generated road map.

[0028] The storage medium access device 6 is, for example, a device that accesses a storage medium 8 such as a magnetic disk, a semiconductor memory card, and an optical storage medium. Further, the storage medium access device 6 constitutes another example of a storage section together with the storage medium 8. The storage medium access device 6, for example, reads in a computer program for the map generation processing executed on the processor 7, or an image that is a target of the map generation processing, stored in the storage medium 8, and hands it over to the processor 7. Alternatively, the storage medium access device 6 can also receive a generated road map from the processor 7, and write the road map to the storage medium 8.

[0029] The processor 7 is an example of a processing section, and has, for example, one or a plurality of CPUs and peripheral circuits thereof. Further, the processor 7 can also have an arithmetic circuit for numerical operation, an arithmetic circuit for graphic processing, and an arithmetic circuit for logical operation. Moreover, the processor 7 controls the entire map generation device 1. In addition, the processor 7 executes the map generation processing with respect to an image representing a road that is a target of the map generation processing.

[0030] Figure 2 is a functional block diagram of the processor 7 relating to the map generation processing. As shown in Figure 2 the processor 7 has a stop line detection section 11, an intersection region detection section 12, a lane detection section 13, a sign structure detection section 14, a lane network generation section 15, and a map generation section 16. These sections possessed by the processor 7 are, for example, functional modules realized by a computer program executed on the processor 7. Alternatively, these sections possessed by the processor 7 can also be dedicated arithmetic circuits provided to the processor 7.

[0031] The stop line detection section 11 detects each temporary stop line (stop line) indicated in the image. To this end, the stop line detection section 11 detects each temporary stop line, for example, by inputting the image to an identifier that is pre-learned to detect a temporary stop line from an image. The stop line detection section 11 can use, for example, a so-called deep neural network (DNN) having a convolutional neural network (CNN) type architecture as such an identifier. More specifically, such an identifier can be configured as a CNN for semantic segmentation such as a fully convolutional network (FCN), U-Net, or PSPNet, or a CNN for instance segmentation such as Mask-RCNN, for example. Alternatively, such an identifier can also be a CNN for object detection such as SSD (Single Shot MultiBox Detector) or Faster R-CNN. Further, such an identifier can also be an identifier in accordance with a machine learning method other than a neural network such as AdaBoost. Alternatively, the stop line detection section 11 can detect a temporary stop line using a method other than machine learning such as template matching.

[0032] The stop line detection section 11 notifies the intersection region detection section 12, the lane detection section 13, and the map generation section 16 of information indicating the position and size on the image of each detected temporary stop line.

[0033] The intersection region detection unit 12 is an example of an intersection detection unit that detects, for each intersection indicated in the image, a predetermined region (hereinafter, simply referred to as an intersection region) that includes the intersection and a plurality of roads connected to the intersection. To this end, the intersection region detection unit 12, for example, detects each intersection marker and a road region by inputting the image to an identifier that is pre-learned to detect a road mark (hereinafter, referred to as an intersection marker) indicating the center of the intersection and a road region indicating a road from the image. The intersection region detection unit 12 can use the same identifier as the identifier explained with respect to the stop line detection unit 11, such as a CNN for semantic segmentation or instance segmentation, as such an identifier. In a case where a CNN for semantic segmentation is used as the identifier, the intersection region detection unit 12 can classify a set of pixels indicating the intersection marker by each intersection marker by applying clustering processing or labeling processing to the set of pixels indicating the intersection marker. Further, it can be that the identifier used in the stop line detection unit 11 is pre-learned to detect not only the temporary stop line from the image but also the intersection marker and the road region from the image. In this case, the intersection region detection unit 12 can receive information indicating the position and size of the temporary stop line on the image from the stop line detection unit 11, and at the same time, receive information indicating the position and size of the intersection marker and the road region on the image.

[0034] Upon detecting the intersection marker and the road region, the intersection region detection unit 12 detects the intersection region with respect to each detected intersection marker in accordance with the following steps.

[0035] The intersection region detection unit 12 sets the position of a predetermined point within the detected intersection marker on the image, such as the center of gravity of the intersection marker, as a reference point within the intersection. Further, the intersection region detection unit 12 sets a plurality of scan lines at equal angular intervals from the reference point in a radial direction, and detects, for each scan line, a position at which the scan line intersects a boundary (hereinafter, referred to as a road end) between the road region and a region other than the road region or a line extending the temporary stop line to the road end, as an outer edge of the intersection region. By thus determining the outer edge of the intersection region, the intersection region is made to include the temporary stop line of each road connected to the intersection, and thus, it becomes easy to obtain the number of lanes possessed by each road connected to the intersection.

[0036] In addition, for each of the scan lines, a position in the scan line that does not have the intersection with the temporary stop line or the like between the reference point from the intersection and a pre-set farthest end, the intersection region detection section 12 detects the position of the farthest end as an outer edge candidate of the intersection region. Furthermore, for the road whose position reaching the farthest end becomes the outer edge candidate, the intersection region detection section 12 gradually approaches the outer edge candidate line in which the outer edge candidates are arranged toward the reference point, and sets the outer edge candidate line of the position that is a predetermined number of pixels away from the reference point when the length of the outer edge candidate line is increased by more than a predetermined ratio compared to the length of the immediately preceding outer edge candidate line as the outer edge of the intersection region. Thus, the intersection region detection section 12 can also include the road connected to the intersection in the intersection region for the road that does not have the temporary stop line among the roads connected to the intersection. Therefore, the lane network related to the road that does not have the temporary stop line can be appropriately set.

[0037] The intersection region detection section 12 detects, for each of the roads connected to the intersection, a region surrounded by a line connecting the outer edges set on the roads to each other as the intersection region.

[0038] Figures 3A-3C is a diagram that outlines the intersection region detection. As shown in Figure 3A Each of the scan lines 302 set in the radial direction from the reference point 301 in the intersection intersects the position of the road end 303 or the line in which the temporary stop line 304 is extended to the road end 303, and is set as the outer edge of the intersection region. However, for the road 305 located on the right side from the reference point 301, since there is no temporary stop line, the candidate line 311 of the outer edge of the intersection region is set at the position of the pre-set farthest end.

[0039] As shown in Figure 3B For the road 305, when the outer edge candidate line is gradually approached toward the reference point 301, the length of the outer edge candidate line is increased by more than a predetermined ratio compared to the length of the immediately preceding outer edge candidate line at the position 306. Then, the outer edge candidate line of the position that is a predetermined number of pixels away from the reference point 301 from the position 306 is set as a part 312 of the outer edge of the intersection region on the road 305.

[0040] As shown in Figure 3C Finally, the region surrounded by the outer edges found for each of the roads becomes the intersection region 313.

[0041] In addition, depending on the intersection, there is a case where the intersection marker is not set. Then, the intersection region detection section 12 can also set the reference point for the intersection where the intersection marker is not set in the following steps.

[0042] The intersection region detection section 12 calculates a skeleton network that represents the connection relationship of each road by applying a skeletonization process to the road region detected by the recognizer. As a result, each road becomes represented by a single line, and thus, at an intersection, becomes a node at which a plurality of lines intersect. The intersection region detection section 12 then sets each node having three or more degrees as a reference point. Alternatively, the intersection region detection section 12 can calculate a network that is obtained by performing a thinning process on the road region instead of skeletonizing the road region, and thus, calculate a network that represents the connection relationship of each road. In this case as well, the intersection region detection section 12 can set each node having three or more degrees as a reference point. By setting the reference point based on the network in this way, the intersection region detection section 12 can set a reference point within an intersection for which no intersection marker is set, and as a result, can appropriately set an intersection region for such an intersection.

[0043] Figure 4 is a diagram that schematically illustrates the reference point detection of an intersection based on skeletonization or thinning. As shown in Figure 4 the network 401 obtained by skeletonizing or thinning the road region shown in the image 400, nodes 402 having three or more degrees appear at each intersection. The intersection region detection section 12 can thus set each node 402 as a reference point for intersection region detection.

[0044] Alternatively, the intersection region detection section 12 can perform a Voronoi partitioning process on the road region using each pixel that represents the end of a road as a mother point. In this case, a Voronoi boundary will be drawn at approximately the center of each road, and thus, at an intersection, a Voronoi point at which a plurality of Voronoi boundaries intersect will appear. The intersection region detection section 12 can thus set each Voronoi point as a reference point. In this case as well, the intersection region detection section 12 can set a reference point within an intersection for which no intersection marker is set, and as a result, can appropriately set an intersection region for such an intersection.

[0045] The intersection region detection section 12 notifies the lane detection section 13, the lane network generation section 15, and the map generation section 16 of information that represents each intersection region detected on the image.

[0046] The lane detection section 13 detects at least one of an entry lane that enters an intersection connected to each road included in each intersection region detected and an exit lane that exits the intersection for each road.

[0047] For example, the lane detection unit 13 determines, for the road of interest, that the road contains both the entrance lane and the exit lane if the length of the temporary stop line is below a value obtained by multiplying the road width by a predetermined coefficient (for example, 0.5). On the other hand, if the length of the temporary stop line is longer than the value obtained by multiplying the road width by the predetermined coefficient, the lane detection unit 13 determines that the road contains only the entrance lane. Further, the lane detection unit 13 can calculate the width of the road of interest as the distance between the road ends in a direction substantially orthogonal to the extension direction of the road. In addition, in a case where the lane division line (lane demarcation line) is detected by the above-described recognizer, the lane detection unit 13 determines, for the entrance lane and the exit lane of the road of interest, that there are the number of lanes divided by the lane division line, respectively. At this time, the lane detection unit 13 can determine the number of entrance lanes and the number of exit lanes for the road on which the temporary stop line is provided, depending on the positional relationship between the lane division line and the temporary stop line. For example, the lane detection unit 13 determines, for the road of interest, the number obtained by adding 1 to the number of lane division lines provided in a direction substantially orthogonal to the extension direction of the road at positions not overlapping the temporary stop line provided on the road, as the number of exit lanes. In addition, the lane detection unit 13 determines, for the road of interest, the number obtained by adding 1 to the number of lane division lines provided in a direction substantially orthogonal to the extension direction of the road at positions overlapping the temporary stop line provided on the road, as the number of entrance lanes. Further, the lane detection unit 13 can determine, for the road of interest, that, in a direction substantially orthogonal to the extension direction of the road, the lane division line within a predetermined range from the road center side of one end of the temporary stop line is not referred to in the counting of the number of entrance lanes and the number of exit lanes as a line distinguishing the entrance lane and the exit lane. Further, the predetermined range is set to a size smaller than the width of the vehicle on the image, for example.

[0048] In addition, in a case where a lane division line is detected on a road on which a temporary stop line is not provided, the lane detection unit 13 can also determine that the road includes two lanes divided by the lane division line. Furthermore, the lane detection unit 13 can determine one of the two lanes that can enter the intersection in accordance with the road regulation of the region indicated in the image as an entry lane and the other lane as an exit lane. Further, for a road on which neither a temporary stop line nor a lane division line is detected (hereinafter, referred to as a stop line-free road for convenience of explanation), the lane detection unit 13 determines that the stop line-free road includes only an exit lane. Alternatively, for the stop line-free road, the lane detection unit 13 can determine whether the lane of the stop line-free road is an entry lane or an exit lane based on the relationship with the lanes of other roads connected to the same intersection. For example, in a case where the intersection included in the intersection region of interest is a crossroad, a temporary stop line is provided on a road located opposite the stop line-free road across the reference point, and the lane of the road is only an entry lane, the lane detection unit 13 determines that the stop line-free road includes only an exit lane. On the other hand, in a case where the intersection included in the intersection region of interest is a crossroad, and the road located opposite the stop line-free road across the reference point is also a stop line-free road, it can also be configured that one of the lanes included in the stop line-free road of interest is a lane that can both enter and exit the intersection of interest, that is, an entry lane and an exit lane. In this way, by detecting the entry lane and the exit lane in accordance with the presence or absence of the temporary stop line and the ratio of the length of the temporary stop line to the width of the road, the lane detection unit 13 can accurately detect the entry lane and the exit lane included in each road connected to the intersection.

[0049] The lane detection unit 13 notifies the lane network generation unit 15 of information indicating the entry lane and the exit lane of each road connected to the intersection included in the intersection region of interest for each intersection region.

[0050] The sign structure detection section 14 detects, from the bird's-eye image, for each intersection, and for each of the plurality of roads connected to the intersection, a road sign (hereinafter referred to as a travel direction indicator) shown on the road indicating a direction in which a vehicle can travel. To this end, the sign structure detection section 14 detects each travel direction indicator from the image, for example, by inputting the image to a recognizer, and recognizes a category of the direction in which a vehicle can travel indicated by the detected travel direction indicator (a straight arrow, a right-turn arrow, a left-turn arrow, a straight-and-left arrow, and the like, hereinafter referred to as a category of the travel direction indicator). The recognizer is pre-learned to detect a travel direction indicator from an image and recognize a category of the travel direction indicator. The sign structure detection section 14 can use, as such a recognizer, a recognizer for object detection, such as an SSD or a Faster R-CNN, which is the same as the recognizer explained with respect to the stop line detection section 11. Alternatively, the sign structure detection section 14 can detect a travel direction indicator and recognize a category of the detected travel direction indicator by a method other than machine learning, such as template matching between a template pre-stored in the memory 5 for each category of the travel direction indicator and an image.

[0051] The sign structure detection section 14 hands over, to the lane network generation section 15, information indicating a position on the image of each detected travel direction indicator (for example, a position of a center of gravity of the travel direction indicator, or positions of a top-left pixel and a bottom-right pixel of a rectangular region enclosing the travel direction indicator) and a category of the travel direction indicator.

[0052] The lane network generation section 15 generates, for each intersection region, a lane network indicating a connection relationship between lanes at the intersection in a manner in which, for each of the plurality of roads connected to the intersection included in the intersection region, an entry lane of the road and an exit lane of another road to which the entry lane of the road is connected at the intersection are connected. Thereby, for each intersection, information indicating a direction in which a vehicle can travel is obtained in units of lanes.

[0053] In the present embodiment, the lane network generation section 15 determines, for each intersection region, an entry lane of each road connected to the intersection and an exit lane of another road to which a vehicle can travel from the entry lane of the road, based on a travel direction indicator indicated on the entry lane of the road. Further, the lane network generation section 15 generates a lane network in a manner in which the entry lane of each road and the exit lane of another road to which a vehicle can travel from the entry lane of the road are connected.

[0054] To this end, the lane network generation section 15 determines, for each of the detected travel direction indication marks, a road on which the travel direction indication mark is provided and an entry lane, and a crossing region including a crossing connected to the road on which the travel direction indication mark is provided, with reference to information indicating a position of the travel direction indication mark on the image and positions of the respective crossing regions on the image. Further, the lane network generation section 15 determines a direction in which a vehicle can travel, indicated by the travel direction indication mark, with reference to a category of the travel direction indication mark. Moreover, the lane network generation section 15 determines an exit lane, which is located in a direction indicated by the travel direction indication mark, from an entry lane on which the travel direction indication mark is provided, as an exit lane connected to the entry lane, and connects the entry lane on which the travel direction indication mark is provided and the determined exit lane.

[0055] Figure 5 is a diagram indicating an example of determining a connection relationship between an entry lane and an exit lane based on a travel direction indication mark. As shown in Figure 5 , a crossing 501 is connected to four roads 502 to 505. Among them, an entry lane 502a of the road 502 connected from the lower side with respect to the crossing 501 is marked with a travel direction indication mark 510 indicating a right turn. Then, the lane network generation section 15 connects, as shown by a line 520, an exit lane 505b of the road 505 located at the rightmost side from the entry lane 502a of the road 502 among the other roads 503 to 505 than the road 502 to the entry lane 502a when generating the lane network.

[0056] In addition, in a case where the travel direction indication mark indicates a plurality of directions, the lane network generation section 15 can connect an entry lane of a road on which the travel direction indication mark is provided and exit lanes of roads located in respective directions indicated by the travel direction indication mark from the entry lane.

[0057] Referring again to Figure 5 , an entry lane 505a of the road 505 is marked with a travel direction indication mark 511 indicating a straight and a left turn. Therefore, the lane network generation section 15 connects, as shown by a line 521 and a line 522, an exit lane 503a of the road 503 located in a straight direction from the entry lane 505a of the road 505 and an exit lane 502b of the road 502 located at the leftmost side from the entry lane 505a to the entry lane 505a among the other roads 502 to 504 than the road 505 when generating the lane network.

[0058] Further, in a case where no travel direction indication marks are marked on the entry lane of any road connected to the intersection, the lane network generating section 15 can also generate the lane network in a manner that connects the entry lane and each of the exit lanes of the other road connected to the intersection.

[0059] Further, in a case where a plurality of entry lanes are provided on the road connected to the intersection and a travel direction indication mark is marked on each of the entry lanes, the lane network generating section 15 can perform the above-described processing for each of the entry lanes. Further, in a case where a plurality of exit lanes are provided on the road in a direction indicated by the travel direction indication mark marked on the entry lane of the road connected to the intersection, the lane network generating section 15 can generate the lane network in a manner that connects the entry lane and each of the plurality of exit lanes.

[0060] The lane network generating section 15 can generate the lane network in a manner that connects the midpoint in the lateral direction of the entry lane, for example, the midpoint of the temporary stop line marked on the entry lane, and the midpoint in the lateral direction of the exit lane, for each of the entry lane and the exit lane that are connected to each other. Further, the lane network generating section 15 can also connect the entry lane and the exit lane that are connected to each other with a curve to enable the vehicle to travel along the lane network.

[0061] The lane network generating section 15 notifies the map generating section 16 of information indicating the lane network for each intersection region.

[0062] The map generating section 16 generates a map containing information indicating the lane network for each intersection region. Further, the map generating section 16 causes the map to contain information about the number of lanes included in each road and the road markings provided on each lane, for each road detected from the image.

[0063] At this time, the map generating section 16 can also associate position information (for example, longitude and latitude) indicating the position of each intersection with the road map, for each intersection region, with reference to the information indicating the geographical range shown in the image that is the object of the map generation processing and the position of the intersection region on the image, to calculate the position of the intersection. Likewise, the map generating section 16 can also associate position information indicating the position of each section with the road map, for each road, by dividing the road into sections of a predetermined length, with reference to the information indicating the geographical range shown in the image that is the object of the map generation processing and the position of the section on the image, to calculate the position of each section.

[0064] Further, the map generation section 16 can generate a road map of a larger range by combining the road maps generated per image. At this time, the map generation section 16 can combine the road maps obtained from the respective images in a manner that the same positions of the same road overlap each other by referring to the information indicating the geographical range shown in each image per image.

[0065] The map generation section 16 stores the generated road map in the storage 5 or writes it to the storage medium 8 via the storage medium access device 6. Alternatively, the map generation section 16 can output the generated road map to another device via the communication interface 2.

[0066] Figure 6 is a workflow diagram of the map generation processing. The processor 7 executes the map generation processing in accordance with the following workflow diagram per aerial image that becomes a target of the map generation processing.

[0067] The stop line detection section 11 of the processor 7 detects each temporary stop line indicated in the aerial image (step S101). In addition, the intersection region detection section 12 of the processor 7 detects each intersection region indicated in the aerial image (step S102).

[0068] The lane detection section 13 of the processor 7 detects at least one of an entry lane and an exit lane for each road connected to the intersection for each detected intersection region, based on the presence or absence of a temporary stop line of the road and the ratio of the length of the temporary stop line to the width of the road (step S103).

[0069] The sign structure detection section 14 of the processor 7 detects a travel direction indication sign shown on each road connected to the intersection included in the intersection region, for each road connected to the intersection, for each intersection region from the aerial image (step S104).

[0070] The lane network generation section 15 of the processor 7 generates a lane network for each road connected to the intersection included in the intersection region, for each intersection region, in a manner that the entry lane of the road is connected to the exit lane of another road on which a vehicle can travel from the entry lane, in a direction indicated by the travel direction indication sign indicated on the entry lane of the road (step S105).

[0071] Further, the map generation section 16 of the processor 7 generates a road map including information indicating the lane network for each intersection region (step S106). Then, the processor 7 ends the map generation processing.

[0072] As explained above, the map generation device detects each intersection region shown in the bird's-eye image in which a road is present, detects, for each of the detected intersection regions, an entering lane and an exiting lane for each road connected to the intersection. Also, the map generation device generates, for each of the roads connected to the intersection, a lane network in which the entering lane of the road is connected to the exiting lane of the road in which a vehicle can travel from the entering lane of the road in such a manner. At this time, the map generation device detects, from the bird's-eye image, for each of the roads connected to the intersection, a road sign indicating a direction in which a vehicle can travel shown on the entering lane of the road, and determines the exiting lane of the other road in which a vehicle can travel from the entering lane in accordance with the direction indicated by the road sign. In this way, the map generation device can automatically extract a connection relationship between lanes in which a vehicle can travel between roads connected to the intersection on the basis of the bird's-eye image.

[0073] Further, depending on the intersection, there can be a lane in which only left turns are possible (in Japan, etc.), or a lane in which only right turns are possible (in the United States, etc.), even without a road sign. For example, sometimes before a road reaches an intersection, a left-turn-only lane is branched from other lanes that are subject to traffic control based on a traffic light and is connected to the exiting lane of a road to the left from the road via a bypass road.

[0074] Accordingly, according to the modification, the sign structure detection section 14 can also detect, for each of the roads connected to the intersection included in the intersection region, for each intersection region, a road structure indicating a direction in which a vehicle can travel (hereinafter referred to as a travel direction designation structure) instead of or in addition to the travel direction indication mark, from the bird's-eye image. The travel direction designation structure can be configured to have a structure in which the entering lane of any one of the roads connected to the intersection and the exiting lane of the other road are connected by a bypass road, as described above.

[0075] For example, the sign structure detection section 14 detects the travel direction designation structure for each intersection region by inputting a predetermined range including the intersection region to an identifier that is pre-learned to detect the travel direction designation structure, from the image. The sign structure detection section 14 can use a CNN for object detection such as SSD or Faster R-CNN as such an identifier. Alternatively, the sign structure detection section 14 can detect the travel direction designation structure by a method other than machine learning, such as matching a template indicating the travel direction designation structure pre-stored in the storage 5 with a template of the predetermined range.

[0076] The sign structure detection section 14 gives information indicating a region including the travel direction designation structure detected on the image to the lane network generation section 15.

[0077] The lane network generation section 15 determines, for the intersection region in which the travel direction designation structure is detected, a road on which the travel direction designation structure is provided and a relative position of the travel direction designation structure with respect to the road (i.e., whether the direction toward the intersection is on the left side or the right side) based on the travel direction designation structure on the image and a positional relationship of each road connected to the intersection. Further, the travel direction designation structure is generally provided so that a vehicle can move from an entry lane to an exit lane without crossing an opposite lane. Therefore, in a country such as Japan that has a traffic regulation in which a vehicle travels on the left side, the travel direction designation structure is provided so as to branch from an entry lane that becomes a left turn at the intersection, i.e., an entry lane located at the left end of the road toward the intersection, and merge into an exit lane located at the right end in another road that crosses the road at the intersection toward the intersection. In contrast, in a country such as the United States that has a traffic regulation in which a vehicle travels on the right side, the travel direction designation structure is provided so as to branch from an entry lane that becomes a right turn at the intersection, i.e., an entry lane located at the right end of the road toward the intersection, and merge into an exit lane located at the left end in another road that crosses the road at the intersection toward the intersection. Thus, the lane network generation section 15 can also determine which of a left turn and a right turn the travel direction designation structure is provided for by referring to information indicating longitude and latitude of a geographical range shown in the image, thereby determining a traffic regulation (left side travel or right side travel) applicable to the geographical range, and determining a relative position of the travel direction designation structure with respect to the road based on the determination result.

[0078] Figure 7 is a diagram indicating an example of determining a connection relationship between an entry lane and an exit lane based on a road structure according to the modified example. As shown in Figure 7 , an intersection 701 is connected with four roads 702 to 705. Among them, between an entry lane 702a at the left end of a road 702 connected from the lower side with respect to the intersection 701 and an exit lane 703a of a road 703 connected from the left side with respect to the intersection 701, a bypass road 706 that branches to the left side from the road 702 and merges with the road 703 while bypassing the intersection itself is provided. This bypass road 706 is an example of a travel direction designation structure. Thus, the sign structure detection section 14 detects the bypass road 706 that directly connects the entry lane 702a and the exit lane 703a, and the lane network generation section 15 generates a lane network in such a manner that the entry lane 702a of the road 702 connected by the bypass road 706 and the exit lane 703a of the road 703 are linked.

[0079] According to this modification, the map generation device can automatically extract the connection relationship between drivable lanes among roads connected to an intersection even without detecting road signs.

[0080] Further, a computer program that causes the processor of the map generation device of the above-described embodiment or modification to have the functions of each part of the processor can be provided in the form of being stored in a recording medium that can be read by a computer. Furthermore, the recording medium that can be read by a computer can be provided as a magnetic recording medium, an optical recording medium, or a semiconductor memory, for example.

Claims

1. A map generation device, comprising: The intersection detection unit detects intersections and multiple roads connected to them by inputting a bird's-eye view image into a first neural network that has been pre-learned to detect road signs representing the center of the intersection and road areas representing road regions. If the first neural network does not detect road signs representing the center of the intersection, it applies skeletonization processing to the road regions detected by the first neural network to obtain a skeleton network representing the connection relationships of each road. Thus, each road becomes represented by a single line, and at the intersection, it becomes a node where multiple lines intersect. The lane detection unit detects, for each of the plurality of roads connected to the intersection, the entry lanes leading to the intersection and the exit lanes exiting the intersection contained in that road. The signage detection unit detects, from the bird's-eye view, road signs or road structures indicating the direction in which vehicles can travel, for each of the plurality of roads; as well as The lane network generation unit generates a lane network representing the connection relationships between lanes at the intersection, based on the road markings or the structure of the road's entry lanes, for each of the plurality of roads, in a manner that connects the entry lanes of that road with the exit lanes of other roads from which vehicles can travel.

2. The map generation apparatus according to claim 1, The signage detection unit detects arrows indicating the direction in which vehicles can travel as road signs. The lane network generation unit generates the lane network for each of the plurality of roads in such a way that the entrance lane of that road is connected only to the exit lane of the road located in the direction indicated by the arrow marked on the entrance lane of that road, in other roads among the plurality of roads.

3. The map generation apparatus according to claim 1, The signage structure detection unit detects bypass lanes that directly connect the entry lane of any one of the multiple roads to the exit lane of any other road in the multiple roads as the road structure. The lane network generation unit generates the lane network by connecting the entry lane and the exit lane connected by the bypass.

4. A map generation method, comprising: Intersections and multiple roads connected to said intersections are detected by inputting a bird's-eye view image into a first neural network that has been pre-learned to detect road signs representing the center of an intersection and road areas representing areas with roads in the image. If the first neural network fails to detect the road sign indicating the center of the intersection, skeletonization is applied to the road area detected by the first neural network to obtain a skeleton network representing the connection relationship of each road. Thus, each road is represented by a single line, and at the intersection, it becomes a node where multiple lines intersect. For each of the plurality of roads connected to the intersection, detect the entry lanes leading to the intersection and the exit lanes exiting the intersection contained in that road; From the bird's-eye view, for each of the multiple roads, detect road signs or road structures that indicate the direction in which vehicles can travel. as well as For each of the plurality of roads, based on the road markings on the road's entry lanes or the road's structure, a lane network representing the connectivity between lanes at the intersection is generated in a manner that connects the road's entry lanes to exit lanes of other roads from which vehicles can travel.

5. A computer program product comprising a computer program for map generation, the program causing a computer to execute: Intersections and multiple roads connected to said intersections are detected by inputting a bird's-eye view image into a first neural network that has been pre-learned to detect road signs representing the center of an intersection and road areas representing areas with roads in the image. If the first neural network fails to detect the road sign indicating the center of the intersection, skeletonization is applied to the road area detected by the first neural network to obtain a skeleton network representing the connection relationship of each road. Thus, each road is represented by a single line, and at the intersection, it becomes a node where multiple lines intersect. For each of the plurality of roads connected to the intersection, detect the entry lanes leading to the intersection and the exit lanes exiting the intersection contained in that road; From the bird's-eye view, for each of the plurality of roads, detect road signs or road structures indicating the direction in which vehicles can travel; and For each of the plurality of roads, based on the road markings on the road's entry lanes or the road's structure, a lane network representing the connectivity between lanes at the intersection is generated in a manner that connects the road's entry lanes to exit lanes of other roads from which vehicles can travel.

Images