Route generation device, route generation method and program

Abstract

To provide a route generation device, route generation method, and program capable of improving the robustness of route generation.SOLUTION: A route generation device includes: an extraction unit that extracts a road region from a bird's-eye image; a setting unit that sets a drivable area with respect to the bird's-eye image; a conversion unit that integrates the extracted road region and the set drivable area and converts them into a graph structure; an addition unit that adds a start point node corresponding to the start point of the route and an end point node corresponding to the end point of the route to the graph structure; a route search unit that searches for the optimal route from the start point to the end point on the basis of the graph structure with the start point node and the end point node added; and an output unit that outputs the optimal route.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a route generation device, a route generation method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technique for correcting a road area extracted from a bird's-eye view image using a machine learning technique called semantic segmentation so as to cover the road section of map information. The technique described in Patent Document 1 is said to be usable, for example, for generating so-called high-precision map information used for automatic driving control of a vehicle. In addition, the map information used for the correction can be, for example, map information used for route search to a destination in a navigation device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7207359 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned route search using map information, the optimum route is usually searched for within a road area. In this case, for example, if a disaster occurs and some road areas become unusable, there is a problem that route search may not be possible.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a route generation device, a route generation method, and a program that can improve the robustness of route generation. [Means for solving the problem]

[0006] In order to solve the above problems, the route generation device of the present disclosure includes an extraction unit that extracts a road area from a bird's-eye image, a setting unit that sets a driveable area for the bird's-eye image, a conversion unit that integrates the extracted road area with the set driveable area and converts them into a graph structure, an addition unit that adds a start node corresponding to a start point of a route and an end node corresponding to an end point of the route to the graph structure, a route search unit that searches for an optimal route from the start point to the end point based on the graph structure to which the start node and the end node have been added, and an output unit that outputs the optimal route.

[0007] The route generation method of the present disclosure includes the steps of extracting a road area from a bird's-eye image, setting a drivable area for the bird's-eye image, integrating the extracted road area with the set drivable area and converting it into a graph structure, adding a start node corresponding to the start point of a route and an end node corresponding to the end point of the route to the graph structure, searching for an optimal route from the start point to the end point based on the graph structure to which the start node and the end node have been added, and outputting the optimal route.

[0008] The program of the present disclosure causes a computer to execute the steps of extracting a road area from a bird's-eye image, setting a drivable area for the bird's-eye image, integrating the extracted road area with the set drivable area and converting it into a graph structure, adding a start node corresponding to the start point of a route and an end node corresponding to the end point of the route to the graph structure, searching for an optimal route from the start point to the end point based on the graph structure to which the start node and the end node have been added, and outputting the optimal route. Effect of the Invention

[0009] According to the route generation device, route generation method, and program of the present disclosure, it is possible to improve the robustness of route generation. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a functional configuration diagram of a route generation device according to an embodiment of the present disclosure. [Diagram 2] 10 is a flowchart illustrating an example of an operation of the route generating device according to an embodiment of the present disclosure. [Diagram 3] FIG. 1 is a diagram illustrating an example of a bird's-eye view image according to an embodiment of the present disclosure. [Figure 4] FIG. 11 is a diagram illustrating an example of extraction of a road area according to an embodiment of the present disclosure. [Diagram 5] 1 is a diagram illustrating an example of an image in which a road area is superimposed on a bird's-eye image according to an embodiment of the present disclosure. [Figure 6] FIG. 11 is a diagram showing an example in which a road area and a drivable area are integrated according to an embodiment of the present disclosure. [Figure 7] FIG. 2 illustrates an example of a graph structure according to an embodiment of the present disclosure. [Figure 8] FIG. 13 is a diagram illustrating an example in which a start point and an end point are added to a graph structure according to an embodiment of the present disclosure. [Figure 9] FIG. 1 is a diagram illustrating an example of an optimal route according to an embodiment of the present disclosure. [Figure 10] FIG. 13 is a diagram for explaining a modified example of an embodiment of the present disclosure. [Figure 11] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a route generation device, a route generation method, and a program according to an embodiment of the present disclosure will be described with reference to FIGS. 1 to 10. FIG. 1 is a functional configuration diagram of a route generation device according to an embodiment of the present disclosure. FIG. 2 is a flowchart showing an example of an operation of the route generation device according to an embodiment of the present disclosure. FIG. 3 is a diagram showing an example of a bird's-eye image according to an embodiment of the present disclosure. FIG. 4 is a diagram showing an example of an extraction of a road area according to an embodiment of the present disclosure. FIG. 5 is a diagram showing an example of an image in which a road area is superimposed on a bird's-eye image according to an embodiment of the present disclosure. FIG. 6 is a diagram showing an example of integrating a road area and a drivable area according to an embodiment of the present disclosure. FIG. 7 is a diagram showing an example of a graph structure according to an embodiment of the present disclosure. FIG. 8 is a diagram showing an example in which a start point and an end point are added to a graph structure according to an embodiment of the present disclosure. FIG. 9 is a diagram showing an example of an optimal route according to an embodiment of the present disclosure. FIG. 10 is a diagram for explaining a modified example of an embodiment of the present disclosure. In addition, the same reference numerals are used for the same or corresponding configurations in each figure, and the description is omitted as appropriate.

[0012] (Example of the configuration of a route generation device) A route generation device according to an embodiment of the present disclosure is, for example, a device that inputs a bird's-eye view image and outputs route information representing an optimal route from a start point to an end point of a route search, as shown in Fig. 1. The route generation device 1 shown in Fig. 1 can be configured using, for example, a computer and peripheral devices of the computer. The route generation device 1 shown in Fig. 1 includes an extraction unit 11, a setting unit 12, a conversion unit 13, an addition unit 14, a cost information association unit 15, a route search unit 16, and an output unit 17 as a functional configuration configured by a combination of hardware such as a computer and software such as a program executed by the computer.

[0013] The extraction unit 11 inputs a bird's-eye image D1 such as a satellite photo or an aerial photo, and extracts a road area included in the input bird's-eye image D1 using a trained machine learning model that has been trained to classify road areas from an image by supervised learning such as semantic segmentation. FIG. 3 shows an example of a bird's-eye image D1 input by the extraction unit 11. FIG. 4 shows an example of a road area 31 extracted by the extraction unit 11 from the bird's-eye image D1 shown in FIG. 3. However, the method of extracting a road area by the extraction unit 11 is not limited to this example. The extraction unit 11 may extract an area having a similar color or texture as a road area by, for example, a person specifying a plurality of points of a road area or by specifying a plurality of points of a road area from road information acquired in advance, or by both. Information indicating the road area extracted by the extraction unit 11 is input to the setting unit 12 and the conversion unit 13. Note that, for example, each image of the bird's-eye image D1 has information indicating longitude and latitude as attribute information.

[0014] The setting unit 12 sets a drivable area for the bird's-eye image D1. The drivable area is an area where a vehicle can travel, and can be set to, for example, a grassland, a sandy beach, a shallow river, or the like that is not a road. For example, as shown in FIG. 5, the setting unit 12 sets a drivable area 41 for the bird's-eye image D1 in a state in which a road area 31 is superimposed. The setting of the drivable area 41 for the bird's-eye image D1 can be performed, for example, by semi-manual setting using deep learning (for example, setting using SAM (Segment Anything Model)), manual specification using a paint tool, setting using graph cutting of image processing, or the like. In the example shown in FIG. 5, an image indicating the drivable area is drawn on the bird's-eye image D1 using a paint tool (for example, drawn on different layers). The drivable area 41 set by the setting unit 12 is a target of route search in a route search described later, similar to the road area 31, etc. Information indicating the drivable area set by the setting unit 12 (information indicating a pixel position, a pixel area, longitude, and latitude, etc.) is input to the conversion unit 13.

[0015] The conversion unit 13 integrates the road area extracted by the extraction unit 11 and the drivable area set by the setting unit 12 to convert into a graph structure. FIG. 6 shows an example of integrating the road area extracted by the extraction unit 11 and the drivable area set by the setting unit 12. In the example shown in FIG. 6, the road area 31 and the drivable area 41 are integrated. FIG. 7 shows a graph structure 50 representing the road area 31 and the drivable area 41 shown in FIG. 6. In this embodiment, the graph structure is a data structure (or a figure) represented by nodes (points) and edges (branches) indicating the connection relationship between the nodes. The graph structure 50 shown in FIG. 7 includes an edge E1 corresponding to the drivable area 41 shown in FIG. 6, and two nodes N1 and N2 connecting the edge E1 to the road area 31 shown in FIG. 6. The graph structure 50 also includes nodes N3 and N4, edges E2, E3 and E4, etc. The conversion unit 13 may, for example, thin or skeletonize the area (image) obtained by integrating the road area and the drivable area, and then convert it into a graph structure.

[0016] The adding unit 14 adds a start node corresponding to the start point of the path to be searched and an end node corresponding to the end point of the path to the graph structure generated by the converting unit 13. FIG. 8 shows an example of a graph structure 50a in the case where a start node NS and an end node NE are added to the graph structure 50 shown in FIG. 7. Compared to the graph structure 50, the graph structure 50a shown in FIG. 8 includes a start node NS and a node NSJ for connecting the start node NS to the edge E3 shown in FIG. 7 via an edge ESJ. Also, the edge E3 shown in FIG. 7 is divided into edges E31 and E32 by the node NSJ in the graph structure 50a. Also, compared to the graph structure 50, the graph structure 50a includes a node NEJ for connecting the end node NE to an existing edge via an edge EEJ. Also, the edge connected to the node NEJ is divided into two edges by the node NSJ. In the example shown in FIG. 8, the start node and the end node are connected to existing edges, but the start node and the end node may be added to the graph structure before addition by connecting them to existing nodes.

[0017] The cost information associating unit 15 associates cost information based on geographic information with the graph structure generated by the adding unit 14. The geographic information is information associated with map information including the bird's-eye view image D1, and includes any information. The geographic information is, for example, altitude, elevation difference, importance of the topography (plains, valleys, mountains, etc.), the probability of the existence of obstacles, a visual field range that can be calculated from altitude data, and the like. The cost information can be used as information that makes it easier to generate a route during route search, such as selecting plains that are easy to travel through, avoiding routes that have obstacles and are long-distance, and ensuring a long-distance visual field. The association of the cost information with the graph structure can be performed, for example, by treating the graph structure as a weighted graph and changing the weight of each edge according to the cost information. The cost information associating unit 15 may be omitted.

[0018] The route search unit 16 searches for an optimal route from a start point to an end point based on a graph structure to which a start point node and an end point node have been added. When cost information based on geographic information is associated with the graph structure, the route search unit 16 searches for an optimal route taking the cost information into consideration. The search method may be an existing method based on graph theory, and is not limited. FIG. 9 shows an example of an optimal route 61 searched for by the route search unit 16 based on the graph structure 50a shown in FIG. 8, with a thick line. The route search unit 16 may search for multiple optimal routes based on multiple conditions (e.g., shortest distance, shortest time, shortest fuel consumption, etc.). Alternatively, the route search unit 16 may search for multiple candidates that can be optimal routes under the same conditions.

[0019] The output unit 17 outputs route information D2 indicating the optimal route searched by the route search unit 16 to a predetermined output device or output medium. The output unit 17 can output the optimal route, for example, as a linear image to be superimposed on the bird's-eye view image D1, or as an image representing a collection of multiple discrete waypoints. A waypoint is information representing a midpoint or a start point or end point of a route, and includes, for example, information representing latitude and longitude. The generated route is an image, and the waypoint is a pixel size. By approximating this with a polygon, it is possible to reduce the waypoints to any point, and it has a mechanism that allows it to be adjusted to the number of waypoints required by the vehicle, for example. The interval between waypoints can be adjusted to the total number of pixels at ground resolution, any distance section, a thinned number of points by approximating a straight line, etc.

[0020] (Example of operation of the route generation device) An example of the operation of the route generation device 1 shown in FIG. 1 will be described with reference to FIG. 2. The process shown in FIG. 2 is started in response to, for example, an instruction from an operator. In the process shown in FIG. 2, first, the extraction unit 11 and the setting unit 12 input the bird's-eye image D1 (step S1). Next, the extraction unit 11 extracts a road area from the bird's-eye image D1 (step S2). Next, the setting unit 12 sets a drivable area for the bird's-eye image D1 (step S3). Next, the conversion unit 13 integrates the road area and the drivable area and converts them into a graph structure (step S4). Next, the addition unit 14 adds a start point and an end point to the graph structure (step S5). Next, the cost information association unit 15 associates cost information based on geographic information with the graph structure (step S6). Next, the route search unit 16 searches for an optimal route from the start point to the end point (step S7). Next, the output unit 17 outputs route information D2 including information representing the optimal route (step S8), and the process shown in FIG. 2 ends.

[0021] (Action and effect) The route generation device, route generation method, and program configured as above extract a road area from a bird's-eye image D1, set a drivable area for the bird's-eye image D1, integrate the extracted road area and the set drivable area to convert into a graph structure, add a start node corresponding to the start point of the route and an end node corresponding to the end point of the route to the graph structure, search for an optimal route from the start point to the end point based on the graph structure to which the start node and the end node have been added, and output the optimal route. According to this configuration, it is possible to search for an optimal route, etc., using the road area and the drivable area as search targets. In this case, for example, even if a necessary road area cannot be used, a route can be generated without using the road area by setting a drivable area separately from the road area. Therefore, it is possible to improve the robustness of route generation.

[0022] (Other embodiments) Although the embodiments of the present disclosure have been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like that do not depart from the gist of the present disclosure are also included.

[0023] In the above embodiment, the extraction unit 11 extracts the road area using a machine learning model trained to classify the road area from the bird's-eye image. However, the configuration of the extraction unit 11 is not limited to this, and for example, the extraction unit 11 may extract the road area and the drivable area using a machine learning model trained to extract the road area and the drivable area from the bird's-eye image. In this case, the machine learning model can learn machine learning using, for example, a bird's-eye image in which the road area to be extracted is specified, a bird's-eye image in which the drivable area to be extracted is specified, a bird's-eye image in which the road area to be extracted and the drivable area are specified, etc. as teacher data. In this case, a learning dataset may be constructed by reflecting the existing setting result by the setting unit 12.

[0024] Also, for example, as shown in Fig. 10, the extraction unit 11 may extract road areas from images divided into tiles M1 so that even high-resolution satellite images can be processed. In this way, extraction can be performed with reduced memory usage. In this case, it is also possible to convert multiple road areas into a graph structure. In this case, for example, load adjustment can be achieved so that the system can operate on computers with different performance.

[0025] Computer Configuration FIG. 11 is a schematic block diagram illustrating a computer configuration according to at least one embodiment. The computer 90 comprises a processor 91, a main memory 92, a storage 93, and an interface 94. The above-mentioned path generating device 1 is implemented in a computer 90. The operations of the above-mentioned processing units are stored in the form of a program in a storage 93. The processor 91 reads the program from the storage 93, loads it in a main memory 92, and executes the above-mentioned processing in accordance with the program. The processor 91 also secures storage areas in the main memory 92 corresponding to the above-mentioned storage units in accordance with the program.

[0026] The program may be for realizing a part of the functions to be performed by the computer 90. For example, the program may be for realizing the functions by combining with other programs already stored in the storage or by combining with other programs implemented in other devices. In another embodiment, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions to be realized by the processor may be realized by the integrated circuit.

[0027] Examples of the storage 93 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), and a semiconductor memory. The storage 93 may be an internal medium directly connected to the bus of the computer 90, or an external medium connected to the computer 90 via an interface 94 or a communication line. In addition, when the program is distributed to the computer 90 via a communication line, the computer 90 that receives the program may load the program into the main memory 92 and execute the above-mentioned process. In at least one embodiment, the storage 93 is a non-transitory tangible storage medium.

[0028] <Additional Notes> The path generating device 1 described in each embodiment can be understood, for example, as follows.

[0029] (1) A route generation device according to a first aspect includes an extraction unit that extracts a road area from a bird's-eye view image, a setting unit that sets a drivable area for the bird's-eye view image, a conversion unit that integrates the extracted road area and the set drivable area to convert them into a graph structure, an addition unit that adds a start node corresponding to a start point of a route and an end node corresponding to an end point of the route to the graph structure, a route search unit that searches for an optimal route from the start point to the end point based on the graph structure to which the start node and the end node have been added, and an output unit that outputs the optimal route. According to this aspect and each of the following aspects, it is possible to improve the robustness of route generation.

[0030] (2) A route generation device according to a second aspect is the route generation device of (1), wherein the output unit outputs the optimal route as a set of a plurality of discrete waypoints.

[0031] (3) A route generation device according to a third aspect is a route generation device according to (1) or (2), further comprising a cost information matching unit that matches cost information based on geographic information with the graph structure, and the route search unit searches for the optimal route taking into account the cost information.

[0032] (4) A route generation device according to a fourth aspect is the route generation device of (1) to (3), wherein the setting unit sets the drivable area on the bird's-eye view image with the road area superimposed thereon. [Explanation of symbols]

[0033] 1...Path generation device 11...Extraction part 12. Setting section 13...Conversion section 14…Additional section 15…Cost information correspondence section 16…Route search section 17...Output section

Claims

1. an extraction unit that extracts a road area from a bird's-eye view image; A setting unit that sets a travelable area for the bird's-eye view image; a conversion unit that integrates the extracted road area and the set drivable area to convert them into a graph structure; an adding unit that adds a start node corresponding to a start point of a route and an end node corresponding to an end point of the route to the graph structure; a route search unit that searches for an optimal route from the start point to the end point based on the graph structure to which the start point node and the end point node have been added; an output unit for outputting the optimal route; A route generating device comprising:

2. The output unit outputs the optimal route as a set of a plurality of discrete waypoints. The route generating device according to claim 1 .

3. a cost information associating unit that associates cost information based on geographic information with the graph structure; The route search unit searches for the optimal route taking into account the cost information. The route generating device according to claim 2 .

4. The setting unit sets the drivable area on the bird's-eye view image with the road area superimposed thereon. The route generating device according to claim 3 .

5. Extracting a road area from a bird's-eye view image; Setting a drivable area for the bird's-eye view image; A step of integrating the extracted road area and the set drivable area into a graph structure; adding a start node corresponding to a start point of a route and an end node corresponding to an end point of the route to the graph structure; searching for an optimal route from the start point to the end point based on the graph structure to which the start point node and the end point node have been added; outputting the optimal route; A route generation method comprising:

6. Extracting a road area from a bird's-eye view image; Setting a drivable area for the bird's-eye view image; A step of integrating the extracted road area and the set drivable area into a graph structure; adding a start node corresponding to a start point of a route and an end node corresponding to an end point of the route to the graph structure; searching for an optimal route from the start point to the end point based on the graph structure to which the start point node and the end point node have been added; outputting the optimal route; A program that causes a computer to execute the following.

Images