Information processing method and information processing device

Abstract

To accurately create a road network from images using rule-based processing.SOLUTION: An information processing method for generating a road network includes steps of: first selecting of a position corresponding to a road as a first point; estimating road information at the first point; second selecting of a second point a predetermined distance from the first point; if it is determined that a link connecting the first point and the second point does not intersect with other road links already stored; storing the link between the first point and the second point as a road link; and, if it is determined that the link connecting the first point and the second point does not intersect with other road links already stored, setting the second point as a new first point and executing the processing from the estimation onwards again.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an information processing method and an information processing device. [Background technology]

[0002] Patent Document 1 discloses detecting lanes using Voronoi division and creating a map representing the lanes. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-201649 A Summary of the Invention [Problem to be solved by the invention]

[0004] One aspect of the present disclosure aims to provide a technology that can accurately create a road network from an image based on rule-based processing. [Means for solving the problem]

[0005] One aspect of the present disclosure is an information processing method for generating a road network, executed by an information processing device, comprising: a first selection step of selecting a position corresponding to a road from a mask image indicating whether each pixel corresponds to a road as a first point; an estimation step of estimating road information at the first point; a second selection step of selecting a second point a predetermined distance away from the first point; and a step of storing the link between the first point and the second point as a road link when it is determined that the link connecting the first point and the second point does not intersect with other road links already stored, wherein when it is determined that the link connecting the first point and the second point does not intersect with other road links already stored, the second point is set as a new first point and the processing from the estimation step onwards is executed again. Effect of the Invention

[0006] According to an aspect of the present disclosure, a road network can be created from an image with high accuracy based on rule-based processing. [Brief description of the drawings]

[0007] [Figure 1] 1A is a diagram illustrating the configuration of a road graph creation device, (B) an overview of a road network creation process, (C) a segmentation mask (mask image), and (D) a road network. [Diagram 2] 4 is a flowchart showing the flow of a road network creation process. [Diagram 3] FIG. 1A is a diagram illustrating a road information acquisition process; FIG. 1B and FIG. 1C are diagrams illustrating road areas. [Figure 4] 11 is a flowchart showing the flow of an intersection area extraction process. [Diagram 5] 5A to 5C are diagrams for explaining intersection area extraction processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] There is a demand for creating road networks that represent the connections between road links. Research and development is being conducted on machine learning technology to create road networks from photographed road images, but machine learning technology is a probabilistic process and does not necessarily output accurate results. In addition, it takes a lot of time to prepare learning data for machine learning, and it is difficult to obtain the required accuracy. There are also issues with the algorithm being difficult to tune.

[0009] Therefore, the present disclosure provides a technique capable of generating a road network by rule-based processing. Note that the above-mentioned problem is particularly problematic in the process of generating a road network from a segmentation mask (mask image) described later, so one aspect of the present disclosure relates to the process of generating a road network from a segmentation mask.

[0010] (Overall composition) 1(A) is a diagram showing the configuration of a road network creation device (information processing device) 100 in one embodiment. The road network creation device 100 includes a processor 110 and a memory 120. The memory 120 non-temporarily stores computer programs for causing the processor 110 to function as an image input unit 111, a road extraction unit 112, an intersection extraction unit 113, and a road network creation unit 114.

[0011] Fig. 1(B) is a diagram for explaining the process of creating a road network from road images such as satellite images and aerial images. Fig. 1(C) is a diagram for explaining a segmentation mask 132. Fig. 1(D) is a diagram for explaining a road network 133. An overview of each functional unit of the road network creation device 100 will be explained with reference to Figs. 1(B) to 1(D).

[0012] The image input unit 111 acquires a road image 131 and stores it in the memory 120. The road image 131 is an image of a road captured by a satellite image, an aerial image, etc. The road image 131 may be an image that has been subjected to pre-processing such as orthogonal transformation, linking of multiple images, and brightness adjustment.

[0013] The road extraction unit 112 performs a road extraction process on the road image 131 to create a segmentation mask (mask image) 312. The road extraction unit 112 judges whether each pixel of the road image 131 corresponds to a road. This judgment process may be performed by a machine learning-based process. Specifically, the road extraction unit 112 has a learning model that has been machine-learned using learning data in which each pixel is labeled as to whether it corresponds to a road or not, and judges whether each pixel of the input road image 131 corresponds to a road or not using this learning model. In another example, the road extraction unit 112 may judge whether a target pixel corresponds to a road or not by a rule-based process based on the color and brightness of the target pixel and its surrounding pixels. The segmentation mask (mask image) 132 is an image that indicates whether each pixel corresponds to a road or not. FIG. 1(C) shows an example of a segmentation image of a binary image in which pixels that correspond to a road are white (pixel value "1") and pixels that do not correspond to a road are black (pixel value "0").

[0014] The intersection extraction unit 113 extracts intersection regions from the segmentation mask 132. The intersection extraction unit 113 will be described in detail later with reference to FIGS.

[0015] The road network creation unit 114 creates a road network 133 from the segmentation mask 132. The road network 133 is data with a graph structure in which links representing roads are connected. The road network creation unit 114 will be described in detail with reference to Figs. 2 and 3.

[0016] (Road network creation process) FIG. 2 is a flowchart showing the overall flow of the road network creation process.

[0017] In step S201, the image input unit 111 acquires a road image. In step S202, the road extraction unit 112 determines whether or not each pixel in the road image corresponds to a road, and creates a segmentation mask (mask image).

[0018] In step S203, the road network creation unit 114 randomly selects an arbitrary pixel from the group of pixels corresponding to the road in the segmentation mask. Note that in this disclosure, "pixel," "position," and "point" have interchangeable meanings. The point selected in step S203 is referred to as point P1. Point P1 corresponds to the "first point" in this disclosure.

[0019] In step S204, the road network creation unit 114 determines whether the point P1 is an unprocessed point. If the point P1 is an unprocessed point, the process proceeds to step S205, and if the point P1 has been processed, the process proceeds to step S219. Note that, if the intersection area has been extracted in advance by the intersection extraction unit 113, the points in the intersection area may be determined to have been processed.

[0020] In order to store whether each pixel has been processed or not, the road network creation unit 114 may use an image (called an auxiliary image) having the same shape and the same number of pixels as the segmentation mask 132. All pixels of the auxiliary image are initialized to a value (e.g., "0") indicating that the pixel is unprocessed, and other values ​​are set to the processed pixels. The value set to the processed pixels may be a value according to the road ID, or, if the pixel is an intersection, a value indicating that the pixel is an intersection. In addition, if the point selected in step S203 is unprocessed, the road network creation unit 114 sets the value of the corresponding pixel in the auxiliary image to a value indicating that the pixel has been processed after step S204. The value according to the road ID or intersection is set to the pixel after it is determined that the point corresponds to a road or an intersection.

[0021] In step S205, the road network creation unit 114 estimates road information at the point P1. In this embodiment, the road information includes the width of the road, the direction of the road, and the center position of the road (center line position). In step S205, the road network creation unit 114 stores the estimated road information in the memory 120.

[0022] FIG. 3(A) is a diagram for explaining the process of estimating road information. In FIG. 3(A), 300 represents a road, and 301 represents a point P1. The road network creation unit 114 obtains the distance to a point other than the road for each direction with the point P1 as the center. Specifically, the road network creation unit 114 obtains the distance La from the point P1 (301) to a position other than the road for the direction 302a, and the distance Lb from the point P1 (301) to a position other than the road for the direction 302b opposite to the direction 302a, and obtains the sum of these distances. Note that the position other than the road includes pixels determined not to be a road and the edge of the mask image. The road network creation unit 114 obtains the sum of the above-mentioned distances for a plurality of directions (for example, all directions at 5-degree intervals) with the point P1 (301) as the center.

[0023] The road network creation unit 114 determines the direction in which the sum of the above-mentioned distances is the largest as the road direction of the road 300, determines the sum of the distances in the directions perpendicular to the road direction as the road width, and determines the center position of the road width as the road center (center line position). In the example of FIG. 3(A), directions 302a and 302b are determined as the road directions. Then, the sum of widths Wa and Wb in directions 303a and 303b perpendicular to the directions 302a and 302b is determined as the width of the road 300. Also, the center position 304 of the road widths Wa and Wb is determined as the center position of the road 300.

[0024] In step S207, the road network creation unit 114 selects a position that is a predetermined distance X from point P1 along the road direction determined in step S206 as point P2. Point P2 corresponds to the "second point" in this disclosure. The predetermined distance X may be any value according to the required accuracy of the system, for example, 1 m, 2 m, 5 m, 10 m, or 20 m (or the number of pixels equivalent thereto).

[0025] In step S208, the road network creation unit 114 determines whether the point P2 is a road. If point P2 is not a road, the road network creation unit 114 increases the predetermined distance X in step S215, and if the increased X does not exceed the maximum value (threshold value) (S216-NO), the process returns to step S207 and reselects point P2. If the increased X exceeds the maximum value (S216-YES), the process proceeds to step S219 to determine whether to continue the process, and if so, returns to step S203 and selects a new point P1. The increment in step S215 may be a fixed value (e.g., 50 cm, 1 m, etc.), a value corresponding to the current X (e.g., 10%, 20% of X, etc.), or a value corresponding to the number of repetitions (e.g., 2 for the nth time). n-2 In this way, by increasing the predetermined distance and reselecting point P2 until point P2 is determined to be a road or the predetermined distance X exceeds the threshold value, it is possible to quickly recover the process when point P2 is erroneously determined to be not a road or point P1 is erroneously determined to be a road in step S202.

[0026] In step S209, the road network creation unit 114 determines whether the link connecting points P1 and P2 intersects with other road links that have already been found and stored. This determination can be made by checking whether any pixel on the line connecting points P1 and P2 is assigned a road ID in the auxiliary image. If the link does not intersect with other roads, the process proceeds to step S210, and if it does intersect, the process proceeds to step S213. Note that in the process of step S209, even if the link connecting points P1 and P2 overlaps with the intersection area extracted by the intersection extraction unit 113, the determination may be regarded as positive and the process may proceed to step S213.

[0027] In step S210, the road network creation unit 114 connects the points P1 and P2 and stores them as a road link. That is, the points P1 and P2 are set as nodes, and new road links connecting both ends of these nodes are generated and stored. In another embodiment, in the loop processing from step S205 to S212, from the second time onwards, the point P2 may be connected to the end node (current point P1) of the processed road link, or the point P2 may be connected to the start node (initial point P1) of the processed road link.

[0028] In step S211, the road network creation unit 114 stores the area of ​​the road link connecting the point P1 and the point P2 as processed. For example, the road network creation unit 114 processes the area connecting the point P1 and the point P2 in the auxiliary image and having the width of the road width obtained in step S205. At this time, the pixels of the area in the auxiliary image are set with values ​​corresponding to the road ID. In addition to the above, the road link area may be determined as an area extending from the road center 304 at the point P1 (301) in the road direction by a predetermined distance X and having the width of the road width obtained in step S205, as shown in FIG. 3(B). Alternatively, as shown in FIG. 3(C), the road link area may be determined as an area extending from the road center 304 at the point P1 (301) in the road direction and in the opposite direction by half the predetermined distance X, respectively, and having the width of the road width obtained in step S205.

[0029] In step S212, the current point P2 is set as the new point P1, and the process returns to step S205 to execute the processes from step S205 (road information estimation process) onwards again.

[0030] Step S213 is a process that is executed when it is determined in step S209 that the link connecting points P1 and P2 intersects with another road. In step S213, the road network creation unit 114 determines whether or not point P2 is located within an intersection area. This determination can be made by checking whether or not point P2 is located within an intersection area extracted by the intersection extraction unit 113. In other embodiments, it may be determined whether or not point P2 is located within an intersection area depending on whether or not point P2 is located on a processed road link, without using the intersection extraction unit 113. If point P2 is not within the intersection area (S213-NO), the road network creation unit 114 decreases the predetermined distance X in step S217, and after the decrease, If X is not smaller than the minimum value (threshold value) (S218-NO), the process returns to step S207 and point P2 is selected again. If X after the decrease is smaller than the minimum value (S218-YES), the process proceeds to step S219 to determine whether to continue the process. If so, the process returns to step S203 and a new point P1 is selected. The decrement in step S217 may be a fixed value (e.g., 50 cm, 1 m, etc.), a value corresponding to the current X (e.g., 10%, 20% of X, etc.), or a value corresponding to the number of repetitions (e.g., 2 for the nth time). 1-n In this way, when the link between point P1 and point P2 intersects with another road but point P2 is not in the intersection area, point P2 can be set as a point in the intersection area by returning the position of point P2.

[0031] Step S214 is a process executed when it is determined in step S213 that point P2 is a point within the intersection area. In step S214, the road network creation unit 114 generates an intersection. Specifically, if an intersection node has not been generated in the intersection area, the road network creation unit 114 generates a new intersection node and connects point P1 to the intersection node. In addition, if there are road links already connected to the intersection node, the road network creation unit 114 generates new road links connecting point P1 to each road link. For example, if three road links are already connected to the intersection, the road network creation unit 114 generates three road links connecting point P1 to the end points of the above three road links.

[0032] The determination of whether to continue the process in step S219 may be determined appropriately. As an example, it may be considered that the process is completed after the selection process (S203) of point P1 is performed a predetermined number of times. This predetermined number of times may be a value (e.g., 10%) according to the number of pixels determined to be roads in the mask image.

[0033] (Intersection area extraction process) FIG. 4 is a diagram for explaining a process of extracting an intersection area from a segmentation mask (mask image). The intersection area extraction process is carried out, for example, by using a road network. This is performed prior to the road network creation process and is used in the road network creation process to determine whether or not the point being processed is included in an intersection area.

[0034] In step S400, the intersection extraction unit 113 acquires the segmentation mask (mask image) created by the road extraction unit 112.

[0035] The process from step S401 onwards can be roughly divided into two parts. One is a process consisting of steps S401 to S405 for determining whether or not a point (a processing target point) is an intersection point. The other is a step of extracting an intersection area based on the intersection point consisting of steps S407 to S408.

[0036] In step S401, the intersection extraction unit 113 selects a processing target point from points in the road area of ​​the segmentation mask (mask image) 132. Any selection method may be used, and for example, random selection may be adopted.

[0037] In step S402, the intersection extraction unit 113 obtains the distance from the processing target point to the road boundary in a plurality of directions. For example, the intersection extraction unit 113 obtains the distance from the processing target point to the road boundary in all directions of 360 degrees centered on the processing target point. For example, the distance to the road boundary is obtained every 5 degrees. The road boundary is a point that is determined to be a non-road in the segmentation mask and an edge of the segmentation mask.

[0038] In step S403, the intersection extraction unit 113 calculates the distance calculated in step S402 as Then, the number of peaks in a graph with the direction (angle) on the horizontal axis and the distance to the road boundary on the vertical axis is obtained. Figures 5(A) to 5(C) show several examples of such graphs, each showing two, three, and four peaks. Note that any existing peak detection algorithm can be used for the process of finding peaks in the graph. As an example, the find_peaks function included in the SciPy library of the Python language can be used.

[0039] In step S404, the intersection extraction unit 113 determines whether the number of peaks is greater than 2. If the determination is affirmative, the process proceeds to step S405, and if the determination is negative, the process proceeds to step S406.

[0040] In step S405, intersection extraction unit 113 stores the processing target point as an intersection point in memory 120. At this time, the number of peaks calculated in step S403 is also stored in association with the intersection point. Of Figures 5(A) to 5(C), Figure 5(A) is determined not to be an intersection point because the number of peaks is 2, and Figures 5(B) and 5(C) are determined to be intersection points because the number of peaks is 3 and 4, respectively.

[0041] In step S406, the intersection extraction unit 113 determines whether to process the next processing point. For example, the process may proceed to step S407 after repeating the processes from steps S401 to S405 a predetermined number of times. The predetermined number may be a value (e.g., 10%) according to the number of pixels determined to be roads in the segmentation mask (mask image).

[0042] In step S408, the intersection extraction unit 113 applies a clustering algorithm to the intersection points to generate a plurality of clusters. Here, an algorithm that can be executed without specifying the number of clusters to be generated, such as DBSCAN, can be suitably used.

[0043] In step S408, the intersection extraction unit 113 stores each area of ​​the obtained cluster as an intersection area in the memory 120. The intersection area is defined as follows: The smallest convex polygon that contains all the intersection points in a cluster (convex hull) ), or may be determined as the smallest rectangle (bounding box) that includes all intersections included in the cluster. Furthermore, the intersection extraction unit 113 stores the cluster centroid as the center of the intersection. Furthermore, the intersection extraction unit 113 determines the number of intersections or the intersection type of the intersection region based on the mode of the number of peaks of the intersection points included in the cluster. For example, an intersection region where the mode of the number of peaks is 3 is determined to be a three-way intersection or a T-type intersection, and an intersection region where the mode of the number of peaks is 4 is determined to be a four-way intersection or a cross type intersection. Furthermore, all intersections where the mode of the number of peaks is 5 or more may be determined to be complex intersections.

[0044] FIG. 5(D) is a diagram showing the results of the clustering process, and shows intersection points and the clustering results in a segmentation mask. The points shown as triangles are intersection points with three peaks, and the points shown as squares are intersection points with four peaks. In this example, two clusters 541 and 542 are obtained as a result of the clustering process. Cluster 541 contains the most intersection points with four peaks, so it is determined to be a cross-type intersection. Cluster 542 contains the most intersection points with three peaks, so it is determined to be a T-type intersection. FIG. 5(E) shows the intersection points contained in one cluster and a convex hull 550 that contains all of them. The inside of the convex hull 550 is stored as the intersection area.

[0045] (About the supplementary images) The auxiliary image is an image used as an auxiliary in the road network creation process, and has the same shape as the segmentation mask 132. The auxiliary image is also called a companion image. The initial values ​​of all pixels of the auxiliary image are values ​​representing unprocessed, and the values ​​of pixels selected for processing are updated to values ​​representing processed. In addition, a road ID or a value corresponding to the road ID is set to pixels determined to be road links in the road network creation process.

[0046] By using such an auxiliary image, it is possible to easily determine whether each point (position, pixel) in the segmentation mask has already been processed or not. Also, since the pixel values ​​of the auxiliary image are set according to the road ID, it is possible to easily determine which area the road link having a specific road ID belongs to and which road link a specific point (position, pixel) belongs to.

[0047] (Effects of the embodiment) According to the above embodiment, intersection areas can be extracted and road networks can be created by rule-based processing. Because it is rule-based processing, the processing can be executed relatively quickly, and there is no need for prior learning processing or collection of learning data. In addition, because it is rule-based processing, it is easy to set according to the required accuracy, that is, it is possible to extract intersection areas and create road networks with high accuracy that meet the required quality.

[0048] (Other embodiments) The above-described embodiment is merely an example, and the present disclosure can be modified and implemented as appropriate without departing from the spirit and scope of the present disclosure.

[0049] The present disclosure can also be realized by supplying a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors of the computer read and execute the program. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (floppy disk, hard disk drive (HDD), etc.), an optical disk (CD-ROM, DVD disk, Blu-ray disk, etc.), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0050] 100: Road network creation device (information processing device) 110: Processor 111: Image input unit 112: Road extraction unit 113: Intersection extraction unit 114: Road network creation unit

Claims

1. An information processing method for generating a road network, the method being executed by an information processing device, comprising: a first selection step of selecting, as a first point, a position corresponding to a road from a mask image which indicates whether each pixel corresponds to a road; an estimation step of estimating road information at the first point; a second selection step of selecting a second point that is a predetermined distance away from the first point; storing the link between the first point and the second point as a road link when it is determined that the link connecting the first point and the second point does not intersect with other road links already stored; Including, when it is determined that the link connecting the first point and the second point does not intersect with another road link already stored, the second point is set as a new first point and the processing from the estimation step onwards is executed again.

23. An information processing method comprising:

2. The estimation step includes: calculating, for a plurality of directions, a sum of a distance from the first point to a position other than a road in the direction and a distance from the first point to a position other than a road in a direction opposite to the direction; determining a direction among the plurality of directions in which the sum of the distances is the largest as a road direction, determining a sum of the distances in directions perpendicular to the road direction as a road width, and determining a center position of the road width as a road center; Including, In the second selection step, a position that is advanced from the first point by the predetermined distance in the road direction is selected as the second point, if it is determined that the link connecting the first point and the second point does not intersect with another road link already stored, storing as a road an area having a width of the road width extending from the road center at the first point in the road direction by the predetermined distance; 2. The information processing method according to claim 1,

3. The second selection step includes: if a position a predetermined distance from the first point is not a road, the predetermined distance is increased and the second point is reselected; repeating the reselection until the second point is determined to be a road or the predetermined distance exceeds a threshold; if the predetermined distance exceeds a threshold, terminating the process or returning to the first selection step to select a new first point; 2. The information processing method according to claim 1, further comprising:

4. If it is determined that the link connecting the first point and the second point intersects with another road link that has already been stored, if the second point is not within the intersection area, returning to the second selection step, shortening the predetermined distance and selecting a new second point; if the second point is within an intersection region, connecting the first point to each of the road links already connected to an intersection node within the intersection region; 2. The information processing method according to claim 1,

5. An information processing device including a processor and a storage device, The storage device stores a program that, when executed by the processor, causes the processor to execute each step of the information processing method according to any one of claims 1 to 4.

23. An information processing apparatus comprising:

Images