Information processing method and information processing device

Abstract

To accurately create a road network from images using rule-based processing.SOLUTION: An information processing method for generating a road network includes steps of: first selecting of a position corresponding to a road as a first point; estimating road information at the first point; second selecting of a second point a predetermined distance from the first point; if it is determined that a link connecting the first point and the second point does not intersect with other road links already stored; storing the link between the first point and the second point as a road link; and, if it is determined that the link connecting the first point and the second point does not intersect with other road links already stored, setting the second point as a new first point and executing the processing from the estimation onwards again.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information processing method and an information processing apparatus.

Background Art

[0002] Patent Document 1 discloses detecting lanes using Voronoi division and creating a map representing the lanes.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One aspect of the present disclosure aims to provide a technique capable of accurately creating a road network from an image based on rule-based processing.

Means for Solving the Problems

[0005] One aspect of the present disclosure is an information processing method for generating a road network, which is executed by an information processing apparatus, and includes a first selection step of selecting, as a first point, a position corresponding to a road from a mask image representing whether each pixel corresponds to a road; an estimation step of estimating road information at the first point; a second selection step of selecting a second point advanced a predetermined distance from the first point; and a step of storing, as a road link, the section between the first point and the second point when it is determined that the link connecting the first point and the second point does not intersect with other road links already stored. When it is determined that the link connecting the first point and the second point does not intersect with other road links already stored, the second point is set as a new first point, and the processing after the estimation step is executed again. [Effects of the Invention]

[0006] According to the aspects of this disclosure, it is possible to accurately create a road network from images based on rule-based processing. [Brief explanation of the drawing]

[0007] [Figure 1] A diagram illustrating (A) the configuration of the road graph generation device, (B) an overview of the road network generation process, (C) a segmentation mask (mask image), and (D) the road network. [Figure 2] A flowchart illustrating the process of creating a road network. [Figure 3] (A) Diagram illustrating the road information acquisition process, (B)(C) Diagram illustrating the road area. [Figure 4] A flowchart illustrating the process of extracting intersection areas. [Figure 5] A diagram illustrating the intersection region extraction process. [Modes for carrying out the invention]

[0008] There is a need to create a road network that represents the connections between road links. Research and development of machine learning techniques to create road networks from images of roads is underway, but machine learning techniques are probabilistic processes and do not always produce accurate results. In addition, preparing training data for machine learning is time-consuming, and achieving the required accuracy is difficult. There are also problems such as the difficulty in adjusting the algorithm.

[0009] Therefore, this disclosure provides a technology that enables the generation of road networks through rule-based processing. The aforementioned issues are particularly problematic in the process of creating road networks from segmentation masks (mask images), as described later; therefore, one aspect of this disclosure relates to the process of creating road networks from segmentation masks.

[0010] (Overall structure) Figure 1(A) shows the configuration of a road network creation device (information processing device) 100 in one embodiment. The road network creation device 100 includes a processor 110 and a memory 120. The memory 120 non-temporarily stores computer programs that cause the processor 110 to function as an image input unit 111, a road extraction unit 112, an intersection extraction unit 113, and a road network creation unit 114.

[0011] Figure 1(B) illustrates the process of creating a road network from road images such as satellite images and aerial images. Figure 1(C) illustrates the segmentation mask 132. Figure 1(D) illustrates the road network 133. The general outline of each functional part of the road network creation device 100 will be explained with reference to Figures 1(B) to 1(D).

[0012] The image input unit 111 acquires a road image 131 and stores it in the memory 120. The road image 131 is an image of a road, such as a satellite image or an aerial image. The road image 131 may be an image that has undergone preprocessing such as orthorectification, concatenation of multiple images, or brightness adjustment.

[0013] The road extraction unit 112 performs road extraction processing on the road image 131 and creates a segmentation mask (mask image) 312. The road extraction unit 112 determines whether each pixel of the road image 131 corresponds to a road. This determination process may be performed by a machine learning-based process. Specifically, the road extraction unit 112 has a learning model that is machine-learned using learning data in which each pixel is labeled as to whether it corresponds to a road, and uses this learning model to determine whether each pixel of the input road image 131 corresponds to a road. In another example, the road extraction unit 112 may determine whether a target pixel corresponds to a road by a rule-based process based on the color and luminance of the target pixel and the surrounding pixels. The segmentation mask (mask image) 132 is an image representing whether each pixel corresponds to a road. FIG. 1(C) shows an example of a segmentation image of a binary image having white (pixel value "1") for pixels corresponding to a road and black (pixel value "0") for pixels not corresponding to a road.

[0014] The intersection extraction unit 113 extracts the intersection area from the segmentation mask 132. Details of the intersection extraction unit 113 will be described in detail later by referring to FIGS. 4 and 5.

[0015] The road network creation unit 114 creates a road network 133 from the segmentation mask 132. The road network 133 is data having a graph structure to which links representing roads are connected. Details of the road network creation unit 114 will be described in detail by referring to FIGS. 2 and 3.

[0016] (Road Network Creation Process) FIG. 2 is a flowchart showing the overall flow of the road network creation process.

[0017] In step S201, the image input unit 111 acquires a road image. In step S202, the road extraction unit 112 determines whether each pixel of the road image corresponds to a road and creates a segmentation mask (mask image).

[0018] In step S203, the road network creation unit 114 randomly selects an arbitrary pixel from the pixel group corresponding to the road in the segmentation mask. In the present disclosure, "pixel", "position", and "point" have interchangeable meanings. The point selected in step S203 is referred to as point P1. Point P1 corresponds to the "first point" in the present disclosure.

[0019] In step S204, the road network creation unit 114 determines whether point P1 is an unprocessed point. If point P1 is unprocessed, the process proceeds to step S205; if it is processed, the process proceeds to step S219. If the intersection extraction unit 113 has previously extracted an intersection area, points within the intersection area may be determined to be processed.

[0020] In order to store whether each pixel has been processed, the road network creation unit 114 may use an image (referred to as an auxiliary image) having the same shape and the same number of pixels as the segmentation mask 132. The auxiliary image is initialized with a value (e.g., "0") representing that all pixels are unprocessed, and other values are set for processed pixels. The value set for a processed pixel may be a value corresponding to the road ID, or may be a value indicating that the pixel is an intersection if the pixel is an intersection. If the point selected in step S203 is unprocessed, the road network creation unit 114 sets the value of the corresponding pixel in the auxiliary image to a value representing processed after step S204. Setting the value corresponding to the road ID or intersection for the pixel is after it is determined that this point corresponds to a road or an intersection.

[0021] In step S205, the road network creation unit 114 estimates the road information at point P1. In this aspect, as the road information, the width of the road, the direction of the road, and the center position (center line position) of the road are estimated. In step S205, the road network creation unit 114 stores the estimated road information in the memory 120.

[0022] Figure 3(A) is a diagram illustrating the process of estimating road information. In Figure 3(A), 300 represents a road and 301 represents point P1. The road network creation unit 114 calculates the distance to points other than roads in each direction, centered on point P1. Specifically, the road network creation unit 114 calculates the distance La from point P1(301) to a location other than a road in direction 302a, and the distance Lb from point P1(301) to a location other than a road in the opposite direction 302b, and calculates the sum of these distances. Note that locations other than roads include pixels determined not to be roads and the edges of the mask image. The road network creation unit 114 calculates the above sum of distances for multiple directions (for example, all directions at 5-degree intervals) centered on point P1(301).

[0023] The road network creation unit 114 determines the direction in which the sum of the above-mentioned distances is greatest as the road direction of road 300, determines the sum of the distances in the direction perpendicular to the road direction as the road width, and determines the center position of the road width as the road center (center line position). In the example in Figure 3(A), directions 302a and 302b are determined as the road directions. Then, the sum of the widths Wa and Wb of directions 303a and 303b perpendicular to directions 302a and 302b is determined as the width of road 300. In addition, the center position 304 of the road widths Wa and Wb is determined as the center position of road 300.

[0024] In step S207, the road network creation unit 114 selects point P2 as a position obtained by advancing a predetermined distance X from point P1 along the road direction determined in step S206. Point P2 corresponds to the "second point" in this disclosure. The predetermined distance X may be any value depending on the required accuracy of the system, for example, 1m, 2m, 5m, 10m, 20m (or an equivalent number of pixels).

[0025] In step S208, the road network creation unit 114 determines whether point P2 is a road. The system determines whether point P2 is on a road or not. If point P2 is not on a road, the road network creation unit 114 increases a predetermined distance X in step S215. If the increased X does not exceed the maximum value (threshold) (S216-NO), the system returns to process S207 to re-select point P2. If the increased X exceeds the maximum value (S216-YES), the system proceeds to step S219 to determine whether to continue processing. If it decides to continue, the system returns to step S203 to select a new point P1. The increment in step S215 may be a fixed value (e.g., 50cm, 1m, etc.), a value corresponding to the current X (e.g., 10%, 20%, etc. of X), or a value corresponding to the number of repetitions (e.g., 2 for the nth time). n-2 m) is also acceptable. By increasing the predetermined distance until point P2 is determined to be a road or the predetermined distance X exceeds a threshold and re-selecting point P2 in this way, the process can be quickly restored if point P2 is incorrectly determined not to be a road or if point P1 is incorrectly determined to be a road in step S202.

[0026] In step S209, the road network creation unit 114 determines whether the link connecting point P1 and point P2 intersects with other road links that have already been found and stored. This determination can be made by checking whether any pixel on the straight line connecting point P1 and point P2 has a road ID assigned to it in the auxiliary image. If it does not intersect with other roads, the process proceeds to step S210; if it does intersect, the process proceeds to step S213. In addition, in the process of step S209, if the link connecting point P1 and point P2 overlaps with an intersection region extracted by the intersection extraction unit 113, the process may be considered positive and proceed to step S213.

[0027] In step S210, the road network creation unit 114 connects point P1 and point P2 and stores them as a road link. That is, point P1 and point P2 are treated as nodes, and a new road link is created and stored with these nodes as both ends. In another embodiment, in the loop processing from step S205 to S212, from the second time onward, point P2 may be connected to the endpoint node of the processed road link (the current point P1), or point P2 may be connected to the starting node of the processed road link (the initial point P1).

[0028] In step S211, the road network creation unit 114 stores the area of ​​the road link connecting point P1 and point P2 as processed. For example, the road network creation unit 114 processes the area in the auxiliary image that connects point P1 and point P2 and has a width equal to the road width determined in step S205. At this time, the pixels of the area in the auxiliary image are set to values ​​corresponding to the road ID. In addition to the above, the area of ​​the road link may also be determined as an area that extends a predetermined distance X in the direction of the road from the road center 304 at point P1(301) and has a width equal to the road width determined in step S205, as shown in Figure 3(B). Alternatively, as shown in Figure 3(C), it may be determined as an area that extends half a predetermined distance X in both the direction of the road and the opposite direction from the road center 304 at point P1(301) and has a width equal to the road width determined in step S205.

[0029] In step S212, the current point P2 is set to a new point P1, and the process returns to step S205 and executes the processes from step S205 (road information estimation process) onward again.

[0030] Step S213 is a process executed when it is determined in step S209 that the link connecting point P1 and point P2 intersects with another road. In step S213, the road network creation unit 114 determines whether point P2 is located within the intersection area. This determination can be made by checking whether point P2 is located within the intersection area extracted by the intersection extraction unit 113. In other embodiments, without using the intersection extraction unit 113, it may be determined whether point P2 is located within the intersection area based on whether point P2 is located on a processed road link. If point P2 is not within the intersection area (S213-NO), the road network creation unit 114 reduces a predetermined distance X in step S217, and the reduced distance If X is not less than the minimum value (threshold) (S218-NO), the process returns to step S207 and point P2 is re-selected. If the reduced X is less than the minimum value (S218-YES), the process proceeds to step S219 to decide whether to continue the process, and if so, the process returns to step S203 to select a new point P1. The reduction in step S217 may be a fixed value (e.g., 50cm, 1m, etc.), a value corresponding to the current X (e.g., 10%, 20%, etc. of X), or a value corresponding to the number of repetitions (e.g., 2 for the nth repetition). 1-n m) is also acceptable. In this way, when the link between point P1 and point P2 intersects with another road, but point P2 is not in the intersection area, point P2 can be made to be a point in the intersection area by moving its position back.

[0031] Step S214 is executed when step S213 determines that point P2 is a point within the intersection area. In step S214, the road network creation unit 114 generates an intersection. Specifically, if an intersection node has not been generated in the intersection area, the road network creation unit 114 generates a new one and connects point P1 to the intersection node. Also, if there are road links already connected to the intersection node, the road network creation unit 114 generates multiple new road links that connect point P1 to each of those road links. For example, if three road links are already connected to the intersection, the road network creation unit 114 generates three road links that connect point P1 to the endpoints of the three road links.

[0032] The determination of whether or not to continue processing in step S219 can be determined as appropriate. For example, it is conceivable to determine that processing is complete after the point selection process (S203) has been executed a predetermined number of times. This predetermined number of times can be a value corresponding to the number of pixels identified as roads in the mask image (for example, 10%).

[0033] (Intersection area extraction process) Figure 4 illustrates the process of extracting intersection areas from a segmentation mask (mask image). Intersection area extraction is performed, for example, in a road network. This process is performed prior to the road creation process and is used in the road network creation process to determine whether or not the point being processed is included in the intersection area.

[0034] In step S400, the intersection extraction unit 113 acquires the segmentation mask (mask image) created by the road extraction unit 112.

[0035] The processing from step S401 onwards can be broadly divided into two parts. One is the process of determining whether a single point (the point to be processed) is an intersection point, consisting of steps S401 to S405. The other is the step of extracting the intersection region based on the intersection point, consisting of steps S407 to S408.

[0036] In step S401, the intersection extraction unit 113 selects points to be processed from the points in the road area of ​​the segmentation mask (mask image) 132. The selection method may be arbitrary; for example, random selection can be used.

[0037] In step S402, the intersection extraction unit 113 calculates the distance from the processing point to the road boundary in multiple directions. For example, the intersection extraction unit 113 calculates the distance from the processing point to the road boundary in all 360 degrees around the processing point. For example, the distance to the road boundary is calculated every 5 degrees. The road boundary is the point that is not a road in the segmentation mask and the edges of the segmentation mask.

[0038] In step S403, the intersection extraction unit 113 uses the distance obtained in step S402. The number of peaks is obtained in a graph where the horizontal axis represents direction (angle) and the vertical axis represents distance to the road boundary. Figures 5(A) to 5A(C) show several examples of such graphs, illustrating cases with 2, 3, and 4 peaks, respectively. Any existing peak detection algorithm can be used to find the peaks in the graph. As an example, the find_peaks function included in the SciPy library of the Python language can be used.

[0039] In step S404, the intersection extraction unit 113 determines whether the number of peaks is greater than 2. If the determination is positive, the process proceeds to step S405; otherwise, the process proceeds to step S406.

[0040] In step S405, the intersection extraction unit 113 stores the processing target point as an intersection point in the memory 120. At this time, the number of peaks obtained in step S403 is also stored in association with the intersection point. Of Figures 5(A) to 5(C), Figure 5(A) is determined not to be an intersection point because it has a peak count of 2, while Figures 5(B) and 5(C) are determined to be intersection points because they have a peak count of 3 and 4, respectively.

[0041] In step S406, the intersection extraction unit 113 determines whether or not to process the next target point. For example, the process can be repeated a predetermined number of times from steps S401 to S405 before proceeding to step S407. The predetermined number can be, for example, a value corresponding to the number of pixels identified as roads in the segmentation mask (mask image) (e.g., 10%).

[0042] In step S408, the intersection extraction unit 113 applies a clustering algorithm to the intersection points to generate multiple clusters. Here, an algorithm that can be executed without specifying the number of clusters to be generated, such as DBSCAN, is preferably used.

[0043] In step S408, the intersection extraction unit 113 stores each of the obtained cluster regions in the memory 120 as an intersection area. The intersection area is The smallest convex polygon (convex hull) that encompasses all intersection points within a cluster. It may be determined as a bounding box, or as the smallest rectangle (bounding box) that encompasses all intersections included in the cluster. The intersection extraction unit 113 also stores the cluster centroid as the center of the intersection. The intersection extraction unit 113 also determines the number of intersections or intersection type of the intersection region based on the mode of the number of peaks of the intersection points included in the cluster. For example, an intersection region with a mode of 3 peaks is determined to be a three-way or T-type intersection, and an intersection region with a mode of 4 peaks is determined to be a four-way or cross-type intersection. All intersections with a mode of 5 or more peaks may be determined to be complex intersections.

[0044] Figure 5(D) shows the results of the clustering process, representing the segmentation mask, intersection points, and clustering results. Points indicated by triangles are intersection points with 3 peaks, and points indicated by squares are intersection points with 4 peaks. In this example, the clustering process yields two clusters, 541 and 542. Cluster 541 is determined to be a cross-type intersection because it contains the most intersection points with 4 peaks. Cluster 542 is determined to be a T-type intersection because it contains the most intersection points with 3 peaks. Figure 5(E) shows the intersection points included in one cluster and the convex hull 550 that encompasses all of them. The interior of the convex hull 550 is stored as the intersection region.

[0045] (Regarding supplementary images) The auxiliary image is an image used supplementarily during the road network creation process and has the same shape as the segmentation mask 132. The auxiliary image is also called a companion image. The initial value of all pixels in the auxiliary image is a value indicating unprocessed, and the value of pixels selected for processing is updated to a value indicating processed. In addition, pixels determined to be road links during the road network creation process are assigned a road ID or a corresponding value.

[0046] By using such auxiliary images, it becomes easy to determine whether each point (location, pixel) within the segmentation mask has already been processed or not. Furthermore, since the pixel values ​​in the auxiliary images are set according to the road ID, it becomes easy to determine which area a road link with a specific road ID belongs to, and which road link a specific point (location, pixel) belongs to.

[0047] (Effects of the embodiment) According to the above embodiment, intersection areas can be extracted and road networks can be created using rule-based processing. Because it is rule-based processing, it can be executed relatively quickly, and prior training processing and collection of training data are unnecessary. Furthermore, because it is rule-based processing, it is easy to configure according to the required accuracy, that is, it is possible to extract intersection areas and create road networks with high accuracy that meet the required quality.

[0048] (Other embodiments) The above embodiments are merely examples, and this disclosure may be modified as appropriate without departing from its essence.

[0049] The present disclosure can also be realized by supplying a computer program implementing the functions described in the embodiments above to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network. Non-temporary computer-readable storage mediums include, for example, any type of disk such as magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions. [Explanation of Symbols]

[0050] 100: Road network creation device (information processing device) 110: Processor 111: Image input unit 112: Road extraction unit 113: Intersection extraction unit 114: Road network creation unit

Claims

1. An information processing method for generating a road network, which is performed by an information processing device, A first selection step involves selecting a location corresponding to a road as the first point from a mask image that indicates whether each pixel corresponds to a road, and An estimation step for estimating road information at the first point, A second selection step involves selecting a second point located a predetermined distance from the first point, If it is determined that the link connecting the first point and the second point does not intersect with any other road links already stored, the step of storing the section between the first point and the second point as a road link, Includes, If it is determined that the link connecting the first point and the second point does not intersect with any other road links already stored, the second point is treated as a new first point and the processing from the estimation step onward is repeated. An information processing method characterized by the following:

2. The estimation step described above is: For multiple directions, the steps include: calculating the sum of the distance from the first point in that direction to a location other than the road and the distance from the first point in the opposite direction to a location other than the road; The steps include determining the direction in which the sum of the distances is greatest among the aforementioned multiple directions as the road direction, determining the sum of the distances in the direction perpendicular to the road direction as the road width, and determining the center position of the road width as the road center, Includes, In the second selection step, the position obtained by advancing a predetermined distance from the first point in the direction of the road is selected as the second point. If it is determined that the link connecting the first point and the second point does not intersect with any other road links already stored, then the area having the width of the road and extending a predetermined distance in the direction of the road from the center of the road at the first point is stored as a road. The information processing method according to feature 1.

3. The second selection step described above is: If the location reached by a predetermined distance from the first point is not a road, the predetermined distance is increased and the second point is re-selected. The re-selection process is repeated until the second point is determined to be a road or the predetermined distance exceeds the threshold. If the predetermined distance exceeds a threshold, the process for generating the road network is terminated, or the process returns to the first selection step to select a new first point. The information processing method according to claim 1, which includes the following:

4. If it is determined that the link connecting the first point and the second point intersects with another road link that is already stored, If the second point is not within the intersection area, return to the second selection step, shorten the predetermined distance, and select a new second point. If the second point is within the intersection area, connect the first point to each of the road links already connected to the intersection node within the intersection area. The information processing method according to feature 1.

5. An information processing device comprising a processor and a memory device, The storage device stores a program that, when executed by the processor, causes the processor to perform each step of the information processing method described in any one of claims 1 to 4. An information processing device characterized by the following:

Images