Information processing method and information processing device

Abstract

To easily and accurately extract intersection areas.SOLUTION: An information processing method for extracting intersection areas includes: an image acquisition step of acquiring a mask image showing whether each pixel corresponds to a road; a determination step of selecting a processing object point from the mask image and determining whether the processing object point is an intersection point; and an intersection area extraction step of applying a clustering algorithm to multiple intersection points obtained by performing the determination step multiple times and determining that an obtained cluster area is an intersection area. The determination step includes: selecting the processing object point from the mask image; obtaining the distance from the processing object point to a road boundary in multiple directions; and determining that the processing object point is an intersection point if the number of distance peaks is greater than 2, and otherwise determining that the processing object point is not an intersection point.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method and an information processing device. [Background technology]

[0002] Patent Document 1 discloses that lanes are detected using Voronoi division and a map representing the lanes is created. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-201649 Summary of the Invention [Problem to be solved by the invention]

[0004] One aspect of the present disclosure aims to provide a technology capable of extracting an intersection area from a mask image that indicates whether each pixel corresponds to a road. [Means for solving the problem]

[0005] One aspect of the present disclosure is an information processing method for extracting intersection areas, executed by an information processing device, comprising: an image acquisition step of acquiring a mask image indicating whether each pixel corresponds to a road; a determination step of selecting a processing target point from the mask image and determining whether the processing target point is an intersection point; and an intersection area extraction step of applying a clustering algorithm to multiple intersection points obtained by performing the determination step multiple times and determining the resulting cluster area to be an intersection area, wherein the determination step comprises the steps of selecting the processing target point from the mask image, calculating the distance from the processing target point to a road boundary in multiple directions, and determining that the processing target point is an intersection point if the number of peaks in a graph with direction on the horizontal axis and distance to the road boundary on the vertical axis is greater than two, and otherwise determining that the processing target point is not an intersection point. [Effects of the Invention]

[0006] According to an aspect of the present disclosure, an intersection area can be extracted from a mask image that indicates whether each pixel corresponds to a road or not. [Brief explanation of the drawings]

[0007] [Figure 1] 1A is a diagram illustrating the configuration of a road graph creation device, (B) an overview of the road network creation process, (C) a segmentation mask (mask image), and (D) a road network. [Figure 2] 10 is a flowchart showing the flow of a road network creation process. [Figure 3] FIG. 1A is a diagram illustrating a road information acquisition process, and FIGS. 1B and 1C are diagrams illustrating road areas. [Figure 4] 10 is a flowchart showing the flow of intersection area extraction processing. [Figure 5] 5A to 5C are diagrams for explaining intersection area extraction processing. DETAILED DESCRIPTION OF THE INVENTION

[0008] There is a demand for the creation of road networks that express the connections between road links. If intersection areas can be recognized in advance when creating a road network, it will be possible to create a highly accurate road network. However, there are currently no methods that can extract intersection areas easily and with high accuracy. No existing technology is known.

[0009] Therefore, the objective of this study is to provide a new technology that can easily and accurately extract intersection areas from a segmentation mask (mask image).

[0010] (Overall composition) 1(A) is a diagram showing the configuration of a road network creation device (information processing device) 100 in one embodiment. The road network creation device 100 includes a processor 110 and a memory 120. The memory 120 non-temporarily stores computer programs for causing the processor 110 to function as an image input unit 111, a road extraction unit 112, an intersection extraction unit 113, and a road network creation unit 114.

[0011] Fig. 1(B) is a diagram illustrating the process of creating a road network from road images such as satellite images and aerial images. Fig. 1(C) is a diagram illustrating a segmentation mask 132. Fig. 1(D) is a diagram illustrating a road network 133. An overview of each functional unit of the road network creation device 100 will be described with reference to Figs. 1(B) to 1(D).

[0012] The image input unit 111 acquires a road image 131 and stores it in the memory 120. The road image 131 is an image of a road captured by a satellite image, an aerial image, etc. The road image 131 may be an image that has been subjected to preprocessing such as orthogonal transformation, linking of multiple images, and brightness adjustment.

[0013] The road extraction unit 112 performs road extraction processing on the road image 131 to create a segmentation mask (mask image) 312. The road extraction unit 112 determines whether each pixel in the road image 131 corresponds to a road. This determination processing may be performed by machine learning processing. Specifically, the road extraction unit 112 has a learning model that has been machine-learned using learning data in which each pixel is labeled as to whether it corresponds to a road, and determines whether each pixel in the input road image 131 corresponds to a road using this learning model. In another example, the road extraction unit 112 may determine whether a target pixel corresponds to a road using rule-based processing based on the color and brightness of the target pixel and its surrounding pixels. The segmentation mask (mask image) 132 is an image that indicates whether each pixel corresponds to a road. FIG. 1C shows an example of a segmentation image of a binary image in which pixels that correspond to a road are white (pixel value "1") and pixels that do not correspond to a road are black (pixel value "0").

[0014] The intersection extraction unit 113 extracts intersection regions from the segmentation mask 132. The intersection extraction unit 113 will be described in detail later with reference to FIGS.

[0015] The road network creation unit 114 creates a road network 133 from the segmentation mask 132. The road network 133 is data with a graph structure in which links representing roads are connected. Details of the road network creation unit 114 will be described in detail with reference to FIGS. 2 and 3.

[0016] (Road network creation process) FIG. 2 is a flowchart showing the overall flow of the road network creation process.

[0017] In step S201, the image input unit 111 acquires a road image. In step S202, the road extraction unit 112 determines whether each pixel in the road image corresponds to a road, and creates a segmentation mask (mask image).

[0018] In step S203, the road network creation unit 114 randomly selects an arbitrary pixel from the pixel group corresponding to the road in the segmentation mask. In this specification, the terms "pixel," "position," and "point" have interchangeable meanings. The point selected in step S203 is referred to as point P1. Point P1 corresponds to the "first point" in this disclosure.

[0019] In step S204, the road network creation unit 114 determines whether the point P1 is an unprocessed point. If the point P1 is an unprocessed point, the process proceeds to step S205, and if it has been processed, the process proceeds to step S219. Note that if the intersection area has been extracted in advance by the intersection extraction unit 113, it is sufficient to determine that the points within the intersection area have been processed.

[0020] In order to store whether each pixel has been processed, the road network creation unit 114 may use an image (referred to as an auxiliary image) that has the same shape and the same number of pixels as the segmentation mask 132. All pixels of the auxiliary image are initialized to a value indicating unprocessed (for example, "0"), and processed pixels are set to other values. The value set for processed pixels may be a value corresponding to the road ID, or, if the pixel is an intersection, may be a value indicating that the pixel is an intersection. In addition, if the point selected in step S203 is unprocessed, the road network creation unit 114 sets the value of the corresponding pixel in the auxiliary image to a value indicating that the point has been processed after step S204. The value corresponding to the road ID or intersection is set for the pixel after it has been determined that the point corresponds to a road or an intersection.

[0021] In step S205, the road network creation unit 114 estimates road information at point P1. In this embodiment, the road information estimated includes the width, direction, and center position of the road (center line position). In step S205, the road network creation unit 114 stores the estimated road information in the memory 120.

[0022] FIG. 3A is a diagram illustrating the process of estimating road information. In FIG. 3A, 300 represents a road, and 301 represents point P1. The road network creation unit 114 calculates the distance to a point other than a road in each direction centered on point P1. Specifically, the road network creation unit 114 calculates the distance La from point P1 (301) to a position other than a road in direction 302a, and the distance Lb from point P1 (301) to a position other than a road in direction 302b opposite to direction 302a, and calculates the sum of these distances. Note that the position other than a road includes pixels determined not to be a road and the edge of the mask image. The road network creation unit 114 calculates the sum of the above-mentioned distances in multiple directions (e.g., all directions at 5-degree intervals) centered on point P1 (301).

[0023] The road network creation unit 114 determines the direction with the largest sum of the above-mentioned distances as the road direction of the road 300, determines the sum of the distances in the directions perpendicular to the road direction as the road width, and determines the center position of the road width as the road center (center line position). In the example of FIG. 3(A), directions 302a and 302b are determined as the road directions. Then, the sum of widths Wa and Wb in directions 303a and 303b perpendicular to directions 302a and 302b is determined as the width of the road 300. Furthermore, center position 304 of the road widths Wa and Wb is determined as the center position of the road 300.

[0024] In step S207, the road network creation unit 114 selects a position that is a predetermined distance X from point P1 along the road direction determined in step S206 as point P2. Point P2 corresponds to the "second point" in this disclosure. The predetermined distance X may be any value depending on the required accuracy of the system, for example, 1 m, 2 m, 5 m, 10 m, or 20 m (or the number of pixels equivalent thereto).

[0025] In step S208, the road network creation unit 114 determines whether or not the point P2 is a road. If the point P2 is not a road, the road network creation unit 114 increases the predetermined distance X in step S215, and if the increased X does not exceed the maximum value (threshold value), If the result is NO in S216, the process returns to step S207 and point P2 is selected again. If the increased X exceeds the maximum value (YES in S216), the process proceeds to step S219, where it is determined whether to continue the process. If so, the process returns to step S203 and a new point P1 is selected. The increment in step S215 may be a fixed value (e.g., 50 cm, 1 m, etc.), a value according to the current X (e.g., 10%, 20% of X, etc.), or a value according to the number of repetitions (e.g., 2 for the nth time). n-2 In this way, by increasing the predetermined distance and reselecting point P2 until point P2 is determined to be a road or the predetermined distance X exceeds the threshold value, it is possible to quickly restore processing when point P2 is erroneously determined not to be a road or point P1 is erroneously determined to be a road in step S202.

[0026] In step S209, the road network creation unit 114 determines whether the link connecting points P1 and P2 intersects with other road links that have already been found and stored. This determination can be made by checking whether any pixel on the line connecting points P1 and P2 has been assigned a road ID in the auxiliary image. If the line does not intersect with other roads, the process proceeds to step S210; if the line does intersect with other roads, the process proceeds to step S213. Note that in the process of step S209, even if the link connecting points P1 and P2 overlaps with the intersection area extracted by the intersection extraction unit 113, the determination may be considered positive, and the process may proceed to step S213.

[0027] In step S210, the road network creation unit 114 connects point P1 and point P2 and stores the result as a road link. That is, point P1 and point P2 are set as nodes, and new road links connecting both ends of these nodes are generated and stored. Note that in another embodiment, in the loop processing from step S205 to S212, from the second time onwards, point P2 may be connected to the end node (current point P1) of the processed road link, or point P2 may be connected to the start node (initial point P1) of the processed road link.

[0028] In step S211, the road network creation unit 114 stores the area of ​​the road link connecting point P1 and point P2 as processed. For example, the road network creation unit 114 processes the area in the auxiliary image connecting point P1 and point P2 and having the width of the road width calculated in step S205. At this time, the pixels in the area in the auxiliary image are set to values ​​corresponding to the road ID. Note that the road link area may be determined as an area extending a predetermined distance X from the road center 304 at point P1 (301) in the road direction and having the width of the road width calculated in step S205, as shown in FIG. 3(B). Alternatively, as shown in FIG. 3(C), the road link area may be determined as an area extending half the predetermined distance X in both the road direction and the opposite direction from the road center 304 at point P1 (301), and having the width of the road width calculated in step S205.

[0029] In step S212, the current point P2 is set as the new point P1, and the process returns to step S205 to execute the processes from step S205 (road information estimation process) onwards again.

[0030] Step S213 is a process that is executed when it is determined in step S209 that the link connecting points P1 and P2 intersects with another road. In step S213, the road network creation unit 114 determines whether point P2 is located within an intersection area. This determination can be made by checking whether point P2 is located within the intersection area extracted by the intersection extraction unit 113. In another embodiment, it may be determined whether point P2 is located within an intersection area based on whether point P2 is located on a processed road link, without using the intersection extraction unit 113. If point P2 is not within the intersection area (S213-NO), the road network creation unit 114 decreases the predetermined distance X in step S217, and if X after the decrease is not smaller than the minimum value (threshold value) (S218-NO), the process returns to step S207 and reselects point P2. If X after the decrease is smaller than the minimum value (S218-YES), the process The process proceeds to step S219 to determine whether to continue the process, and if so, the process returns to step S203 to select a new point P1. The decrement in step S217 may be a fixed value (for example, 50 cm, 1 m, etc.), a value corresponding to the current X (for example, 10%, 20% of X, etc.), or a value corresponding to the number of repetitions (for example, 2 for the nth time). 1-n In this way, when the link between points P1 and P2 intersects with another road but point P2 is not in the intersection area, point P2 can be set as a point in the intersection area by returning the position of point P2.

[0031] Step S214 is a process that is executed when it is determined in step S213 that point P2 is a point within the intersection area. In step S214, the road network creation unit 114 generates an intersection. Specifically, if an intersection node has not been generated in the intersection area, the road network creation unit 114 generates a new intersection node and connects point P1 to the intersection node. Furthermore, if there are road links already connected to the intersection node, the road network creation unit 114 generates multiple new road links that connect point P1 to each road link. For example, if three road links are already connected to the intersection, the road network creation unit 114 generates three road links that connect point P1 to the endpoints of the three road links.

[0032] The determination of whether to continue the process in step S219 may be determined as appropriate. As an example, it may be considered that the process is completed when the selection process (S203) of point P1 is performed a predetermined number of times. This predetermined number of times may be set to a value (e.g., 10%) corresponding to the number of pixels determined to be roads in the mask image.

[0033] (Intersection area extraction processing) FIG. 4 is a diagram for explaining the process of extracting an intersection area from a segmentation mask (mask image). The intersection area extraction process is carried out, for example, by extracting an intersection area from a road network. This is performed prior to the road network creation process and is used to determine whether the point to be processed in the road network creation process is included in the intersection area.

[0034] In step S400, the intersection extraction unit 113 acquires the segmentation mask (mask image) created by the road extraction unit 112.

[0035] The processing from step S401 onwards can be broadly divided into two parts. One is the processing consisting of steps S401 to S405 to determine whether or not a point (processing target point) is an intersection point. The other is the step of extracting an intersection area based on the intersection point consisting of steps S407 to S408.

[0036] In step S401, the intersection extraction unit 113 selects a processing target point from points in the road area of ​​the segmentation mask (mask image) 132. Any selection method may be used, and for example, random selection may be adopted.

[0037] In step S402, the intersection extraction unit 113 calculates the distance from the processing target point to the road boundary in multiple directions. For example, the intersection extraction unit 113 calculates the distance from the processing target point to the road boundary in all directions of 360 degrees around the processing target point. For example, the distance to the road boundary is calculated every 5 degrees. The road boundary is defined as a point that is not a road in the segmentation mask and the edge of the segmentation mask.

[0038] In step S403, the intersection extraction unit 113 obtains the number of peaks in a graph of the distance calculated in step S402, with the direction (angle) on the horizontal axis and the distance to the road boundary on the vertical axis. Figures 5(A) to 5(C) show some examples of such graphs, each of which shows The graphs show cases with two, three, and four peaks. Any existing peak detection algorithm can be used to find peaks in the graph. For example, the find_peaks function included in the SciPy library of the Python language can be used.

[0039] In step S404, the intersection extraction unit 113 determines whether the number of peaks is greater than 2. If the determination is affirmative, the process proceeds to step S405, and if the determination is negative, the process proceeds to step S406.

[0040] In step S405, intersection extraction unit 113 stores the processing target point as an intersection point in memory 120. At this time, the number of peaks calculated in step S403 is also stored in association with the intersection point. Of Figures 5(A) to 5(C), Figure 5(A) is determined not to be an intersection point because the number of peaks is 2, while Figures 5(B) and 5(C) are determined to be intersection points because the number of peaks is 3 and 4, respectively.

[0041] In step S406, the intersection extraction unit 113 determines whether to process the next processing point. For example, the process may proceed to step S407 after repeating the processes from steps S401 to S405 a predetermined number of times. The predetermined number may be set to a value (e.g., 10%) corresponding to the number of pixels determined to be roads in the segmentation mask (mask image).

[0042] In step S408, the intersection extraction unit 113 applies a clustering algorithm to the intersection points to generate multiple clusters. Here, an algorithm that can be executed without specifying the number of clusters to be generated, such as DBSCAN, can be suitably used.

[0043] In step S408, the intersection extraction unit 113 stores each area of ​​the obtained cluster as an intersection area in the memory 120. An intersection area is defined as follows: The smallest convex polygon (convex hull) that contains all the intersection points in the cluster ), or may be determined as the smallest rectangle (bounding box) that includes all of the intersections included in the cluster. The intersection extraction unit 113 stores the cluster centroid as the center of the intersection. The intersection extraction unit 113 also determines the number of intersections or the intersection type of the intersection area based on the most frequent value of the number of peaks of the intersection points included in the cluster. For example, an intersection area where the most frequent value of the number of peaks is 3 is determined to be a three-way intersection or a T-type intersection, and an intersection area where the most frequent value of the number of peaks is 4 is determined to be a four-way intersection or a cross-type intersection. All intersections where the most frequent value of the number of peaks is 5 or more may be determined to be complex intersections.

[0044] FIG. 5(D) is a diagram showing the results of the clustering process, with intersection points and the clustering results shown on a segmentation mask. Points indicated by triangles are intersection points with three peaks, and points indicated by squares are intersection points with four peaks. In this example, two clusters 541 and 542 are obtained as a result of the clustering process. Cluster 541 contains the most intersection points with four peaks, and is therefore determined to be a cross-type intersection. Cluster 542 contains the most intersection points with three peaks, and is therefore determined to be a T-type intersection. FIG. 5(E) shows the intersection points contained in one cluster and the convex hull 550 that encompasses them all. The interior of convex hull 550 is stored as the intersection region.

[0045] (About the supplementary images) The auxiliary image is an image used as an auxiliary in the road network creation process, and has the same shape as the segmentation mask 132. The auxiliary image is also called a companion image. The initial values ​​of all pixels in the auxiliary image are values ​​that indicate unprocessed, and the values ​​of pixels selected for processing are updated to values ​​that indicate processed. In addition, pixels that are determined to be road links in the road network creation process are set with a road ID or a value corresponding to the road ID.

[0046] By using such an auxiliary image, it is possible to easily determine whether each point (position, pixel) in the segmentation mask has already been processed or not. In addition, since the pixel values ​​of the auxiliary image are set according to the road ID, it is possible to easily determine which area a road link having a specific road ID belongs to and which road link a specific point (position, pixel) belongs to.

[0047] (Effects of the embodiment) According to the above embodiment, intersection areas can be extracted and road networks can be created using rule-based processing. Because it is rule-based processing, the processing can be performed relatively quickly and there is no need for prior learning processing or collection of learning data. Furthermore, because it is rule-based processing, it is easy to set up according to the required accuracy, which means that it is possible to extract intersection areas and create road networks with high accuracy that meet the required quality.

[0048] (Other embodiments) The above-described embodiment is merely an example, and the present disclosure can be modified and implemented as appropriate within the scope that does not deviate from the gist of the disclosure.

[0049] The present disclosure can also be realized by providing a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer via a non-transitory computer-readable storage medium connectable to the computer's system bus or via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (e.g., a floppy disk, a hard disk drive (HDD), etc.), an optical disk (e.g., a CD-ROM, a DVD disk, a Blu-ray disk), a read-only memory (ROM), a random-access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, or any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0050] 100: Road network creation device (information processing device) 110: Processor 111: Image input unit 112: Road extraction unit 113: Intersection extraction unit 114: Road network creation unit

Claims

1. An information processing method for extracting an intersection area, which is executed by an information processing device, comprising: an image acquisition step of acquiring a mask image that indicates whether each pixel corresponds to a road; a determination step of selecting a processing target point from the mask image and determining whether the processing target point is an intersection point; an intersection area extraction step of applying a clustering algorithm to a plurality of intersection points obtained by performing the determination step a plurality of times, and determining that the resulting cluster area is an intersection area; Including, The determining step selecting the target points from the mask image; calculating distances from the processing point to a road boundary in a plurality of directions; a step of determining that the processing target point is an intersection point if the number of peaks in a graph with the direction on the horizontal axis and the distance to the road boundary on the vertical axis is greater than two, and otherwise determining that the processing target point is not an intersection point; Including, 1. An information processing method comprising:

2. the determining step includes storing the number of peaks at the intersection point; the intersection area extraction step includes a step of determining a mode value of the number of peaks of intersection points included in the intersection area as the number of intersections in the intersection area, 2. The information processing method according to claim 1,

3. 2. The information processing method according to claim 1, wherein said intersection region extraction step further comprises a step of determining a cluster centroid as a center of the intersection region.

4. An information processing method for generating a road network, executed by an information processing device, comprising: a first selection step of selecting a position corresponding to a road as a first point from a mask image that indicates whether each pixel corresponds to a road; an estimation step of estimating road information at the first point; a second selection step of selecting a second point that is a predetermined distance away from the first point; a step of storing the link between the first point and the second point as a road when it is determined that the link connecting the first point and the second point does not intersect with other road links already stored; Including, When it is determined that the link connecting the first point and the second point intersects with another road link that has already been stored, it is determined whether or not the second point is within an intersection area, and processing is performed according to the determination result; The determination of whether the second point is within an intersection area is made based on an intersection area extracted in advance by the method of claim 1.

1. An information processing method comprising:

5. An information processing device including a processor and a storage device, The storage device stores a program that, when executed by the processor, causes the processor to execute each step of the information processing method according to any one of claims 1 to 4.

1. An information processing device comprising:

Images