Intersection extraction in binary road aerial images based on tensor voting

Abstract

The application discloses a kind of intersection extraction method in binary road overhead image based on tensor voting, comprising: the image preprocessing of road mask image is carried out to suppress noise and enhance the continuity of road structure;Based on the Hessian matrix of preprocessed road mask to obtain the local curvature information of road area;The feature decomposition of the Hessian matrix is carried out to obtain corresponding eigenvalue and eigenvector, and road structure saliency response is constructed accordingly, to highlight multidirectional structure area;By thresholding processing to the structure saliency response, candidate intersection area is extracted;Density clustering algorithm is used to cluster analysis to candidate intersection area, and the accurate road intersection node position is obtained.The application can effectively extract intersection and other key structure information from pixel-level road semantic segmentation result, improve the accuracy and robustness of intersection detection, and be suitable for robot autonomous exploration, path planning and road structure analysis.

Description

Technical Field

[0001] This invention relates to the field of robot environmental perception and intersection extraction technology, specifically, a method for intersection extraction from binary road top-view images based on tensor voting. Background Technology

[0002] Semantic segmentation can imbue images with explicit semantic information and effectively separate road regions from complex backgrounds. Its segmentation results provide pixel-level road region masks, laying the foundation for subsequent road structure analysis. However, semantic segmentation is essentially still a dense prediction task in the image domain. Its output mainly describes pixel-level category relationships, lacking explicit expression of road geometry, structural features, and spatial organization, making it difficult to directly support higher-level tasks such as road structure understanding.

[0003] In real-world scenarios, roads not only represent continuous regions but also contain explicit geometric and structural information. Intersections, as key structural units in the road network, embody multi-directional connectivity and spatial organization characteristics. Relying solely on road masks obtained through semantic segmentation is insufficient to accurately identify intersection locations and their structural features. Therefore, it is necessary to further extract geometric features from the segmentation results, elevating the representation level from the "pixel layer" to the "structural layer," thereby enabling the effective extraction of key road structures.

[0004] Therefore, structural analysis of the road binary mask is necessary. By introducing geometric information such as curvature and directional consistency, the response capability to multi-directional intersection areas can be enhanced, thereby enabling explicit modeling of key structures such as intersections. Analysis of local road geometric features can effectively distinguish between linear road trunks and multi-directional intersection areas, providing a reliable basis for subsequent intersection detection.

[0005] Compared to methods based solely on morphological operations or simple connectivity analysis, intersection extraction methods based on road geometry features can more fully utilize the inherent structural information of road regions. They also exhibit good robustness and generalization ability even in complex scenarios, with noise interference, or significant changes in road morphology. Therefore, extracting road geometry from semantic segmentation results and achieving accurate intersection localization is of great significance for road structure analysis, environmental understanding, and subsequent high-level tasks. Summary of the Invention

[0006] The purpose of this invention is to propose an intersection extraction method based on tensor voting in binary road top-view images. Addressing the problem that existing methods rely solely on pixel-level semantic segmentation results and struggle to accurately characterize road geometry and intersection features, this invention introduces a road geometry analysis mechanism to further mine local structural information from semantic segmentation results, enabling precise detection of road intersection locations and thus improving the accuracy and robustness of intersection extraction in complex road scenarios.

[0007] The technical solution for achieving the objective of this invention is: a method for extracting intersections from binary road top-view images based on tensor voting, comprising:

[0008] Step 1: Perform semantic segmentation on the acquired environmental image, extract the road region from the image and generate a road binary mask; perform image preprocessing on the road binary mask;

[0009] Step 2: Calculate the Hessian matrix based on the preprocessed road binary mask to obtain curvature change information in local road regions; perform feature decomposition on the Hessian matrix corresponding to each pixel position to obtain eigenvalues ​​and eigenvectors, and construct the road structure saliency response accordingly;

[0010] Step 3: Normalize the saliency response of the road structure and use a thresholding method to filter high response areas and extract candidate intersection areas;

[0011] Step 4: Use density clustering algorithm to perform cluster analysis on candidate intersection areas, group spatially adjacent high saliency pixels into the same cluster, and automatically remove isolated noise points; calculate the geometric center position of each cluster to obtain the road intersection node.

[0012] Furthermore, the image preprocessing includes:

[0013] The road mask obtained from semantic segmentation is preprocessed, and a fixed threshold binarization is used to convert the mask into a clear binary image. Gaussian filtering is then applied to smooth the boundaries and suppress noise.

[0014] Furthermore, the step of calculating the Hessian matrix based on the preprocessed road mask to obtain curvature variation information of local road regions specifically includes:

[0015] Let the preprocessed image be Its Hessian matrix is ​​defined as:

[0016]

[0017] , Let represent the second-order partial derivatives of the image in the x and y directions, and the mixed partial derivatives, respectively, used to construct the Hessian matrix; that is:

[0018] , , .

[0020] Furthermore, the eigenvalue and eigenvector decomposition of the Hessian matrix specifically includes:

[0021] The analytic eigenvalues ​​of a Hessian matrix are obtained by the following characteristic equation:

[0022]

[0023] in Represents the Hessian matrix. This represents the second-order partial derivative of the image in different directions;

[0024] Two analytical eigenvalues ​​are derived:

[0025]

[0026] A value close to 0 indicates a linear road. Large and similar sizes indicate the intersection structure;

[0027] Min-Max normalization is applied to the response map to compress it to... Interval:

[0028]

[0029] I() is an indicator function that takes the value 1 when the condition is true and 0 otherwise; the larger the response value, the more likely the location is to be an intersection or a multi-directional structure.

[0030] Use a fixed threshold Binarize the normalized significance map to discretize the continuous response into several candidate intersection regions:

[0031]

[0032] in This is the screening threshold for structural saliency.

[0033] The process of using density clustering algorithm to cluster candidate intersection regions to obtain road intersection nodes specifically includes:

[0034] All satisfied The pixel coordinates are collected into a two-dimensional point cloud set.

[0035]

[0036] Indicates all that satisfy A set of pixel coordinates;

[0037] DBSCAN is used to cluster the point cloud; for candidate points ,That - Neighborhood is defined as:

[0038]

[0039] Point of Neighborhood, i.e., distance not exceeding The set of all points;

[0040] For each cluster obtained by clustering

[0041] This represents the set of pixels in the k-th cluster;

[0042] .

[0044] Compared with the prior art, the present invention has the following significant advantages: (1) The present invention makes full use of the prior road information provided by the semantic segmentation results, and further combines Hessian geometric structure analysis and tensor voting ideas to realize the effective transition from pixel-level road semantic information to key road structure information. (2) The method can stably identify the intersection position in the road when the road shape is complex, the noise interference is strong, or the road width changes significantly, avoiding the false detection and false negative detection problems caused by relying solely on morphological operations or simple connectivity analysis. (3) It significantly improves the accuracy and robustness of the intersection extraction results and is suitable for robot environmental perception, road structure analysis and related visual perception application scenarios. Attached Figure Description

[0045] Figure 1 This is a comparison between the pixel representation of roads and the topological representation.

[0046] Figure 2 Comparison of preprocessing effects for road segmentation maps.

[0047] Figure 3 The structural differences between linear trunk lines and intersection nodes.

[0048] Figure 4 These are the Hessian eigenvalue patterns corresponding to different road structures.

[0049] Figure 5 This refers to the Hessian eigenvalue decomposition and structural orientation.

[0050] Figure 6 This is due to the effect of significantly enhancing structural significance.

[0051] Figure 7 For thresholding and candidate region extraction.

[0052] Figure 8 Generate intersection nodes for DBSCAN clustering.

[0053] Figure 9 This is the overall flowchart of the present invention. Detailed Implementation

[0054] The present invention provides a method for extracting intersections from binary road top-view images based on tensor voting, comprising the following steps:

[0055] Step 1: Road Region Extraction and Preprocessing Based on Semantic Segmentation

[0056] A road semantic layer is constructed based on the acquired environmental images, and a road binary mask is generated for structural analysis, providing basic semantic information for subsequent road geometric structure analysis and intersection extraction. Specifically, the steps include:

[0057] 1) Environmental images captured by cameras or drones are fed into a pre-trained semantic segmentation network. The network performs pixel-level semantic classification of road regions within the images, outputting semantic segmentation results that include information on road and non-road categories. Semantic segmentation effectively eliminates interference from buildings, vegetation, and other background areas, retaining only regions relevant to the road structure.

[0058] 2) The semantic segmentation results are binarized, with pixels belonging to the road category labeled as foreground pixels and pixels not belonging to the road category labeled as background pixels, thereby generating a road binary mask image. The road binary mask explicitly represents the spatial distribution of the road area in binary form, providing basic data for subsequent geometric feature calculations.

[0059] 3) Since there may still be noisy pixels, jagged edges, or local discontinuities in the semantic segmentation results, image preprocessing is performed on the road binary mask. Gaussian filtering is applied to the road binary mask to reduce high-frequency noise and enhance the structural continuity of the road area, making the road area more spatially smooth and differentiable, thus providing stable input conditions for subsequent structural analysis based on the second derivative.

[0060] Step 2: Hessian matrix calculation and local structural feature extraction

[0061] The local curvature information of the road region is calculated based on the preprocessed binary road mask, and the principal orientation features of the road structure are extracted to characterize the geometric structure of the road. Specifically, this includes the following steps:

[0062] 1) Calculate the second-order partial derivatives of the smoothed road binary mask and construct the corresponding Hessian matrix at each pixel position to describe the second-order characteristics of gray-level changes in the neighborhood of that pixel, thereby reflecting the local curvature information of the road area.

[0063] 2) Perform eigenvalue decomposition on the Hessian matrix corresponding to each pixel position to obtain a set of eigenvalues ​​and corresponding eigenvectors, wherein the eigenvalues ​​are used to characterize the curvature intensity of the local road structure in different directions, and the eigenvectors are used to characterize the main direction information of the local structure.

[0064] 3) Based on the magnitude relationship and differences between Hessian eigenvalues, different structural types in road areas are distinguished: when eigenvalues ​​show significant directional differences, it indicates that the area is a linear road trunk; when eigenvalues ​​in multiple directions are large and similar, it indicates that the area has multi-directional structural characteristics; and areas with eigenvalues ​​close to zero correspond to background or areas without significant structure. This analysis provides a foundation for subsequent structural saliency modeling.

[0065] Step 3: Extracting candidate intersection regions based on structural saliency

[0066] A road structure saliency response is constructed based on the relationships between Hessian eigenvalues ​​to highlight intersection areas with significant multi-directional structures. Specifically, the following steps are included:

[0067] 1) Based on the relationship between the minimum and maximum eigenvalues ​​in the Hessian matrix, a road structure saliency response function is constructed to quantify the degree of multi-directional consistency of local structures in a road area. The stronger the multi-directional consistency, the higher the corresponding saliency response value.

[0068] 2) The road image is enhanced by using the structural saliency response, so that areas with multi-directional intersection features (such as intersections and corner areas) show a higher response value in the saliency map, while suppressing the main linear road area and background area, thereby highlighting potential intersection areas in space.

[0069] 3) Normalize the structural saliency response results and use thresholding to filter out high saliency regions. Extract the pixels that meet the saliency conditions as candidate intersection point sets to reduce the interference of non-intersection regions on subsequent processing.

[0070] Step 4: Generation of road intersection nodes based on density clustering

[0071] Spatial clustering analysis is performed on the candidate intersection point set to generate accurate and stable road intersection nodes. Specifically, the steps include:

[0072] 1) Extract the pixel coordinates of all pixels that meet the saliency threshold from the candidate intersection point set to form point cloud data in two-dimensional space. This point cloud shows obvious spatial clustering characteristics near the actual intersection location.

[0073] 2) The two-dimensional point cloud data is clustered using a density clustering algorithm. Based on the spatial density of the point cloud, neighboring candidate points are automatically divided into different point clusters, and isolated points with insufficient density are regarded as noise points and removed, thereby avoiding false detection.

[0074] 3) Calculate the geometric center position of each cluster of points and use the geometric center position as the final detection result of the corresponding road intersection to achieve accurate mapping from pixel-level candidate regions to discrete intersection nodes.

[0075] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0076] This embodiment is based on the following hardware platform.

[0077] ① Data Acquisition Platform: The Livox MID-360 LiDAR is used to acquire 3D point cloud data of the experimental environment; the processing unit uses an Intel Core i5-8259U processor, 16GB of DDR4 2400MHz memory, and a Kioxia EXCERIA G2 SSD 500GB for storage. The LiDAR and IMU are connected to the NUC industrial control host, which communicates with the mobile robot control chassis via a USB interface. The system is powered by a lithium battery carried by the mobile robot itself.

[0078] ② Experimental Platform: The experimental computing platform uses an Intel Core i5-8259U processor, 16GB of DDR4 2400MHz memory, and a Kioxia EXCERIA G2 SSD 500GB for storage, which is used for algorithm running, data processing, and experimental result analysis.

[0079] This embodiment is implemented based on the following software environment: the computer operating system is Ubuntu 18.04, paired with the ROSME Lodic robot operating system, serving as the basic framework for multi-sensor data management and algorithm execution. The semantic segmentation network is implemented based on a deep learning framework and is used to perform road semantic segmentation processing on the acquired environmental images; the image processing and numerical calculation module is implemented based on OpenCV and a linear algebra library and is used to complete image preprocessing, Hessian matrix calculation, feature decomposition, and saliency response construction; the clustering analysis module is implemented based on a density clustering algorithm and is used to complete spatial clustering of candidate intersection areas and intersection node extraction.

[0080] Combination Figure 3 This invention relates to a method for extracting intersections from binary road top-view images based on tensor voting. The specific implementation steps are as follows:

[0081] Step 1: Road semantic segmentation and road binary mask generation

[0082] First, aerial view images of the experimental environment are collected using a camera or drone. The collected images are then input into a semantic segmentation network to perform pixel-level semantic segmentation on the road regions in the images, obtaining semantic segmentation results that include information on road and non-road categories.

[0083] Subsequently, the semantic segmentation results are binarized, with pixels belonging to the road category labeled as foreground pixels and pixels not belonging to the road category labeled as background pixels, generating a road binary mask image. This road binary mask explicitly represents the spatial distribution of road areas in binary form, providing fundamental data for subsequent road geometry analysis.

[0084] To reduce noise pixels and boundary discontinuities generated during semantic segmentation, the generated binary road mask is smoothed using Gaussian filtering to suppress high-frequency noise and enhance the overall structural continuity of the road region, making the road region smoother in space and providing stable input for subsequent structural feature calculations based on the second derivative.

[0085] Step 2: Hessian matrix calculation and local road structure analysis

[0086] After completing the road binary mask preprocessing, the second-order partial derivatives of the smoothed road binary mask image are calculated, and a corresponding Hessian matrix is ​​constructed at each pixel location to characterize the local gray-level second-order variation characteristics of the road area at that location.

[0087] The Hessian matrix corresponding to each pixel position is subjected to eigenvalue decomposition to obtain the corresponding eigenvalues ​​and eigenvectors. The eigenvalues ​​reflect the curvature intensity of the local structure in different directions, while the eigenvectors represent the main directional information of the road structure.

[0088] By analyzing the magnitude and interrelationships of Hessian eigenvalues, we can distinguish between linear and multi-directional structural regions in road areas, providing a foundation for subsequent intersection structural saliency modeling.

[0089] Step 3: Construction of salient road structure response and extraction of candidate intersection regions

[0090] A saliency response function for road structure is constructed based on the relationship between Hessian eigenvalues ​​to characterize the degree of multidirectional consistency of local structures within a road region. Regions with multidirectional intersection characteristics exhibit higher saliency response values, while linear road trunk areas and background areas show lower saliency response values.

[0091] The road binary mask is enhanced using the structural saliency response, making multi-directional structural regions such as intersections and corners stand out in the saliency map, while suppressing the response of linear road regions.

[0092] The structural saliency response results are normalized, and high saliency regions are extracted through thresholding. Pixels that meet the saliency threshold are selected as candidate intersection point sets to reduce false detections and improve the reliability of intersection detection.

[0093] Step 4: Road intersection extraction based on density clustering

[0094] The coordinates of all pixels in the candidate intersection point set that meet the saliency threshold condition are extracted to form a candidate intersection point set in two-dimensional space.

[0095] Density clustering algorithm is used to cluster the candidate intersection point set. Based on the spatial distribution density of the points, the candidate points that are close to each other are automatically divided into different point clusters, and spatially isolated noise points are removed.

[0096] The geometric center position of each cluster of points is calculated, and the geometric center position is used as the final detection result of the corresponding road intersection, thereby realizing the accurate extraction from pixel-level candidate regions to discrete road intersection nodes.

Claims

1. A method for extracting intersections from binary road top-view images based on tensor voting, characterized in that, include: Semantic segmentation is performed on the acquired environmental image to extract the road region in the image and generate a binary road mask; Perform image preprocessing on the road binary mask; The Hessian matrix is ​​calculated based on the preprocessed road binary mask to obtain curvature change information in local road regions; the Hessian matrix corresponding to each pixel position is subjected to feature decomposition to obtain eigenvalues ​​and eigenvectors, and the road structure saliency response is constructed accordingly. The saliency response of the road structure is normalized, and a thresholding method is used to screen high response areas and extract candidate intersection areas. Density clustering algorithm is used to perform cluster analysis on candidate intersection areas, grouping spatially adjacent high-salience pixels into the same cluster and automatically removing isolated noise points; the geometric center position of each cluster is calculated to obtain the road intersection node.

2. The method as described in claim 1, characterized in that, The image preprocessing includes: The road mask obtained from semantic segmentation is preprocessed, and a fixed threshold binarization is used to convert the mask into a clear binary image. Gaussian filtering is then applied to smooth the boundaries and suppress noise.

3. The method as described in claim 1, characterized in that, The calculation of the Hessian matrix based on the preprocessed road mask to obtain curvature variation information in local road regions specifically includes: Let the preprocessed image be Its Hessian matrix is ​​defined as: , Let represent the second-order partial derivatives of the image in the x and y directions, and the mixed partial derivatives, respectively, used to construct the Hessian matrix; that is: , , 。 4. The method as described in claim 1, characterized in that, The eigenvalue and eigenvector decomposition of the Hessian matrix specifically includes: The analytic eigenvalues ​​of a Hessian matrix are obtained by the following characteristic equation: in Represents the Hessian matrix. The second-order partial derivatives of the image are represented in different directions; Two analytical eigenvalues ​​are derived: A value close to 0 indicates a linear road. Large and similar sizes indicate the intersection structure; Min-Max normalization is applied to the response map to compress it to... Interval: I() is an indicator function that takes the value 1 when the condition is true and 0 otherwise; the larger the response value, the more likely the location is to be an intersection or a multi-directional structure. Use a fixed threshold Binarize the normalized significance map to discretize the continuous response into several candidate intersection regions: in This is the screening threshold for structural saliency.

5. The method as described in claim 1, characterized in that, The process of using density clustering algorithm to cluster candidate intersection regions to obtain road intersection nodes specifically includes: All satisfied The pixel coordinates are collected into a two-dimensional point cloud set. Indicates all that satisfy A set of pixel coordinates; DBSCAN is used to cluster the point cloud; for candidate points ,That - Neighborhood is defined as: Point of Neighborhood, i.e., distance not exceeding The set of all points; For each cluster obtained by clustering This represents the set of pixels in the k-th cluster; in For each valid cluster, use the cluster labels obtained from DBSCAN clustering as an example. Calculate the arithmetic mean of all pixel coordinates to obtain: , here This represents the number of pixels in the k-th cluster. , Represents the geometric center coordinates of the k-th cluster , The final coordinates of the k-th intersection node on the image are obtained as follows: , The discrete pixel coordinates of the k-th intersection node are obtained by rounding. All nodes together constitute the intersection node set: 。

Images