Method and system for determining road centerline based on deep neural network satellite map

Abstract

The application discloses a road centerline determination method and system based on a deep neural network satellite map, and relates to the technical field of satellite image processing; the method comprises the following steps: preprocessing time-phase satellite images of a collected to-be-processed region, vehicle GPS track data and street view image sequences to obtain a dynamic region in the to-be-processed region, a track density map and a street view feature mapping table; inputting the time-phase satellite images into a pre-trained road segmentation model to obtain multiple road probability maps; based on the multiple road probability maps, performing dynamic region weighted fusion, non-dynamic region logical or operation and threshold determination to obtain an initial road mask; vectorizing the initial road mask into a centerline network and extracting cross nodes to obtain an initial road network topology; and the application has the advantages of improving the accuracy of the road network topology and guaranteeing the rationality of path planning.

Description

Technical Field

[0001] This invention relates to the field of satellite image processing technology, and more specifically, to a method and system for determining the centerline of a road based on a deep neural network satellite image. Background Technology

[0002] In urban transportation networks, elevated roads, ground-level roads, and tunnel entrances / exits are often distributed in a multi-layered structure, appearing as a superimposed projection on a two-dimensional plane in satellite imagery. Existing road centerline determination techniques based on deep neural networks typically first use semantic segmentation networks to perform pixel-level identification of road regions in satellite imagery, then use skeleton extraction algorithms to convert the segmented road regions into single-pixel-width centerlines, and finally attempt to separate different levels of road structure through post-processing techniques such as thresholding and cropping. The core principle of this type of technology relies on the pixel grayscale, texture, and local connectivity features in two-dimensional images, using a network model to learn the distinguishing rules between road and non-road areas, thereby achieving automatic centerline extraction.

[0003] However, because satellite imagery only provides two-dimensional spatial information and lacks absolute elevation data for each road level and the logical relationships of occlusion between them, the segmentation network is prone to misclassifying roads of different levels with similar projection positions as the same connected region when processing multi-layered overlay scenes. Subsequent skeleton extraction and threshold clipping operations, unable to determine the actual level of roads based on three-dimensional spatial relationships, can only segment based on distance and direction on a two-dimensional plane. This ultimately leads to false intersections and connections on the centerline between elevated and ground roads, tunnel entrances and exits, and ground roads, which should be independent of each other.

[0004] Such spurious connections can severely disrupt traffic flow models. Traffic flow models rely on the topological relationships built from road centerlines for path planning and traffic simulation. False intersections can be misidentified as actual intersections, leading to turning paths that do not conform to physical rules, such as planning a perpendicular turning route from an elevated road directly onto a ground-level road. This type of error is more difficult to detect than road centerline interruptions because spurious intersections often exhibit reasonable continuity in a two-dimensional plane, making them difficult to identify by conventional topology verification algorithms. This not only reduces the path planning accuracy of navigation systems but may also increase collision risks in scenarios such as autonomous driving that rely on precise road network topology.

[0005] In view of this, the present invention proposes a method and system for determining the centerline of a road based on deep neural network satellite imagery to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art and achieve the above objectives, the present invention provides the following technical solution: a method for determining the road centerline based on deep neural network satellite imagery, comprising:

[0007] Preprocessing is performed on the collected temporal satellite images, vehicle GPS trajectory data and street view image sequences of the area to be processed to obtain the dynamic region, trajectory density map and street view feature mapping table in the area to be processed;

[0008] The temporal satellite imagery is input into a pre-trained road segmentation model to obtain multiple road probability maps. Based on these multiple road probability maps, dynamic region weighted fusion, non-dynamic region logical OR operation, and threshold determination are performed to obtain an initial road mask.

[0009] The initial road mask is vectorized into a centerline network and intersection nodes are extracted to obtain the initial road network topology; based on the trajectory density map and street scene feature mapping table, intersection nodes that meet the preset conditions are marked as pseudo-intersection nodes;

[0010] Based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables, multimodal data blocks of the local area corresponding to each pseudo-crossing node are extracted;

[0011] Input the multimodal data block into the pre-trained occlusion detection model to obtain the judgment result set;

[0012] By associating the initial road network topology with the decision result set, a complete hierarchical road network is formed.

[0013] Furthermore, the initial road mask is a binary raster image generated by fusing road detection results from multi-temporal satellite imagery, wherein regions with a pixel value of 1 represent regions identified as roads, and regions with a pixel value of 0 represent non-road regions;

[0014] Methods for obtaining the initial road mask include:

[0015] For dynamic regions, the weights of each phase satellite image are assigned according to the acquisition time of the phase satellite image and a preset weight allocation method. The road probability of each pixel in the dynamic region is the sum of the road probabilities of the corresponding pixels in each phase satellite image multiplied by the corresponding weight. If the sum is greater than or equal to the set road determination threshold, the corresponding pixel is determined to be a road pixel.

[0016] For non-dynamic regions, when the road probability of a pixel in the road probability map of any temporal satellite image is greater than or equal to the set road determination threshold, the corresponding pixel is determined to be a road pixel; when the road probability values ​​of all temporal satellite images are less than the set road determination threshold, the corresponding pixel is determined to be a non-road pixel.

[0017] By fusing all road pixels in the road probability map, an initial road mask is obtained.

[0018] Furthermore, methods for obtaining dynamic regions within the region to be processed include:

[0019] Acquire temporal satellite images covering different lighting conditions, and after radiometric correction, atmospheric correction and geometric registration, select temporal satellite images within a preset time interval;

[0020] By using the change vector analysis algorithm, the radiation level of satellite images from one time phase to the previous time phase is corrected in adjacent time phase satellite images;

[0021] The difference in spectral values ​​of each band of satellite images in two adjacent time phases is calculated to form multiple change vectors for the corresponding pixels. Candidate change thresholds are selected based on the value range of the change vectors. The inter-class variance of the candidate change thresholds is maximized to determine the optimal change threshold. If the magnitude of any change vector of the corresponding pixel is greater than the optimal change threshold, the corresponding pixel is determined to be a changed pixel.

[0022] All changed pixels are morphologically filtered to remove isolated changed pixels and adjacent changed pixels are merged to obtain the changed region.

[0023] The edge pixels of the changing region are connected into a closed polygon boundary by a polygon fitting algorithm. The area enclosed by the polygon boundary is the dynamic region.

[0024] Furthermore, the method for marking intersection nodes that meet preset conditions as pseudo-intersection nodes includes:

[0025] The preset conditions include:

[0026] The intersection node is located within the area marked on the uncertain region map;

[0027] If the high-value areas of trajectory density of two roads in the initial road network topology do not overlap at the intersection node and the high-value area of ​​trajectory density of one road is interrupted at the intersection node, the high-value area of ​​trajectory density of the other road starts from the intersection node; the high-value area of ​​trajectory density is the area in the trajectory density map where the grid value is greater than the preset grid threshold.

[0028] By querying the street view feature mapping table, we can find the street view features within a set range around the intersection node and identify the existence of upper and lower layer road sign text or structural features.

[0029] If a cross node meets at least two of the preset conditions, it is marked as a pseudo cross node; otherwise, it is marked as a true cross node.

[0030] Furthermore, the method for generating the uncertain region map includes:

[0031] The consistency ratio of each pixel in the road probability map is obtained by dividing the number of times each pixel is identified as a road pixel in all road probability maps by the number of road probability maps. The consistency ratio ranges from 0 to 1.

[0032] The regions containing pixels with a consistency ratio lower than a preset consistency threshold are marked as uncertain regions, and an uncertain region map is generated.

[0033] Furthermore, based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables, methods for extracting multimodal data blocks corresponding to the local area of ​​each pseudo-crossing node include:

[0034] Traverse each temporal satellite image in the spatiotemporally consistent image set, crop the image block corresponding to the local area of ​​each pseudo-cross node, and superimpose the image blocks from different temporal phases according to the channel dimension to form a multi-channel local block; the channel dimension is the dimension that carries specific information in the temporal satellite image.

[0035] Based on the coordinate range of the local area corresponding to each pseudo-cross node, crop the raster data with the same coordinate range in the trajectory density map;

[0036] Based on the coordinate range of the local area corresponding to each pseudo-intersection node, retrieve all road sign text and structural features within the coordinate range of the street view feature mapping table; construct a hierarchical feature vector based on the retrieved road sign text and structural features; the length of the hierarchical feature vector is determined by the preset number of feature types;

[0037] Each pseudo-cross node corresponds to a local region multimodal data block consisting of a multi-channel image local block, a trajectory density block, and a hierarchical feature vector.

[0038] Furthermore, the structure of the occlusion detection model includes:

[0039] The occlusion detection model is a three-branch network consisting of a main branch, an auxiliary branch, and a feature enhancement branch; the input of the main branch is a local image patch; the input of the auxiliary branch is a trajectory density block corresponding to the spatial location of the local image patch; and the input of the feature enhancement branch is a hierarchical feature vector.

[0040] The bottom feature extraction layer of the three-branch network is used to extract the corresponding features of each branch. For local image patches, image features are extracted; for trajectory density features, trajectory features are extracted; and for hierarchical feature vectors, street scene features are extracted.

[0041] Image features, trajectory features, and street view features are input into the upper layer of the network of each branch to obtain image detail features, trajectory detail features, and street view detail features, respectively.

[0042] Image detail features, trajectory detail features, and street view detail features are input into the feature fusion layer to obtain comprehensive features;

[0043] The final output is a hierarchical mask and the corresponding prediction confidence. The hierarchical mask includes an upper road mask and a lower road mask, both of which are binary raster images. A road pixel of 1 indicates that it belongs to the corresponding level road, and a road pixel of 0 indicates that it does not belong to the corresponding level road.

[0044] Furthermore, methods for obtaining the judgment result set include:

[0045] If the prediction confidence is higher than the preset confidence threshold, and the percentage of pixels intersecting the upper and lower road masks is less than the preset intersection threshold, the intersection type of the corresponding pseudo-intersection node is marked as a three-dimensional cross-layer intersection and as a true intersection node; if the percentage of pixels intersecting the upper and lower road masks is greater than or equal to the preset intersection threshold, or if there is only a single-level mask with no distinction between upper and lower layers, the intersection type of the corresponding intersection node is marked as a planar intersection.

[0046] If the prediction confidence is lower than the preset confidence threshold, the street view image sequence of the local area of ​​the corresponding pseudo-intersection node is called, and the hierarchical mask output by the occlusion detection model is corrected by combining the structural features in the street view feature mapping table and the road sign text. The correction method includes at least the following: if there is a bridge pier in the street view image sequence and it is located below a certain road, the road pixels in the corresponding hierarchical mask are marked as upper-layer road pixels, and the road pixels in another hierarchical mask are marked as lower-layer road pixels; if the road sign text contains a tunnel exit and shows a connection to the ground road, the intersection type of the corresponding pseudo-intersection node is corrected from planar intersection to three-dimensional cross-layer.

[0047] The final output set of judgment results includes the upper road mask, lower road mask, corresponding prediction confidence, and intersection type of the pseudo-intersection node for the local area corresponding to each pseudo-intersection node.

[0048] Furthermore, the area to be processed is a polygonal region of the geographical range to be processed in the latitude and longitude coordinate system.

[0049] A road centerline determination system based on deep neural network satellite imagery includes:

[0050] The data preprocessing module is used to preprocess the collected temporal satellite images, vehicle GPS trajectory data and street view image sequences of the area to be processed, and to obtain the dynamic area, trajectory density map and street view feature mapping table in the area to be processed.

[0051] The road recognition module is used to input temporal satellite imagery into a pre-trained road segmentation model to obtain multiple road probability maps. Based on the multiple road probability maps, dynamic region weighted fusion, non-dynamic region logical OR operation, and threshold determination are performed to obtain an initial road mask.

[0052] The node labeling module is used to vectorize the initial road mask into a centerline network and extract the intersection nodes to obtain the initial road network topology; based on the trajectory density map and street scene feature mapping table, the intersection nodes that meet the preset conditions are labeled as pseudo-intersection nodes;

[0053] The feature extraction module extracts multimodal data blocks of the local area corresponding to each pseudo-crossing node based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables.

[0054] The node determination module is used to input multimodal data blocks into the pre-trained occlusion detection model to obtain a determination result set;

[0055] The road generation module is used to associate the initial road network topology with the judgment result set to form a complete hierarchical road network.

[0056] Compared with existing technologies, the technical effects and advantages of the road centerline determination method and system based on deep neural network satellite imagery of the present invention are as follows:

[0057] This invention preprocesses temporal satellite imagery, vehicle GPS trajectory data, and street view image sequences to obtain dynamic regions, trajectory density maps, and street view feature mapping tables. The temporal satellite imagery is then input into a pre-trained road segmentation model to obtain a road probability map. An initial road mask is generated through dynamic region weighted fusion, a logical OR operation on non-dynamic regions, and threshold judgment. This initial road mask is vectorized into a centerline network, and intersection nodes are extracted. Pseudo-intersection nodes are marked using the trajectory density map and street view feature mapping table. Multimodal data blocks corresponding to the local areas of the pseudo-intersection nodes are extracted and input into an occlusion detection model to obtain a judgment result set. Finally, the initial road network topology and the judgment result set are associated to form a complete hierarchical road network.

[0058] This invention can accurately distinguish the spatial hierarchy of multiple road layers such as elevated roads, ground-level roads, and tunnels, effectively identify and eliminate false intersections caused by overlapping two-dimensional projections, and restore the true road network topology. This solves the problems in the prior art where the inability to distinguish the three-dimensional relationship of multiple roads in satellite imagery leads to false intersections interfering with traffic flow models, generating unreasonable paths, reducing navigation accuracy, and posing collision risks. It has the advantages of improving the accuracy of road network topology and ensuring the rationality of path planning. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of a road centerline determination system based on deep neural network satellite imagery according to an embodiment of the present invention;

[0060] Figure 2 This is a flowchart of the road centerline determination method based on deep neural network satellite imagery according to an embodiment of the present invention;

[0061] Figure 3This is a structural diagram of the occlusion detection model according to an embodiment of the present invention;

[0062] Figure 4 This is a flowchart of a method for obtaining a dynamic region in a region to be processed according to an embodiment of the present invention;

[0063] Figure 5 This is a schematic diagram of the upper and lower road masking layers according to an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be described in detail, clearly, and completely below with reference to the accompanying drawings. It should be particularly noted that the specific embodiments described below are only for better illustrating and explaining the technical solutions of the present invention, and are intended to enable those skilled in the art to better understand and implement the present invention, and should not be construed as limiting the scope of protection of the present invention. Without departing from the spirit and substance of the present invention, those skilled in the art can modify, adjust, or make equivalent substitutions based on the content disclosed in the present invention, and these should all be considered within the scope of protection of the present invention.

[0065] Example 1:

[0066] Please see Figure 1 As shown, this embodiment discloses a road centerline determination system based on deep neural network satellite imagery, including a data preprocessing module, a road identification module, a node marking module, a feature extraction module, a node determination module, and a road generation module. Each module is connected via wired and / or wireless means to achieve data transmission.

[0067] The data preprocessing module is used to preprocess the collected temporal satellite images, vehicle GPS trajectory data and street view image sequences of the area to be processed, and to obtain the dynamic area, trajectory density map and street view feature mapping table of the area to be processed.

[0068] Collect multi-source data on the geographic boundary range of the area to be processed; the multi-source data includes temporal satellite imagery, vehicle GPS trajectory data, road occlusion training data, and street view image sequences; the geographic boundary range of the area to be processed is a polygonal boundary in a latitude and longitude coordinate system.

[0069] Methods for acquiring temporal satellite imagery include: acquiring high-resolution optical satellite images of the target area at different time periods, prioritizing early morning and midday times within a one-month interval, with a resolution better than 0.5 meters. Multiple images can be acquired using commercial or high-resolution satellites, and radiometric correction, geometric correction, and temporal consistency filtering are used to ensure image alignment across different dates. Simultaneously, change detection algorithms such as CVA (Cardiometric Variation Vector Analysis) are employed to mark areas of difference between images, such as road construction and temporary occlusion, generating a spatiotemporally consistent image set. Temporal satellite imagery provides the appearance of roads under different lighting angles and environments, helping to identify occluded road sections and the shadow features of elevated structures.

[0070] Methods for acquiring vehicle GPS trajectory data include: collecting GPS driving trajectory data of a large number of vehicles within the target area, such as historical location sequence sequences of taxis and ride-hailing vehicles. Trajectory data can be obtained through traffic management departments or crowdsourcing applications. After coordinate projection transformation, removal of speed anomalies, and supplementation of sparse trajectories in the lower level of elevated roads / tunnels through interpolation of trajectories from adjacent road segments, a rasterized trajectory density map is generated. The trajectory density map reflects the frequency of vehicle traffic on the road and is used to assist in determining road connectivity; at the intersection of multiple road layers, different layers typically correspond to independent trajectory clusters, and vehicles do not directly cross layers to turn.

[0071] The methods for obtaining road occlusion training data include: preparing a sample dataset for training the "road occlusion / saliency" model. This is achieved through two methods: first, manually labeling several satellite image segments containing viaducts / tunnels, marking which road segment is on the upper layer and which is on the lower layer; second, using existing single-layer road image composite data, overlaying one road image onto another and adding shadow effects to generate training samples with known occlusion labels; simultaneously, 10% "extreme case samples" are added, such as nested multi-level interchanges or temporary construction occlusion scenarios. The road occlusion training data is used to supervise the learning model, enabling it to learn to identify road occlusion relationships and hierarchical information from satellite image segments.

[0072] The method for acquiring street view image sequences includes: collecting street view images along the road using a vehicle-mounted panoramic camera, including road signs and spatial relationships, and simultaneously recording GPS coordinates and shooting angles. Priority is given to collecting street view images of multi-level road intersections such as elevated road entrances and exits, and tunnel entrances and exits. The focus is on extracting text signs such as "elevated road entrance" and "tunnel exit," as well as structural features such as bridge piers and guardrails, to supplement the determination of hierarchical attributes.

[0073] Please see Figure 4As shown, the geographic boundary of the area to be processed is input through a commercial satellite data interface or the satellite imagery service API of an open-source geographic data platform to obtain temporal satellite imagery of the area. This temporal satellite imagery must cover time periods with different lighting conditions. Radiometric correction is used to eliminate sensor response differences, and atmospheric correction is used to remove interference from clouds, fog, aerosols, etc. Then, a corresponding point matching algorithm is used to complete the geometric registration of the temporal satellite imagery, ensuring accurate spatial alignment. Subsequently, the acquired temporal satellite imagery is time-filtered, retaining only those captured within one month to ensure temporal consistency. Simultaneously, the Comparative Vector Analysis (CVA) algorithm is used to compare the spectral differences between different temporal satellite imagery. Specifically, a linear regression algorithm is used to correct the spectral values ​​of the later temporal satellite imagery to match the radiometric level of the earlier temporal satellite imagery, eliminating systematic spectral deviations caused by differences in light intensity and atmospheric conditions. Then, for each pixel in the temporal satellite imagery, the difference in spectral values ​​of each band between the two temporal satellite imagery is calculated to form the corresponding pixel's comparative vector. The magnitude of the change vector is obtained by taking the square root of the sum of the squares of the differences between each band. The magnitude directly reflects the severity of the spectral change. The direction of the change vector is determined by the proportional relationship between the differences between each band, used to distinguish the type of change. Subsequently, the magnitudes of the change vectors of all pixels in the temporal satellite image are statistically analyzed, and the range of change vector magnitudes is divided into continuous gray-level intervals. Specific classifications are set based on the actual distribution range of the magnitudes. Each value within a gray-level interval is used as a candidate threshold, dividing all pixels in the temporal satellite image into two categories: one category consists of pixels whose change vector magnitude is less than or equal to the candidate threshold, considered as pixels with insignificant changes; the other category consists of pixels whose change vector magnitude is greater than the candidate threshold, considered as pixels with potentially significant changes. For each candidate threshold, the mean of the change vector magnitudes of insignificant and potentially significant change pixels and their respective pixel counts are calculated to obtain the proportion of insignificant and potentially significant change pixels. The inter-class variance is calculated by summing the products of the proportions of insignificant and potentially significant change pixels. The inter-class variance corresponding to each candidate threshold is calculated one by one. The candidate threshold that maximizes the inter-class variance is the optimal threshold. Pixels whose change vector magnitude is greater than the optimal threshold are initially identified as changed pixels. The physical significance of the optimal threshold is that the difference in magnitude between significant and insignificant change regions is most obvious at this point, resulting in the highest separation between insignificant and potentially significant change pixels. To reduce noise interference, morphological filtering is applied to the initially identified changed pixels. First, isolated changed pixels with an area of ​​less than 3 pixels are removed by screening. Then, adjacent changed pixel regions are merged through a concatenation operation to form continuous changed regions. Finally, a chain code tracing algorithm is used to traverse the edge pixels of the changed regions, recording the coordinates of each edge pixel. Then, a polygon fitting algorithm is used to connect the discrete edge pixels into closed polygon boundaries.By associating the polygon boundary with the geographic coordinates of the temporal satellite image, a polygon region containing the boundary coordinates, the area of ​​the changed region, and the mean magnitude of the change vector is generated and marked as a dynamic region.

[0074] Marking dynamic regions helps accurately capture temporal changes in roads within the processing area. Existing technologies, relying on single or static satellite imagery, struggle to handle interference from dynamic changes such as road construction and temporary obstructions, easily misjudging temporary overlaps caused by dynamic changes as fixed multi-layered road intersections, thus creating false connections. Dynamic regions, by comparing satellite imagery from different time periods, identify areas where spectral characteristics change significantly. When generating the initial road mask, a time-weighted fusion strategy is used for dynamic regions, prioritizing the retention of the current state reflected in the latest temporal image. This avoids interference from static information in older images solidifying dynamic changes, ensuring road recognition responds promptly to actual changes. Simultaneously, the distinction between dynamic and non-dynamic regions allows for the stable preservation of road information in non-dynamic areas, maintaining the integrity of the overall road network. This precise grasp of dynamic changes reduces misjudgments of road areas due to temporary changes, lowers the probability of false intersections, and provides a more reliable foundation for accurate identification of false intersections and correction of road network topology, thereby mitigating the interference of false connections on traffic flow models and improving the accuracy of path planning.

[0075] The WGS84 coordinate system of the original GPS trajectory data points is converted to a projected coordinate system consistent with the time-phase satellite imagery, such as UTM partitioning, using a coordinate transformation tool. Then, statistical filtering is used to remove speed anomalies and drift points from the original trajectory points. Speed ​​anomalies and drift points are determined by a distance threshold between adjacent points. Speed ​​anomalies are points in the original trajectory data where the calculated speed, based on the time interval and distance between adjacent trajectory points, exceeds the reasonable range for normal vehicle driving. For example, if the time interval between two adjacent original trajectory points is 1 second, but the distance exceeds 50 meters, the corresponding speed is 180 km / h, far exceeding the speed limit on ordinary roads, then this point will be identified as a speed anomaly. Drift points are points in the original trajectory data where the distance between them and adjacent original trajectory points far exceeds the distance threshold under normal driving conditions. For example, if the threshold for the distance between adjacent points is set to 50 meters, and the distance between a certain original trajectory point and its previous adjacent point reaches 200 meters, and does not conform to the actual driving path of the vehicle, such as suddenly jumping to an area outside the road, then this point will be judged as a drift point, usually caused by GPS signal obstruction or interference leading to positional deviation; the threshold for the distance between adjacent points is adjusted accordingly based on the road type. For sparse trajectory segments in the lower level of elevated roads or tunnels, linear interpolation is used to uniformly insert supplementary points between adjacent valid trajectory points in the original trajectory points at time intervals to ensure trajectory continuity. Finally, a kernel density estimation algorithm is used to convert the processed original trajectory points into a rasterized trajectory density map. The raster resolution is consistent with the temporal satellite imagery, that is, the side length of each grid in the trajectory density map (e.g., 0.5 meters) is the same as the pixel side length of the corresponding area temporal satellite imagery. The raster value reflects the density of vehicle GPS trajectory points in the corresponding ground area, used to reflect the spatial distribution of vehicle driving trajectories. In this embodiment, the raster value is normalized to the range of 0 to 1 through a maximum-minimum normalization. For example, if the resolution of the temporal satellite image is 0.5 meters, that is, each pixel corresponds to a 0.5-meter × 0.5-meter area on the ground, then the grid of the trajectory density map is also set to 0.5 meters × 0.5 meters to ensure that the two correspond precisely in spatial scale, which is convenient for subsequent multi-source data overlay analysis and feature association.

[0076] By spatially associating image GPS coordinates with the area, street view data covering the region to be processed is selected from a street view image sequence. Optical automatic recognition (OIR) technology is used to extract road sign text from each image in the street view image sequence, such as "elevated entrance" and "tunnel exit." Pre-trained object detection models such as YOLO are used to identify structural features such as bridge piers and guardrails. The road sign text and structural features are then bound to their corresponding geographic coordinates to establish a street view feature mapping table. This table includes a list of road sign texts and structural feature types corresponding to each geographic coordinate. The method for binding road sign text and structural features to their corresponding geographic coordinates includes: each image in the street view image sequence contains GPS coordinates recorded at the time of capture, as well as camera attitude parameters such as heading and pitch angles. When extracting a specific road sign text using OIR technology, the pixel coordinate range of the text in the image must first be determined; then, combining the image's GPS coordinates, camera intrinsic parameters, and attitude parameters, a spatial projection transformation method is used to convert the pixel coordinate range of the text into a ground geographic coordinate range. For example, if road sign text is located near the bottom of the image, the camera's tilt angle indicates that the actual distance between the road sign text and the shooting point is relatively close. The corresponding ground coordinate range is then calculated through projection. Similarly, for structural features such as bridge piers and guardrails identified by the object detection model, their pixel coordinate range in the image is first determined, and then converted to geographic coordinates using the aforementioned spatial projection transformation method. Subsequently, the converted geographic coordinates are associated and stored with the extracted road sign text content and structural feature type: if both the "elevated entrance" road sign text and the bridge pier structural feature are extracted within a certain geographic coordinate range, the binding relationship of the corresponding coordinate range is recorded in the street view feature mapping table as: Sign Text: Elevated Entrance; Structural Feature: Bridge Pier. In this way, a one-to-one spatial binding relationship is established between the road sign text and structural features and their actual ground locations.

[0077] The road recognition module is used to input temporal satellite imagery into a pre-trained road segmentation model to obtain multiple road probability maps. Based on the multiple road probability maps, dynamic region weighted fusion, non-dynamic region logical OR operation, and threshold determination are performed to obtain the initial road mask.

[0078] A pre-trained road segmentation model is run on each temporal satellite image in the spatiotemporally consistent image set. The road segmentation model has been trained with a large amount of road sample data and can identify road pixels in the image. The output road probability map is in raster form, where the value of each pixel represents the probability that the corresponding pixel belongs to a road. The probability ranges from 0 to 1, and the higher the value, the more likely the corresponding pixel belongs to a road.

[0079] Methods for obtaining the initial road mask include:

[0080] For dynamic regions, the weights of each satellite image are assigned according to the acquisition time of the time-phase satellite imagery and a preset weighting method. The road probability of each pixel in the dynamic region is the sum of the road probabilities of the corresponding pixels in each time-phase satellite imagery multiplied by their corresponding weights. When the sum is greater than or equal to the set road determination threshold, the corresponding pixel is determined to be a road pixel. For non-dynamic regions, when the road probability of a pixel in the road probability map of any time-phase satellite imagery is greater than or equal to the set road determination threshold, the corresponding pixel is determined to be a road pixel. When the road probability values ​​of all time-phase satellite images are less than the set road determination threshold, the corresponding pixel is determined to be a non-road pixel. All road pixels in the road probability map are fused to obtain the initial road mask.

[0081] The road determination threshold is set based on a reference sample set of road pixels with manual annotations. Specifically, sample pixels known to be road pixels and non-road pixels are selected, and the probability values ​​of these samples in the road probability map of each time phase are extracted. The probability distribution of road pixels and non-road pixels is statistically analyzed. Candidate thresholds in the range of 0 to 1 are traversed. For each candidate threshold, the matching situation between its determination result and the manual annotation is calculated, including the proportion of road pixels judged as roads to the actual road samples, i.e., the correct recognition rate, and the proportion of non-road pixels judged as non-roads to the actual non-road samples, i.e., the correct exclusion rate. The smallest candidate value that makes both the correct recognition rate and the correct exclusion rate reach the preset target (e.g., both not less than 95%) is selected as the road determination threshold to ensure effective recognition of road pixels and accurate exclusion of non-road pixels.

[0082] The initial road mask is a binary raster image generated by fusing road detection results from multi-temporal satellite imagery. Regions with a pixel value of 1 represent areas identified as roads, while regions with a pixel value of 0 represent non-road areas. Through this fusion method, the initial road mask integrates road information from different temporal satellite imagery, prioritizing the preservation of dynamic changes reflected in the latest temporal satellite imagery while ensuring the integrity of roads in non-dynamic areas.

[0083] The detection consistency of each pixel in the road probability map is calculated in different time phases. Specifically, for each pixel, the number of times it is identified as a road pixel in all road probability maps is counted, and the result is divided by the number of road probability maps to obtain the consistency ratio of the corresponding pixel, which ranges from 0 to 1. The regions where the consistency ratio is lower than the preset consistency threshold are marked as uncertain regions, and an uncertain region map is generated. The uncertain region map is used to mark the regions where the detection results differ greatly in different time phases.

[0084] The preset consistency threshold is determined based on a reference sample set of road areas with manual annotations. Specifically, it involves selecting definite road areas (those consistently identified as roads across different time phases by manual annotation), definite non-road areas (those consistently identified as non-roads across different time phases by detection results), and ambiguous areas where detection results differ across different time phases. Areas marked as uncertain regions within these definite road, non-road, and ambiguous areas are used as sample areas. The consistency ratio of all pixels within these definite road, non-road, and uncertain regions is calculated, which is the ratio of the number of times a pixel is identified as a road to the number of road probability maps. The distribution of the consistency ratio among definite road, non-road, and uncertain regions is then statistically analyzed. The consistency ratio of road areas is determined to be concentrated in the high value range (e.g., 0.8–1.0), the consistency ratio of non-road areas is determined to be concentrated in the low value range (e.g., 0–0.2), and the consistency ratio of uncertain areas is concentrated in the middle range (e.g., 0.3–0.7). Candidate thresholds within the range of 0–1 are iterated, and for each candidate threshold, its correct labeling rate for uncertain areas and its mislabeling rate for certain areas are calculated. The correct labeling rate is the proportion of pixels labeled as uncertain out of the total pixels in the uncertain area; the mislabeling rate is the proportion of pixels mislabeled as uncertain out of the total pixels in the certain area. The smallest candidate value that ensures the correct labeling rate is not lower than a preset target (e.g., 90%) and the mislabeling rate is not higher than a preset upper limit (e.g., 5%) is selected as the final consistency threshold.

[0085] The node labeling module is used to vectorize the initial road mask into a centerline network and extract the intersection nodes to obtain the initial road network topology; based on the trajectory density map and street scene feature mapping table, the intersection nodes that meet the preset conditions are labeled as pseudo intersection nodes.

[0086] The initial road mask's road pixel region is refined into single-pixel-width centerlines using a skeleton extraction algorithm. Then, a line connection algorithm connects these discrete centerlines into a continuous network, forming a complete road centerline network and yielding the initial road network topology. Points where three or more centerlines intersect, or points where two centerlines intersect at a certain angle, are extracted from this initial road network topology to obtain intersection nodes. The geographic coordinates of each intersection node are recorded.

[0087] For each intersection node, the following three conditions are checked sequentially: First, it is determined whether the intersection node is located within the area marked on the uncertain area map; second, it is analyzed whether the trajectory density clusters corresponding to the roads connected by the intersection node are independent, which is achieved by calculating the connectivity of each road in the trajectory density map. Specifically, if the high-value areas of trajectory density of two roads do not overlap at the intersection node and the high-value area of ​​trajectory density of one road is interrupted at the node, while the high-value area of ​​trajectory density of the other road starts from the intersection node, then it is determined that there is no trajectory interconnection and the trajectory density clusters are independent; the high-value area of ​​trajectory density is the area in the trajectory density map where the grid value is greater than a preset grid threshold, for example, the grid value is greater than or equal to 0.6; third, the street scene features within a certain range (e.g., a 50-meter radius) around the intersection node are queried through the street scene feature mapping table. If there are upper and lower level road identification text such as "elevated" or "tunnel", or structural features such as "bridge pier" or "layered guardrail", then it is determined that the street scene features show upper and lower level identification.

[0088] The preset raster values ​​are set based on the actual distribution characteristics of trajectory density and the need to distinguish road trajectory clusters. The specific implementation steps are as follows: Select known layered road areas containing elevated roads, ground surfaces, tunnels, and their connecting ramps as layered reference samples. Extract raster value data from the trajectory density map within the layered road areas, while also including raster values ​​from non-road areas as a background comparison. Statistically analyze the raster values ​​of the road areas in the layered reference samples to generate a frequency distribution histogram and analyze its concentration range. For example, the raster values ​​of most road trajectories are concentrated in the range of 0.4 to 0.8. At the same time, statistically analyze the distribution range of raster values ​​in non-road areas. For example, the raster values ​​of most road trajectories are concentrated in the range of 0.1 to 0.3, clarifying the numerical differences between road trajectories and background trajectories. Based on the statistical results, initially set the candidate raster value range within the boundary interval between the raster values ​​of road areas and non-road areas, such as 0.3 to 0.5, ensuring that the candidate raster value range can cover the critical values ​​that may distinguish dense trajectory clusters from sparse background trajectories. For each candidate raster value, its effectiveness in dividing high-value trajectory density areas is examined: the overlap rate between high-value areas with raster values ​​greater than the candidate raster value and the actual road trajectories in the reference sample is calculated. For example, an overlap rate greater than or equal to 90% can be required, meaning that the main trajectory segments on the road are all included in the high-value area; at the same time, the proportion of non-road area rasteres in the high-value area should be less than or equal to 5%, thus avoiding misclassifying sparse non-road trajectories as high-value areas. If a candidate raster value does not meet the above indicators, it is adjusted. If the overlap rate is insufficient, the candidate raster value is reduced to include more road trajectories; if the proportion of non-road areas is too high, the candidate raster value is increased to eliminate redundant background. After multiple verifications, the smallest candidate raster value that simultaneously meets the overlap rate and non-road proportion requirements is selected as the final preset raster value to ensure that it can stably distinguish independent trajectory clusters of roads at different levels, providing an accurate basis for judging the trajectory interconnectivity at intersection nodes. For example, the setting of "raster value greater than or equal to 0.6 in this embodiment" needs to be verified for its adaptability through the above steps.

[0089] A node is marked as a pseudo-intersection node if it meets at least two of the three preset conditions mentioned above; otherwise, it is marked as a true intersection node. Each pseudo-intersection node is accompanied by a corresponding local region, such as a rectangular area of ​​200×200 pixels centered on the node, which provides a spatial range basis for subsequent extraction of local data blocks.

[0090] The core purpose of marking false intersections is to identify and locate spurious intersections in satellite imagery caused by the overlapping two-dimensional projections of multiple road layers, such as elevated roads, ground-level roads, and tunnels. This provides precise targets for subsequent correction of road network topology, thereby addressing the core issues in the background technology. Existing technologies rely on the two-dimensional information of satellite imagery and cannot distinguish the three-dimensional spatial relationships of multiple road layers. This leads to the formation of spurious intersections on the two-dimensional projection of the centerlines of different road levels. These spurious intersections are misjudged as actual intersections by traffic flow models, resulting in path planning that does not conform to physical rules, reducing navigation accuracy and even increasing the risk of collisions. By identifying false intersections based on conditions such as uncertain regions, independent trajectory density clusters, and street view layer-by-layer features, the marking of false intersection nodes can accurately identify intersections formed by layer overlap rather than actual connectivity, providing a basis for distinguishing between real planar intersections and multi-layer projection overlaps. On this basis, subsequent occlusion detection models can be used to further determine the corresponding road layer relationships and delete false connections between roads of different layers, making the centerlines of roads at each layer topologically independent, restoring the real three-dimensional road network structure, thereby preventing traffic flow models from misjudging false connections as actual intersections, eliminating the generation of unreasonable turning paths, improving the path planning accuracy of navigation systems, and reducing collision risks in scenarios such as autonomous driving.

[0091] The feature extraction module extracts multimodal data blocks of the local area corresponding to each pseudo-cross node based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables.

[0092] For each pseudo-intersection node, corresponding local area image data is extracted from the spatiotemporally consistent image set. This involves traversing each temporal satellite image in the set, cropping image blocks for the corresponding temporal phase and region, and then superimposing these image blocks from different temporal phases along the channel dimension to form a multi-channel local block. The channel dimension refers to the dimension in the temporal satellite image that carries visual information related to road hierarchy identification, such as shadow boundaries and illumination, at different temporal phases. Because the illumination angles differ across temporal phases, the shadow boundaries of roads in the images will vary. The superimposed multi-channel data can completely preserve the spatiotemporal changes in shadow features, providing a basis for subsequent identification of upper and lower road layers.

[0093] Simultaneously, trajectory density blocks corresponding to the local area of ​​each pseudo-intersection node are extracted from the trajectory density map. Based on the coordinate range of the local area, raster data with the same coordinate range in the trajectory density map are cropped to ensure that the trajectory density blocks are perfectly aligned spatially with the local blocks of the multi-channel image. The pixel values ​​in the trajectory density blocks reflect the density of vehicle trajectories in that area, complementing the image features.

[0094] By associating the street view features of the local area corresponding to each pseudo-intersection node with the street view feature mapping table, a hierarchical feature vector is obtained. Specifically, based on the coordinate range of the local area, all road sign text and structural features within that range are retrieved from the street view feature mapping table. If road sign text related to "elevated road" exists, the corresponding dimension of the hierarchical feature vector is increased by 1; if road sign text related to "tunnel" exists, the corresponding dimension of the hierarchical feature vector is increased by 1; if upper-level road structural features such as bridge piers exist, the corresponding dimension of the hierarchical feature vector is increased by 1; if features distinguishing upper and lower layers such as guardrail layers exist, the corresponding dimension of the hierarchical feature vector is increased by 1. The length of the hierarchical feature vector is determined by the preset number of feature types. For example, if there are 4 types of features, the length of the hierarchical feature vector is 4. The numerical value directly reflects the richness of hierarchical features contained in the area.

[0095] Ultimately, the multimodal data block corresponding to the local region of each pseudo-cross node consists of three parts: a multi-channel image local block superimposed with shadow features from different temporal phases, a trajectory density block spatially aligned with the image block, and a quantized hierarchical feature vector. These three parts of data strictly correspond in spatial range and together constitute the multi-source input information used for occlusion relationship discrimination.

[0096] The node determination module is used to input multimodal data blocks into the pre-trained occlusion detection model to obtain a determination result set.

[0097] Training methods for occlusion detection models include:

[0098] Please see Figure 3As shown, a three-branch network is constructed, comprising a main branch, an auxiliary branch, and a feature enhancement branch. The input to the main branch is an image local patch extracted from the road occlusion training data. This image local patch is used to crop satellite image segments from the road occlusion training data into fixed-size sub-maps, which include superimposed and occluded scenes of roads at different levels. The input to the auxiliary branch is a trajectory density block corresponding to the spatial location of the image local patch. The trajectory density block is converted into a feature matrix and then input to the auxiliary branch. The input to the feature enhancement branch is a hierarchical feature vector spatially aligned with the image local patch. This feature vector is converted into a fixed-dimensional feature vector through a fully connected layer and then input to the feature enhancement branch. The bottom feature extraction layer of the three-branch network employs a shared weight design, aiming to synchronously extract basic features from the input data. Specifically: for the image local patches input by the main branch, the bottom layer extracts image features reflecting the road's appearance, including grayscale distribution, edge texture, and shadow morphology of roads at different levels. These image features originate from the visual differences between the road and its surrounding environment in temporal satellite imagery. For the trajectory density features input by the auxiliary branch, the bottom layer extracts trajectory features reflecting vehicle movement patterns, including trajectory density, continuous distribution, and interruption locations. These trajectory features originate from the spatial distribution patterns left by vehicle passage in the trajectory density map. The feature enhancement branch extracts street scene features reflecting street scene hierarchy attributes, such as the presence or absence of upper and lower layer markers and the type of structural features. Through shared weights, image features, trajectory features, and street scene features can be correlated during bottom-level learning. For example, the shadow boundaries in the image local patches can be correlated with the interruption locations of the trajectory density features, serving as a basis for determining the road hierarchy. The upper layers of the network employ separate convolutional layers tailored to the characteristics of different inputs, processing branch-specific features. Specifically: the upper convolutional layers of the main branch focus on refining layer-related details in image features, such as shadow gradient changes at the bottom of elevated roads and brightness differences at tunnel entrances and exits, outputting image detail features; the upper convolutional layers of the auxiliary branches focus on refining layer-related details in trajectory features, such as the boundaries between high-density trajectory clusters corresponding to upper-level roads and low-density trajectory clusters corresponding to lower-level roads, and the parallel distribution of trajectories at different levels, outputting trajectory detail features; the upper layer of the feature enhancement branch refines street scene features through a multilayer perceptron, outputting street scene detail features. Finally, in the feature fusion layer, image detail features, trajectory detail features, and street scene detail features are concatenated to obtain comprehensive features. These comprehensive features preserve differences in image appearance and vehicle movement patterns while integrating street scene layer attributes, providing a comprehensive basis for distinguishing road layers.

[0099] To ensure the occlusion detection model prioritizes shadow boundaries, trajectory discontinuities, and street view signs, a hierarchical attention mechanism is introduced after the feature fusion layer. The weights of image detail features, trajectory detail features, and street view detail features in the integrated feature set are calculated, with the weight reflecting the importance of the corresponding feature to the hierarchical judgment. The weights are converted into an attention mask using a sigmoid activation function. The value at each position in the attention mask represents the degree of attention the occlusion detection model pays to that position. Positions corresponding to shadow boundaries, trajectory discontinuities, and street view signs are assigned higher values, causing the occlusion detection model to prioritize these features during feature extraction. The positions of shadow boundaries, trajectory discontinuities, and street view signs are derived from manually labeled road occlusion training data. These labels include the specific locations of shadows in the image, the specific locations of trajectory discontinuities in the trajectory density map, and the spatial coordinates of street view signs.

[0100] The weighting methods for image detail features, trajectory detail features, and street view detail features in the comprehensive feature set include:

[0101] Global average pooling is performed on each feature in the comprehensive feature set. The spatial dimension (H×W) of image detail features (spatial dimension H×W×C1), trajectory detail features (H×W×C2), and street scene detail features (H×W×C3) is compressed to 1×1, resulting in their respective global feature vectors with dimensions of 1×1×C1, 1×1×C2, and 1×1×C3, respectively. Here, C1, C2, and C3 are the number of channels for the corresponding feature, H is the height of the corresponding feature, and W is the width of the corresponding feature. The three global feature vectors are concatenated into a 1×1×(C1+C2+C3) global descriptor, which is input into a fully connected layer containing a ReLU activation function. This is mapped to an initial weight vector of length 3 (ω1, ω2, ω3), corresponding to the initial importance of image detail features, trajectory detail features, and street scene detail features, respectively. ω1, ω2, and ω3 are the initial weights of the three feature classes, respectively.

[0102] Manually labeled content from the road occlusion training data is incorporated to calculate the response intensity of three types of features in the labeled areas. Within the shadow boundary labeled area, the average pixel value S1 of image detail features is calculated; within the trajectory interruption labeled area, the average pixel value S2 of trajectory detail features is calculated; and within the street view marker labeled area, the average pixel value S3 of street view detail features is calculated. S1, S2, and S3 are multiplied by ω1, ω2, and ω3 in the initial weight vector, respectively, to obtain the adjusted weights (ω1', ω2', ω3'), where ω1' = ω1 × S1, ω2' = ω2 × S2, and ω3' = ω3 × S3. ω1', ω2', and ω3' represent the adjusted weights for the three types of features, ensuring that features with stronger responses in the labeled key areas receive higher weights.

[0103] The adjusted weights (ω1', ω2', ω3') are L2 normalized to obtain normalized weights (ω1'', ω2'', ω3'') with a sum of 1. ω1'', ω2'', and ω3'' are the normalized weights for the three types of features, ensuring that the weights for the three types of features can be directly used for feature fusion. The normalized weights are then mapped to the 0-1 interval using the sigmoid activation function, ultimately yielding feature weights that can be used as attention mask coefficients. Higher values ​​indicate a more significant contribution of the corresponding feature to the hierarchical decision.

[0104] The occlusion detection model is optimized using a hybrid loss function: cross-entropy loss measures the difference between the hierarchical segmentation result output by the occlusion detection model and the actual hierarchical levels manually labeled in the road occlusion training data. The hierarchical segmentation result output by the occlusion detection model is the judgment of each pixel in the input image local block, i.e., whether each pixel belongs to the upper road, lower road, or non-road region, presented in probability form; the hierarchical information labeled in the road occlusion training data is the actual hierarchical level of each pixel manually labeled. During calculation, for each pixel, the three probabilities predicted by the occlusion detection model are cross-entropy calculated with the manually labeled actual hierarchical level. The actual hierarchical level is represented by one-hot encoding, such as [1,0,0] for upper roads. The results of the calculation for all pixels are then summed to obtain the cross-entropy loss value. The smaller the cross-entropy loss value, the closer the classification result of the occlusion detection model is to the actual situation. Hierarchical contrast loss is used to enhance the occlusion detection model's ability to distinguish road features at different hierarchical levels. During calculation, the pixels belonging to the upper and lower road regions are first determined based on the boundaries of the upper and lower roads marked in the road occlusion training data. Then, the feature vectors output by the intermediate layer of the occlusion detection model for the corresponding pixels are extracted. These feature vectors are the numerical representation of the pixel-level features by the occlusion detection model, and the feature vectors of all pixels together constitute the feature space. For pixels within the upper road region, the average distance between their feature vectors is calculated to obtain the clustering distance; for pixels in the upper and lower road regions, the average distance between the feature vectors of the two classes is calculated to obtain the dissimilar separation distance. The hierarchical contrast loss value is the weighted sum of the clustering distance and the dissimilar separation distance. By minimizing the hierarchical contrast loss value, the occlusion detection model learns more discriminative hierarchical features, thereby improving the recognition accuracy of upper and lower roads.

[0105] When using road occlusion training data for supervised training, the data is divided into a training set and a validation set in a 7:3 ratio. The training set is used for parameter updates, and the validation set is used to evaluate the performance of the occlusion detection model. In this embodiment, the stochastic gradient descent algorithm is used to optimize the network parameters, with an initial learning rate of 0.001, which decays to 0.1 every 5 epochs. During training, the training set loss and validation set loss are monitored in real time. When the validation set loss does not decrease for 10 consecutive epochs, the occlusion detection model is considered to have converged, and training is stopped, resulting in a well-trained occlusion detection model. Finally, in the trained occlusion detection model, the variance of the prediction results is calculated by performing multiple forward propagations on the same input during the inference phase. The reciprocal of the variance is used as the prediction confidence. The smaller the variance, the higher the certainty and confidence of the occlusion detection model in predicting the results.

[0106] The multimodal data block corresponding to the local region of each pseudo-cross node is input into the trained occlusion detection model. The multi-channel image local block, trajectory density block, and hierarchical feature vector in the multimodal data block are combined into an input tensor according to the format required by the occlusion detection model. The multi-channel image local block and trajectory density block maintain the same spatial dimension, and the hierarchical feature vector is integrated into the feature fusion layer of the occlusion detection model as an additional feature, together providing multi-source basis for hierarchical judgment.

[0107] The occlusion detection model performs feature extraction and hierarchical segmentation on the input data. Specifically, it processes image, trajectory features, and hierarchical feature vectors through a three-branch network, and strengthens the responses to shadow boundaries, trajectory discontinuities, and street view markers through a hierarchical attention mechanism. The final output consists of two types of results: an upper road mask and a lower road mask. (See also...) Figure 5 The images shown are all binary raster images. A pixel of 1 indicates that the image belongs to the corresponding road level, and a pixel of 0 indicates that the image does not belong to the corresponding road level. Figure 5 White represents 1 and black represents 0; the second is the prediction confidence level, ranging from 0 to 1, reflecting the reliability of the segmentation results.

[0108] If the prediction confidence is higher than the preset confidence threshold, and the percentage of pixels intersecting the upper and lower road masks is less than the preset intersection threshold, the intersection type of the corresponding intersection node is marked as a three-dimensional cross-layer and as a true intersection node; if the percentage of pixels intersecting the upper and lower road masks is greater than or equal to the preset intersection threshold, or if there is only a single-level mask without upper and lower layer distinction, the intersection type of the corresponding intersection node is marked as a planar intersection.

[0109] If the prediction confidence is lower than a preset confidence threshold, the street view image sequence of the corresponding pseudo-intersection node's local area is called. Combining the structural features in the street view feature mapping table and the road sign text, the hierarchical mask output by the occlusion detection model is corrected. The correction method includes at least the following: if a bridge pier exists in the street view image sequence and is located below a road, the road pixels in the corresponding hierarchical mask are marked as upper-layer road pixels, and the road pixels in another hierarchical mask are marked as lower-layer road pixels; if the road sign text contains a tunnel exit and shows a connection to a ground road, the intersection type of the corresponding pseudo-intersection node is corrected from planar intersection to three-dimensional cross-layer. For example, if a bridge pier clearly exists in the street view and is located below a road, the road can be determined to be upper-layer, and the pixel range of the corresponding hierarchical mask is adjusted; if the road sign text shows a "tunnel exit" connecting to a ground road, the corresponding local area can be confirmed as a transition between upper and lower layers, and the intersection type of the corresponding pseudo-intersection node is corrected. The process and results of manual review are recorded in the manual review and correction record.

[0110] The preset confidence threshold is determined based on road occlusion training data. Specifically, it involves statistically analyzing the matching rate between the model output and manually labeled data under different candidate confidence levels, and selecting the candidate value that achieves the preset target (e.g., 90%) as the confidence threshold. The preset intersection threshold is determined based on manually labeled samples in the training data that are three-dimensional cross-layers. Specifically, it involves calculating the percentage of intersection pixels between the upper and lower road masks of such samples, i.e., the ratio of the number of intersection pixels to the total number of pixels in the two masks, and taking the maximum value of the percentage of intersection pixels as the intersection threshold. This ensures that the percentage of intersection pixels in real three-dimensional cross-layers is less than this value, while the percentage of intersection pixels in planar intersections or only single-layer masks is greater than or equal to this value.

[0111] For results determined to be three-dimensional cross-layer, the shadow boundaries of the upper road are further extracted from the local blocks of the multi-channel image, and the trajectory range of the lower road is extracted from the trajectory density block, such as connected regions with trajectory density values ​​greater than or equal to 0.5. These spatial ranges are then associated with and stored with the hierarchical mask.

[0112] The final output set of judgment results includes the upper road mask, lower road mask, corresponding prediction confidence, and intersection type of the pseudo-intersection node for the local area corresponding to each pseudo-intersection node; the manual review and correction record includes the original judgment of the low confidence area, the corrected result, and the corresponding street scene feature basis, ensuring that the entire discrimination process is traceable.

[0113] The road generation module is used to associate the initial road network topology with the judgment result set to form a complete hierarchical road network.

[0114] The initial road network topology is correlated and matched with the judgment result set. The initial road network topology is a vector structure composed of a centerline network, including all pseudo-intersection nodes and the road connections between them. The judgment result set is marked as true intersection nodes that span three layers, and the corresponding upper and lower layer road ranges can be determined through a hierarchical mask. Based on the spatial distribution of road pixels in the hierarchical mask, it is determined whether the corresponding pseudo-intersection node belongs to the centerline of the upper layer road or the centerline of the lower layer road.

[0115] For true intersection nodes marked as three-dimensional cross-levels, a topology correction operation is performed to obtain corrected nodes. Specifically, the direct connection between the centerlines of the upper and lower level roads at the corresponding true intersection nodes is deleted from the initial road network topology. This makes the centerlines of the two levels of roads independent of each other at the corresponding true intersection nodes, but maintaining continuity. That is, the centerline of the upper level road extends from one side of the node to the other, and the same applies to the lower level road, only canceling the cross connection between the upper and lower levels. At the same time, a virtual associated node is introduced at the location of the corresponding true intersection node. The virtual associated node does not participate in the actual road network traffic connection; it is only used to record spatial hierarchy relationships, including the associated upper level road ID, lower level road ID, and hierarchy type, to identify the spatial vertical position relationship between the two levels of roads.

[0116] After completing the topology correction, the corrected road network topology is obtained. The correction effect is verified using a trajectory density map. Specifically, the centerline of each road in the corrected road network topology is traversed, and the continuity of the trajectory density values ​​of the grid within a preset width range on both sides of the centerline is calculated. If the trajectory density value of a certain road segment shows a significant abrupt change at the correction node, for example, it suddenly drops from a high value area corresponding to the upper road to near zero, and then suddenly rises to a high value area in the lower road area, without any transitional trajectory density distribution in between, then the corresponding correction node is determined to be a trajectory jump node, and the location of the corresponding correction node is marked as a potential ramp point. Its geographical coordinates and the associated upper and lower road IDs are recorded, indicating that there is a ramp connecting the two roads at the location of the corresponding correction node.

[0117] The final corrected road network topology output is a vector network including independent centerlines at each level, removed pseudo-intersection nodes, and virtual associated nodes; the potential ramp point list contains the coordinates of all marked trajectory jump nodes and associated road information, providing locational basis for subsequent road network connectivity completion.

[0118] The process involves centerline completion for potential ramp points. Specifically, each corrected node recorded in the potential ramp point list is associated with the IDs and geographic coordinates of the upper and lower roads. Starting from the coordinates of each potential ramp point, the nearest upper road centerline endpoint and the nearest lower road centerline endpoint are extracted from the corrected road network topology. These endpoints are then used as the start and end points for path searching. The Dijkstra's shortest path algorithm is run to calculate the optimal connection path in the area between the nearest upper road centerline endpoint and the nearest lower road centerline endpoint, excluding existing roads. The algorithm's cost function considers the smoothness of the path and the angle with surrounding roads; for example, it prioritizes routes with the least curvature and connections with upper and lower roads forming angles of 30° to 60°. Meanwhile, by retrieving street view signs for areas that are not existing roads through the street view feature mapping table, if there are road sign texts related to "ramp" or structural features unique to ramps such as curved guardrails and gradient widths, the path direction is adjusted according to the spatial distribution of the corresponding road sign texts or structural features so that the completed ramp centerline fits the actual physical location.

[0119] For other road segments not marked as potential ramp points but with centerline breaks in the corrected road network topology, a completion method based on road orientation and trajectory trends is used. Specifically: First, the extension direction of the road centerlines on both sides of the centerline break is determined by the slope of the line segment. If the orientations on both sides are consistent, for example, the angle is less than 10°, a straight line segment is generated to connect the break point. If there is a certain angle, for example, between 10° and 45°, a cubic Bézier curve is used to generate a smooth curve segment, making the curvature transition of the connecting line segment natural. At the same time, the trajectory trend corresponding to the centerline break point is referenced in the trajectory density map. If the trajectory density on both sides of the centerline break point shows a continuous distribution, for example, a slow transition from a high-value area on one side of the break point to the other side, the position of the completed line segment is adjusted to align it with the center of the trajectory density area, ensuring that the completed road conforms to the actual driving path of the vehicle.

[0120] After completing all completion operations, the connectivity of the entire road network is verified. The road network is treated as a graph structure composed of nodes and edges, where nodes are true intersection nodes and edges are centerline segments. A breadth-first search algorithm is used to traverse all nodes and count the number of connected components. A connected component is a subnetwork within the road network's graph structure, consisting of interconnected true intersection nodes and road centerline segments. Any two intersection nodes within a subnetwork can be connected by road segments, and this subnetwork has no road segment connections to other subnetworks. If a connected component exists that has no connecting segments to other roads, the distance between the corresponding connected component and surrounding roads is checked. If the distance is less than a preset distance threshold (e.g., 50 meters), a supplementary connecting segment is generated; if the distance is greater than or equal to the preset distance threshold, it is marked as an abnormal road segment awaiting manual verification. Ultimately, it is ensured that all road segments in the completed road network belong to the same connected component, and there are no isolated road segments.

[0121] The final output layered road network includes all completed ramp centerlines, connecting segments at interruptions, and the original centerlines of each level of road. Virtual associated nodes are retained in the network to record hierarchical relationships, and all segments are classified and labeled according to the hierarchical attributes of elevated, ground, and tunnel.

[0122] Example 2:

[0123] Please see Figure 2 As shown, this embodiment provides a method for determining the road centerline based on deep neural network satellite imagery, including:

[0124] Preprocessing is performed on the collected temporal satellite images, vehicle GPS trajectory data and street view image sequences of the area to be processed to obtain the dynamic region, trajectory density map and street view feature mapping table in the area to be processed;

[0125] The temporal satellite imagery is input into a pre-trained road segmentation model to obtain multiple road probability maps. Based on these multiple road probability maps, dynamic region weighted fusion, non-dynamic region logical OR operation, and threshold determination are performed to obtain an initial road mask.

[0126] The initial road mask is vectorized into a centerline network and intersection nodes are extracted to obtain the initial road network topology; based on the trajectory density map and street scene feature mapping table, intersection nodes that meet the preset conditions are marked as pseudo-intersection nodes;

[0127] Based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables, multimodal data blocks of the local area corresponding to each pseudo-crossing node are extracted;

[0128] Input the multimodal data block into the pre-trained occlusion detection model to obtain the judgment result set;

[0129] By associating the initial road network topology with the decision result set, a complete hierarchical road network is formed.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0131] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for determining the centerline of a road based on satellite imagery using a deep neural network, characterized in that: include: Preprocessing is performed on the collected temporal satellite images, vehicle GPS trajectory data and street view image sequences of the area to be processed to obtain the dynamic region, trajectory density map and street view feature mapping table in the area to be processed; The temporal satellite imagery is input into a pre-trained road segmentation model to obtain multiple road probability maps. Based on these multiple road probability maps, dynamic region weighted fusion, non-dynamic region logical OR operation, and threshold determination are performed to obtain an initial road mask. The initial road mask is vectorized into a centerline network and intersection nodes are extracted to obtain the initial road network topology; based on the trajectory density map and street scene feature mapping table, intersection nodes that meet the preset conditions are marked as pseudo-intersection nodes; Based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables, multimodal data blocks of the local area corresponding to each pseudo-crossing node are extracted; Input the multimodal data block into the pre-trained occlusion detection model to obtain the judgment result set; The structure of the occlusion detection model includes: The occlusion detection model is a three-branch network consisting of a main branch, an auxiliary branch, and a feature enhancement branch; the input of the main branch is a local image patch; the input of the auxiliary branch is a trajectory density block corresponding to the spatial location of the local image patch; and the input of the feature enhancement branch is a hierarchical feature vector. The bottom feature extraction layer of the three-branch network is used to extract the corresponding features of each branch. For local image patches, image features are extracted; for trajectory density features, trajectory features are extracted; and for... Hierarchical feature vectors are used to extract street scene features. Image features, trajectory features, and street view features are input into the upper layer of the network of each branch to obtain image detail features, trajectory detail features, and street view detail features, respectively. Image detail features, trajectory detail features, and street view detail features are input into the feature fusion layer to obtain comprehensive features; The final output is a hierarchical mask and the corresponding prediction confidence. The hierarchical mask includes an upper road mask and a lower road mask, both of which are binary raster images. A road pixel of 1 indicates that it belongs to the corresponding level road, and a road pixel of 0 indicates that it does not belong to the corresponding level road. Based on the predicted confidence level and the preset confidence level threshold, the cross-node cross-type is labeled to obtain the judgment result set; By associating the initial road network topology with the decision result set, a complete hierarchical road network is formed.

2. The method for determining the road centerline based on satellite imagery using a deep neural network according to claim 1, characterized in that, The initial road mask is a binary raster image generated by fusing road detection results from multi-temporal satellite imagery, where areas with a pixel value of 1 represent areas identified as roads, and areas with a pixel value of 0 represent non-road areas. Methods for obtaining the initial road mask include: For dynamic regions, the weights of each phase satellite image are assigned according to the acquisition time of the phase satellite image and a preset weight allocation method. The road probability of each pixel in the dynamic region is the sum of the road probabilities of the corresponding pixels in each phase satellite image multiplied by the corresponding weight. If the sum is greater than or equal to the set road determination threshold, the corresponding pixel is determined to be a road pixel. For non-dynamic regions, when the road probability of a pixel in the road probability map of any temporal satellite image is greater than or equal to the set road determination threshold, the corresponding pixel is determined to be a road pixel; when the road probability values ​​of all temporal satellite images are less than the set road determination threshold, the corresponding pixel is determined to be a non-road pixel. By fusing all road pixels in the road probability map, an initial road mask is obtained.

3. The method for determining the road centerline based on deep neural network satellite imagery according to claim 1, characterized in that, Methods for obtaining the dynamic region within the region to be processed include: Acquire temporal satellite images covering different lighting conditions, and after radiometric correction, atmospheric correction and geometric registration, select temporal satellite images within a preset time interval; By using the change vector analysis algorithm, the radiation level of satellite images from one time phase to the previous time phase is corrected in adjacent time phase satellite images; The difference in spectral values ​​of each band of satellite images in two adjacent time phases is calculated to form multiple change vectors for the corresponding pixels. Candidate change thresholds are selected based on the value range of the change vectors. The inter-class variance of the candidate change thresholds is maximized to determine the optimal change threshold. If the magnitude of any change vector of the corresponding pixel is greater than the optimal change threshold, the corresponding pixel is determined to be a changed pixel. All changed pixels are morphologically filtered to remove isolated changed pixels and adjacent changed pixels are merged to obtain the changed region. The edge pixels of the changing region are connected into a closed polygon boundary by a polygon fitting algorithm. The area enclosed by the polygon boundary is the dynamic region.

4. The method for determining the road centerline based on deep neural network satellite imagery according to claim 1, characterized in that, Methods for marking intersection nodes that meet preset conditions as pseudo-intersection nodes include: The preset conditions include: The intersection node is located within the area marked on the uncertain region map; If the high-value areas of trajectory density of two roads in the initial road network topology do not overlap at the intersection node and the high-value area of ​​trajectory density of one road is interrupted at the intersection node, then the high-value area of ​​trajectory density of the other road starts from the intersection node; the high-value area of ​​trajectory density is the area in the trajectory density map where the grid value is greater than the preset grid threshold. By querying the street view feature mapping table, we can find the street view features within a set range around the intersection node and identify the existence of upper and lower layer road sign text or structural features. If a cross node meets at least two of the preset conditions, it is marked as a pseudo cross node; otherwise, it is marked as a true cross node.

5. The method for determining the road centerline based on deep neural network satellite imagery according to claim 4, characterized in that, The method for generating the uncertain region map includes: The consistency ratio of each pixel in the road probability map is obtained by dividing the number of times each pixel is identified as a road pixel in all road probability maps by the number of road probability maps. The consistency ratio ranges from 0 to 1. The regions containing pixels with a consistency ratio lower than a preset consistency threshold are marked as uncertain regions, and an uncertain region map is generated.

6. The method for determining the road centerline based on deep neural network satellite imagery according to claim 1, characterized in that, The method for extracting multimodal data blocks of the local region corresponding to each pseudo-crossing node based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables includes: Traverse each temporal satellite image in the spatiotemporally consistent image set, crop the image block corresponding to the local area of ​​each pseudo-cross node, and superimpose the image blocks from different temporal phases according to the channel dimension to form a multi-channel local block; the channel dimension is the dimension that carries specific information in the temporal satellite image. Based on the coordinate range of the local area corresponding to each pseudo-cross node, crop the raster data with the same coordinate range in the trajectory density map; Based on the coordinate range of the local area corresponding to each pseudo-intersection node, retrieve all road sign text and structural features within the coordinate range of the street view feature mapping table; construct a hierarchical feature vector based on the retrieved road sign text and structural features; the length of the hierarchical feature vector is determined by the preset number of feature types; Each pseudo-cross node corresponds to a local region multimodal data block consisting of a multi-channel image local block, a trajectory density block, and a hierarchical feature vector.

7. The method for determining the road centerline based on deep neural network satellite imagery according to claim 1, characterized in that, Methods for obtaining the judgment result set include: If the prediction confidence is higher than the preset confidence threshold, and the percentage of pixels intersecting the upper and lower road masks is less than the preset intersection threshold, the intersection type of the corresponding pseudo-intersection node is marked as a three-dimensional cross-layer intersection and as a true intersection node; if the percentage of pixels intersecting the upper and lower road masks is greater than or equal to the preset intersection threshold, or if there is only a single-level mask with no distinction between upper and lower layers, the intersection type of the corresponding intersection node is marked as a planar intersection. If the prediction confidence is lower than the preset confidence threshold, the street view image sequence of the local area of ​​the corresponding pseudo-intersection node is called, and the hierarchical mask output by the occlusion detection model is corrected by combining the structural features in the street view feature mapping table and the road sign text. The correction method includes at least the following: if there is a bridge pier in the street view image sequence and it is located below a certain road, the road pixels in the corresponding hierarchical mask are marked as upper-layer road pixels, and the road pixels in another hierarchical mask are marked as lower-layer road pixels; if the road sign text contains a tunnel exit and shows a connection to the ground road, the intersection type of the corresponding pseudo-intersection node is corrected from planar intersection to three-dimensional cross-layer. The final output set of judgment results includes the upper road mask, lower road mask, corresponding prediction confidence, and intersection type of the pseudo-intersection node for the local area corresponding to each pseudo-intersection node.

8. The method for determining the road centerline based on deep neural network satellite imagery according to claim 1, characterized in that, The area to be processed is a polygonal region in the latitude and longitude coordinate system of the geographical range to be processed.

9. A road centerline determination system based on deep neural network satellite imagery, used to implement the road centerline determination method based on deep neural network satellite imagery as described in any one of claims 1-8, characterized in that, include: The data preprocessing module is used to preprocess the collected temporal satellite images, vehicle GPS trajectory data and street view image sequences of the area to be processed, and to obtain the dynamic area, trajectory density map and street view feature mapping table in the area to be processed. The road recognition module is used to input temporal satellite imagery into a pre-trained road segmentation model to obtain multiple road probability maps. Based on the multiple road probability maps, dynamic region weighted fusion, non-dynamic region logical OR operation, and threshold determination are performed to obtain an initial road mask. The node labeling module is used to vectorize the initial road mask into a centerline network and extract the intersection nodes to obtain the initial road network topology; based on the trajectory density map and street scene feature mapping table, the intersection nodes that meet the preset conditions are labeled as pseudo-intersection nodes; The feature extraction module extracts multimodal data blocks of the local area corresponding to each pseudo-crossing node based on temporal satellite imagery, trajectory density maps, and street view feature mapping tables. The node determination module is used to input multimodal data blocks into the pre-trained occlusion detection model to obtain a determination result set; The structure of the occlusion detection model includes: The occlusion detection model is a three-branch network consisting of a main branch, an auxiliary branch, and a feature enhancement branch; the input of the main branch is a local image patch; the input of the auxiliary branch is a trajectory density block corresponding to the spatial location of the local image patch; and the input of the feature enhancement branch is a hierarchical feature vector. The bottom feature extraction layer of the three-branch network is used to extract the corresponding features of each branch. For local image patches, image features are extracted; for trajectory density features, trajectory features are extracted; and for hierarchical feature vectors, street scene features are extracted. Image features, trajectory features, and street view features are input into the upper layer of the network of each branch to obtain image detail features, trajectory detail features, and street view detail features, respectively. Image detail features, trajectory detail features, and street view detail features are input into the feature fusion layer to obtain comprehensive features; The final output is a hierarchical mask and the corresponding prediction confidence. The hierarchical mask includes an upper road mask and a lower road mask, both of which are binary raster images. A road pixel of 1 indicates that it belongs to the corresponding level road, and a road pixel of 0 indicates that it does not belong to the corresponding level road. Based on the predicted confidence level and the preset confidence level threshold, the cross-node cross-type is labeled to obtain the judgment result set; The road generation module is used to associate the initial road network topology with the judgment result set to form a complete hierarchical road network.

Images