An aerial image road extraction method, a storage medium and a computer program product

Abstract

This invention discloses a method, storage medium, and computer program product for road extraction from aerial imagery, belonging to the field of remote sensing image information extraction technology. The method includes: inputting aerial images of multiple types of roads to be detected into an aerial image road extraction model to obtain a road segmentation mask; the aerial image road extraction model includes an input module, a regressor module, a multi-scale feature extraction module, a decoding module, and an output module connected in sequence; the input module is used to input the aerial images of multiple types of roads to be detected into the regressor module; the regressor module is used to perform initial feature extraction on the aerial images of multiple types of roads to be detected to obtain initial features; the multi-scale feature extraction module is used to perform multi-scale feature extraction on the initial features to obtain enhanced features; the decoding module is used to decode the multi-scale features to obtain a road mask; the output module is used to output the road mask, which can improve the completeness and topological accuracy of the extraction of complex road networks.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image information extraction technology, and in particular to a method, storage medium, and computer program product for extracting roads from aerial images. Background Technology

[0002] Road networks from high-resolution remote sensing imagery are becoming increasingly important in numerous applications such as urban planning, autonomous driving, vehicle navigation, and geographic information analysis. These images possess high spatial resolution and complex road structures, containing a wealth of spatial features spanning a wide range of scales. Furthermore, because remote sensing imagery is captured by top-down cameras, it can be acquired from any direction, meaning that spatial features can exist in any orientation. Therefore, the ability to perform global contextual modeling of high-resolution remote sensing imagery and extract large-scale spatial features from multiple directions is crucial for dense prediction tasks.

[0003] Deep learning models based on CNNs or Transformers have been widely applied to road segmentation in high-resolution remote sensing imagery. CNNs, by performing convolutional operations on image data, can effectively extract semantic features from shallow to deep layers. However, CNNs have limited receptive fields, posing challenges when processing high-resolution images. Transformers are renowned for their ability to effectively capture global contextual information and model spatial dependencies through self-attention mechanisms, but Transformers are not always accurate in focusing on tubular patterns around each road image query pixel and are relatively slow on standard GPUs. Specifically, current state-of-the-art Transformer-based methods struggle to accurately model such topological features using self-attention mechanisms when dealing with elongated, tubular structures similar to road structures. Furthermore, due to the quadratic complexity of Transformers, inference speeds on standard GPUs are relatively slow. Training and inference of these models require significant computational resources, especially when dealing with large-scale datasets. These shortcomings make it difficult for models to effectively model common complex spatial patterns of roads in remote sensing imagery, such as irregular orientations, sharp bends, intersection topological changes, and discontinuous road segments obscured by vegetation, buildings, etc., resulting in insufficient completeness and accuracy in extracting road information in complex scenes.

[0004] In recent years, a novel approach called Mamba has been proposed. Mamba is a new deep learning architecture that utilizes a state-space model (SSM) to capture global semantic information with low computational complexity. Unlike the Transformer, the Mamba architecture has linear complexity, offering significant advantages when processing long sequences. However, the Mamba architecture is designed to process data along a specific direction, meaning that preceding data cannot establish connections with subsequent data. This directional limitation makes it less suitable for image data without a specific orientation where spatial relationships across dimensions are crucial. Summary of the Invention

[0005] The purpose of this invention is to provide a method, storage medium, and computer program product for road extraction from aerial imagery. Through the constructed aerial imagery road extraction model, global contextual features of various types of road aerial imagery can be extracted from multiple dimensions, and the model adaptively fits the tubular geometric features of roads, improving the completeness and topological accuracy of complex road network extraction. This invention is achieved through the following technical solutions.

[0006] In a first aspect, the present invention provides a method for road extraction from aerial imagery, comprising:

[0007] The aerial images of various types of roads to be detected are input into the aerial image road extraction model to obtain the road segmentation mask;

[0008] The aerial image road extraction model includes a sequentially connected input module, a regressor module, a multi-scale feature extraction module, a decoding module, and an output module.

[0009] The input module is used to input the various types of road aerial images to be detected into the regressor module;

[0010] The regressor module is used to extract initial features from the multi-type road aerial images to be detected, and obtain initial features.

[0011] The multi-scale feature extraction module is used to perform multi-scale feature extraction on the initial features to obtain enhanced features;

[0012] The decoding module is used to decode the multi-scale features to obtain the road mask;

[0013] The output module is used to output the road mask.

[0014] In practical applications, extracting roads from satellite imagery to update the dynamic changes of road networks has become a research hotspot in recent years. To overcome the shortcomings of the aforementioned background technologies, this invention improves upon the original Mamba architecture. To better extract road shape information and model its directional context, the Conv and SSM modules of the original Mamba architecture are replaced with a tracking convolutional subunit TConv and a tracking selective scan subunit TSSM, used to model the global context of the image from multiple directions. The improved Mamba architecture is named the tracking state space unit Tamba, retaining the efficiency of the original Mamba while enhancing its focus on the tubular patterns of road structures. These two subunits together enable the tracking state space unit Tamba to handle road extraction tasks more accurately and efficiently. Experiments will demonstrate the effectiveness of this method in this task and its superior performance on multiple remote sensing benchmark datasets.

[0015] Optionally, the cutoff distance function learned by the regressor module is calculated using the following formula:

[0016] ,

[0017] In the formula, For random pixels in aerial images of various road types, This is a function representing the cutoff distance from random pixels in aerial images of various road types to the road centerline. This is the width threshold from the road centerline to the road edge. This represents the distance from a random pixel in a multi-type aerial image of a road to the road's centerline. This is the exponential decay coefficient.

[0018] Optionally, the initial features include pixels in multi-type road aerial imagery that fall within a width threshold from the road centerline to the road edge.

[0019] In this invention, the initial feature is a feature with an additional truncation distance channel dimension added to the original feature map, that is, to the original features of the input multi-type road aerial image to be detected. The truncation distance refers to the distance from a pixel in the multi-type road aerial image to the road centerline. When this distance is within the width threshold from the road centerline to the road edge, it is considered a road pixel and has a non-zero value; otherwise, it is determined to be a background pixel and has a value of 0.

[0020] Optionally, the multi-scale feature extraction module includes multiple sequentially connected tracking state space units (Tamba), each Tamba comprising a first layer of normalization subunits, parallel branches, and a first layer of linear transformation subunits connected sequentially.

[0021] The parallel branch includes a first branch and a second branch connected in parallel. The first branch includes a second-layer linear transformation subunit, a tracking convolution subunit TConv, a tracking selective scan module subunit TSSM, and a second-layer normalization subunit connected in sequence. The second branch includes a third-layer linear transformation subunit.

[0022] Optionally, the tracking state space unit Tamba performs the following operations:

[0023] The initial features are input into the first layer-level normalization subunit for standardization along the channel dimension to obtain the first layer-level normalized features. The first layer-level normalized features are input into the second layer-level linear transformation subunit for linear projection operation to adjust the channel dimension, to obtain the first projection features. The first projection features are input into the tracking convolution subunit TConv for deformable convolution operation to obtain geometrically aware features. The geometrically aware features are input into the tracking selective scanning module subunit TSSM for multi-directional scanning to obtain features that fuse multi-directional global context information. The features that fuse multi-directional global context information are input into the second layer-level normalization subunit for standardization along the channel dimension to obtain the second layer-level normalized features.

[0024] The first layer of normalized features is input into the third layer of linear transformation subunit to perform a linear projection operation to adjust the channel dimension, resulting in the second projection feature; the second projection feature and the second layer of normalized features are then multiplied element-wise to obtain the interactive feature.

[0025] The interactive features are input into the first-layer linear transformation subunit to perform channel dimension transformation operation, thereby obtaining the transformed features;

[0026] The transformed features are added element-wise to the initial features to obtain the final enhanced features.

[0027] Optionally, the step of inputting the first projection feature into the tracking convolution subunit TConv for deformable convolution operation to obtain geometrically perceptual features includes: the tracking convolution subunit dynamically generates a sampling offset based on the first projection feature, adaptively adjusts the sampling position of the standard convolution kernel based on the sampling offset, and performs deformable convolution operation to extract structured features that fit the road geometry and obtain geometrically perceptual features.

[0028] Optionally, the multi-directional scanning refers to the tracking selective scanning module subunit TSSM scanning the geometric sensing features along the horizontal, vertical, diagonal, anti-diagonal directions and their respective reverse directions.

[0029] Optionally, the step of inputting geometric perception features into the Tracking Selective Scanning Module (TSSM) subunit for multi-directional scanning to obtain features that integrate multi-directional global context information includes: inputting geometric perception features into the TSSM subunit for multi-directional scanning to obtain multiple geometric perception feature sequences, inputting multiple geometric perception feature sequences into the existing state space model (SSM) for multi-directional feature extraction, and adding the extracted features to obtain features that integrate multi-directional global context information.

[0030] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, when the computer program / instructions are executed by a processor, they implement the steps of the aerial image road extraction method described in the first aspect.

[0031] Thirdly, the present invention provides a computer program product, including a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, they implement the steps of the aerial image road extraction method described in the first aspect.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] The aerial image road extraction method provided by this invention can extract tubular geometric features that fit the road from aerial images of various types of roads by constructing an aerial image road extraction model, thereby extracting more accurate road information;

[0034] The multi-scale feature extraction module introduced in this invention replaces the Convolutional Subunit (Conv) and State Space Model (SSM) of the original Mamba architecture with the Tracking Convolutional Subunit (TConv) and the Tracking Selective Scanning Module (TSSM). This module can model the global context of various types of road aerial imagery from multiple dimensions and adaptively fit the tubular geometric features of roads, improving the extraction completeness and topological accuracy of complex road networks. Furthermore, this invention reconstructs the road information segmentation task into a centerline distance regression problem through a regressor module, effectively alleviating the pixel class imbalance problem in road aerial imagery and enhancing the robustness of extraction for irregular and occluded roads. The regressor module adopts a lightweight and differentiable design, making it easy to integrate into existing segmentation frameworks and possessing good scalability and transfer potential. Simulation experiments demonstrate that the proposed aerial imagery road extraction model achieves segmentation accuracy comparable to or even higher than advanced Transformer models while maintaining the linear computational complexity of the state space model. It also boasts faster inference speed and lower computational overhead, providing a feasible solution for real-time processing of large-scale, high-resolution remote sensing images. Attached Figure Description

[0035] Figure 1 The diagram shown is a schematic flowchart of an aerial image road extraction method in one embodiment of the present invention.

[0036] Figure 2 The diagram shown is a schematic diagram of the aerial image road extraction model structure in one embodiment of the present invention;

[0037] Figure 3 The diagram shown is a comparison of simulation results of different segmentation networks on a multi-type road remote sensing dataset in one embodiment of the present invention. Detailed Implementation

[0038] The following description, in conjunction with the accompanying drawings and specific embodiments, provides further details. In this description, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature.

[0039] Example 1 This embodiment describes a method for road extraction from aerial imagery, such as... Figure 1 As shown, it includes:

[0040] The aerial images of various types of roads to be detected are input into the aerial image road extraction model to obtain the road segmentation mask;

[0041] The aerial image road extraction model includes a sequentially connected input module, a regressor module, a multi-scale feature extraction module, a decoding module, and an output module.

[0042] The input module is used to input the various types of road aerial images to be detected into the regressor module;

[0043] The regressor module is used to extract initial features from the multi-type road aerial images to be detected, and obtain initial features.

[0044] The multi-scale feature extraction module is used to perform multi-scale feature extraction on the initial features to obtain enhanced features;

[0045] The decoding module is used to decode the enhanced features to obtain the road mask;

[0046] The output module is used to output the road mask.

[0047] In practical applications, extracting roads from satellite imagery to update the dynamic changes of road networks has become a research hotspot in recent years. To overcome the shortcomings of the aforementioned background technologies, this invention improves upon the original Mamba architecture. To better extract road shape information and model its directional context, the Conv and SSM modules of the original Mamba architecture are replaced with tracking convolutional subunits TConv and tracking selective scan subunits TSSM, used to model the global context of the image from multiple directions. The improved Mamba architecture is named the tracking state space unit Tamba, retaining the efficiency of the original Mamba while enhancing its focus on the tubular patterns of road structures. These two subunits together enable the tracking state space unit Tamba to handle road extraction tasks more accurately and efficiently. Experiments will demonstrate the effectiveness of this method in this task and its superior performance on multiple remote sensing benchmark datasets.

[0048] Example 2

[0049] Based on Example 1, this example describes the specific implementation process of an aerial image road extraction method, which includes the following:

[0050] I. Road Extraction Model from Aerial Imagery

[0051] In one specific embodiment of the present invention, such as Figure 2 The diagram shows the structure of the road extraction model from aerial imagery. Figure 2 In this context, Input represents the input module, Stem represents the regressor module, Tamba Block represents the tracking state space unit Tamba, four serially connected Tamba Blocks represent the multi-scale feature extraction module, UpperNet represents the decoding module, Output represents the output module, LN represents the layer normalization sub-unit, and Linear represents the linear transformation sub-unit.

[0052] In one specific embodiment of this invention, due to the irregularity and complexity of road structures, road segmentation based on classification is inaccurate. This invention solves the road segmentation problem by reconstructing road centerline detection and calculating distance maps, transforming the classification problem into a regression problem. Specifically, it first acquires pixels from aerial images of various road types that fall within a width threshold from the road centerline to the road edge. Pixels not falling within this threshold range are considered background points and are filtered out.

[0053] In one specific embodiment of the present invention, the cutoff distance function learned by the regressor module is calculated using the following formula:

[0054] ,

[0055] In the formula, For random pixels in aerial images of various road types, This is a function representing the cutoff distance from random pixels in aerial images of various road types to the road centerline. This is the width threshold from the road centerline to the road edge. This represents the distance from a random pixel in a multi-type aerial image of a road to the road's centerline. This is the exponential decay coefficient.

[0056] In this embodiment, the initial feature is a feature with an additional truncation distance channel dimension added to the original feature map, that is, to the original features of the input multi-type road aerial image to be detected. The truncation distance refers to the distance from a pixel in the multi-type road aerial image to the road centerline. When this distance is within the width threshold from the road centerline to the road edge, it is considered a road pixel and has a non-zero value; otherwise, it is determined to be a background pixel and has a value of 0.

[0057] In practical applications, the initial features, including the additional truncation distance dimension, are first extracted using the above formula.

[0058] In one specific embodiment of the present invention, the multi-scale feature extraction module includes multiple sequentially connected tracking state space units (Tamba), and the tracking state space unit (Tamba) includes a first layer normalization subunit, a parallel branch, and a first layer linear transformation subunit connected in sequence.

[0059] The parallel branch includes a first branch and a second branch connected in parallel. The first branch includes a second-layer linear transformation subunit, a tracking convolution subunit TConv, a tracking selective scan module subunit TSSM, and a second-layer normalization subunit connected in sequence. The second branch includes a third-layer linear transformation subunit.

[0060] In one specific embodiment of the present invention, the tracking state space unit Tamba performs the following operations:

[0061] The initial features are input into the first layer-level normalization subunit for standardization along the channel dimension to obtain the first layer-level normalized features. The first layer-level normalized features are input into the second layer-level linear transformation subunit for linear projection operation to adjust the channel dimension, to obtain the first projection features. The first projection features are input into the tracking convolution subunit TConv for deformable convolution operation to obtain geometrically aware features. The geometrically aware features are input into the tracking selective scanning module subunit TSSM for multi-directional scanning to obtain features that fuse multi-directional global context information. The features that fuse multi-directional global context information are input into the second layer-level normalization subunit for standardization along the channel dimension to obtain the second layer-level normalized features.

[0062] The first layer of normalized features is input into the third layer of linear transformation subunit to perform a linear projection operation to adjust the channel dimension, resulting in the second projection feature; the second projection feature and the second layer of normalized features are then multiplied element-wise to obtain the interactive feature.

[0063] The interactive features are input into the first-layer linear transformation subunit to perform channel dimension transformation operation, thereby obtaining the transformed features;

[0064] The transformed features are added element-wise to the initial features to obtain the final enhanced features.

[0065] In one specific embodiment of the present invention, the first projection feature is input to the tracking convolution subunit TConv for deformable convolution operation to obtain geometric perception features. This includes: the tracking convolution subunit dynamically generates a sampling offset based on the first projection feature, adaptively adjusts the sampling position of the standard convolution kernel based on the sampling offset, and performs deformable convolution operation to extract structured features that fit the road geometry and obtain geometric perception features.

[0066] In one specific embodiment of the present invention, multi-directional scanning refers to the tracking selective scanning module subunit TSSM scanning the geometric sensing features along the horizontal, vertical, diagonal, anti-diagonal directions and their respective reverse directions.

[0067] In one specific embodiment of the present invention, the step of inputting geometric perception features into the Tracking Selective Scanning Module (TSSM) subunit for multi-directional scanning to obtain features fused with multi-directional global context information includes: inputting geometric perception features into the TSSM subunit for multi-directional scanning to obtain multiple geometric perception feature sequences; inputting the multiple geometric perception feature sequences into an existing state space model (SSM) for multi-directional feature extraction; and adding the extracted features to obtain features fused with multi-directional global context information.

[0068] Combination Figure 2 The structure of the tracking state space unit Tamba is shown. The data flow described in this embodiment is based on the data flow of the first tracking state space unit Tamba. The data flows of the other three tracking state space units Tamba are similar.

[0069] from Figure 2As can be seen, the Selective Scanning Module (TSSM) subunit first scans the geometrically sensed features in eight directions: horizontal, vertical, diagonal, anti-diagonal, and their respective reverse directions, obtaining geometrically sensed feature sequences in these eight directions. These sequences are then input into the S6 block of the existing state-space model (SSM) for feature extraction. The extracted features are then summed to obtain features that incorporate global contextual information from multiple directions. Furthermore, the four tracking state-space units (Tamba) extract abstract semantic features at different levels: shallow layers capture local details and low-level features, while deep layers understand high-level semantics and global context, ultimately improving the accuracy and completeness of road extraction. This approach can extract multi-dimensional, multi-type road aerial image features, thereby improving the completeness and accuracy of road information extraction in complex scenes. It also overcomes the limitation of the existing Mamba architecture, which can only process data along a single direction, resulting in ineffective interaction of information between preceding and following positions in the sequence. Figure 2 The numbers 1-9 in the text mean that the geometric sensing features of the Selective Scanning Subunit (TSSM) of the Input Tracking module are divided into 9 blocks.

[0070] II. Simulation Experiment

[0071] 2.1 Dataset

[0072] The simulation experiments used DeepGlobe, SpaceNet, and a large-scale road dataset. The DeepGlobe road dataset is the benchmark dataset for the Satellite Image Understanding Challenge, containing 6226 images with pixel-level annotations from India, Indonesia, and Thailand. The image size is 1024×1024 pixels, with a spatial resolution of 0.5 meters per pixel. The SpaceNet road dataset uses 2549 images from four cities: Shanghai, Paris, Khartoum, and Las Vegas. It employs line segment labeling to represent the centerlines of unpaved roads, paved roads, and dirt roads, respectively. The image size is 1300×1300 pixels, with a spatial resolution of 0.3 meters per pixel.

[0073] 2.2 Evaluation Indicators and Simulation Conditions

[0074] The evaluation metrics used in the experiments were overall accuracy (OA), precision (P), recall (R), F1 score, and intersection-over-union (IoU). All networks shared the same training and testing framework, implemented in PyTorch 1.8.1, and ran on a standard workstation equipped with an NVIDIA GeForce RTX 2080 Ti® GPU, an Intel® Core™ i9-10920X (3.50GHz) CPU, and 64GB of RAM. The Adam optimizer was used with an initial learning rate of 0.0001. The learning rate was halved when the validation set loss did not decrease for 10 consecutive epochs. All models were trained for 200 epochs with a batch size of 2, using a combination of mean squared error loss and dice loss as the loss function.

[0075] Tables 1 and 2 present the quantitative results of simulation experiments on the DeepGlobe and SpaceNet road datasets for the method of this invention and its comparative methods. Experimental results show that, under the same parameter settings, the method of this invention, by introducing a regressor module, a tracking convolution subunit, and a tracking selective scan module subunit, significantly improves the F1-score and IoU values. The "Proposed" section in the table represents the aerial image road extraction method proposed in this invention. Compared to U-Net variants (D-LinkNet, RecurrUNet, DSCNet, and OARENet in the table), existing Transformer encoding / decoding segmentation models (SegFormer, RoadFormer, and UCTransNet in the table), and the latest Mamba model (RS-Mamba in the table), it improves the F1-score values ​​on the two datasets by 9.47 / 14.03 / 10.77 / 1.73 / 0.68 / 8.30 / 6.45 / 4.57 and 20.80 / 20.37 / 13.44 / 3.48 / 8.26 / 13.45 / 14.84 / 1.70, respectively. The IoU values ​​were improved by 9.47 / 14.03 / 10.77 / 1.73 / 0.68 / 8.30 / 6.45 / 4.57 and 26.63 / 26.13 / 17.97 / 2.96 / 11.71 / 18.10 / 19.61 / 2.44, respectively. This demonstrates that the aerial image road extraction method proposed in this invention significantly surpasses the comparative methods in improving road segmentation accuracy.

[0076] Table 3 compares the computational complexity of the proposed method with other methods. The aerial image road extraction model proposed in this invention has 32.4M parameters and 28.8G FLOPs. Combining Tables 1-3, it can be seen that the proposed method achieves better simulation results in terms of F1-score and IoU value than segmentation methods (i.e., D-LinkNet, OARENet, and RS-Mamba) with similar parameter counts (M) and floating-point operations per second (FLOPs) (G). Furthermore, RecurrUNet's parameter / FLOPs ratio is 5.2 / 16.6 times that of this method. This demonstrates that the proposed method balances accuracy and computational efficiency, achieving a practical balance.

[0077] Table 1. Simulation comparison results of different methods on the DeepGlobe dataset.

[0078] Table 2. Simulation comparison results of different methods on the SpaceNet dataset.

[0079] Table 3 Comparison of parameter count and floating-point arithmetic results for different methods

[0080] A comparative illustration of simulation results of different segmentation networks on various types of road remote sensing datasets is shown below. Figure 3 As shown, Figure 3 In the diagram, (a) is the original image, i.e., the aerial image of the various types of roads to be detected; (b) is the segmentation effect of manual segmentation on the road dataset; (c) is the segmentation effect of D-LinkNet on the road dataset; (d) is the segmentation effect of RecurrUNet on the road dataset; (e) is the segmentation effect of DSCNet on the road dataset; (f) is the segmentation effect of OARENet on the road dataset; (g) is the segmentation effect of SegFormer on the road dataset; (h) is the segmentation effect of UCTransNet on the road dataset; (i) is the segmentation effect of RoadFormer on the road dataset; (j) is the segmentation effect of RS-Mamba on the road dataset; and (k) is the segmentation effect of the remote sensing image road extraction method proposed in this invention on the road dataset. Figure 3 As can be seen, the remote sensing image road extraction method proposed in this invention can effectively handle problems such as road breaks, occlusions, and shadows, producing continuous and complete extraction results that more closely resemble manual segmentation. In the above, the road datasets refer to the DeepGlobe dataset and the SpaceNet dataset. Figure 3In the diagram, the effects of the first and second rows are based on the DeepGlobe dataset, while the effects of the third and fourth rows are based on the SpaceNet dataset.

[0081] Example 3

[0082] This embodiment describes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aerial image road extraction method as described in Embodiment 1 or 2.

[0083] Example 4

[0084] This embodiment describes a computer program product. When the computer program / instruction is executed by a processor, it implements the steps of the aerial image road extraction method as described in Embodiment 1 or 2.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for road extraction from aerial imagery, characterized in that, include: The aerial images of various types of roads to be detected are input into the aerial image road extraction model to obtain the road segmentation mask; The aerial image road extraction model includes a sequentially connected input module, a regressor module, a multi-scale feature extraction module, a decoding module, and an output module. The input module is used to input the various types of road aerial images to be detected into the regressor module; The regressor module is used to extract initial features from the multi-type road aerial images to be detected, and obtain initial features. The multi-scale feature extraction module is used to perform multi-scale feature extraction on the initial features to obtain enhanced features; The decoding module is used to decode the enhanced features to obtain the road mask; The output module is used to output the road mask.

2. The aerial image road extraction method according to claim 1, characterized in that, The cutoff distance function learned by the regressor module is calculated using the following formula: ， In the formula, For random pixels in aerial images of various road types, This is a function representing the cutoff distance from random pixels in aerial images of various road types to the road centerline. This is the width threshold from the road centerline to the road edge. This represents the distance from a random pixel in a multi-type aerial image of a road to the road's centerline. This is the exponential decay coefficient.

3. The aerial image road extraction method according to claim 2, characterized in that, The initial features include pixels in multi-type road aerial imagery that fall within a width threshold from the road centerline to the road edge.

4. The aerial image road extraction method according to claim 1, characterized in that, The multi-scale feature extraction module includes multiple sequentially connected tracking state space units, each tracking state space unit including a first layer of normalization sub-units, parallel branches, and a first layer of linear transformation sub-units connected in sequence. The parallel branch includes a first branch and a second branch connected in parallel. The first branch includes a second-layer linear transformation subunit, a tracking convolution subunit, a tracking selective scan module subunit, and a second-layer normalization subunit connected in sequence. The second branch includes a third-layer linear transformation subunit.

5. The aerial image road extraction method according to claim 4, characterized in that, The tracking state space unit performs the following operations: The initial features are input into the first layer of normalization subunits for standardization along the channel dimension to obtain the first layer of normalized features; the first layer of normalized features are input into the second layer of linear transformation subunits for linear projection operation to adjust the channel dimension to obtain the first projection features. The first projection feature is input into the tracking convolutional subunit for deformable convolution operation to obtain geometrically perceptual features; The geometric perception features are input into the tracking selective scanning module subunit for multi-directional scanning to obtain features that integrate multi-directional global context information; the features that integrate multi-directional global context information are input into the second-layer normalization subunit for standardization processing along the channel dimension to obtain the second-layer normalized features. The first layer of normalized features is input into the third layer of linear transformation subunit to perform a linear projection operation to adjust the channel dimension, thus obtaining the second projection feature. Perform element-wise multiplication on the second projected features and the second-level normalized features to obtain the interactive features; The interactive features are input into the first-layer linear transformation subunit to perform channel dimension transformation operation, thereby obtaining the transformed features; The transformed features are added element-wise to the initial features to obtain the final enhanced features.

6. The aerial image road extraction method according to claim 5, characterized in that, The step of inputting the first projection feature into the tracking convolution subunit for deformable convolution operation to obtain geometric perception features includes: the tracking convolution subunit dynamically generates a sampling offset based on the first projection feature, adaptively adjusts the sampling position of the standard convolution kernel based on the sampling offset, and performs deformable convolution operation to extract structured features that fit the road geometry and obtain geometric perception features.

7. The aerial image road extraction method according to claim 5, characterized in that, The multi-directional scanning refers to the tracking selective scanning module subunit scanning the geometric sensing features along the horizontal, vertical, diagonal, anti-diagonal directions and their respective reverse directions.

8. The aerial image road extraction method according to claim 5, characterized in that, The step of inputting geometric perception features into the tracking selective scanning module subunit to perform multi-directional scanning to obtain features that integrate multi-directional global context information includes: inputting geometric perception features into the tracking selective scanning module subunit to perform multi-directional scanning to obtain multiple geometric perception feature sequences, inputting multiple geometric perception feature sequences into an existing state space model to perform multi-directional feature extraction, and adding the extracted features to obtain features that integrate multi-directional global context information.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the aerial image road extraction method according to any one of claims 1-8.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the aerial image road extraction method according to any one of claims 1-8.

Images