Dual convolution network road extraction method based on hypergraph and multi-level feature fusion

Abstract

This invention discloses a road extraction method based on hypergraphs and multi-level feature fusion using dual convolutional networks. It first constructs a road extraction model based on hypergraphs and multi-level feature fusion, then inputs HRSI (High-Resolution Indicator Sensor) into the model for road extraction prediction to obtain ground-based real road information. The extraction model includes a hypergraph construction module, a hypergraph feature extraction module, a multi-scale feature extraction module, and a dual-modal fusion module. This invention proposes a road extraction method based on hypergraphs and multi-level feature fusion using dual convolutional networks. This method combines the effectiveness of hypergraphs in modeling road network topology, the efficiency of hypergraph neural networks in extracting long-term contextual information, and the rich semantics extracted by multi-level feature fusion. It effectively addresses the current problem that road extraction accuracy still needs improvement and that road features are not utilized in data-driven methods, ensuring the stability and accuracy of the road extraction results.

Description

Technical Field

[0001] This invention is a road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network. It belongs to the field of high-resolution remote sensing image processing and is particularly suitable for road extraction based on hypergraph and multi-level feature fusion using a dual convolutional network. Background Technology

[0002] With the rapid development of remote sensing technology and continuous innovation in engineering techniques, more and more remote sensing images are playing a role in daily life and production. High-resolution remote sensing images (HRSI), as a part of remote sensing images, are widely used in many fields such as urban planning and land conservation. Roads, as an important feature in HRSI, can be accurately extracted for convenient and timely updates to map road networks, or to provide decision support during emergency earthquakes. Furthermore, with the booming development of intelligent driving technology, there is an increasing need for accurate and real-time road data to assist autonomous driving. Therefore, how to reliably and accurately extract roads from HRSI has become an urgent problem to be solved in the research field.

[0003] Traditional road extraction methods primarily utilize mathematical morphology and texture features for analysis and extraction. However, these methods suffer from low accuracy in complex road scenarios and require significant manual preprocessing and post-processing, making them unsuitable for HRSI road extraction. With the development of deep learning, methods based on deep convolutional neural networks (CNNs) are widely used in road extraction. These CNN-based methods can be broadly categorized into two types: ① prediction based on CNN models; and ② prediction based on attention mechanisms or Transformer models.

[0004] All the methods described above require feature normalization before inputting HRSI into the model. Method ①, CNN, can effectively process image data, but its structure, limited by regular convolutional kernels, cannot well match the long, winding features of roads, resulting in poor performance when extracting small and occluded road regions. Method ②, benefiting from positional encoding and fully connected graphs, can better capture long-distance features, improving semantic richness and thus enhancing road extraction results, but it requires more computation time and resources.

[0005] In summary, given the current shortcomings in road extraction accuracy and the lack of utilization of road features in data-driven methods, how to better leverage road structure and semantic information to improve road extraction accuracy is a key challenge and focus of research in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a road extraction method based on hypergraphs and multi-level feature fusion using dual convolutional networks. This method leverages the irregular structure of hypergraphs and the matching properties of road networks, the semantic richness of multi-scale features, and the efficiency of Hyper Graph Neural Networks (HGNN) models in extracting long-term contextual information, as well as the accuracy of deep convolutional neural network models in extracting multi-scale features. By incorporating prior knowledge of road shape into the deep learning model, this method improves the accuracy of road extraction and reduces computation time, addressing the issues of existing road extraction results having insufficient accuracy and failing to consider road features.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A road extraction method based on hypergraph and multi-level feature fusion using a dual-convolutional network (DCN) is proposed. This method first constructs a DCN road extraction model based on hypergraph and multi-level feature fusion. Then, HRSI (High-Speed ​​Index System) is input into the DCN road extraction model to perform road extraction and prediction, obtaining real-world road information. The DCN road extraction model based on hypergraph and multi-level feature fusion includes the following modules:

[0009] Hypergraph Construction Module: Employs the hypergraph construction method to construct a hypergraph structure with road features from HRSI;

[0010] Hypergraph feature extraction module: Employs hypergraph convolution to perform remote semantic extraction on the hypergraph structure constructed by the hypergraph construction module, thereby obtaining hypergraph-level semantic features;

[0011] The multi-scale feature extraction module uses an encoder-decoder structure to extract semantic features at multiple scales from HRSI.

[0012] The dual-modal fusion module connects the hypergraph feature extraction module and the multi-scale feature extraction module, fusing and transforming semantic features at the hypergraph level and semantic features at multiple scales to enhance the semantic richness of the features.

[0013] The aforementioned road extraction method based on hypergraph and multi-level feature fusion of dual convolutional networks further includes a data preprocessing module, which performs denoising and filtering on HRSI and normalizes the filtered HRSI and its corresponding label.

[0014] The aforementioned road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network is characterized by the following training steps for the road extraction model based on hypergraph and multi-level feature fusion:

[0015] Step 1: The data preprocessing module performs cropping, noise reduction, and filtering on the HRSI dataset;

[0016] Step 2: The data preprocessing module performs normalization operations on the dataset and corresponding labels after cropping, denoising, and filtering in Step 1;

[0017] Step 3: The hypergraph construction module performs feature analysis on the dataset after the normalization operation in Step 2, and constructs the dataset into a corresponding hypergraph structure dataset by combining the road shape features.

[0018] Step 4: The hypergraph feature extraction module uses hypergraph convolution to perform long-range semantic extraction on the hypergraph structure dataset from Step 3; the multi-scale feature extraction module uses a CNN-based method to train the hypergraph structure dataset from Step 3 and the HRSI dataset in parallel.

[0019] Step 5: The dual-modal fusion module fuses the intermediate results of the hypergraph feature extraction module and the multi-scale feature extraction module, and inputs the fused features into the dual-convolutional network road extraction model based on hypergraph and multi-level feature fusion for subsequent training.

[0020] The aforementioned road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network involves the cropping and denoising filtering process described in step 1, which involves cropping and dividing the original HRSI dataset acquired by the satellite into HRSI datasets of appropriate size, and then using RGB analysis to remove half-black, half-white, or all-black and all-white boundary images from the dataset.

[0021] The aforementioned dual-convolutional network road extraction method based on hypergraph and multi-level feature fusion uses zero-mean normalization in step 2 to accelerate model convergence.

[0022] The aforementioned road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network involves the following steps for constructing the hypergraph structure dataset:

[0023] (1) First, the HRSI dataset that has been normalized in step 2 is divided into L regular 2×2 pixel blocks from left to right and from top to bottom;

[0024] (2) Based on the obtained L pixel blocks, sequentially from each pixel block L i Starting from, select the pixel blocks to its right, below, and below right (L) i+1 ,L i+W / 2 L i+W / 2+1) as a neighboring node, where W is the width of HRSI;

[0025] (3) Using pixel block L i Based on this, and using the F-norm, the similarity between the node and its neighboring nodes is calculated sequentially. The calculation formula is as follows: F in public notice sub L represents i The F-norm of a node i and its neighbor node L j F-norm F j The difference;

[0026] (4) Based on the similarity between nodes, first determine its original node L. i Whether a node belongs to a certain superedge, if node L i Exists on some superedge e k , then sim ij Neighbor nodes with similarity greater than the similarity threshold r are added to the hyperedge e. k In the middle, if node L i If no hyperedge exists, then add a new hyperedge e in the hypergraph G. m and sim ij Neighbor nodes with similarity greater than the similarity threshold r are added to the hyperedge e. m middle;

[0027] (5) After traversing all starting pixel block nodes L i Afterwards, a hypergraph structure corresponding to HRSI is obtained.

[0028] The aforementioned road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network consists of a hypergraph feature extraction module composed of three layers of hypergraph convolution, and a multi-scale feature extraction module composed of a ResNet-50 encoder and a multi-scale decoder. Cross-entropy and Dice coefficients are used as loss functions, and the Adam optimizer is selected to train and optimize the model, with an initial learning rate of 0.001.

[0029] The aforementioned dual-convolutional network road extraction method based on hypergraph and multi-level feature fusion is described in that the process of the dual-modal fusion module is divided into fusion from hypergraph to image and fusion from image to hypergraph: the fusion from hypergraph to image is to restore the image structure by the reverse process of constructing the hypergraph structure using the hypergraph, and then merge the image structure with the semantic features of multiple scales in the multi-scale feature extraction module, and then use 1*1 convolution to extract features before inputting them into the subsequent process;

[0030] The image-to-hypergraph fusion process involves creating a hypergraph structure from the semantic features at multiple scales in the multi-scale feature extraction module using the hypergraph construction process, merging the hypergraph structure with the hypergraph-level semantic features in the hypergraph feature extraction module, extracting features using 1*1 convolution, and then inputting the extracted features into subsequent processes.

[0031] The aforementioned road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network is described in the context of real road information being a black-and-white binary image, with roads represented by RGB-255 and other ground features by RGB-0.

[0032] The aforementioned road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network involves the RGB analysis method reading the image as an RGB matrix, counting (255, 255, 255) or (0, 0, 0) pixels, and using thresholding to remove noise from the image.

[0033] The beneficial effects of this invention are as follows: This invention proposes a road extraction method based on hypergraphs and multi-level feature fusion using dual convolutional networks. This method combines the effectiveness of hypergraphs in modeling the topology of road networks, the efficiency of hypergraph neural networks in extracting long-term contextual information, and the rich semantics extracted by multi-level feature fusion. It can effectively solve the problem that the accuracy of current road extraction still needs to be improved and that road features are not utilized in data-driven methods, thus ensuring the stability and accuracy of road extraction results. Attached Figure Description

[0034] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0035] Figure 1 The flowchart of the road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network provided by this invention is shown below.

[0036] Figure 2 A model diagram of the road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network provided by this invention;

[0037] Figure 3 The prediction results for the Massachusetts dataset in Embodiment 1 of this invention;

[0038] Figure 4 This is the prediction effect of the Gansu road dataset in Embodiment 2 of the present invention. Detailed Implementation

[0039] The experimental implementation methods of the present invention are illustrated below through specific examples. Those skilled in the art can easily understand the advantages of the present invention based on the content of this specification. The present invention can also be implemented through other different specific cases, and the details in this specification can also be modified or changed based on different viewpoints without departing from the spirit of the present invention.

[0040] Example 1: A road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network. The method is implemented by a road extraction model based on hypergraph and multi-level feature fusion, which includes a data preprocessing module, a hypergraph construction module, a hypergraph feature extraction module, a multi-scale feature extraction module, a dual-modal fusion module, and a prediction module. Specifically, the data preprocessing module performs pruning and noise reduction filtering on the HRSI and normalizes the filtered HRSI and its corresponding labels; the hypergraph construction module uses a hypergraph construction method to construct a hypergraph structure with road features from the HRSI; the hypergraph feature extraction module uses hypergraph convolution to perform remote semantic extraction on the hypergraph structure constructed by the hypergraph construction module to obtain hypergraph-level semantic features; the multi-scale feature extraction module uses an encoder-decoder structure to extract semantic features at multiple scales from the HRSI; and the dual-modal fusion module is responsible for connecting the hypergraph feature extraction module and the multi-scale feature extraction module, fusing and transforming the hypergraph-level semantic features and the semantic features at multiple scales to improve the semantic richness of the features.

[0041] The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network includes the following steps:

[0042] Step 1: The data preprocessing module performs cropping, denoising, and filtering on the HRSI dataset. Specifically, the cropping and denoising filtering process involves cropping the original HRSI images acquired by the satellite into appropriately sized HRSI images, and then using RGB analysis to remove boundary images (half-black / half-white) or completely black / completely white images from the dataset. Specifically, the images are read as RGB matrices, and the number of pixels at (255, 255, 255) or (0, 0, 0) points is counted. Thresholding is then applied to remove noise from the images.

[0043] Step 2: The data preprocessing module performs a normalization operation on the dataset and corresponding labels after cropping, denoising, and filtering in Step 1; the normalization operation adopts zero-mean normalization to accelerate the model convergence speed.

[0044] Step 3: The hypergraph construction module performs feature analysis on the dataset after normalization in Step 2, and combines road shape features to construct the corresponding hypergraph structure dataset; the construction process of the hypergraph structure dataset is as follows:

[0045] (1) First, the HRSI dataset that has been normalized in step 2 is divided into L regular 2×2 pixel blocks from left to right and from top to bottom;

[0046] (2) Based on the obtained L pixel blocks, sequentially from each pixel block L i Starting from, select the pixel blocks to its right, below, and below right (L) i+1 ,L i+W / 2 L i+W / 2+1 ) as a neighboring node, where W is the width of HRSI;

[0047] (3) Using pixel block L i Based on this, and using the F-norm, the similarity between the node and its neighboring nodes is calculated sequentially. The calculation formula is as follows: F in public notice sub L represents i The F-norm of a node i and its neighbor node L j F-norm F j The difference;

[0048] (4) Based on the similarity between nodes, first determine its original node L. i Whether a node belongs to a certain superedge, if node L i Exists on some superedge e k , then sim ij Neighbor nodes with similarity greater than the similarity threshold r are added to the hyperedge e. k In the middle, if node L i If no hyperedge exists, then add a new hyperedge e in the hypergraph G. m and sim ij Neighbor nodes with similarity greater than the similarity threshold r are added to the hyperedge e. m The hypergraph feature extraction module consists of three layers of hypergraph convolution, and the multi-scale feature extraction module consists of a ResNet-50 encoder and a multi-scale decoder. Cross-entropy and Dice coefficients are used as loss functions, and the Adam optimizer is selected for training and optimization of the model. The initial learning rate is set to 0.001.

[0049] (5) After traversing all starting pixel block nodes L i Afterwards, a hypergraph structure corresponding to HRSI is obtained.

[0050] Step 4: The hypergraph feature extraction module uses hypergraph convolution to perform long-range semantic extraction on the hypergraph structure dataset from Step 3; the multi-scale feature extraction module uses a CNN-based method to train the hypergraph structure dataset from Step 3 and the HRSI dataset in parallel.

[0051] Step 5: The dual-modal fusion module fuses the intermediate results of the hypergraph feature extraction module and the multi-scale feature extraction module, and inputs the fused features into the dual-convolutional network road extraction model based on hypergraph and multi-level feature fusion for subsequent training.

[0052] The dual-modal fusion module process is divided into fusion from hypergraph to image and fusion from image to hypergraph: the fusion from hypergraph to image is to restore the image structure by reversing the hypergraph structure using the hypergraph construction process, and then merge the image structure with the semantic features of multiple scales in the multi-scale feature extraction module, and then extract the features using 1*1 convolution before inputting them into the subsequent process; the fusion from image to hypergraph is to create a hypergraph structure by using the semantic features of multiple scales in the multi-scale feature extraction module using the hypergraph construction process, and then merge the hypergraph structure with the hypergraph-level semantic features in the hypergraph feature extraction module, and then extract the features using 1*1 convolution before inputting them into the subsequent process.

[0053] Step 6: The prediction module inputs the HRSI test set into the trained road extraction model to perform road extraction prediction and obtain the real road information on the ground; the real road information is a black and white binary image, with roads in RGB-255 and other ground features in RGB-0.

[0054] Example 2: Segmenting roads from aerial imagery is a challenging task. Occlusion by nearby trees, shadows from adjacent buildings, variations in road texture and color, and road class imbalance (due to the relatively small number of pixels in road images) are among the other challenges hindering current models from segmenting clear road boundaries extending from one end of the road. Volodymyr Mnih introduces the Massachusetts Roads Dataset in Chapter 6 of his doctoral dissertation. This dataset covers urban, suburban, and rural areas, spanning over 2600 square kilometers. The test field alone covers more than 110 square kilometers. The challenge lies in extracting complete and accurate roads from this aerial imagery.

[0055] For the Massachusetts Roads dataset, this invention provides a "road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network". The method is implemented by a dual convolutional network road extraction system based on hypergraph and multi-level feature fusion. The dual convolutional network road extraction model based on hypergraph and multi-level feature fusion includes a data preprocessing module, a hypergraph construction module, a hypergraph feature extraction module, a multi-scale feature extraction module, a dual-modal fusion module, and a prediction module.

[0056] The entire model was trained on a supercomputer, specifically on a Linux server running an Intel(R) Xeon(R) Silver 4316 CPU with a frequency of 2.3 GHz and four NVIDIA TESLA A100 GPUs.

[0057] This embodiment is implemented based on the PyTorch deep learning framework.

[0058] Combination Figure 1 and Figure 2 The method includes the following steps:

[0059] S1: The data preprocessing module performs cropping, noise reduction, and filtering on the Massachusetts road dataset;

[0060] S2: The data preprocessing module performs normalization operations on the filtered dataset and corresponding labels;

[0061] S3: The hypergraph construction module performs feature analysis on the dataset normalized by S2, and combines the road shape features to construct the dataset into a corresponding hypergraph structure dataset.

[0062] S4: The hypergraph feature extraction module and the multi-scale feature extraction module are trained using the data generated by S3 and S2, respectively;

[0063] S5: The dual-modal fusion module fuses the intermediate results of the two modules in S3 and then inputs them into the fusion module for subsequent training.

[0064] S6: The prediction module inputs the HRSI test set into the trained model to extract and predict roads, thereby obtaining real road information on the ground.

[0065] In step S1:

[0066] Crop-and-denoise filtering is performed on the Massachusetts Roads dataset, which consists of 1171 aerial images. Each image is 1500×1500 pixels in size. The dataset is divided into a training set of 1108 images, a validation set of 14 images, and a test set of 49 images. After cropping each image to 224×224 pixels and performing denoising filtering, the resulting dataset contains 48791 images for training, 686 images for validation, and 2401 images for testing.

[0067] In step S2:

[0068] The mean and variance of all images in the dataset are calculated, and then each image is normalized to zero-mean. Labeled images are set to 0 to 1, with label 1 representing roads and label 0 representing background information.

[0069] In step S3:

[0070] The specific hypergraph construction process is as follows:

[0071] (1) First, divide the input HRSI into L regular 2×2 pixel blocks from left to right and from top to bottom;

[0072] (2) Based on the obtained L pixel blocks, sequentially from each pixel block L i Starting from, select the pixel blocks to its right, below, and below right (L) i+1 ,L i+W / 2 L i+W / 2+1 ) as a neighboring node. Where W is the width of HRSI.

[0073] (3) Using pixel block L i Based on this, and using the F-norm, the similarity between the node and its neighboring nodes is calculated sequentially. The specific calculation formula is as follows: Where F sub L represents i The F-norm of a node i and its neighbor node L j F-norm F j The difference.

[0074] (4) Based on the similarity between nodes, first determine its original node L. i Does node L belong to a certain superedge? i Exists on some superedge e k , then sim ij Neighbor nodes with similarity greater than the similarity threshold r are added to the hyperedge e. k In the middle. If node L i If no hyperedge exists, then add a new hyperedge e in the hypergraph G. m and sim ij Neighbor nodes with similarity greater than the similarity threshold r are added to the hyperedge e. m middle.

[0075] (5) After traversing all starting pixel block nodes L i Then, a hypergraph structure corresponding to HRSI is obtained.

[0076] In steps S4 and S5: Combining Figure 2 In the experiment, the Adam optimizer was used to optimize the loss during the training process. The initial learning rate was 0.001, the learning rate tolerance was set to 10, and the decay factor was 0.5.

[0077] In step S6: The test set is input into the trained model file to obtain the predicted image file and various indicators.

[0078] To better demonstrate the advantages of the method of this invention, we use the U-Net model based on CNN and the RCFS-Net model based on the attention mechanism to predict images in the Massachusetts Roads dataset as a control group. The prediction results are as follows: Figure 3 As shown in the figure. Furthermore, the results for Road IOU, mean IOU, precision, recall, and F-score (F1) are shown in Table 1.

[0079] Table 1 Performance Comparison of Embodiment 1 of the Invention

[0080] Method of the present invention 0.6437 0.8113 0.7841 0.7824 0.7832 U-Net 0.6297 0.8039 0.7810 0.7647 0.7728 RCFS-Net 0.6337 0.8063 0.7944 0.7588 0.7758

[0081] It is evident that the proposed dual-convolutional network road extraction method based on hypergraph and multi-level feature fusion is far superior in all metrics.

[0082] Example 3: To better demonstrate the advantages of the road extraction method based on hypergraphs and multi-level feature fusion using dual convolutional networks, the road dataset in Example 1 was changed to the Gansu road dataset. Collected by the ZY-3 satellite and created by Gaodian Zhou, this dataset consists of 204 training images, 40 validation images, and 11 test images. This dataset focuses on mountainous road scenes, which are more complex and present greater challenges for road extraction.

[0083] It is particularly important to note that the steps and system descriptions in this embodiment are the same as those in Embodiment 2, and will not be elaborated further here.

[0084] Input the test set into the trained model file to obtain the predicted image file. Figure 4 The various indicators are shown in Table 2.

[0085] Table 2 Performance Comparison of Embodiment 2 of the Invention

[0086] Method of the present invention 0.4483 0.7061 0.6375 0.6116 0.6191 U-Net 0.2188 0.5609 0.2652 0.5557 0.3597 RCFS-Net 0.3538 0.6522 0.4950 0.5536 0.5227

[0087] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network. This method first constructs a road extraction model based on hypergraph and multi-level feature fusion using a dual convolutional network. Then, HRSI (High-Speed ​​Indexed Sensor) is input into this model for road extraction and prediction to obtain real-world road information. The road extraction model based on hypergraph and multi-level feature fusion using a dual convolutional network includes the following modules: Hypergraph Construction Module: Employs the hypergraph construction method to construct a hypergraph structure with road features from HRSI; Hypergraph feature extraction module: Employs hypergraph convolution to perform remote semantic extraction on the hypergraph structure constructed by the hypergraph construction module, thereby obtaining hypergraph-level semantic features; The multi-scale feature extraction module uses an encoder-decoder structure to extract semantic features at multiple scales from HRSI. The dual-modal fusion module is responsible for connecting the hypergraph feature extraction module and the multi-scale feature extraction module, fusing and transforming the semantic features at the hypergraph level and the semantic features at multiple scales to improve the semantic richness of the features; The training steps of the dual convolutional network road extraction model based on hypergraph and multi-level feature fusion include: Step 1: The data preprocessing module performs cropping, noise reduction, and filtering on the HRSI dataset; Step 2: The data preprocessing module performs normalization operations on the dataset and corresponding labels after cropping, denoising, and filtering in Step 1; Step 3: The hypergraph construction module performs feature analysis on the dataset after the normalization operation in Step 2, and constructs the dataset into a corresponding hypergraph structure dataset by combining the road shape features. The construction process of the hypergraph structure dataset mentioned in step 3 is as follows: (1) First, the HRSI dataset which has been normalized in step 2 is divided into L regular pixel blocks from left to right and from top to bottom; (2) According to the obtained L pixel blocks, the pixel blocks right to, below, and right below each pixel block are selected as neighbor nodes, where W is the width of the HRSI. (2) According to the obtained L pixel blocks, the pixel blocks right to, below, and right below each pixel block are selected as neighbor nodes, where W is the width of the HRSI. ​ (3) Using pixel blocks Based on this, and using the F-norm, the similarity between the node and its neighboring nodes is calculated sequentially. The calculation formula is as follows: During the public announcement period express F-norm of a node and its neighboring nodes F-norm The difference; (4) Based on the similarity between nodes, first determine its original node. Whether a node belongs to a certain superedge, if the node Exists on a certain superedge Then Greater than the similarity threshold Neighboring nodes are added to the superedge In, if node If no hyperedge exists, then add a new hyperedge in the hypergraph G. and will Greater than the similarity threshold Neighboring nodes are added to the superedge middle; (5) After traversing all starting pixel block nodes Afterwards, a hypergraph structure corresponding to HRSI is obtained.

2. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 1, characterized in that: The road extraction model based on hypergraph and multi-level feature fusion of the dual convolutional network also includes a data preprocessing module. The data preprocessing module performs pruning and noise reduction filtering on HRSI and normalizes the filtered HRSI and its corresponding label.

3. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 2, characterized in that: The training steps of the dual convolutional network road extraction model based on hypergraph and multi-level feature fusion also include: Step 4: The hypergraph feature extraction module uses hypergraph convolution to perform long-range semantic extraction on the hypergraph structure dataset from Step 3; the multi-scale feature extraction module uses a CNN-based method to train the hypergraph structure dataset from Step 3 and the HRSI dataset in parallel. Step 5: The dual-modal fusion module fuses the intermediate results of the hypergraph feature extraction module and the multi-scale feature extraction module, and inputs the fused features into the dual-convolutional network road extraction model based on hypergraph and multi-level feature fusion for subsequent training.

4. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 3, characterized in that: The cropping and denoising filtering process described in step 1 involves cropping and dividing the original HRSI dataset acquired by the satellite into HRSI datasets of appropriate sizes, and then using the RGB analysis method to remove half-black, half-white, or completely black and completely white boundary images in the dataset.

5. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 4, characterized in that: The normalization operation described in step 2 uses zero-mean normalization to accelerate model convergence.

6. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 5, characterized in that: The hypergraph feature extraction module consists of three layers of hypergraph convolution, and the multi-scale feature extraction module consists of a ResNet-50 encoder and a multi-scale decoder, using cross-entropy and... Dice The coefficients are used as the loss function. The Adam optimizer is used to train and optimize the model, and the initial learning rate is set to 0.

001.

7. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 6, characterized in that: The dual-modal fusion module process is divided into hypergraph-to-image fusion and image-to-hypergraph fusion: the hypergraph-to-image fusion is the reverse process of constructing the hypergraph structure using the hypergraph to restore the image structure, and then merging the image structure with the semantic features of multiple scales in the multi-scale feature extraction module. After feature extraction via convolution, the data is input into subsequent processes. The image-to-hypergraph fusion process involves creating a hypergraph structure from the semantic features at multiple scales in the multi-scale feature extraction module using a hypergraph construction process, and then merging the hypergraph structure with the hypergraph-level semantic features in the hypergraph feature extraction module. Features extracted by convolution are then input into subsequent processes.

8. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 7, characterized in that: The actual road information is a black and white binary image, with roads represented by RGB-255 and other ground features represented by RGB-0.

9. The road extraction method based on hypergraph and multi-level feature fusion using a dual convolutional network according to claim 4, characterized in that: The RGB analysis method reads the image as an RGB matrix, counts the (255, 255, 255) or (0, 0, 0) pixels, and achieves the effect of removing noise from the image through threshold processing.

Images