Remote sensing image road extraction method and system based on lightweight network structure

Abstract

The present invention discloses a remote sensing image road extraction method and system based on a lightweight network structure, the method comprising: receiving a remote sensing image and extracting features through a lightweight structure network to obtain high-level features of the image; modeling based on the high-level features of the image to obtain global features and local features of the remote sensing image; generating multi-scale perceived semantic information based on the global features and local features of the remote sensing image, capturing the dependency between distant regions based on the semantic information, and classifying and extracting road targets and backgrounds in the remote sensing image through specific functions based on the dependency between distant regions. The basic pixel features of the remote sensing image can be obtained while the spatial information of the remote sensing image can be obtained from the spatial scale to quickly and accurately extract the road features in the remote sensing image, thereby ensuring data accuracy and integrity and improving practicality.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a remote sensing image road extraction method and system based on a lightweight network structure. Background Art

[0002] As a common and important digital image, remote sensing images are the product of the rapid development of space measurement technology, earth science theory, computer technology, sensor technology and aerospace technology in the 1960s. Since the 1970s, some foreign scholars and institutions have begun to use image processing and other methods to extract ground truth information from remote sensing images. With the development of remote sensing technology, remote sensing images have begun to transition from low resolution to high resolution, forming a development trend characterized by high spatial resolution, high spectral resolution and high temporal resolution.

[0003] Due to the development of remote sensing technology, the amount of remote sensing image data has increased dramatically, and computer technology has also been developing continuously during this period. The research focus of remote sensing image information extraction has gradually shifted to semi-automatic and fully automatic extraction methods. In optical remote sensing images, roads, as an important infrastructure, are of great significance in military strategy, spatial mapping, urban construction, traffic management and navigation. Especially in the military field, it is crucial to quickly obtain information such as road distribution, road material, road undulations and road capacity for land combat scenes without contact. Extracting road information from remote sensing images provides fast and effective data and technical support for the formulation of combat plans and action deployment. At present, road extraction from high-resolution remote sensing images can be regarded as a semantic segmentation task, which classifies all pixels of roads and non-roads to achieve road extraction. It has the following problems: classification only from the pixel feature dimension results in the lack of spatial information of road features, which makes road extraction incomplete, poor connectivity, and the existence of false extraction, wrong extraction, and omission, which reduces practicality. Summary of the invention

[0004] In response to the above-mentioned problems, the present invention provides a remote sensing image road extraction method and system based on a lightweight network structure to solve the problem mentioned in the background technology that classification is only performed from the pixel feature dimension, resulting in the lack of road feature spatial information, thus making the road extraction incomplete, the connectivity poor, and the existence of false extraction, wrong extraction, omission and the like, which reduces the practicality.

[0005] A remote sensing image road extraction method based on a lightweight network structure comprises the following steps:

[0006] Receive remote sensing images and extract features through a lightweight structure network to obtain high-level features of the image;

[0007] Modeling is performed based on high-level image features to obtain global and local features of remote sensing images;

[0008] Generate multi-scale perceived semantic information based on the global and local features of remote sensing images, and capture the dependencies between distant regions based on semantic information;

[0009] Based on the dependency between long-distance regions, road targets and backgrounds in remote sensing images are classified and extracted through specific functions.

[0010] Preferably, the receiving of remote sensing images and extracting features through a lightweight structure network to obtain high-level features of the images include:

[0011] The remote sensing image is received through an encoder, and the number of structural layers and output channels of the encoder network are reduced and adjusted to obtain a lightweight structural network;

[0012] Use lightweight structure network to perform multi-channel preliminary feature extraction on remote sensing images and obtain the extracted image features of each channel;

[0013] Fusing the extracted image features of each channel to obtain fused image features;

[0014] The feature layer of the lightweight structure network is used to reduce the dimension of the fused image features based on deep separable convolution to obtain high-level image features.

[0015] Preferably, the modeling based on the high-level features of the image to obtain the global features and local features of the remote sensing image includes:

[0016] Determine the image domain according to the high-level features of the image, and perceive the high-level features of the image in the image domain through a multi-head attention perception algorithm to obtain a multi-head perception value;

[0017] The multi-head perception values ​​are spliced, output and projected to obtain an output feature modeling graph;

[0018] The global context information is captured by the output feature modeling graph, and local features and global features are obtained based on the global context information.

[0019] Preferably, generating multi-scale perceived semantic information according to global features and local features of remote sensing images, and capturing dependencies between distant regions based on the semantic information, includes:

[0020] The local and global features are processed through residual mapping to enhance the semantic expression;

[0021] The processed local features and global features are perceived at multiple scales through the transformer model to obtain perceptual semantic information;

[0022] Capturing dependencies between distant regions based on perceptual semantic information.

[0023] Preferably, the classification and extraction of road targets and backgrounds in the remote sensing image by using a specific function based on the dependency relationship between the long-distance regions includes:

[0024] The spatial information of remote sensing images is obtained based on the dependency relationship between distant regions, and convolution sampling is performed on the spatial information;

[0025] Perform linear correction and normalization processing on the sampling results to obtain processed sampling data;

[0026] The si gmo id function is used to divide the road target and background of the processed sampling data based on the road characteristics to obtain the division result;

[0027] According to the segmentation results, the road targets and background in the remote sensing image are determined and classified and extracted respectively.

[0028] A remote sensing image road extraction system based on a lightweight network structure, the system comprising:

[0029] The first extraction module is used to receive the remote sensing image and perform feature extraction through a lightweight structure network to obtain high-level features of the image;

[0030] The acquisition module is used to build models based on high-level features of images and obtain global and local features of remote sensing images;

[0031] A generation module is used to generate multi-scale perceived semantic information based on the global and local features of remote sensing images, and to capture the dependencies between distant regions based on the semantic information;

[0032] The second extraction module is used to classify and extract the road target and background in the remote sensing image through a specific function based on the dependency relationship between the long-distance regions.

[0033] Preferably, the first extraction module includes:

[0034] An adjustment submodule is used to receive remote sensing images through an encoder, reduce the number of structural layers and output channels of the encoder network, and obtain a lightweight structural network;

[0035] The first extraction submodule is used to perform multi-channel preliminary feature extraction on the remote sensing image using a lightweight structure network to obtain extracted image features of each channel;

[0036] A fusion submodule is used to fuse the extracted image features of each channel to obtain fused image features;

[0037] The dimension reduction submodule is used to reduce the dimension of the fused image features based on the deep separable convolution through the feature layer of the lightweight structure network to obtain high-level image features.

[0038] Preferably, the acquisition module includes:

[0039] The perception submodule is used to determine the image domain according to the high-level features of the image, and perceive the high-level features of the image in the image domain through a multi-head attention perception algorithm to obtain a multi-head perception value;

[0040] The first acquisition submodule is used to splice and output the multi-head perception values ​​and project them to obtain an output feature modeling graph;

[0041] The second acquisition submodule is used to capture global context information by outputting a feature modeling graph, and to acquire local features and global features according to the global context information.

[0042] Preferably, the generating module comprises:

[0043] A first processing submodule, for processing local features and global features through residual mapping to enhance semantic expression;

[0044] The third acquisition submodule is used to perform multi-scale perception on the processed local features and global features through the transformer model to obtain perceptual semantic information;

[0045] The fourth acquisition submodule is used to capture the dependencies between distant regions based on perceptual semantic information.

[0046] Preferably, the second extraction module comprises:

[0047] The sampling submodule is used to obtain the spatial information of the remote sensing image based on the dependency relationship between distant regions and perform convolution sampling on the spatial information;

[0048] The second processing submodule is used to perform linear correction and normalization processing on the sampling results to obtain processed sampling data;

[0049] The segmentation submodule is used to segment the road target and background of the processed sampled data based on the road features through the si gmo id function to obtain the segmentation result;

[0050] The second extraction submodule is used to determine the road target and background in the remote sensing image according to the segmentation result and classify and extract them respectively.

[0051] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0054] Figure 1 A workflow diagram of a remote sensing image road extraction method based on a lightweight network structure provided by the present invention;

[0055] Figure 2 Another workflow diagram of a remote sensing image road extraction method based on a lightweight network structure provided by the present invention;

[0056] Figure 3 A backneck structure diagram of an encoder in a remote sensing image road extraction method based on a lightweight network structure provided by the present invention;

[0057] Figure 4 A schematic diagram of the structure of a remote sensing image road extraction system based on a lightweight network structure provided by the present invention;

[0058] Figure 5 This is a schematic diagram of the structure of the first extraction module in a remote sensing image road extraction system based on a lightweight network structure provided by the present invention. DETAILED DESCRIPTION

[0059] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0060] As a common and important digital image, remote sensing images are the product of the rapid development of space measurement technology, earth science theory, computer technology, sensor technology and aerospace technology in the 1960s. Since the 1970s, some foreign scholars and institutions have begun to use image processing and other methods to extract ground truth information from remote sensing images. With the development of remote sensing technology, remote sensing images have begun to transition from low resolution to high resolution, forming a development trend characterized by high spatial resolution, high spectral resolution and high temporal resolution.

[0061] Due to the development of remote sensing technology, the amount of remote sensing image data has increased dramatically, and computer technology has also been developing continuously during this period. The research focus of remote sensing image information extraction has gradually shifted to semi-automatic and fully automatic extraction methods. In optical remote sensing images, roads, as an important infrastructure, are of great significance in military strategy, spatial mapping, urban construction, traffic management and navigation. Especially in the military field, it is crucial to quickly obtain information such as road distribution, road material, road undulations and road capacity for land battle scenes without contact. Extracting road information through remote sensing images provides fast and effective data and technical support for the formulation of combat plans and action deployment. At present, road extraction from high-resolution remote sensing images can be regarded as a semantic segmentation task, which classifies all pixels of roads and non-roads to achieve road extraction. It has the following problems: classification only from the pixel feature dimension causes the lack of road feature spatial information, resulting in incomplete road extraction, poor connectivity, and the existence of false extraction, wrong extraction, and omission, which reduces practicality. In order to solve the above problems, this embodiment discloses a remote sensing image road extraction method based on a lightweight network structure.

[0062] A road extraction method for remote sensing images based on a lightweight network structure, such as Figure 1 As shown, the following steps are included:

[0063] Step S101, receiving a remote sensing image and performing feature extraction through a lightweight structure network to obtain high-level features of the image;

[0064] Step S102: Modeling is performed based on high-level features of the image to obtain global features and local features of the remote sensing image;

[0065] Step S103, generating multi-scale perceived semantic information according to the global features and local features of the remote sensing image, and capturing the dependency relationship between distant regions based on the semantic information;

[0066] Step S104: classify and extract the road targets and background in the remote sensing image through a specific function based on the dependency relationship between the long-distance regions.

[0067] In this embodiment, the lightweight structure network consists of three parts: encoder, multi-scale semantic parsing (MSSP) network and decoder. The encoder accepts the image as input, extracts high-level features, and inputs them into MSSP to provide a basis for subsequent road extraction tasks. The improved ViT is used as MSSP to model global relationships, capture dependencies between long-distance regions, and generate multi-scale perceived semantic information. The semantic information enhances the feature representation capability through the decoder. Finally, the segmentation task is performed to achieve the binary classification of road targets and backgrounds.

[0068] The working principle of the above technical solution is: receiving remote sensing images and extracting features through a lightweight structure network to obtain high-level features of the image; modeling based on the high-level features of the image to obtain global and local features of the remote sensing image; generating multi-scale perceived semantic information based on the global and local features of the remote sensing image, capturing the dependencies between distant regions based on the semantic information, and classifying and extracting road targets and backgrounds in the remote sensing image through specific functions based on the dependencies between distant regions.

[0069] The beneficial effects of the above technical solution are as follows: by acquiring the global features and local features of the remote sensing image and then performing multi-scale semantic perception, it is possible to obtain the basic pixel features of the remote sensing image and at the same time obtain the spatial information of the remote sensing image from the spatial scale, thereby quickly and accurately extracting the road features in the remote sensing image, thereby ensuring data accuracy and integrity, improving practicality, and solving the problem mentioned in the prior art that classification is performed only from the pixel feature dimension, resulting in the lack of spatial information of road features, thereby making the road extraction incomplete, the connectivity poor, and the existence of false extraction, wrong extraction, and omission, thereby reducing the practicality.

[0070] In one embodiment, Figure 2 As shown, the remote sensing image is received and feature extraction is performed through a lightweight structure network to obtain high-level image features, including:

[0071] Step S201: receiving a remote sensing image through an encoder, reducing and adjusting the number of structural layers and the number of output channels of the encoder network to obtain a lightweight structural network;

[0072] Step S202: Perform multi-channel preliminary feature extraction on the remote sensing image using a lightweight structure network to obtain extracted image features of each channel;

[0073] Step S203, fusing the extracted image features of each channel to obtain fused image features;

[0074] Step S204: Reduce the dimension of the fused image features based on the depth-separable convolution through the feature layer of the lightweight structure network to obtain high-level image features.

[0075] In this embodiment, the encoder network uses the improved MobileNetv3 network as the backbone feature extraction network. First, the SE module in the bneck structure is replaced by a convolutional attention module; then, the network structure of MobileNetv3 is adjusted to reduce the number of layers and output channels of the bneck structure. This makes the model in this paper more lightweight and improves the model's road feature extraction capability.

[0076] The backneck structure in the MobileNetv3 model consists of a backbone and a residual edge. The backbone first increases the dimension of the input features through a 1×1 convolution, expands the input feature layer channel, and then inputs the output features into a 3×3 depthwise separable convolution. Then, an attention mechanism module is added. The attention mechanism is added to the channel, and the feature layer after the depthwise separable convolution is globally averaged and pooled to obtain the channel weight. The feature is then fully connected twice to obtain the channel weight, and the length of the channel weight is consistent with the number of channels in the feature layer after the 3×3 depthwise separable convolution. Therefore, the channel weight is multiplied by the feature after the depthwise separable convolution to obtain the corresponding feature layer, and then a 1×1 convolution is used for dimensionality reduction. The residual edge is directly connected to the output.

[0077] The bneck structure has a large number of redundant features to ensure that the model has a comprehensive explanation of the input. These features calculated by redundant convolutions not only ensure the generalization ability of the model, but also occupy a lot of computing resources and increase the complexity of the model. Therefore, this paper adjusts the network structure of MobileNetv3, reduces the number of layers of the bneck structure and the number of output channels, making the network lighter and more efficient. At the same time, since the SE module in the bneck structure only focuses on channel information, the spatial information of road features is missing. In order to further improve the ability of MobileNetv3 model to extract spatial information of road feature maps, CBAM is used to replace the SE module. CBAM introduces two analysis dimensions, channel attention and spatial attention, to realize the inference of attention maps from channel and spatial dimensions in turn, and then multiply the attention map with the input feature map for adaptive feature optimization, integrate spatial and channel information, highlight the key information in the feature map, and discard irrelevant features. Compared with the SE module, the maximum pooling and average pooling operations are used in the channel attention to produce more refined attention than SE, and finally the spatial attention is used to promote the improvement of the model road extraction performance. At the same time, there is no large number of convolution structures inside the CBAM module, which avoids the large amount of calculations caused by convolution multiplication, making its module complexity low and the amount of calculation small. Therefore, compared with the SE module, it reduces the number of model parameters, the amount of calculation and the complexity, and speeds up the training and reasoning of the model. The improved bneck structure diagram in this paper is shown in the figure. Figure 3 shown.

[0078] The encoder consists of an improved MobileNetv3 block. First, the Conv block performs preliminary feature extraction on the input image. The Conv block consists of a 3×3 convolution, a batch normalization layer (BN), and an H-Swish activation function. Then the bneck structure group fuses the features of each channel to achieve high-level feature extraction and data dimensionality reduction. Finally, the extracted features are input into the multi-scale semantic parsing network.

[0079] The beneficial effects of the above technical solution are: receiving remote sensing images through an encoder and obtaining a lightweight structure network, thereby performing multi-channel preliminary feature extraction on the remote sensing images, which can ensure improved efficiency and reduced computing resource requirements when processing a large number of remote sensing images. Further, the extracted image features of each channel are obtained, the features are fused and the dimension is reduced to obtain high-level image features, thereby improving the effectiveness of the high-level features we obtain and improving the accuracy of road extraction.

[0080] In one embodiment, the modeling based on the high-level features of the image to obtain the global features and local features of the remote sensing image includes:

[0081] Determine the image domain according to the high-level features of the image, and perceive the high-level features of the image in the image domain through a multi-head attention perception algorithm to obtain a multi-head perception value;

[0082] The multi-head perception values ​​are spliced, output and projected to obtain an output feature modeling graph;

[0083] The global context information is captured by the output feature modeling graph, and local features and global features are obtained based on the global context information.

[0084] The beneficial effects of the above technical solution are: acquiring the image domain and perceiving high-level features through a multi-head attention perception algorithm, simultaneously considering different regions and different types of information in the image, thereby obtaining a more comprehensive feature representation, obtaining multi-head perception values ​​and splicing them, and obtaining an output feature modeling graph, which can integrate different regions and different types of information into a unified feature space, enhancing the model's ability to understand the global image and improving its classification and detection accuracy.

[0085] In this embodiment, global context information is captured by outputting a feature modeling graph, and local features and global features are obtained according to the global context information, including:

[0086] According to the output feature modeling graph, point cloud features of the background area and point cloud features of the road target area are obtained and a point cloud feature set is constructed;

[0087] Construct a target feature fusion network through the cross-attention mechanism algorithm;

[0088] Determine the correlation matrix between the point cloud feature set of the background area point cloud feature and the point cloud feature set of the road target area point cloud feature based on the target feature fusion network;

[0089] Determine the background area view frame and the road target area view frame according to the relative relationship matrix and align the two view frames based on the output feature modeling graph;

[0090] According to the registered view frame, the regional image quantity parameters corresponding to the background area and the road target area are extracted from the output feature modeling map;

[0091] Acquire global context information according to the area map parameters corresponding to the background area and the road target area, and extract edge direction features according to the global context information;

[0092] Respectively obtain the spatial color features corresponding to the background area and the road target area;

[0093] Determine a local feature description vector according to edge direction features and spatial color features corresponding to the background area and the road target area;

[0094] Vector mapping is performed on the local feature description vector to obtain local features, and the local features are integrated to obtain global features.

[0095] In this embodiment, the background area point cloud feature represents the spatial point cloud feature used to represent the background other than the road in the output feature modeling graph;

[0096] The beneficial effects of the above technical solution are as follows: by respectively determining the regional map parameters of the road target area and the Beijing area in the output feature modeling map, the road and the background can be accurately segmented and judged, which lays a good reference factor for subsequent road extraction, improves accuracy and stability, and ensures the accuracy and rationality of subsequent local feature acquisition. Furthermore, by obtaining local features and global features through vector mapping, accurate and high-quality image features can be fully obtained, ensuring data quality and effectiveness, and further improving practicality and reliability.

[0097] In one embodiment, generating multi-scale perceived semantic information based on global features and local features of remote sensing images, and capturing dependencies between distant regions based on the semantic information, includes:

[0098] The local and global features are processed through residual mapping to enhance the semantic expression;

[0099] The processed local features and global features are perceived at multiple scales through the transformer model to obtain perceptual semantic information;

[0100] Capturing dependencies between distant regions based on perceptual semantic information.

[0101] The beneficial effects of the above technical solution are: by processing local features and global features and performing multi-scale perception, the boundaries and categories of roads can be more accurately identified in tasks such as target detection and segmentation, and perceptual semantic information can be obtained, thereby capturing the dependencies between long-distance areas, and the relationships between roads can be obtained, thereby improving the efficiency of road analysis.

[0102] In one embodiment, the classification and extraction of road targets and backgrounds in remote sensing images by using a specific function based on the dependency relationship between long-distance regions includes:

[0103] The spatial information of remote sensing images is obtained based on the dependency relationship between distant regions, and convolution sampling is performed on the spatial information;

[0104] Perform linear correction and normalization processing on the sampling results to obtain processed sampling data;

[0105] The si gmo id function is used to divide the road target and background of the processed sampling data based on the road characteristics to obtain the division result;

[0106] According to the segmentation results, the road targets and background in the remote sensing image are determined and classified and extracted respectively.

[0107] The beneficial effects of the above technical solution are: by acquiring the spatial information of the remote sensing image through the dependency relationship between long distances and performing convolution sampling, the positions and relationships of different areas and objects in the image can be better understood, and the sampling results are processed to obtain processed sampling data, which can further improve the stability and reliability of the data and provide a more stable data set. Furthermore, the processed sampling data is divided into road targets and backgrounds through the si gmo id function to determine the road targets and backgrounds in the remote sensing image, which can improve the accuracy of the division and facilitate the determination of different areas and classifications of the road.

[0108] In one embodiment, this embodiment also discloses a remote sensing image road extraction system based on a lightweight network structure, such as Figure 4 As shown, the system includes:

[0109] The first extraction module 301 is used to receive remote sensing images and perform feature extraction through a lightweight structure network to obtain high-level features of the image;

[0110] An acquisition module 302 is used to perform modeling based on high-level features of the image and acquire global features and local features of the remote sensing image;

[0111] The generation module 303 is used to generate multi-scale perceived semantic information based on the global features and local features of the remote sensing image, and to capture the dependency between distant regions based on the semantic information.

[0112] The second extraction module 304 is used to classify and extract the road target and background in the remote sensing image through a specific function based on the dependency relationship between the long-distance regions.

[0113] The working principle of the above technical solution is: first, the remote sensing image is received through the first extraction module and feature extraction is performed through a lightweight structure network to obtain high-level features of the image; secondly, the acquisition module is used to model according to the high-level features of the image to obtain the global features and local features of the remote sensing image; then, based on the generation module, multi-scale perceived semantic information is generated according to the global features and local features of the remote sensing image, and the dependency relationship between long-distance regions is captured based on the semantic information. Finally, the second extraction module is used to classify and extract road targets and backgrounds in the remote sensing image through specific functions based on the dependency relationship between long-distance regions.

[0114] The beneficial effects of the above technical solution are: by acquiring the global features and local features of the remote sensing image and then performing multi-scale semantic perception, it is possible to obtain the basic pixel features of the remote sensing image while also acquiring the spatial information of the remote sensing image from the spatial scale, and then quickly and accurately extract the road features in the remote sensing image, thereby ensuring data accuracy and integrity and improving practicality.

[0115] In one embodiment, Figure 5 As shown, the first extraction module 301 includes:

[0116] The adjustment submodule 3011 is used to receive the remote sensing image through the encoder, reduce the number of structural layers and output channels of the encoder network, and obtain a lightweight structural network;

[0117] The first extraction submodule 3012 is used to perform multi-channel preliminary feature extraction on the remote sensing image using a lightweight structure network to obtain extracted image features of each channel;

[0118] A fusion submodule 3013 is used to fuse the extracted image features of each channel to obtain a fused image feature;

[0119] The dimension reduction submodule 3014 is used to reduce the dimension of the fused image features based on the depth-separable convolution through the feature layer of the lightweight structure network to obtain high-level image features.

[0120] The beneficial effects of the above technical solution are: receiving remote sensing images through an encoder and obtaining a lightweight structure network, thereby performing multi-channel preliminary feature extraction on the remote sensing images, which can ensure improved efficiency and reduced computing resource requirements when processing a large number of remote sensing images. Further, the extracted image features of each channel are obtained, the features are fused and the dimension is reduced to obtain high-level image features, thereby improving the effectiveness of the high-level features we obtain and improving the accuracy of road extraction.

[0121] In one embodiment, the acquisition module includes:

[0122] The perception submodule is used to determine the image domain according to the high-level features of the image, and perceive the high-level features of the image in the image domain through a multi-head attention perception algorithm to obtain a multi-head perception value;

[0123] The first acquisition submodule is used to splice and output the multi-head perception values ​​and project them to obtain an output feature modeling graph;

[0124] The second acquisition submodule is used to capture global context information by outputting a feature modeling graph, and to acquire local features and global features according to the global context information.

[0125] The beneficial effects of the above technical solution are: acquiring the image domain and perceiving high-level features through a multi-head attention perception algorithm, simultaneously considering different regions and different types of information in the image, thereby obtaining a more comprehensive feature representation, obtaining multi-head perception values ​​and splicing them, and obtaining an output feature modeling graph, which can integrate different regions and different types of information into a unified feature space, enhancing the model's ability to understand the global image and improving its classification and detection accuracy.

[0126] In one embodiment, the generating module comprises:

[0127] A first processing submodule, for processing local features and global features through residual mapping to enhance semantic expression;

[0128] The third acquisition submodule is used to perform multi-scale perception on the processed local features and global features through the transformer model to obtain perceptual semantic information;

[0129] The fourth acquisition submodule is used to capture the dependencies between distant regions based on perceptual semantic information.

[0130] The beneficial effects of the above technical solution are: by processing local features and global features and performing multi-scale perception, the boundaries and categories of roads can be more accurately identified in tasks such as target detection and segmentation, and perceptual semantic information can be obtained, thereby capturing the dependencies between long-distance areas, and the relationships between roads can be obtained, thereby improving the efficiency of road analysis.

[0131] In one embodiment, the second extraction module includes:

[0132] The sampling submodule is used to obtain the spatial information of the remote sensing image based on the dependency relationship between distant regions and perform convolution sampling on the spatial information;

[0133] The second processing submodule is used to perform linear correction and normalization processing on the sampling results to obtain processed sampling data;

[0134] The segmentation submodule is used to segment the road target and background of the processed sampled data based on the road features through the sigmo id function to obtain the segmentation result;

[0135] The second extraction submodule is used to determine the road target and background in the remote sensing image according to the segmentation result and classify and extract them respectively.

[0136] The beneficial effects of the above technical solution are: by acquiring the spatial information of the remote sensing image through the dependency between long distances and performing convolution sampling, the positions and relationships of different areas and objects in the image can be better understood, and the sampling results are processed to obtain processed sampling data, which can further improve the stability and reliability of the data and provide a more stable data set. Furthermore, the processed sampling data is divided into road targets and backgrounds through the sigmoid function to determine the road targets and backgrounds in the remote sensing image, which can improve the accuracy of the division and facilitate the determination of different areas and classifications of the road.

[0137] Those skilled in the art should understand that the first and second in the present invention merely refer to different application stages.

[0138] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0139] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A remote sensing image road extraction method based on a lightweight network structure, characterized in that: The following steps are involved: Receive remote sensing images and extract features through a lightweight structure network to obtain high-level features of the image; Modeling is performed based on high-level image features to obtain global and local features of remote sensing images; Generate multi-scale perceived semantic information based on the global and local features of remote sensing images, and capture the dependencies between distant regions based on semantic information; Based on the dependency between long-distance regions, the road targets and background in the remote sensing image are classified and extracted through specific functions; The modeling is performed based on the high-level features of the image to obtain the global features and local features of the remote sensing image, including: Determine the image domain according to the high-level features of the image, and perceive the high-level features of the image in the image domain through a multi-head attention perception algorithm to obtain a multi-head perception value; The multi-head perception values ​​are spliced, output and projected to obtain an output feature modeling graph; Capture global context information by outputting feature modeling graphs, and obtain local features and global features based on the global context information; The global context information is captured by the output feature modeling graph, and local features and global features are obtained based on the global context information, including: According to the output feature modeling graph, point cloud features of the background area and point cloud features of the road target area are obtained and a point cloud feature set is constructed; Construct a target feature fusion network through the cross-attention mechanism algorithm; Determine the correlation matrix between the point cloud feature set of the background area point cloud feature and the point cloud feature set of the road target area point cloud feature based on the target feature fusion network; Determine the background area view frame and the road target area view frame according to the relative relationship matrix and align the two view frames based on the output feature modeling graph; According to the registered view frame, the regional image quantity parameters corresponding to the background area and the road target area are extracted from the output feature modeling map; Acquire global context information according to the area map parameters corresponding to the background area and the road target area, and extract edge direction features according to the global context information; Respectively obtain the spatial color features corresponding to the background area and the road target area; Determine a local feature description vector according to edge direction features and spatial color features corresponding to the background area and the road target area; Vector mapping is performed on the local feature description vector to obtain local features, and the local features are integrated to obtain global features.

2. According to claim 1, the remote sensing image road extraction method based on lightweight network structure is characterized in that: The receiving of remote sensing images and feature extraction through a lightweight structure network to obtain high-level features of the image include: The remote sensing image is received through an encoder, and the number of structural layers and output channels of the encoder network are reduced and adjusted to obtain a lightweight structural network; Use lightweight structure network to perform multi-channel preliminary feature extraction on remote sensing images and obtain the extracted image features of each channel; Fusing the extracted image features of each channel to obtain fused image features; The feature layer of the lightweight structure network is used to reduce the dimension of the fused image features based on deep separable convolution to obtain high-level image features.

3. The remote sensing image road extraction method based on a lightweight network structure according to claim 1 is characterized in that: The method generates multi-scale perceived semantic information based on global features and local features of remote sensing images, and captures dependencies between distant regions based on the semantic information, including: The local and global features are processed through residual mapping to enhance the semantic expression; The processed local features and global features are perceived at multiple scales through the transformer model to obtain perceptual semantic information; Capturing dependencies between distant regions based on perceptual semantic information.

4. The remote sensing image road extraction method based on a lightweight network structure according to claim 1 is characterized in that: The method of classifying and extracting the road target and the background in the remote sensing image by using a specific function based on the dependency relationship between the long-distance regions includes: The spatial information of remote sensing images is obtained based on the dependency relationship between distant regions, and convolution sampling is performed on the spatial information; Perform linear correction and normalization processing on the sampling results to obtain processed sampling data; The road target and background are divided into two parts based on the road features by using the sigmoid function to process the sampled data and obtain the division result; According to the segmentation results, the road targets and background in the remote sensing image are determined and classified and extracted respectively.

5. A remote sensing image road extraction system based on a lightweight network structure, characterized in that: The system includes: The first extraction module is used to receive the remote sensing image and perform feature extraction through a lightweight structure network to obtain high-level features of the image; The acquisition module is used to build models based on high-level features of images and obtain global and local features of remote sensing images; A generation module is used to generate multi-scale perceived semantic information based on the global and local features of remote sensing images, and to capture dependencies between distant regions based on semantic information; The second extraction module is used to classify and extract the road target and background in the remote sensing image through a specific function based on the dependency relationship between the long-distance regions; The acquisition module comprises: The perception submodule is used to determine the image domain according to the high-level features of the image, and perceive the high-level features of the image in the image domain through a multi-head attention perception algorithm to obtain a multi-head perception value; The first acquisition submodule is used to splice and output the multi-head perception values ​​and project them to obtain an output feature modeling graph; The second acquisition submodule is used to capture global context information by outputting a feature modeling graph, and to acquire local features and global features according to the global context information; The global context information is captured by the output feature modeling graph, and local features and global features are obtained based on the global context information, including: According to the output feature modeling graph, point cloud features of the background area and point cloud features of the road target area are obtained and a point cloud feature set is constructed; Construct a target feature fusion network through the cross-attention mechanism algorithm; Determine the correlation matrix between the point cloud feature set of the background area point cloud feature and the point cloud feature set of the road target area point cloud feature based on the target feature fusion network; Determine the background area view frame and the road target area view frame according to the relative relationship matrix and align the two view frames based on the output feature modeling graph; According to the registered view frame, the regional image quantity parameters corresponding to the background area and the road target area are extracted from the output feature modeling map; Acquire global context information according to the area map parameters corresponding to the background area and the road target area, and extract edge direction features according to the global context information; Respectively obtain the spatial color features corresponding to the background area and the road target area; Determine a local feature description vector according to edge direction features and spatial color features corresponding to the background area and the road target area; Vector mapping is performed on the local feature description vector to obtain local features, and the local features are integrated to obtain global features.

6. The remote sensing image road extraction system based on a lightweight network structure according to claim 5 is characterized in that: The first extraction module comprises: An adjustment submodule is used to receive remote sensing images through an encoder, reduce the number of structural layers and output channels of the encoder network, and obtain a lightweight structural network; The first extraction submodule is used to perform multi-channel preliminary feature extraction on the remote sensing image using a lightweight structure network to obtain extracted image features of each channel; A fusion submodule is used to fuse the extracted image features of each channel to obtain fused image features; The dimension reduction submodule is used to reduce the dimension of the fused image features based on the deep separable convolution through the feature layer of the lightweight structure network to obtain high-level image features.

7. The remote sensing image road extraction system based on lightweight network structure according to claim 5 is characterized in that: The generating module comprises: A first processing submodule, for processing local features and global features through residual mapping to enhance semantic expression; The third acquisition submodule is used to perform multi-scale perception on the processed local features and global features through the transformer model to obtain perceptual semantic information; The fourth acquisition submodule is used to capture the dependencies between distant regions based on perceptual semantic information.

8. The remote sensing image road extraction system based on a lightweight network structure according to claim 5 is characterized in that: The second extraction module comprises: The sampling submodule is used to obtain the spatial information of the remote sensing image based on the dependency relationship between distant regions and perform convolution sampling on the spatial information; The second processing submodule is used to perform linear correction and normalization processing on the sampling results to obtain processed sampling data; The segmentation submodule is used to segment the road target and background of the processed sampled data based on the road features through the sigmoid function to obtain the segmentation result; The second extraction submodule is used to determine the road target and background in the remote sensing image according to the segmentation result and classify and extract them respectively.

Images