Road change detection method based on mixed bar convolution and related equipment

Abstract

The invention provides a road change detection method based on hybrid bar convolution and related equipment. The method comprises the following steps: acquiring two to-be-detected images for the same position; extracting road linear features in each to-be-detected image through a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale bar convolution unit; performing fusion processing on the road linear features in the to-be-detected images through a cross fusion module to obtain fusion features; and performing road change detection based on the fusion features. According to the invention, the long-distance dependency relationship of the road features is captured through the feature enhancement module, and the spatial information and the time information of the linear features of the two roads are interactively propagated through the cross fusion module, so that the accuracy of road change detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a road change detection method based on hybrid strip convolution and related equipment. Background Art

[0002] Remote Sensing Change Detection (RSCD) aims to locate and segment land surface changes from co-registered image pairs taken at different times over the same area. It is an important problem in remote sensing understanding tasks and a key step in many real-world tasks such as global resource monitoring, land use change detection, damage assessment, and urban management.

[0003] Road change detection based on dual-temporal remote sensing images identifies newly built, disappeared, damaged or rebuilt roads based on images collected at different times. Existing road change detection methods based on deep learning are mainly based on general change detection neural networks. General change detection networks can effectively detect land cover changes, however, complex post-processing steps are required to find changes related to roads. Since roads in remote sensing images are usually distributed throughout the image and have large spans and narrow shapes, general change detection methods use square convolution kernels and cannot capture the linear features of roads well. In addition, since roads are easily affected by environmental interference and are easily blocked by tree shadows and buildings, the same road presents different appearance characteristics in dual-temporal images. Summary of the invention

[0004] The present invention provides a road change detection method and related equipment based on hybrid strip convolution, which are used to solve the defects of the prior art that the road changes detected by remote sensing images are discontinuous, incomplete, and easily blocked, which leads to low detection accuracy, so as to improve the accuracy of road change detection.

[0005] The present invention provides a road change detection method based on hybrid strip convolution, comprising: Obtain two images to be detected for the same position; Extracting the linear features of the road in each of the images to be detected by a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; The road linear features in each of the images to be detected are fused by a cross fusion module to obtain fused features; Road change detection is performed based on the fused features.

[0006] According to a road change detection method based on hybrid strip convolution provided by the present invention, the multi-scale strip convolution unit includes a plurality of strip convolution units for different directions; the road linear features in each of the images to be detected are extracted by the feature enhancement module, including: Extracting the first feature in the image to be detected by each of the strip convolution units respectively; The road linear feature of each of the images to be detected is obtained based on the first feature corresponding to each of the images to be detected.

[0007] According to a road change detection method based on hybrid strip convolution provided by the present invention, the multi-scale strip convolution unit includes strip convolution units for the horizontal direction, vertical direction, left diagonal direction and right diagonal direction respectively.

[0008] According to a road change detection method based on hybrid strip convolution provided by the present invention, the feature enhancement module further includes a multi-scale hole convolution unit, and the road linear features in each of the images to be detected are extracted by the feature enhancement module, including: Extracting the second feature of each of the images to be detected by the multi-scale strip convolution unit, and extracting the third feature of each of the images to be detected by the multi-scale hole convolution unit; The second feature and the third feature of each of the images to be detected are connected to obtain the road linear feature of each of the images to be detected.

[0009] According to a road change detection method based on hybrid strip convolution provided by the present invention, the cross fusion module includes a self-attention mechanism and a depth-separable convolution; the cross fusion module is used to fuse the road linear features in each of the images to be detected to obtain a fusion feature, including: Performing splicing processing on the road linear features to obtain road linear splicing features; Inputting the road linear splicing features into the self-attention mechanism and the depthwise separable convolution for processing respectively, and performing channel interaction and spatial interaction during the processing to obtain output features corresponding to the self-attention mechanism and the depthwise separable convolution respectively; A fusion process is performed based on each of the output features to obtain the fusion feature.

[0010] According to a road change detection method based on hybrid strip convolution provided by the present invention, the fusion processing is performed based on each of the output features to obtain the fusion feature, including: Based on pixels, each of the output features is summed, and the summed results are convolved to obtain a mixed feature; Based on the pixels, the mixed features are subjected to difference processing to obtain the fused features.

[0011] According to a road change detection method based on hybrid strip convolution provided by the present invention, the road change detection based on the fusion feature includes: The fused features are upsampled and spliced, and a road change map is obtained based on the upsampled and spliced ​​fused features.

[0012] The present invention also provides a road change detection device based on hybrid strip convolution, comprising: An acquisition module, configured to acquire two images to be detected for the same position; An extraction module configured to extract the road linear features in each of the images to be detected through a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; A fusion processing module, configured to perform fusion processing on the road linear features in each of the images to be detected through a cross fusion module to obtain fusion features; The detection module is configured to perform road change detection based on the fused features.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the road change detection method based on hybrid strip convolution as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described road change detection methods based on hybrid strip convolution.

[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned road change detection methods based on hybrid strip convolution.

[0016] The road change detection method and related equipment based on hybrid strip convolution provided by the present invention construct a feature enhancement module through multi-scale strip convolution units, so that the feature enhancement module can capture the long-distance dependency of road features, avoid the blurring of narrow object boundaries by traditional square convolution, better adapt to the road distribution characteristics, reduce the blurring of road edges caused by irrelevant areas, and improve the integrity and continuity of road extraction. The cross-fusion module fuses two road linear features, and interactively propagates their respective spatial information and temporal information during the fusion process, guides the network to focus on road-related change areas, suppresses the influence of distribution differences in the two images to be detected, and thus improves the accuracy of road change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the road change detection method based on hybrid strip convolution provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of the road change detection method based on hybrid strip convolution provided by the present invention.

[0020] Figure 3 It is one of the structural schematic diagrams of the feature enhancement module provided by the present invention.

[0021] Figure 4 This is the second structural schematic diagram of the feature enhancement module provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the cross-fusion module provided by the present invention.

[0023] Figure 6 It is an experimental comparison diagram provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of a road change detection device based on hybrid strip convolution provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Early traditional RSCD methods mainly used artificially designed features to obtain change results, such as transformation-based and image algebra-based RSCD methods to detect changes in these images. For image algebra-based methods, image difference and image regression are usually used to generate difference images (DIs), in which a suitable threshold needs to be selected to determine the change area. Transformation-based methods use transformation techniques such as principal component analysis (PCA) and Tasseled Cap transformation to transform remote sensing images into other spaces and enhance the change information of dual-phase images. The above methods mainly obtain change clues based on artificially designed features. However, artificially designed features have low robustness and insufficient semantics, and are only suitable for low-resolution or medium-resolution RSCD tasks.

[0028] In recent years, deep convolutional neural networks (CNNs) have become the mainstream solution for RSCD tasks with their powerful feature learning capabilities. Compared with traditional methods, deep learning-based methods require less human intervention and can automatically learn features from annotated data. Most of them regard RSCD as a binary segmentation task and detect changes by classifying the extracted bi-temporal features into different categories. At present, most of the RSCD backbone networks based on deep learning adopt encoder-decoder structures such as Fully Convolutional Network (FCN) and U-Net. Due to the scale variation and complex background of remote sensing images, various multi-scale feature fusion modules and attention mechanisms are introduced into deep neural networks for remote sensing image CD. In the prior art, there are schemes that impose constraints on different feature maps recorded at various stages of the encoder to produce more refined representations. The prior art proposes a dense attention refinement network based on the U-type encoder-decoder architecture, which supports progressive adjustment of the predicted change map at each stage of the decoding process. DARNe uses dense skip connections and mixed attention to aggregate multi-layer feature representations and gradually refine the predicted change map from high level to low level. The emergence of ViT provides another approach for the CD task. Specifically, ViT divides the image into a series of patches, treats all patches as a sequence, and uses the attention mechanism to capture the global dependencies between patches in the image. The prior art also uses SwinTransformer to construct the encoder and decoder, and designs a pure transformer network with a Siamese U-shaped structure. ChangeFormer is a change detection model based on the Segformer architecture. In the basic module of ChangeFormer, the query, key, and value are spatially downsampled to reduce the amount of computation. On the other hand, some studies use parallel CNN and Transformer structures. WNet (Windows Net) is proposed, which uses Siamese CNN and Siamese Transformer as encoders and fuses all features in the decoder. Similarly, ICIF-Net (Integrated Column and Instance Fusion Network) is also proposed, which constructs a dual-branch structure of CNN and Transformer to capture multi-scale local and global features respectively, and uses cross attention for fusion.

[0029] Limited by the road change detection dataset, the existing road change detection methods based on deep learning are mainly based on general change detection neural networks. A convolutional Siamese network is proposed, which uses time series images to identify change areas, uses a contrast loss function to minimize the distance between two feature vectors in the unchanged area, and uses deliberate training samples to overcome the road-irrelevant noise in the image. The general change detection network requires complex post-processing steps to find road-related changes. The post-processing steps are complex and lack robustness. In addition, the general change detection network mainly focuses on spectral changes and lacks geometric constraints on boundaries. Therefore, in order to improve the accuracy of road change detection, the present invention provides a road change detection method based on hybrid strip convolution and related equipment.

[0030] Combine the following Figure 1-Figure 8 The present invention describes a road change detection method and related equipment based on hybrid strip convolution.

[0031] Figure 1 FIG. 1 is a flow chart showing a method for detecting road changes based on hybrid strip convolution according to an exemplary embodiment. Figure 1 As shown, in an exemplary embodiment, the road change detection method based on hybrid strip convolution includes steps 110 to 140, which are described in detail as follows.

[0032] Step 110: Acquire two images to be detected for the same position.

[0033] In the embodiment of the present invention, the two images to be detected are images taken at the same location at different times.

[0034] Step 120, extracting the road linear features in each of the images to be detected through a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit.

[0035] In the embodiment of the present invention, a feature enhancement module is constructed by multi-scale strip convolution units, which can also be called a mixed strip convolution feature enhancement module (MSCFE). The feature enhancement module uses multi-scale strip convolution units to capture the long-distance dependencies of road features and avoid the blurring of narrow object boundaries by traditional square convolution. The present invention uses multi-scale strip convolution units to better adapt to the characteristics of road distribution, reduce the blurring of road edges caused by irrelevant areas, and improve the integrity and continuity of road extraction.

[0036] Step 130 , fusing the road linear features in each of the to-be-detected images through a cross-fusion module to obtain fused features.

[0037] In the embodiment of the present invention, the cross fusion module performs fusion processing on two road linear features, interactively propagates their respective spatial information and temporal information during the fusion process, guides the network to focus on road-related change areas, and suppresses the influence of distribution differences in the two images to be detected.

[0038] Step 140: Perform road change detection based on the fused features.

[0039] In the embodiment of the present invention, road change detection with fine details is performed based on fusion features.

[0040] In the embodiment of the present invention, Figure 2 As shown in the figure, the feature enhancement module uses the MobileNetV2 network fused with multi-scale strip convolution units as the feature enhancement module to extract road linear features. Then the cross fusion module (CrossFusion) is used to fuse the road linear features to achieve the transmission of spatial and temporal information between the two road linear features. Then the change feature is obtained by using the feature difference method, and the decoder is used to achieve top-down multi-level change feature fusion to predict the road change map with fine details.

[0041] The MobileNetV2 network used discards the AvgPool layer and the fully connected layer to adapt to the change detection task. The output of the feature enhancement module is H / 32×W / 32×320, where H and W are the input image sizes. Figure 2 Element-wiseSub&Abs in it means element-wise subtraction and element-wise absolute value, Skip Connection means skip connection, and Upsample means upsampling.

[0042] In an exemplary embodiment of the present invention, the multi-scale strip convolution unit includes multiple strip convolution units for different directions; the extraction of road linear features in each of the images to be detected through the feature enhancement module includes the following steps, which are described in detail as follows.

[0043] The first feature in the image to be detected is extracted by each of the strip convolution units respectively.

[0044] The road linear feature of each of the images to be detected is obtained based on the first feature corresponding to each of the images to be detected.

[0045] In the embodiment of the present invention, in the detection of road changes, the road shape has the characteristics of large span and narrowness, but the square convolution kernel will additionally calculate the non-road area when extracting features, and cannot capture the linear features of the road well. At the same time, the square convolution kernel is insensitive to the direction of the road and cannot effectively extract the oblique road features. Therefore, in view of the inherent characteristics of the road, the present invention proposes a feature enhancement module based on multi-scale strip convolution units, which uses strip convolution units in different directions to adapt to roads distributed in different directions. The strip convolution unit is not only conducive to extracting linear features, but also can reduce the amount of calculation.

[0046] In the embodiment of the present invention, each strip convolution unit extracts the first feature in the image to be detected, and all the first features of each image to be detected are mixed to obtain the road linear features of each image to be detected.

[0047] In an exemplary embodiment of the present invention, the multi-scale strip convolution unit includes strip convolution units for the horizontal direction, the vertical direction, the left diagonal direction and the right diagonal direction respectively.

[0048] In the embodiment of the present invention, Figure 3 As shown in the figure, a feature enhancement module is constructed based on the strip convolution units in the horizontal direction, vertical direction, left diagonal direction and right diagonal direction. Suppose the feature dimension of the image to be detected is , D is the number of channels. First, strip pooling is performed along the horizontal and vertical dimensions to obtain features and , the long-range contextual relationship in the spatial dimension is obtained through global average pooling operations in the horizontal and vertical dimensions, and the local information is retained through local convolution (1D Conv) with convolution kernel sizes of H×1 and 1×W. Then, the channel dimension is aggregated through global average pooling (Channel Avg) and global maximum pooling (Channel Max), and left diagonal convolution (Left Diagonal Conv) and right diagonal convolution (Right Diagonal Conv) are used to adapt to the distribution of roads in different directions. Finally, the output features of the four directions are mixed, that is, the first feature. The weight map of the feature map is then obtained through 1×1 convolution (1×1 Conv) and sigmoid activation function, and the input is weighted using the weight map to obtain the feature map enhanced by strip convolution. . Figure 3 Element-wiseAdd in it means pixel-by-pixel addition, and Matrix Multiplication means matrix multiplication.

[0049] In an exemplary embodiment of the present invention, the feature enhancement module further includes a multi-scale hole convolution unit, and the extraction of road linear features in each of the images to be detected by the feature enhancement module includes the following steps, which are described in detail as follows.

[0050] The second feature of each of the images to be detected is extracted by the multi-scale strip convolution unit, and the third feature of each of the images to be detected is extracted by the multi-scale hole convolution unit.

[0051] The second feature and the third feature of each of the images to be detected are connected to obtain the road linear feature of each of the images to be detected.

[0052] In the embodiment of the present invention, in order to effectively learn road features of different scales, a multi-scale dilated convolution unit is added to the feature enhancement module. The dilated convolution enables the model to observe contextual information of a larger area through its enlarged receptive field, which is particularly important for understanding the overall layout of the road. Using a higher dilation rate in the convolution layer can help the model capture the long-range semantic relationship of the road, even when the road is visually discontinuous (such as being blocked by trees or intersecting with other roads). Specifically, Figure 4 As shown, the multi-scale dilated convolution unit uses strip space convolutions with expansion rates of 3, 5, and 7 to capture features of different scales, and then connects the third feature captured by the strip space convolution and the second feature captured by the multi-scale strip convolution unit, and uses 1×1 convolution to map it to a new channel space to obtain the road linear feature. In this embodiment, the second feature is consistent with the first feature mentioned above. Figure 4 Concatenation in it means connection, and Dilation Conv means dilation convolution.

[0053] In an exemplary embodiment of the present invention, the cross-fusion module includes a self-attention mechanism and a depth-wise separable convolution; the cross-fusion module is used to fuse the road linear features in each of the images to be detected to obtain fused features, including the following steps, which are described in detail as follows.

[0054] The road linear features are spliced ​​to obtain road linear splicing features.

[0055] The road linear splicing features are respectively input into the self-attention mechanism and the depthwise separable convolution for processing, and channel interaction and spatial interaction are performed during the processing to obtain output features corresponding to the self-attention mechanism and the depthwise separable convolution, respectively.

[0056] A fusion process is performed based on each of the output features to obtain the fusion feature.

[0057] In the embodiment of the present invention, the linear features of the road have spatiotemporal dependencies. Now, the self-attention mechanism and the temporal attention mechanism can be used to generate attention weight maps in the spatial dimension and the channel dimension to capture the spatiotemporal relationship between the linear features of the road. The self-attention mechanism dynamically calculates weights in the spatial dimension, which can well capture the spatial relationship between pixels, but has weaker modeling capabilities in the channel dimension; the deep separable convolution calculates weights in the channel dimension, which can well capture the spatial relationship between features, but has weaker modeling capabilities in the spatial dimension. In order to explore the spatial correlation and temporal correlation between the linear features of the road, this embodiment proposes a cross-fusion module consisting of a self-attention mechanism and a deep separable convolution. The module structure is as follows: Figure 5 shown.

[0058] like Figure 2 and Figure 5 As shown, the two road linear features are concatenated (Concat) to obtain the road linear splicing feature. The parallel self-attention mechanism and depthwise separable convolution are respectively fed into the network to model the spatial and temporal dependencies between the two road linear features. Channel interaction is used to enhance the channel modeling capability of the self-attention mechanism, and spatial interaction is used to provide spatial clues for the depthwise separable convolution. Based on the output features of the self-attention mechanism and the depthwise separable convolution, the network is constructed. and Get fusion features .

[0059] In the process of channel interaction, global average pooling is used to aggregate features, and then two convolution kernels (1,1) are used for channel interaction. The sigmoid activation function is used to obtain the channel attention map, and the channel attention map is applied to the value to obtain the output features of the self-attention mechanism. In the spatial interaction process, two convolution kernels are used to The convolution reduces the number of channels to 1, and finally obtains the spatial attention map through the sigmoid activation function. Apply the spatial attention map to the depthwise separable convolution to obtain the output features of the depthwise separable convolution. .

[0060] In an exemplary embodiment of the present invention, the fusion processing is performed based on each of the output features to obtain the fusion feature, which includes the following steps, which are described in detail as follows.

[0061] Based on pixels, each of the output features is summed, and the summed results are convolved to obtain a mixed feature; Based on the pixels, the mixed features are subjected to difference processing to obtain the fused features.

[0062] In the embodiment of the present invention, Figure 5 As shown, the two output features are added pixel by pixel to obtain the feature and through Convolution mixed features, get and , and finally and Subtract pixel by pixel to get fusion features .

[0063] In an exemplary embodiment of the present invention, the road change detection based on the fusion feature includes the following steps, which are described in detail as follows.

[0064] The fused features are upsampled and spliced, and a road change map is obtained based on the upsampled and spliced ​​fused features.

[0065] In the embodiment of the present invention, Figure 2 As shown in the figure, the change features at different levels are upsampled (D5-D1) and concatenated. The low-level features have fine-grained detail information, but also contain a lot of noise. The high-level features have more abstract semantic information, which helps to locate the road and suppress noise. The high-to-low difference feature aggregation method is adopted, and the change features at different levels are gradually obtained using deep supervision to predict the road change map with fine details.

[0066] In the embodiment of the present invention, RSCD can essentially be regarded as a pixel-level classification task, and binary cross entropy loss is usually used to optimize the network weights. However, in the road change detection task, there is a class imbalance problem, that is, the proportion of the changed area is much smaller than the proportion of the unchanged area. The binary cross entropy loss treats each pixel equally, so when there are fewer positive samples, only using binary cross entropy loss will cause the network to be dominated by negative samples, resulting in training deviation and the inability to identify positive samples. Due to the complex background of high-resolution remote sensing images, many roads cannot be successfully identified, that is, the number of FNs (False Negatives) far exceeds the number of false positives (False Positives, FPs). In this case, in order to improve the recall rate, FNs need to be punished more than FPs. HEM loss (Hybrid Embedding Loss) controls the balance between FNs and FPs through hyperparameters α and β. At the same time, VHR (Very High Resolution) remote sensing images are complexly distributed, and interference such as occlusion and shadows inevitably make some roads difficult to identify. In order to alleviate this problem, the exponent γ is used in the HEM loss to pay more attention to difficult samples detected with lower probability. Therefore, the present invention combines the binary cross entropy loss and HEM loss , which can be described as: ; ; in, represents the dot product operation, represents the L1 norm, P is the predicted road change map, Y is the corresponding true label, W and H are the width and height of the image to be detected, respectively. The values ​​of α, β, and γ are 0.3, 0.7, and 0.75, respectively.

[0067] Total loss for road change detection It can be expressed as: .

[0068] In an exemplary embodiment of the present invention, experiments are conducted on a change detection dataset WRCD (Wuhan Road Change Detection) containing only roads and a general change detection dataset SYSU (South China Normal University) containing various change scenarios of roads.

[0069] WRCD is a 17-level remote sensing image downloaded from Google Earth. This dataset only contains road changes. The image resolution of each phase is 10884×13655. The image is cropped into a 512×512 patch with an overlap rate of 0.25. The first 48 columns of each phase are taken as test data, and the rest are used as training data, resulting in a total of 1995 pairs of training images and 980 pairs of test images.

[0070] The SYSU dataset consists of 20,000 pairs of dual-temporal remote sensing image patches with a spatial size of 256×256 and a spatial resolution of 0.5m. The ratio of training, validation, and testing is formally set to 6:2:2. The dataset contains various types of complex change scenes, including road expansion, new urban buildings, vegetation changes, suburban expansion, and pre-construction foundation work.

[0071] The present invention uses MobileNetV2 as the backbone, uses random cropping and random flipping to perform data enhancement on the input image, uses the Adam optimizer, and the momentum is 0.9. is 0.9, is 0.99, the initial learning rate lr is 5e-4, and the poly strategy is used to decay the learning rate. As the training progresses, the learning rate is adjusted to ,The total training rounds are set to 100, the power value is 0.9, and the batch size is set to 10. All codes are implemented based on the PyTorch framework, and the experiments are carried out on a server configured with GeForce RTX 4090, the device video memory is 24GB, the operating system version is Ubuntu22.04, and the cuda version is 2.1.2.

[0072] The performance of the road change detection method based on hybrid strip convolution is evaluated by using four widely used evaluation indicators: Intersection over Union (IoU), F1-score (F1), Recall (Rec) and Precision (Pre). Pre reflects the proportion of correct predictions in the positive samples of the prediction results, which measures the precision of the model; Rec reflects the proportion of actual positive samples predicted as positive samples, which measures the recall of the model; F1 is the weighted average of the two; OA reflects the proportion of all correctly classified pixels to the total pixels. The specific definitions of the above evaluation indicators are as follows: ; ; ; ; Among them, Tp, Fp, Tn and Fn represent the number of true positives, false positives, true negatives and false negatives respectively.

[0073] The hybrid strip convolution-based road change detection method proposed in this paper is compared with five state-of-the-art RSCD methods, including four CNN-based methods: TinyCD, USSFCNet, DMINet, A2Net, and a Transformer-based method: BiT. In order to verify the robustness of the proposed method, experiments are carried out on the road change detection dataset WRCD and the general change detection dataset SYSU.

[0074] The change detection results on the WRCD dataset are visualized as follows: Figure 6 The quantification results are shown in Table 1 below. To make the visualization effect more intuitive, four white colors are used to represent the true positive road change detection results.

[0075] Table 1

[0076] From visualization Figure 6 It can be seen that the present invention performs better in three aspects: road detection accuracy, road continuity and distinguishing false changes. In terms of detection accuracy, compared with other methods, the present invention has higher road change detection accuracy and can locate the change target more accurately. Figure 6 In the first and second rows, BiT, TinyCD, USSFCNet, and DMINet can hardly identify road changes. In contrast, the present invention can more accurately identify the changed area. In terms of road continuity, the roads in remote sensing images are long and narrow, resulting in that road changes cannot be fully identified. Compared with other methods, the present invention has more advantages in terms of road continuity and integrity. Figure 6 In the fifth and sixth rows of , the road change regions identified by many methods have poor continuity. In contrast, the present invention can well identify the road change edges. In terms of distinguishing pseudo changes, due to interference from factors such as season, background, and surrounding buildings, pseudo changes are prone to occur in road change detection in remote sensing images. For example, Figure 6 In the fourth and ninth rows of , USSFCNet and A2Net cannot distinguish the pseudo changes caused by buildings. In comparison, the present invention can better identify pseudo changes.

[0077] From Table 1, we can see that the present invention achieves the best recall (0.7395), IoU (0.5742) and F1 score (0.7295) on the WRCD road dataset. Compared with BiT, the recall of the present invention is improved by 13.37%, and compared with A2Net, the recall is improved by 8.57%, indicating that our model can detect more complete road changes. Compared with BiT, our IoU is improved by 9.24%, indicating that the road changes detected by the present invention are more in line with the actual situation.

[0078] In order to further verify the effectiveness of the proposed model in the general change detection task, experiments were conducted on the general change detection dataset SYSU. The quantitative results are shown in Table 2 below.

[0079] Table 2

[0080] By comparison, the present invention also has better results on the SYSU dataset, achieving the best recall rate (0.8484), IoU (0.7200) and F1 score (0.8372). Compared with BiT, the recall rate of the present invention is improved by 8.16%, and the IoU is improved by 6.16%. Compared with A2Net, the recall rate is improved by 4.73%, and the IoU is improved by 0.62%, indicating the effectiveness of the present invention in general change detection tasks.

[0081] In order to verify the effectiveness of MSCFE and CrossFusion, a series of ablation experiments are conducted on the WRCD and SYSU datasets, and the quantitative results are shown in Table 3. The present invention using MSCFE and CrossFusion achieves the best IoU, F1 and recall rates on both the WRCD and SYSU datasets, proving that the present invention can detect complete road changes and is also competitive in general change detection.

[0082] Table 3

[0083] After removing MSCFE and CrossFusion, the F1 score and recall score on both datasets drop significantly, demonstrating the effectiveness of our invention. After removing MSCFE, the recall rate on the WRCD dataset drops from 73.95% to 70.87%, indicating that MSCFE is useful for road change detection. The recall rate on the SYSU dataset drops from 84.84% to 82.50%, indicating that MSCFE is useful for general change detection.

[0084] After removing CrossFusion, the IoU and F1 scores of the model on the WRCD dataset decreased by 2.06% and 1.69%, respectively, and the IoU and F1 scores on the SYSU dataset decreased by 0.73% and 0.50%, respectively, which indicates that CrossFusion improves the change detection results of the model.

[0085] The road change detection device based on hybrid strip convolution provided by the present invention is described below. The road change detection device based on hybrid strip convolution described below and the road change detection method based on hybrid strip convolution described above can be referred to each other. It should be noted that the device provided in the following embodiment and the method provided in the above embodiment belong to the same concept, and the specific way in which each module and unit performs the operation has been described in detail in the method embodiment, which will not be repeated here.

[0086] In an exemplary embodiment of the present invention, see Figure 7 , Figure 7 A road change detection device based on hybrid strip convolution is shown according to an exemplary embodiment, and includes the following modules.

[0087] An acquisition module 710 is configured to acquire two images to be detected for the same position; The extraction module 720 is configured to extract the road linear features in each of the images to be detected through a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; A fusion processing module 730 is configured to perform fusion processing on the road linear features in each of the images to be detected through a cross fusion module to obtain fusion features; The detection module 740 is configured to perform road change detection based on the fused features.

[0088] In an exemplary embodiment of the present invention, the multi-scale strip convolution unit includes a plurality of strip convolution units for different directions; the extraction module 720 includes: A first extraction submodule, configured to extract a first feature in the image to be detected through each of the strip convolution units respectively; The feature submodule is configured to obtain the road linear feature of each of the images to be detected based on the first feature corresponding to each of the images to be detected.

[0089] In an exemplary embodiment of the present invention, the multi-scale strip convolution unit includes strip convolution units for the horizontal direction, the vertical direction, the left diagonal direction and the right diagonal direction respectively.

[0090] In an exemplary embodiment of the present invention, the feature enhancement module further includes a multi-scale dilated convolution unit, and the extraction module 720 includes: A second extraction submodule is configured to extract the second feature of each of the images to be detected through the multi-scale strip convolution unit, and to extract the third feature of each of the images to be detected through the multi-scale hole convolution unit; The connection submodule is configured to connect the second feature and the third feature of each of the images to be detected to obtain the road linear feature of each of the images to be detected.

[0091] In an exemplary embodiment of the present invention, the cross fusion module includes a self-attention mechanism and a depth-separable convolution; the fusion processing module 730 includes: A splicing processing submodule is configured to splice the road linear features to obtain road linear splicing features; An input submodule, configured to input the road linear splicing features into the self-attention mechanism and the depthwise separable convolution for processing, and perform channel interaction and spatial interaction during the processing to obtain output features corresponding to the self-attention mechanism and the depthwise separable convolution respectively; The fusion feature submodule is configured to perform fusion processing based on each of the output features to obtain the fusion feature.

[0092] In an exemplary embodiment of the present invention, the fusion feature submodule includes: A summing processing unit is configured to sum the output features based on pixels, and perform convolution mixing on the summing results to obtain mixed features; The difference processing unit is configured to perform difference processing on the mixed feature based on pixels to obtain the fused feature.

[0093] In an exemplary embodiment of the present invention, the detection module 740 includes: The processing submodule is configured to upsample and splice the fused features, and obtain a road change map based on the fused features after the upsampling and splicing.

[0094] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the road change detection method based on hybrid strip convolution, the method comprising: obtaining two images to be detected for the same position; Extracting the linear features of the road in each of the images to be detected by a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; The road linear features in each of the images to be detected are fused by a cross fusion module to obtain fused features; Road change detection is performed based on the fused features.

[0095] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0096] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the road change detection method based on hybrid strip convolution provided by the above methods, the method includes: obtaining two images to be detected for the same position; Extracting the linear features of the road in each of the images to be detected by a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; The road linear features in each of the images to be detected are fused by a cross fusion module to obtain fused features; Road change detection is performed based on the fused features.

[0097] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the road change detection method based on hybrid strip convolution provided by the above methods, the method comprising: acquiring two images to be detected for the same position; Extracting the linear features of the road in each of the images to be detected by a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; The road linear features in each of the images to be detected are fused by a cross fusion module to obtain fused features; Road change detection is performed based on the fused features.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road change detection method based on hybrid strip convolution, characterized in that: include: Obtain two images to be detected for the same position; Extracting the linear features of the road in each of the images to be detected by a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; The road linear features in each of the images to be detected are fused by a cross fusion module to obtain fused features; Road change detection is performed based on the fused features.

2. The road change detection method based on hybrid strip convolution according to claim 1 is characterized in that: The multi-scale strip convolution unit includes a plurality of strip convolution units for different directions; the feature enhancement module extracts the road linear features in each of the images to be detected, including: Extracting the first feature in the image to be detected by each of the strip convolution units respectively; The road linear feature of each of the images to be detected is obtained based on the first feature corresponding to each of the images to be detected.

3. The road change detection method based on hybrid strip convolution according to claim 2 is characterized in that: The multi-scale strip convolution unit includes strip convolution units for the horizontal direction, the vertical direction, the left diagonal direction and the right diagonal direction respectively.

4. The road change detection method based on hybrid strip convolution according to claim 1, characterized in that: The feature enhancement module further includes a multi-scale hole convolution unit, and the step of extracting the road linear features in each of the images to be detected by the feature enhancement module includes: Extracting the second feature of each of the images to be detected by the multi-scale strip convolution unit, and extracting the third feature of each of the images to be detected by the multi-scale hole convolution unit; The second feature and the third feature of each of the images to be detected are connected to obtain the road linear feature of each of the images to be detected.

5. The road change detection method based on hybrid strip convolution according to claim 1, characterized in that: The cross-fusion module includes a self-attention mechanism and a depth-separable convolution; the cross-fusion module is used to fuse the road linear features in each of the images to be detected to obtain fused features, including: Performing splicing processing on the road linear features to obtain road linear splicing features; Inputting the road linear splicing features into the self-attention mechanism and the depthwise separable convolution for processing respectively, and performing channel interaction and spatial interaction during the processing to obtain output features corresponding to the self-attention mechanism and the depthwise separable convolution respectively; A fusion process is performed based on each of the output features to obtain the fusion feature.

6. The road change detection method based on hybrid strip convolution according to claim 5 is characterized in that: The performing fusion processing based on each of the output features to obtain the fusion feature includes: Based on pixels, each of the output features is summed, and the summed results are convolved to obtain a mixed feature; Based on the pixels, the mixed features are subjected to difference processing to obtain the fused features.

7. The road change detection method based on hybrid strip convolution according to any one of claims 1 to 6, characterized in that: The road change detection based on the fusion feature includes: The fused features are upsampled and spliced, and a road change map is obtained based on the upsampled and spliced ​​fused features.

8. A road change detection device based on hybrid strip convolution, characterized in that: include: An acquisition module, configured to acquire two images to be detected for the same position; An extraction module configured to extract the road linear features in each of the images to be detected through a feature enhancement module; wherein the feature enhancement module is obtained based on a multi-scale strip convolution unit; A fusion processing module, configured to perform fusion processing on the road linear features in each of the images to be detected through a cross fusion module to obtain fusion features; The detection module is configured to perform road change detection based on the fused features.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the road change detection method based on hybrid strip convolution as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the road change detection method based on hybrid strip convolution as described in any one of claims 1 to 7 is implemented.

Images