Remote sensing image semantic change detection method based on hierarchical semantic reasoning

Abstract

The invention discloses a remote sensing image semantic change detection method based on hierarchical semantic reasoning. The method comprises the following steps: step 1, collecting and processing a remote sensing semantic change detection sample; 2, providing a specific construction method of a remote sensing image semantic change detection model; step 3, realizing the model and training a model algorithm; and step 4, testing and precision evaluation are carried out on the trained model, the remote sensing image semantic change detection method based on hierarchical semantic reasoning enhances the precise recognition capability of tiny ground feature changes, further improves the accuracy and stability of the whole remote sensing image semantic change detection, and improves the accuracy and stability of the remote sensing image semantic change detection. Through cross-level feature collaboration and semantic perception modeling, a self-adaptive aggregation mechanism of multi-level associated features is constructed, the detection robustness and generalization ability of tiny ground feature changes in a complex scene are comprehensively improved, and a remote sensing image semantic change detection technology is promoted to develop towards the direction of higher precision and higher adaptability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing information extraction, in particular to a remote sensing image semantic change detection method based on hierarchical semantic reasoning. BACKGROUND

[0002] Change detection, as a basic technology in the field of earth observation, aims to realize quantitative and qualitative evaluation of the same geographical area in different periods through comparative analysis of multi-temporal remote sensing images. Traditional change detection methods are mainly divided into two categories: pixel-based algorithms and object-oriented algorithms. The former, such as image difference, ratio method, principal component analysis and change vector analysis, performs well in medium and low resolution images due to its simplicity and efficiency, but the minimum analysis unit is a single pixel, which is easily disturbed by imaging conditions and complex feature shapes. The latter combines spectral and spatial information to improve detail consistency and detection accuracy through object direct comparison, comparison after classification or multi-temporal segmentation strategies. However, its effectiveness often depends on the quality of prior classification or segmentation, and it still faces the problems of parameter sensitivity and over-segmentation in multi-source heterogeneous images. With the popularity of high-resolution satellite and unmanned aerial vehicle images, the accuracy and robustness of change detection tasks have put forward higher requirements for algorithms, prompting researchers to continuously explore more adaptive technical paths.

[0003] In recent years, the introduction of deep learning technology has brought revolutionary progress to remote sensing change detection. Based on convolutional neural networks (CNN), generative adversarial networks (GAN) and joint sparse representation, through automatic learning of statistical consistency and complex nonlinear features between multi-temporal images, the detection ability of change areas is significantly improved, and preliminary understanding is achieved at the semantic level. However, most existing deep models can only distinguish the binary classification task of "whether change has occurred", and are difficult to further analyze the change type, and have inherent deficiencies in different scale feature fusion and small target detection, which easily leads to semantic inconsistency between change categories and missing or false detection of small feature elements (such as small buildings, narrow roads, small farmland, etc.), restricting its practical value in fine application scenarios.

[0004] To break through the above bottleneck, semantic change detection (also known as multi-class change detection) has gradually become a research hotspot. Semantic change detection not only requires to identify "where" the change occurs, but also needs to analyze "what" type of change, so multi-scale feature fusion and hierarchical semantic modeling become the core technology route. Although the encoding-decoding structure and the long-distance dependence model of the Transformer have made significant progress in large-scale scenarios, in terms of weak target representation, down-sampling information loss and low-contrast environment adaptability, the existing methods are still difficult to balance accuracy and robustness. Based on this, a unified framework that can realize dynamic interaction between deep semantics and shallow geometric details, retain local details and strengthen change prior guidance is needed to further improve the detection performance of small ground elements in remote sensing images and promote the development of semantic change detection technology to higher accuracy and stronger universality.

[0005] Based on this, the present application designs a remote sensing image semantic change detection method based on hierarchical semantic reasoning to solve the above problems. SUMMARY

[0006] The present application aims to provide a remote sensing image semantic change detection method based on hierarchical semantic reasoning to solve the technical problems raised in the background art.

[0007] To achieve the above purpose, the present application provides the following technical solution: a remote sensing image semantic change detection method based on hierarchical semantic reasoning, comprising the following steps:

[0008] Step one, collect and process remote sensing semantic change detection samples;

[0009] Step two, provide a specific construction method of the remote sensing image semantic change detection model;

[0010] Step three, implementation of the model of the present application and model algorithm training;

[0011] Step four, test and accuracy evaluation of the trained model.

[0012] Preferably, in step one, it includes:

[0013] 1) Analysis of ground feature characteristics of remote sensing images at different time phases;

[0014] 2) Ground feature change analysis method of remote sensing images.

[0015] Preferably, in step two, the construction of the framework of the remote sensing image semantic change detection method based on deep learning is realized.

[0016] Preferably, in step three, the twin convolutional neural network built in step two is used as the basic framework, that is, the neural network architecture of two or more identical subnetworks is used, and the model is trained in the Windows system and Pytorch environment, and the weight information is shared in the training, so that fewer parameters are required and overfitting is not easy to occur.

[0017] Preferably, in step four, the results of the model test are comprehensively and systematically evaluated, and quantitative and qualitative results are analyzed and studied.

[0018] Compared with the prior art, the beneficial effects of the present application are:

[0019] The change detection precision of micro-geographical elements in remote sensing images is improved, the accurate identification ability of micro-geographical changes is enhanced, and the accuracy and stability of the overall remote sensing image semantic change detection are further improved. Under the guidance of this idea, a remote sensing image semantic change detection method based on hierarchical semantic reasoning is proposed, which builds an adaptive aggregation mechanism of multi-level associated features through cross-level feature collaboration and semantic perception modeling, comprehensively improves the detection robustness and generalization ability of micro-geographical changes in complex scenes, and promotes the development of remote sensing image semantic change detection technology to higher precision and stronger adaptability. BRIEF DESCRIPTION OF DRAWINGS

[0020] Fig. 1 The model structure of the present application is provided;

[0021] Fig. 2 The hierarchical feature coupling module structure diagram in the present application is provided;

[0022] Fig. 3 The multi-dimensional adaptive reasoning module structure diagram in the present application is provided. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0024] Please refer to Figs. 1-3 The present application provides a technical solution: a remote sensing image weak target change detection method based on hierarchical semantic reasoning. The method comprises the following steps:

[0025] Step one, collect remote sensing image semantic change detection samples;

[0026] The present application is based on remote sensing image to make a sample library. First, various types of remote sensing image public data sets are downloaded from the Internet. Secondly, the collected large amount of image data is sorted, and the damaged or poor quality files are removed. Then data preprocessing is carried out, which includes image cropping, scaling, rotation and other operations to remove irrelevant content and adjust the picture to the appropriate size, and the construction of the sample library is completed.

[0027] Step two, remote sensing image weak target change detection model construction based on hierarchical semantic reasoning;

[0028] In order to overcome the limitations of the existing method in the detection of micro-geographical element change, the present application proposes a remote sensing image semantic change detection method based on hierarchical semantic reasoning. In order to enhance the feature expression of micro-geographical elements in remote sensing images, the present application embeds a hierarchical interactive enhancement module at the end of the double encoder. By cross-level feature reorganization to construct geometric-semantic correlation prior, after multi-scale context feature extraction, the present application innovatively designs a bidirectional feature conduction mechanism to realize the collaborative enhancement of deep semantic understanding and shallow geometric details, and generates more discriminative multi-level semantic feature representation for micro-geographical elements. At the same time, in view of the problem that micro-geographical elements are easy to lose in the down-sampling stage, the present application proposes a multi-granularity dynamic calibration module at the feature fusion stage. Based on the heterogeneous granular perception strategy, the present application constructs a parallel fusion architecture to realize the dynamic balance of local detail response enhancement and global semantic constraint, and realizes feature adaptive fusion combined with channel-space dual-domain attention weighting, which effectively preserves the spatial continuity of micro-geographical elements.

[0029] The main structure and composition of the network model of the present application will be described in detail below. The model of the present application mainly includes: hierarchical interactive enhancement module, multi-granularity dynamic calibration module.

[0030] Hierarchical interactive enhancement module: in view of the feature expression bottleneck of low brightness, fuzzy boundary and low contrast of micro-geographical elements, in order to strengthen the perception ability of the network to the key area, the present application introduces geometric-semantic correlation prior before feature fusion to guide the network to focus on the significant change area.

[0031] As Fig. 2As shown, the feature extraction stage adopts a pre-trained ResNet34 as a learnable backbone network (twin weight sharing structure) to extract multi-level local features from the dual temporal remote sensing image pair. Subsequently, the hierarchical interactive enhancement module receives the triple feature input of the shallow F1 (high-resolution detail features), the middle F2 (spatial structure features), and the deep F3 (high-level semantic features) extracted from the encoder and performs preliminary fusion. By fusing high-level semantic features with shallow detail information, the hierarchical interactive enhancement module can compensate for the lack of low-level features in global information while enhancing the network's ability to depict small ground elements. Subsequently, the fused features are output through residual connection and bidirectional feature transmission mechanism, strengthening the dynamic interaction between deep semantic features and shallow geometric details, providing prior knowledge about the location, shape, and context environment of small ground elements for subsequent feature fusion, and also effectively distinguishing targets from backgrounds, improving the feature expression ability of small ground elements. The relevant formulas are shown as follows:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037] Multi-granularity dynamic calibration module: To overcome the problem of layer-by-layer loss of small ground elements under the traditional encoding-decoding architecture, and to address the challenges of complex background interference and detail loss in the encoding stage, this module constructs a parallel fusion architecture based on a heterogeneous granular perception strategy. The multi-granularity dynamic calibration module adopts a multi-scale parallel fusion architecture to realize adaptive calibration of local and global features, and performs multi-granularity reasoning through dynamic weight distribution, thereby enhancing the feature expression ability.

[0038] As shown in Fig. 3 , the module design includes three parallel branches: local branch, global branch, and skip branch. First, the same level but different time two tensors are added and aggregated, then different patch sizes (such as 2x2, 4x4) are controlled to distinguish between local and global branches. Finally, the outputs of the three branches are added and fused, and the final output is obtained by combining the weight calculation of channel and spatial attention mechanism, realizing the dynamic integration of local and global features, i.e., suppressing complex background interference and strengthening the detail response of small ground elements. The relevant formulas are shown as follows:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] Step three, model implementation and training;

[0045] In terms of software environment configuration for the experiment, all experiments in this paper are operated on the Windows operating system. The network model and the model used in the comparative experiment are based on PyTorch as a deep learning framework, and the deep learning environment for model training is configured by Anaconda. The corresponding Python version used is 3.9, and the final development platform of the experiment is Pycharm. In the training and testing process of the network model, Numpy, OpenCV and other dependent libraries are mainly used. At the same time, in order to improve the model running rate, GPU acceleration technology is adopted, and parallel programming model CUDA and acceleration library cnDNN are used for parallel computing acceleration of GPU to speed up the processing of complex image processing tasks.

[0046] In terms of hardware environment configuration for the experiment, all experiments in this paper are run on a high-performance computer configured with two Nvidia A6000 graphics cards for parallel acceleration, an Intel Core i7-13700F CPU with 16 cores, and Kingston DDR4 3600MHz 128GB memory. In addition, a Kingston 2T solid state disk and a Seagate 4T mechanical hard disk are configured.

[0047] In the training part of the model, the proposed model is trained on the SECOND dataset using the Sgdm optimizer, with an initial learning rate of 0.1. The learning rate is exponentially decayed according to the set epochs, with a weight decay of 0.0005 and a total batch size of 8. Data augmentation techniques, including horizontal / vertical flipping and random rotation, are used during training to increase the scale and diversity of training data.

[0048] Step four, model testing and evaluation;

[0049] The model is tested using the test set of SECOND, which covers the following 6 object classes: water, ground, low vegetation, trees, buildings, and sports fields. Finally, the experiment evaluates the results on all classes using mIoU, Sek, and OA as evaluation indicators, and takes the average value as the final accuracy.

[0050] Among them, step one is to collect and process remote sensing semantic change detection samples; step two provides a remote sensing image semantic change detection method based on hierarchical semantic reasoning, which is the focus of the present application, step three is mainly the implementation of the model and the model algorithm training content, and step four tests and evaluates the precision of the trained model.

[0051] Although embodiments of the present application have been shown and described, it is to be understood that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A remote sensing image semantic change detection method based on hierarchical semantic reasoning, characterized in that, It comprises the following steps: Step one, collecting and processing remote sensing semantic change detection samples; Step two, providing a specific construction method of the remote sensing image semantic change detection model; Step three, implementation of the model and model algorithm training; Step four, testing and precision evaluation of the trained model. 2.The hierarchical semantic reasoning based remote sensing image semantic change detection method according to claim 1, characterized in that: In step one, it comprises: 1) analysis of feature characteristics of remote sensing images at different time; 2) analysis method of feature changes of remote sensing images. 3.The hierarchical semantic reasoning based remote sensing image semantic change detection method according to claim 2, characterized in that: In step two, the construction of a remote sensing image semantic change detection method framework based on deep learning is realized.

4. The method of claim 3, wherein the hierarchical semantic reasoning based remote sensing image semantic change detection method is characterized by: In step three, the twin convolutional neural network built in step two is used as the basic framework, that is, the neural network architecture of two or more identical subnetworks is used, and the model is trained in the Windows system and Pytorch environment, and the weight information is shared in the training, so that fewer parameters are required and overfitting is not prone to occur.

5. The method of claim 4, wherein the hierarchical semantic reasoning based remote sensing image semantic change detection method is characterized by: In step four, the results of the model test are comprehensively and systematically evaluated, and quantitative and qualitative results are analyzed and studied.

Images