Semantic change detection method and device considering geoscience attributes

Abstract

The invention relates to the technical field of semantic change detection, in particular to a geoscience attribute-considered semantic change detection method and device, and the method comprises the steps: obtaining dual-temporal remote sensing image data, and carrying out the data enhancement processing of the dual-temporal remote sensing image data, so as to obtain a dual-temporal semantic change detection data set, the two-time-phase images comprise semantic tags of the two-time-phase images and corresponding change areas; inputting the dual-temporal semantic change detection data set into a preset semantic change detection network model considering geoscience attributes, so as to output a semantic change position and a semantic change category of the dual-temporal image by utilizing the preset semantic change detection network model considering the geoscience attributes; and generating a semantic change result of the double-temporal image according to the semantic change position and the semantic change category. According to the method, the semantic change detection model considering the geoscience attributes is constructed, the spatial-temporal correlation of the earth surface coverage can be enhanced while the spatial-temporal heterogeneity of the earth surface coverage is inhibited, and therefore high-precision semantic change detection is achieved.

Description

Technical Field

[0001] This application relates to the technical field of semantic change detection, and particularly relates to a semantic change detection method and device considering geoscience attributes. Background Art

[0002] Surface cover change is a geographical phenomenon in the macroscopic and low-speed Earth system, which reflects the intricate interaction between ecological environment change and human activities. Remote sensing large spatio-temporal data has multi-dimensional characteristics such as time, space, and attributes, providing guarantees for discovering and analyzing this geographical phenomenon. The research focus of previous work was binary change detection and object change detection. The former emphasizes the changed area without caring about the category, while the latter emphasizes the change of specific categories. Recent work has focused on semantic change detection, which pays attention to both the changed area and the changed category, that is, "from - to" multiple change types. Therefore, semantic change detection is a valuable and challenging task for detecting surface cover change.

[0003] The specific geoscience attributes of surface cover will have mixed effects on the semantic change detection task. Specifically, the spatio-temporal heterogeneity of surface cover will introduce significant within-class variance, while the spatio-temporal correlation of surface cover will provide a reference for surface cover change.

[0004] However, although the semantic change detection work in related technologies mainly uses a multi-task framework and achieves considerable semantic-change detection results by constraining the consistency between the two sub-tasks of semantic segmentation and binary change detection, these works lack sufficient exploration of the geoscience attributes of ground objects that play a key role in semantic change detection and urgently need to be solved. Summary of the Invention

[0005] This application provides a semantic change detection method and device considering geoscience attributes to solve the problem that the semantic change detection work in related technologies mainly uses a multi-task framework and achieves considerable semantic-change detection results by constraining the consistency between the two sub-tasks of semantic segmentation and binary change detection, but these works lack sufficient exploration of the geoscience attributes of ground objects that play a key role in semantic change detection.

[0006] An embodiment of the first aspect of the present application provides a semantic change detection method considering geoscience attributes, including the following steps: obtaining dual-temporal remote sensing image data, and performing data augmentation processing on the dual-temporal remote sensing image data to obtain a dual-temporal semantic change detection data set, where the dual-temporal semantic change detection data set includes dual-temporal images and semantic labels of their corresponding change regions; inputting the dual-temporal semantic change detection data set into a preset semantic change detection network model considering geoscience attributes to output the semantic change positions and semantic change categories of the dual-temporal images by using the preset semantic change detection network model considering geoscience attributes; generating a semantic change result of the dual-temporal images according to the semantic change positions and the semantic change categories.

[0007] Optionally, in an embodiment of the present application, the performing data augmentation processing on the dual-temporal remote sensing image data to obtain the dual-temporal semantic change detection data set includes: performing random histogram equalization processing, random rotation processing, and random color jitter processing on the dual-temporal remote sensing image data to obtain the dual-temporal semantic change detection data set.

[0008] Optionally, in an embodiment of the present application, before inputting the dual-temporal semantic change detection data set into the preset semantic change detection network model considering geoscience attributes, it further includes: collecting optical remote sensing image pairs of the same region and different temporal phases in the historical land cover area, registering the optical remote sensing image pairs to obtain registered image pairs; generating change samples through the registered image pairs to construct a training set and a test set by using the change samples, and constructing the preset semantic change detection network model considering geoscience attributes by using the training set and the test set.

[0009] Optionally, in an embodiment of the present application, before inputting the dual-temporal semantic change detection data set into the preset semantic change detection network model considering geoscience attributes, it further includes: performing the data augmentation processing on the training data in the training set to obtain dual-temporal training images, inputting the dual-temporal training images and semantic labels of their corresponding change regions into a pre-trained semantic change detection network model to generate temporal semantic features through a siamese semantic encoder in the pre-trained semantic change detection network model; generating implicit semantic change features based on the temporal semantic features, generating a semantic change map of one temporal phase based on an implicit change detection decoder and the implicit semantic change features in the pre-trained semantic change detection network model, and generating a semantic segmentation map of another temporal phase except the one temporal phase through a semantic decoder and the temporal semantic features in the pre-trained semantic change detection network model; using the semantic change map to mask the semantic segmentation map to generate a semantic change training result to construct the preset semantic change detection network model considering geoscience attributes.

[0010] Optionally, in an embodiment of the present application, before inputting the dual-temporal semantic change detection data set into the preset semantic change detection network model considering geospatial attributes, it further includes: calculating the semantic segmentation loss and the direction-independent implicit change detection loss of the pre-trained semantic change detection network model; calculating the semantic change detection loss according to the average of the semantic segmentation loss and the direction-independent implicit change detection loss, so as to determine the preset semantic change detection network model considering geospatial attributes through the semantic change detection loss.

[0011] An embodiment of the second aspect of the present application provides a semantic change detection device considering geospatial attributes, including: a first processing module, configured to obtain dual-temporal remote sensing image data and perform data augmentation processing on the dual-temporal remote sensing image data to obtain the dual-temporal semantic change detection data set, where the dual-temporal semantic change detection data set includes dual-temporal images and semantic labels of their corresponding change regions; a detection module, configured to input the dual-temporal semantic change detection data set into a preset semantic change detection network model considering geospatial attributes, so as to use the preset semantic change detection network model considering geospatial attributes to output the semantic change positions and semantic change categories of the dual-temporal images; a first generation module, configured to generate a semantic change result of the dual-temporal images according to the semantic change positions and the semantic change categories.

[0012] Optionally, in an embodiment of the present application, the first processing module includes: a processing unit, configured to perform random histogram equalization processing, random rotation processing, and random color jitter processing on the dual-temporal remote sensing image data to obtain the dual-temporal semantic change detection data set.

[0013] Optionally, in an embodiment of the present application, it further includes: a collection module, configured to collect optical remote sensing image pairs of the same region and different temporal phases in the historical land cover area before inputting the dual-temporal semantic change detection data set into the preset semantic change detection network model considering geospatial attributes, and register the optical remote sensing image pairs to obtain a registered image pair; a second generation module, configured to generate change samples through the registered image pair, so as to use the change samples to construct a training set and a test set, and use the training set and the test set to construct the preset semantic change detection network model considering geospatial attributes.

[0014] Optionally, in an embodiment of the present application, it further includes: a second processing module, configured to perform the data augmentation processing on the training data in the training set before inputting the dual-temporal semantic change detection data set into the preset semantic change detection network model considering geospatial attributes, to obtain dual-temporal training images, and input the dual-temporal training images and the semantic labels of their corresponding change regions into the pre-trained semantic change detection network model, so as to generate temporal semantic features through the siamese semantic encoder in the pre-trained semantic change detection network model; a third generation module, configured to generate implicit semantic change features based on the temporal semantic features, so as to generate a semantic change map of one temporal phase based on the implicit change detection decoder and the implicit semantic change features in the pre-trained semantic change detection network model, and generate a semantic segmentation map of another temporal phase except the one temporal phase through the semantic decoder and the temporal semantic features in the pre-trained semantic change detection network model; a construction module, configured to use the semantic change map to mask the semantic segmentation map to generate a semantic change training result, so as to construct the preset semantic change detection network model considering geospatial attributes.

[0015] Optionally, in an embodiment of the present application, it further includes: a calculation module, configured to calculate the semantic segmentation loss and the direction-independent implicit change detection loss of the pre-trained semantic change detection network model before inputting the dual-temporal semantic change detection data set into the preset semantic change detection network model considering geospatial attributes; a determination module, configured to calculate the semantic change detection loss according to the average of the semantic segmentation loss and the direction-independent implicit change detection loss, so as to determine the preset semantic change detection network model considering geospatial attributes through the semantic change detection loss.

[0016] An embodiment of the third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the semantic change detection method considering geospatial attributes as described in the above embodiment.

[0017] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the program is executed by a processor, it implements the semantic change detection method considering geospatial attributes as above.

[0018] An embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, and when the computer program is executed, it is used to implement the semantic change detection method considering geospatial attributes as above.

[0019] The embodiments of the present application can generate the semantic change positions and semantic change categories of the dual-temporal images through the dual-temporal remote sensing image data of surface coverage and the semantic change detection network model considering geoscience attributes, and finally generate the semantic change results of the dual-temporal images. Thus, the semantic change detection task of surface coverage is decoupled into two interrelated subtasks: semantic segmentation in one temporal phase and semantic-centered implicit change detection in the other temporal phase. While suppressing the spatio-temporal heterogeneity of surface coverage, the spatio-temporal correlation of surface coverage is enhanced, and the learned specific information is mutually supplemented through joint representation learning, so as to achieve high-precision semantic change detection. Moreover, based on this process, the present application constructs a semantic change detection network model considering geoscience attributes. The disentangled semantic encoder generated by the contrastive learning strategy in the semantic change detection network model can provide an accurate understanding of semantic categories for the semantic segmentation and implicit change detection branches, so as to simply and efficiently implement dual-temporal semantic change detection, accurately locate the change positions of dual-temporal images and identify the semantic categories of changes. Thus, it solves the problem that the semantic change detection work in the related art mainly focuses on the multi-task framework, and achieves considerable semantic-change detection results by constraining the consistency between the two subtasks of semantic segmentation and binary change detection. However, these works lack sufficient exploration of the geoscience attributes of the ground objects that play a key role in semantic change detection.

[0020] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0022] Figure 1 is a flowchart of a method for semantic change detection considering geoscience attributes according to an embodiment of the present application;

[0023] Figure 2 is a flowchart of a method for semantic change detection considering geoscience attributes according to an embodiment of the present application;

[0024] Figure 3 is a schematic structural diagram of a semantic change detection network model considering geoscience attributes according to an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of a semantic change detection result considering geoscience attributes according to an embodiment of the present application;

[0026] Figure 5 is a schematic structural diagram of a semantic change detection device considering geoscience attributes according to an embodiment of the present application;

[0027] Figure 6 Schematic structural diagram of an electronic device provided according to an embodiment of the present application.

[0028] Reference numerals:

[0029] 10 - Semantic change detection device considering geoscience attributes: 100 - First processing module, 200 - Detection module, and 300 - First generation module; 601 - Memory, 602 - Processor, and 603 - Communication interface. Detailed implementation manners

[0030] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as a limitation to the present application.

[0031] The method and device for semantic change detection considering geoscience attributes according to embodiments of the present application will be described below with reference to the accompanying drawings. In the related art for semantic change detection mentioned in the above background art, the multi - task framework is mainly used, and considerable semantic - change detection results are achieved by constraining the consistency between the two sub - tasks of semantic segmentation and binary change detection. However, these works lack exploration of the geoscience attributes of ground objects that play a key role in semantic change detection. The present application provides a method for semantic change detection considering geoscience attributes. In this method, the semantic change positions and semantic change categories of the two - temporal remote sensing image data of land cover can be generated through a semantic change detection network model considering geoscience attributes, and finally the semantic change results of the two - temporal images are generated. Thus, the semantic change detection task of land cover is decoupled into two interrelated sub - tasks: semantic segmentation in one temporal phase and semantic - centered implicit change detection in another temporal phase. While suppressing the spatio - temporal heterogeneity of land cover, the spatio - temporal correlation of land cover is enhanced, and the learned specific information is mutually supplemented through joint representation learning, thereby achieving high - precision semantic change detection. Moreover, the present application constructs a semantic change detection network model considering geoscience attributes based on this process. The disentangled semantic encoder generated by the contrastive learning strategy in this semantic change detection network model can provide an accurate understanding of semantic categories for the semantic segmentation and implicit change detection branches, so that the two - temporal semantic change detection can be simply and efficiently realized, accurately locating the change positions of the two - temporal images and identifying the semantic categories of the changes. Thus, it solves the problem that in the related art for semantic change detection, the multi - task framework is mainly used, and considerable semantic - change detection results are achieved by constraining the consistency between the two sub - tasks of semantic segmentation and binary change detection, but these works lack exploration of the geoscience attributes of ground objects that play a key role in semantic change detection.

[0032] Specifically, Figure 1 is a flowchart of a semantic change detection method considering geoscience attributes provided by an embodiment of the present application.

[0033] As Figure 1 shown, the semantic change detection method considering geoscience attributes includes the following steps:

[0034] In step S101, obtain dual-temporal remote sensing image data of land cover, and perform data enhancement processing on the dual-temporal remote sensing image data to obtain a dual-temporal semantic change detection data set, where the dual-temporal semantic change detection data set includes dual-temporal images and semantic labels of their corresponding change regions.

[0035] It can be understood that land cover generally refers to various features covering the Earth's surface by natural and artificial factors, and these features can be identified through direct observation or remote sensing means. It reflects the natural attributes of the land, and its nature mainly depends on natural factors such as climate, topography, soil, hydrology, and geological conditions. Land cover change is a geographical phenomenon in a macro low-speed Earth system, which reflects the intricate interaction between ecological environment change and human activities. Remote sensing large spatio-temporal data has multi-dimensional characteristics such as time, space, and attributes, which can provide guarantee for discovering and analyzing this geographical phenomenon.

[0036] In some embodiments, the present application can obtain dual-temporal remote sensing image data, where the dual-temporal remote sensing image data can be understood here as remote sensing images of land cover obtained at different times. Data enhancement processing can be understood here as image enhancement processing performed on the dual-temporal remote sensing images.

[0037] After the dual-temporal remote sensing image data is subjected to data enhancement processing in an embodiment of the present application, a dual-temporal semantic change detection data set can be obtained, which includes dual-temporal images and semantic labels of their corresponding change regions. Among them, the dual-temporal images can be understood here as two remote sensing images of the same region observed by land cover at different time points respectively. By comparing the differences between these two images, the change information of land cover can be captured.

[0038] Then, the embodiment of the present application can input the dual-temporal images and semantic labels of their corresponding change regions (land cover change regions) into a preset semantic change detection network model considering geoscience attributes to detect the semantic change of land cover by using the preset semantic change detection network model considering geoscience attributes. Among them, the preset semantic change detection network model considering geoscience attributes can be understood here as a preset and trained semantic change detection network model considering geoscience attributes, which can detect the semantic change of land cover.

[0039] Optionally, in an embodiment of the present application, data augmentation processing is performed on the dual-temporal remote sensing image data to obtain a dual-temporal semantic change detection dataset, including: performing random histogram equalization processing, random rotation processing, and random color jitter processing on the dual-temporal remote sensing image data to obtain a dual-temporal semantic change detection dataset.

[0040] Based on the relevant descriptions of other embodiments, it can be understood that the present application can perform data augmentation processing on the dual-temporal remote sensing image data of land cover to obtain a dual-temporal semantic change detection dataset for semantic change detection of land cover.

[0041] In some embodiments, when the present application performs data augmentation processing on the dual-temporal remote sensing image data of land cover, it mainly but not limited to performing data processing such as random histogram equalization processing, random rotation processing, and random color jitter processing on the dual-temporal remote sensing image data, so as to complete the image enhancement of the dual-temporal remote sensing image and obtain dual-temporal image data.

[0042] Among them, the random histogram equalization processing can reallocate the image pixel values by non-linearly stretching the image, so that the gray histogram of the image changes from a relatively concentrated gray interval to a uniform distribution within the entire gray range. This processing aims to improve the generalization ability of the deep learning model.

[0043] The random rotation processing refers to rotating the image at a certain random angle, which can perform data augmentation on the image, thereby improving the generalization ability of the deep learning model.

[0044] The random color jitter processing can change the color values of pixels by adding random noise between adjacent pixels of the image, so as to achieve a visual jitter effect.

[0045] Optionally, in an embodiment of the present application, before inputting the dual-temporal semantic change detection dataset into a preset semantic change detection network model considering geoscience attributes, it further includes: collecting optical remote sensing image pairs of the same area and different time phases in the historical land cover area, registering the optical remote sensing image pairs to obtain registered image pairs; generating change samples through the registered image pairs, so as to construct a training set and a test set by using the change samples, and constructing a preset semantic change detection network model considering geoscience attributes by using the training set and the test set.

[0046] In the actual execution process, before inputting the dual-temporal image data and the semantic labels of its corresponding land cover change area into a certain semantic change detection network model, it is necessary to first construct a certain semantic change detection network model considering geoscience attributes. In the construction process of the semantic change detection network model considering geoscience attributes, a certain amount of data is required for training and testing.

[0047] Based on this, embodiments of the present application can collect optical remote sensing image pairs of the same area at different time phases in the historical land cover area and process them. For example, the optical remote sensing image pairs are registered to obtain registered image pairs, and certain change samples are generated from the remote sensing image pairs, so as to divide the change samples into a training set and a test set, and use them respectively for training and testing in the construction process of the semantic change detection network model.

[0048] For example, the present application can collect optical remote sensing image pairs of the same area at different time phases where land cover changes have occurred for the image pairs perform registration to align pixels at the same geographical location; then use sliding cropping to crop the registered image pairs into a number of image patches of a fixed size, and an image patch of one time phase and an image patch of another time phase form a change sample; finally, use all the change samples to construct a two-time-phase semantic change detection data set, and divide this data set into a training set and a test set, so as to use the training set to train the semantic change detection network model and use the test set to test the effectiveness of the semantic change detection network model.

[0049] Figure 2 is a flowchart for constructing a semantic change detection network model considering geoscience attributes according to an embodiment of the present application. As Figure 2 shown:

[0050] Step S201, construct a semantic change detection data set, including a training set and a test set;

[0051] Step S202, construct a pre-trained semantic change detection network model considering geoscience attributes;

[0052] Step S203, use the training set to retrain the pre-trained semantic change detection network model considering geoscience attributes;

[0053] Step S204, test the trained semantic change detection network model considering geoscience attributes;

[0054] Step S205, test the trained semantic change detection network model considering geoscience attributes through the two-time-phase remote sensing image pairs of the test set;

[0055] Step S206, output the test results to obtain the effectiveness of the semantic change detection network model considering geoscience attributes.

[0056] Optionally, in an embodiment of the present application, before inputting the dual-temporal semantic change detection dataset into the preset semantic change detection network model considering geoscience attributes, it further includes: performing data augmentation on the training data in the training set to obtain dual-temporal training images, and inputting the dual-temporal training images and the semantic labels of their corresponding change regions into the pre-trained semantic change detection network model, so as to generate temporal semantic features through the siamese semantic encoder in the pre-trained semantic change detection network model; based on the temporal semantic features, generating implicit semantic change features, so as to generate a semantic change map of one temporal phase based on the implicit change detection decoder and the implicit semantic change features in the pre-trained semantic change detection network model, and generating a semantic segmentation map of another temporal phase except one temporal phase through the semantic decoder and the temporal semantic features in the pre-trained semantic change detection network model; using the semantic change map to mask the semantic segmentation map to generate a semantic change training result, so as to construct the preset semantic change detection network model.

[0057] In other embodiments, the data in the training set and the test set also need to be subjected to data augmentation processing, namely random histogram equalization processing, random rotation processing, random color jitter processing, etc., to perform image enhancement on the remote sensing image (image) to obtain dual-temporal training images. Then, the dual-temporal training images and the semantic labels of the corresponding change regions are used as the input of the pre-trained semantic change detection network model considering geoscience attributes, and the pre-trained semantic change detection network model is trained again. Figure 3 Schematic diagram of the architecture of the semantic change detection network model considering geoscience attributes according to an embodiment of the present application. As Figure 3 shown:

[0058] In the embodiment of the present application, the semantic change detection network model considering geoscience attributes mainly but not limited to consists of two parts: an encoder structure and a decoder structure. Among them, the encoder includes but not limited to a semantic encoder and a spatio-temporal encoder. The semantic encoder can but not limited to be composed of a convolution, a regularization, and a ReLU activation function module. Among them, the spatio-temporal encoder can but not limited to be composed of a relative position encoding, a self-attention, and a multi-layer perception module; the decoder includes but not limited to a multi-task decoder composed of an implicit change detection decoder and a semantic decoder, which mainly but not limited to is composed of a convolution, an upsampling, and a ReLU activation function module. And, 6 repeated residual blocks are used for connection in both the encoder and the decoder.

[0059] First, the embodiment of the present application can generate temporal semantic features through the basic siamese semantic encoder in the pre-trained semantic change detection network model; then abstract the category prototype P in the temporal semantic features through the self-prototype estimation algorithm C, a disentangled semantic encoder is trained through a pixel-to-prototype and prototype-to-prototype contrast learning strategy within a time phase to overcome the spatio-temporal heterogeneity of land cover changes and obtain disentangled time-phase semantic features

[0060] In the embodiments of the present application, the disentangled semantic encoder generated by the contrast learning strategy can provide an accurate understanding of semantic categories for the semantic segmentation and implicit change detection branches. Compared with the traditional method that indirectly learns the relative distance between different category embeddings by optimizing pixel-level predictions through cross-entropy loss, the contrast learning strategy uses category prototypes as anchors for pixel features, directly pulling closer pixel-level features of the same category and pushing away heterogeneous features.

[0061] Subsequently, the embodiments of the present application can perform channel splicing on the time-phase semantic features, use them as the input of the spatio-temporal correlation transformer, and generate implicit semantic change features and integrate them into the implicit change detection decoder and the semantic decoder through a certain decoder to achieve multi-task decoding, that is, a semantic change map of one time phase output by the implicit change detection decoder Generate a semantic change map by masking the semantic segmentation map of another time phase output by the semantic decoder Obtain the semantic change detection results of the dual-time-phase images.

[0062] Finally, the embodiments of the present application can decouple semantic change detection into two interrelated subtasks: semantic segmentation in one time phase and semantic-centered implicit change detection in another time phase, suppress the spatio-temporal heterogeneity of land cover while enhancing the spatio-temporal correlation of land cover, and mutually complement the learned specific information through joint representation learning to construct a semantic change detection network model considering geoscience attributes and complete high-precision semantic change detection.

[0063] Optionally, in an embodiment of the present application, before inputting the dual-time-phase semantic change detection dataset into a preset semantic change detection network model considering geoscience attributes, it further includes: calculating the semantic segmentation loss and the direction-independent implicit change detection loss of the pre-trained semantic change detection network model; calculating the semantic change detection loss according to the average of the semantic segmentation loss and the direction-independent implicit change detection loss to determine the preset semantic change detection network model considering geoscience attributes through the semantic change detection loss.

[0064] In some embodiments, the present application can also construct a certain objective loss function to calculate the semantic change detection loss of the semantic change detection network model. By minimizing the multi-task loss function and combining the stochastic gradient descent optimization algorithm to iteratively update the model parameters of the semantic segmentation and semantic change detection branches, the final semantic change detection network model is determined.

[0065] For example, the present application may adopt a semantic segmentation loss and an orientation-independent implicit change detection loss and use their average to determine the semantic change detection loss The formula can be but is not limited to being expressed as follows:

[0066]

[0067] Among them, in order to address the challenge of limited semantic annotation, in the embodiments of the present application Based on the cross-entropy loss, an intra-phase pixel-to-prototype and an inter-phase prototype-to-prototype contrast loss are introduced. The formula can be but is not limited to being expressed as follows:

[0068]

[0069] Among them, C is the number of semantic categories, y i and p i respectively represent the true value and the predicted probability that the sample belongs to the i-th category, P + and P _ are the positive prototype and negative prototype of pixel i respectively, e i is the pixel-level temporal semantic feature, and τ is the temperature parameter.

[0070] Furthermore, the orientation-independent implicit change detection loss consists of two parts. One is the implicit semantic change loss in the direction from t 1 to t 2 , and the other is the implicit semantic change loss in the direction from t 2 to t 1 . Both use cross-entropy to calculate the loss. The formula can be but is not limited to being expressed as follows:

[0071]

[0072] Among them, the weight w i is the proportion of each semantic change category. In , y i is the semantic change label of t 1 , and in , y i is the semantic change label of t 2 .

[0073] So far, the construction of the semantic change detection network model considering geoscience attributes has been completed. The constructed semantic change detection model considering geoscience attributes can use the class prototype as the anchor point of pixel features, adopt the pixel-to-prototype and prototype-to-prototype contrast learning strategies to suppress the spatio-temporal heterogeneity of land cover, provide a shared and accurate semantic class understanding for semantic segmentation and semantic change detection, and at the same time use the spatio-temporal transformer to associate spatio-temporal change dependencies to enhance the spatio-temporal correlation of land cover, which helps to achieve high-precision "from-to" semantic change detection and output the "from-to" semantic change result.

[0074] In the test stage, the embodiments of the present application can use the test set to test the constructed semantic change detection network model considering geoscience attributes. Compare the semantic change detection method considering geoscience attributes (S-cCDNet) including the semantic change detection network model considering geoscience attributes in the embodiments of the present application with other deep learning-based semantic change detection methods on the Second dataset. The precision metrics of semantic change include the overall accuracy OA, intersection over union IoU, separable kappa coefficient Sek, recall rate Pscd, and the harmonic mean Fscd of the precision rate Rscd, etc. Table 1 is a comparison table of the quantitative results of S-cCDNet and other comparison methods on the Second dataset, which can be shown as follows:

[0075] Table 1

[0076]

[0077] As can be seen from Table 1, compared with other methods, the semantic change detection method considering geoscience attributes (S-cCDNet) in the embodiments of the present application has achieved good results in each index, with the highest recall rate of 68.92%, the highest intersection over union of 59.35%, and the highest harmonic mean of 64.99%. These results all prove the ability of the semantic change detection method considering geoscience attributes (S-cCDNet) in this embodiment in semantic and semantic change information mining. Figure 4 It is a schematic diagram of the semantic change detection result considering geoscience attributes of an embodiment of the present application, and part of the visualization results are as Figure 4 shown.

[0078] Step S102, input the dual-temporal semantic change detection dataset into a preset semantic change detection network model considering geoscience attributes, so as to use the preset semantic change detection network model considering geoscience attributes to output the semantic change positions and semantic change categories of the dual-temporal images.

[0079] Those skilled in the art can understand that specific geoscience attributes of land cover will have uneven impacts on the semantic change detection task. Specifically, the spatio-temporal heterogeneity of land cover will introduce significant within-class variances, while the spatio-temporal correlation of land cover will provide references for land cover changes.

[0080] As a possible implementation, after constructing a semantic change detection network model that takes into account geoscience attributes, the present application can use the trained semantic change detection network model that takes into account geoscience attributes to detect the semantic changes in the bi-temporal images.

[0081] During the detection process, the embodiments of the present application mainly use the trained semantic change detection network model that takes into account geoscience attributes to suppress the spatio-temporal heterogeneity of land cover and enhance the spatio-temporal correlation of land cover. In this case, the semantic change positions and semantic change categories of the land cover with high precision are output.

[0082] Step S103, generate the semantic change result of the land cover according to the semantic change position and the semantic change category.

[0083] In some embodiments, after obtaining the semantic change position and the semantic change category of the land cover, the present application can obtain the final semantic change result of the land cover according to the semantic change position and the semantic change category.

[0084] The embodiments of the present application can implement bi-temporal semantic change detection by using a semantic change detection network model that takes into account geoscience attributes, can locate the change region and the semantic category of the change, focus on the semantic-change of interest, and can achieve a high recall rate in semantic change detection.

[0085] According to the semantic change detection method considering geoscience attributes proposed in the embodiments of the present application, the semantic change positions and semantic change categories of the bi-temporal images can be generated through the bi-temporal remote sensing image data of land cover and the semantic change detection network model considering geoscience attributes, and finally the semantic change results of the bi-temporal images can be generated. Thus, the semantic change detection task of land cover is decoupled into two interrelated subtasks: semantic segmentation in one phase and semantics-centered implicit change detection in the other phase, which suppresses the spatio-temporal heterogeneity of land cover while enhancing the spatio-temporal correlation of land cover, and mutually complements the learned specific information through joint representation learning, so as to achieve high-precision semantic change detection; moreover, the present application constructs a semantic change detection network model considering geoscience attributes based on this process. The disentangled semantic encoder generated by adopting a contrastive learning strategy in the semantic change detection network model can provide an accurate understanding of semantic categories for the semantic segmentation and implicit change detection branches, so that the bi-temporal semantic change detection can be simply and efficiently realized, the change positions of the bi-temporal images can be accurately located, and the semantic categories of the changes can be identified. Thus, the problems in the related art that the semantic change detection work mainly uses a multi-task framework and achieves considerable semantic-change detection results by constraining the consistency between the two subtasks of semantic segmentation and binary change detection, but these works lack exploration of the geoscience attributes of the ground objects that play a key role in semantic change detection are solved.

[0086] Next, a semantic change detection device considering geoscience attributes proposed in the embodiments of the present application is described with reference to the accompanying drawings.

[0087] Figure 5 It is a schematic structural diagram of a semantic change detection device considering geoscience attributes according to an embodiment of the present application.

[0088] As Figure 5 shown, the semantic change detection device 10 considering geoscience attributes includes: a first processing module 100, a detection module 200, and a first generation module 300.

[0089] Among them, the first processing module 100 is configured to obtain bi-temporal remote sensing image data and perform data enhancement processing on the bi-temporal remote sensing image data to obtain a bi-temporal semantic change detection data set, where the bi-temporal semantic change detection data set includes bi-temporal images and semantic labels of their corresponding change regions.

[0090] The detection module 200 is configured to input the bi-temporal semantic change detection data set into a preset semantic change detection network model considering geoscience attributes, so as to output the semantic change positions and semantic change categories of the bi-temporal images by using the preset semantic change detection network model considering geoscience attributes.

[0091] The first generation module 300 is configured to generate the semantic change result of the dual-temporal image according to the semantic change position and the semantic change category.

[0092] Optionally, in an embodiment of the present application, the first processing module 100 includes: a processing unit configured to perform random histogram equalization processing, random rotation processing, and random color jitter processing on the dual-temporal remote sensing image data to obtain a dual-temporal semantic change detection data set.

[0093] Optionally, in an embodiment of the present application, it further includes: an acquisition module and a second generation module.

[0094] Among them, the acquisition module is configured to, before inputting the dual-temporal semantic change detection data set into a preset semantic change detection network model considering geoscience attributes, acquire optical remote sensing image pairs of the same area and different time phases in the historical land cover area, and register the optical remote sensing image pairs to obtain a registered image pair.

[0095] The second generation module is configured to generate change samples through the registered image pair, so as to construct a training set and a test set by using the change samples, and construct a preset semantic change detection network model considering geoscience attributes by using the training set and the test set.

[0096] Optionally, in an embodiment of the present application, it further includes: a second processing module, a third generation module, and a construction module.

[0097] Among them, the second processing module is configured to, before inputting the dual-temporal semantic change detection data set into a preset semantic change detection network model considering geoscience attributes, perform data augmentation processing on the training data in the training set to obtain dual-temporal training images, and input the dual-temporal training images and the semantic labels of their corresponding change regions into a pre-trained semantic change detection network model, so as to generate temporal semantic features through the siamese semantic encoder in the pre-trained semantic change detection network model.

[0098] The third generation module is configured to generate implicit semantic change features based on the temporal semantic features, so as to generate a semantic change map of one time phase based on the implicit change detection decoder and the implicit semantic change features in the pre-trained semantic change detection network model, and generate a semantic segmentation map of another time phase except one time phase through the semantic decoder and the temporal semantic features in the pre-trained semantic change detection network model.

[0099] The construction module is configured to generate a semantic change training result by using the semantic change map to mask the semantic segmentation map, so as to construct a preset semantic change detection network model considering geoscience attributes.

[0100] Optionally, in an embodiment of the present application, it further includes: a calculation module and a determination module.

[0101] Among them, a calculation module is configured to calculate a semantic segmentation loss and an orientation-independent implicit change detection loss of a pre-trained semantic change detection network model before inputting a dual-temporal semantic change detection dataset into a preset semantic change detection network model considering geospatial attributes.

[0102] A determination module is configured to calculate a semantic change detection loss based on the average of the semantic segmentation loss and the orientation-independent implicit change detection loss, so as to determine a preset semantic change detection network model considering geospatial attributes through the semantic change detection loss.

[0103] It should be noted that the foregoing explanation of the embodiment of the semantic change detection method considering geospatial attributes also applies to the semantic change detection device considering geospatial attributes in this embodiment, and will not be elaborated here.

[0104] The semantic change detection device considering geospatial attributes proposed according to the embodiments of the present application can generate the semantic change position and semantic change category of the land cover through the dual-temporal remote sensing image data of the land cover and the semantic change detection network model considering geospatial attributes, and finally generate the semantic change result of the dual-temporal image. Thus, the semantic change detection task of the dual-temporal image is decoupled into two interrelated subtasks: semantic segmentation in one temporal phase and semantic-centered implicit change detection in the other temporal phase, while suppressing the spatio-temporal heterogeneity of the land cover and enhancing the spatio-temporal correlation of the land cover, and mutually complementing the learned specific information through joint representation learning, so as to achieve high-precision semantic change detection; moreover, the present application constructs a semantic change detection network model considering geospatial attributes based on this process, and the disentangled semantic encoder generated by adopting a contrastive learning strategy in the semantic change detection network model can provide an accurate understanding of semantic categories for the semantic segmentation and implicit change detection branches, so that dual-temporal semantic change detection can be simply and efficiently realized, the change position of the dual-temporal image can be accurately located, and the semantic category of the change can be identified. Thus, the problem that the semantic change detection work in the related art mainly uses a multi-task framework and achieves considerable semantic-change detection results by constraining the consistency between two subtasks of semantic segmentation and binary change detection, but these works lack exploration of the geospatial attributes of the ground objects that play a key role in semantic change detection is solved.

[0105] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include:

[0106] A memory 601, a processor 602, and a computer program stored on the memory 601 and executable on the processor 602.

[0107] When the processor 602 executes the program, it implements the semantic change detection method considering geoscience attributes provided in the above embodiments.

[0108] Furthermore, the electronic device further includes:

[0109] A communication interface 603, used for communication between the memory 601 and the processor 602.

[0110] A memory 601, used for storing a computer program that can run on the processor 602.

[0111] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0112] If the memory 601, the processor 602, and the communication interface 603 are implemented independently, the communication interface 603, the memory 601, and the processor 602 can be interconnected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 6 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0113] Optionally, in a specific implementation, if the memory 601, the processor 602, and the communication interface 603 are integrated on a chip, the memory 601, the processor 602, and the communication interface 603 can communicate with each other through an internal interface.

[0114] The processor 602 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0115] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the above semantic change detection method considering geoscience attributes.

[0116] An embodiment of the present application also provides a computer program product, including a computer program, which can run computer instructions, and when the computer instructions are executed by a processor, the semantic change detection method considering geoscience attributes provided by the embodiment of the present application is implemented.

[0117] In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0118] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically and clearly defined.

[0119] Any process or method description in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or N executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art of the embodiments of the present application.

[0120] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection part (electronic device) having one or N wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0121] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented by a combination of any one or more of the following techniques known in the art: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), and the like.

[0122] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of the above-described embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0123] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0124] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A semantic change detection method taking into account geographic attributes, characterized in that: The following steps are involved: Acquire dual-phase remote sensing image data, and perform data enhancement processing on the dual-phase remote sensing image data to obtain a dual-phase semantic change detection dataset, wherein the dual-phase semantic change detection dataset includes the dual-phase images and the semantic labels of the corresponding change areas; Inputting the dual-temporal semantic change detection data set into a preset semantic change detection network model taking into account geoscientific attributes, so as to output the semantic change position and semantic change category of the dual-temporal image by using the preset semantic change detection network model taking into account geoscientific attributes; A semantic change result of the dual-phase image is generated according to the semantic change position and the semantic change category.

2. The method according to claim 1, characterized in that The performing data enhancement processing on the dual-temporal remote sensing image data to obtain a dual-temporal semantic change detection data set includes: The dual-temporal remote sensing image data is subjected to random histogram equalization processing, random rotation processing and random color dithering processing to obtain the dual-temporal semantic change detection data set.

3. The method according to claim 1, characterized in that Before inputting the bi-temporal semantic change detection data set into the preset semantic change detection network model taking into account geoscientific attributes, the method further includes: Collecting optical remote sensing image pairs of the same area and different phases in the historical surface coverage area, and registering the optical remote sensing image pairs to obtain registered image pairs; A change sample is generated through the registered image pair, and a training set and a test set are constructed by using the change sample, and the preset semantic change detection network model taking into account geographic attributes is constructed by using the training set and the test set.

4. The method according to claim 3, characterized in that Before inputting the bi-temporal semantic change detection data set into the preset semantic change detection network model taking into account geoscientific attributes, the method further includes: Performing the data enhancement processing on the training data in the training set to obtain a dual-phase training image, inputting the dual-phase training image and the semantic labels of the corresponding change regions into a pre-trained semantic change detection network model, so as to generate a phase semantic feature through a twin semantic encoder in the pre-trained semantic change detection network model; Based on the time phase semantic features, an implicit semantic change feature is generated to generate a semantic change map of a time phase based on the implicit change detection decoder in the pre-trained semantic change detection network model and the implicit semantic change feature, and a semantic segmentation map of another time phase other than the one time phase is generated through the semantic decoder in the pre-trained semantic change detection network model and the time phase semantic feature; The semantic change map is used to mask the semantic segmentation map to generate a semantic change training result, so as to construct the preset semantic change detection network model taking into account the geographical attributes.

5. The method according to claim 4, characterized in that Before inputting the bi-temporal semantic change detection data set into the preset semantic change detection network model taking into account geoscientific attributes, the method further includes: Calculating the semantic segmentation loss and the direction-independent implicit change detection loss of the pre-trained semantic change detection network model; The semantic change detection loss is calculated according to the average sum of the semantic segmentation loss and the direction-independent implicit change detection loss, so as to determine the preset semantic change detection network model taking into account geographical attributes through the semantic change detection loss.

6. A semantic change detection device taking into account geographic attributes, characterized in that: include: A processing module, used for acquiring dual-phase remote sensing image data, and performing data enhancement processing on the dual-phase remote sensing image data to obtain the dual-phase semantic change detection data set, wherein the dual-phase semantic change detection data set includes the dual-phase images and the semantic labels of the corresponding change areas; A detection module, used for inputting the dual-temporal semantic change detection data set into a preset semantic change detection network model taking into account geoscientific attributes, so as to output the semantic change position and semantic change category of the dual-temporal image by using the preset semantic change detection network model taking into account geoscientific attributes; A generating module is used to generate a semantic change result of the dual-phase image according to the semantic change position and the semantic change category.

7. The device according to claim 6, characterized in that The processing module comprises: The processing unit is used to perform random histogram equalization processing, random rotation processing and random color dithering processing on the dual-temporal remote sensing image data to obtain the dual-temporal semantic change detection data set.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the semantic change detection method taking into account geographic attributes as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the semantic change detection method taking into account geographic attributes as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, it is used to implement the semantic change detection method taking into account geographical attributes as described in any one of claims 1 to 5.

Images