Method and device for detecting semantic change considering geoscientific attributes
By constructing a semantic change detection network model that takes into account geoscientific attributes and using a contrastive learning strategy to generate an unwrapped semantic encoder, the problem of insufficient exploration of ground feature attributes in semantic change detection is solved, and high-precision semantic change detection is achieved.
Patent Information
- Application Number
- CN202411960369.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2026-02-17
- Estimated Expiration
- 2044-12-27
AI Technical Summary
Existing semantic change detection technologies lack exploration of the geoscientific attributes of ground features within a multi-task framework, resulting in inaccurate semantic change detection results.
By constructing a semantic change detection network model that takes into account geoscientific attributes, data augmentation is performed using dual-temporal remote sensing image data, and a contrastive learning strategy is adopted to generate an unentangled semantic encoder, thereby decoupling semantic segmentation and implicit change detection, enhancing spatiotemporal correlation, and providing accurate semantic category understanding.
It achieves high-precision semantic change detection, accurately locates the changed position and identifies the semantic category, thus improving the accuracy and efficiency of semantic change detection.
Smart Images
Figure CN120071122B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of semantic change detection, and particularly relates to a semantic change detection method and device considering geosciences attributes. BACKGROUND
[0002] Land cover change is a geographical phenomenon in the macro-slow earth system, which reflects the complex interaction between ecological environment change and human activities. Remote sensing big spatio-temporal data has multi-dimensional characteristics such as time, space and attribute, which provides guarantee for discovering and analyzing this geographical phenomenon. The previous research focuses on binary change detection and object change detection, the former emphasizes the changed area and does not care about the category, and the latter emphasizes the change of a specific category. Recent work focuses on semantic change detection, which pays attention to both the changed area and the changed category, that is, the 'from-to' multiple change types. Therefore, semantic change detection is a valuable and challenging task to detect land cover change.
[0003] The specific geosciences attributes of land cover will bring mixed effects to the semantic change detection task. Specifically, the spatio-temporal heterogeneity of land cover will introduce significant intra-class variance, and the spatio-temporal correlation of land cover will provide a reference for land cover change.
[0004] However, although the semantic change detection work in the related art mainly uses a multi-task framework, by constraining the consistency between the two sub-tasks of semantic segmentation and binary change detection, it achieves an observable semantic-change detection result, but these works lack exploration of the geosciences attributes of the key features in semantic change detection, which needs to be solved urgently. SUMMARY
[0005] The present application provides a semantic change detection method and device considering geosciences attributes, to solve the problem that the semantic change detection work in the related art mainly uses a multi-task framework, by constraining the consistency between the two sub-tasks of semantic segmentation and binary change detection, it achieves an observable semantic-change detection result, but these works lack exploration of the geosciences attributes of the key features in semantic change detection.
[0006] The first aspect embodiment of the present application provides a semantic change detection method considering geosciences attributes, comprising the following steps: acquiring double-time remote sensing image data, and performing data enhancement processing on the double-time remote sensing image data to obtain a double-time semantic change detection data set, wherein the double-time semantic change detection data set comprises double-time images and semantic labels of corresponding change regions of the double-time images; inputting the double-time semantic change detection data set into a preset semantic change detection network model considering geosciences attributes to output semantic change positions and semantic change categories of the double-time images by using the preset semantic change detection network model considering geosciences attributes; and generating a semantic change result of the double-time images according to the semantic change positions and the semantic change categories.
[0007] Optionally, in an embodiment of the present application, the data enhancement processing on the double-time remote sensing image data to obtain the double-time semantic change detection data set comprises: performing random histogram equalization processing, random rotation processing and random color jittering processing on the double-time remote sensing image data to obtain the double-time semantic change detection data set.
[0008] Optionally, in an embodiment of the present application, before the double-time semantic change detection data set is input into the preset semantic change detection network model considering geosciences attributes, the method further comprises: collecting an optical remote sensing image pair of the same region and different time phases in a historical land surface cover region, performing registration on the optical remote sensing image pair to obtain a registered image pair; generating a change sample through the registered image pair, using the change sample to construct a training set and a test set, and using the training set and the test set to construct the preset semantic change detection network model considering geosciences attributes.
[0009] Optionally, in an embodiment of the present application, before the double-time semantic change detection data set is input into the preset semantic change detection network model considering geosciences attributes, the method further comprises: performing the data enhancement processing on training data in the training set to obtain double-time training images, inputting the double-time training images and the semantic labels of corresponding change regions of the double-time training images into a pre-trained semantic change detection network model to generate time phase semantic features through a twin semantic encoder in the pre-trained semantic change detection network model; generating implicit semantic change features based on the time phase semantic features, generating a semantic change map of one time phase based on an implicit change detection decoder in the pre-trained semantic change detection network model and the implicit semantic change features, and generating a semantic segmentation map of another time phase except the one time phase through a semantic decoder in the pre-trained semantic change detection network model and the time phase semantic features; and using the semantic change map to mask the semantic segmentation map to generate a semantic change training result, and using the semantic change training result to construct the preset semantic change detection network model considering geosciences attributes.
[0010] Optionally, in an embodiment of the present application, before the double-time-phase semantic change detection data set is input into the preset semantic change detection network model considering geographical attributes, further comprising: calculating a semantic segmentation loss and a direction-independent implicit change detection loss of the pre-trained semantic change detection network model; calculating a semantic change detection loss according to the average of the semantic segmentation loss and the direction-independent implicit change detection loss, so as to determine the preset semantic change detection network model considering geographical attributes by the semantic change detection loss.
[0011] The second aspect embodiment of the present application provides a semantic change detection device considering geographical attributes, comprising: a first processing module configured to obtain double-time-phase remote sensing image data and perform data enhancement processing on the double-time-phase remote sensing image data to obtain a double-time-phase semantic change detection data set, wherein the double-time-phase semantic change detection data set comprises double-time-phase images and semantic labels of corresponding change regions; a detection module configured to input the double-time-phase semantic change detection data set into a preset semantic change detection network model considering geographical attributes, so as to output semantic change positions and semantic change categories of the double-time-phase images by using the preset semantic change detection network model considering geographical attributes; and a first generation module configured to generate a semantic change result of the double-time-phase images according to the semantic change positions and the semantic change categories.
[0012] Optionally, in an embodiment of the present application, the first processing module comprises: a processing unit configured to perform random histogram equalization processing, random rotation processing and random color jittering processing on the double-time-phase remote sensing image data to obtain the double-time-phase semantic change detection data set.
[0013] Optionally, in an embodiment of the present application, further comprising: a collection module configured to, before the double-time-phase semantic change detection data set is input into the preset semantic change detection network model considering geographical attributes, collect a pair of optical remote sensing images of the same region and different time phases in a historical land surface cover region, perform registration on the pair of optical remote sensing images to obtain a registered image pair; and a second generation module configured to generate change samples by using the registered image pair, to construct a training set and a test set by using the change samples, and to construct the preset semantic change detection network model considering geographical attributes by using the training set and the test set.
[0014] Optionally, in an embodiment of the present application, further comprising: a second processing module, configured to perform the data enhancement processing on the training data in the training set before inputting the dual-temporal semantic change detection dataset into the preset geology attribute considering semantic change detection network model, to obtain dual-temporal training images, input the dual-temporal training images and the semantic labels of the corresponding change regions of the dual-temporal training images into the pre-trained semantic change detection network model, and generate temporal semantic features through a twin semantic encoder in the pre-trained semantic change detection network model; a third generation module, configured to generate implicit semantic change features based on the temporal semantic features, generate a semantic change map of one temporal phase based on an implicit change detection decoder in the pre-trained semantic change detection network model and the implicit semantic change features, and generate a semantic segmentation map of another temporal phase except the one temporal phase through a semantic decoder in the pre-trained semantic change detection network model and the temporal semantic features; a construction module, configured to mask the semantic segmentation map to generate a semantic change training result using the semantic change map, and construct the preset geology attribute considering semantic change detection network model.
[0015] Optionally, in an embodiment of the present application, further comprising: a calculation module, configured to calculate a semantic segmentation loss and a direction-independent implicit change detection loss of the pre-trained semantic change detection network model before inputting the dual-temporal semantic change detection dataset into the preset geology attribute considering semantic change detection network model; and a determination module, configured to calculate a semantic change detection loss according to an average of the semantic segmentation loss and the direction-independent implicit change detection loss, and determine the preset geology attribute considering semantic change detection network model through the semantic change detection loss.
[0016] The third aspect embodiment of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the geology attribute considering semantic change detection method as described in the above embodiments.
[0017] The fourth aspect embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the geology attribute considering semantic change detection method as described above.
[0018] The fifth aspect embodiment of the present application provides a computer program product, comprising a computer program, and the computer program is executed to implement the geology attribute considering semantic change detection method as described above.
[0019] The embodiment of the application can generate the semantic change position and the semantic change category of the double-time-phase image through the double-time-phase remote sensing image data of the land cover and the semantic change detection network model considering the geosciences attribute, and finally generate the semantic change result of the double-time-phase image. Thus, the semantic change detection task of the land cover is decoupled into two interrelated sub-tasks: semantic segmentation in one time phase and semantic-centered implicit change detection in another time phase, the spatio-temporal heterogeneity of the land cover is suppressed while the spatio-temporal correlation of the land cover is enhanced, and the specific information learned by the joint representation learning is mutually complementary, so as to realize high-precision semantic change detection; moreover, the embodiment constructs the semantic change detection network model considering the geosciences attribute based on the process, and the disentangled semantic encoder generated by the contrast learning strategy in the semantic change detection network model can provide accurate semantic category understanding for the semantic segmentation and the implicit change detection branch, so that the double-time-phase semantic change detection can be simply and efficiently realized, the change position of the double-time-phase image can be accurately located and the semantic category of the change can be recognized. Thus, the semantic change detection work in the related art mainly uses the multi-task framework, and the consistency between the semantic segmentation and the binary change detection is constrained to realize the observable semantic-change detection result, but the geosciences attribute of the ground object playing a key role in the semantic change detection is insufficiently explored.
[0020] Additional aspects and advantages of the application will be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS
[0021] The above and / or additional aspects and advantages of the application will become apparent and be readily understood by considering the following detailed description, including the accompanying drawings, in which:
[0022] Figure 1 A flowchart of a semantic change detection method considering geosciences attribute according to an embodiment of the application;
[0023] Figure 2 A flowchart of a semantic change detection method considering geosciences attribute according to an embodiment of the application;
[0024] Figure 3 An architecture schematic diagram of a semantic change detection network model considering geosciences attribute according to an embodiment of the application;
[0025] Figure 4 A semantic change detection result schematic diagram considering geosciences attribute according to an embodiment of the application;
[0026] Figure 5 A structure schematic diagram of a semantic change detection device considering geosciences attribute according to an embodiment of the application;
[0027] Figure 6 FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application.
[0028] Reference signs:
[0029] 10- semantic change detection device considering geosciences attributes: 100- first processing module, 200- detection module and 300- first generation module; 601- memory, 602- processor and 603- communication interface. DETAILED DESCRIPTION
[0030] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which the same or similar numerals or symbols throughout the drawings denote the same or similar elements or elements having the same or similar functions. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.
[0031] The semantic change detection method and device considering geosciences attributes of the embodiments of the present application are described below with reference to the accompanying drawings. The semantic change detection work in the related art mentioned in the above background technology is mainly based on a multi-task framework, and a consistent semantic segmentation and binary change detection sub-task is used to achieve a satisfactory semantic-change detection result. However, these works lack exploration of the geosciences attributes of the key ground features in semantic change detection. The present application provides a semantic change detection method considering geosciences attributes. In the method, the semantic change location and semantic change category of the dual-time-phase remote sensing image data of the ground cover can be generated by a semantic change detection network model considering geosciences attributes, and finally the semantic change result of the dual-time-phase image is generated. Thus, the semantic change detection task of the ground cover is decoupled into two interrelated sub-tasks: semantic segmentation in one time phase and semantic-centered implicit change detection in another time phase, which suppresses the spatio-temporal heterogeneity of the ground cover while enhancing the spatio-temporal correlation of the ground cover, and mutually complements the specific information learned through joint representation learning, thereby achieving high-precision semantic change detection. Moreover, the present application constructs a semantic change detection network model considering geosciences attributes based on the process. The disentangled semantic encoder generated by the contrastive learning strategy in the semantic change detection network model can provide accurate semantic category understanding for the semantic segmentation and implicit change detection branches, thereby achieving simple and efficient dual-time-phase semantic change detection, accurately locating the change location of the dual-time-phase image and identifying the semantic category of the change. Thus, the semantic change detection work in the related art is mainly based on a multi-task framework, and a consistent semantic segmentation and binary change detection sub-task is used to achieve a satisfactory semantic-change detection result. However, these works lack exploration of the geosciences attributes of the key ground features in semantic change detection.
[0032] In particular, Figure 1 A flowchart of a semantic change detection method considering geosciences attributes provided by an embodiment of the present application.
[0033] As Figure 1 shown, the semantic change detection method considering geosciences attributes includes the following steps:
[0034] In step S101, dual-time-phase remote sensing image data of land cover is acquired, and data enhancement processing is performed on the dual-time-phase remote sensing image data to obtain a dual-time-phase semantic change detection data set, wherein the dual-time-phase semantic change detection data set includes dual-time-phase images and semantic labels of corresponding change regions.
[0035] It can be understood that land cover generally refers to various features of the earth's surface covered by natural and artificial factors, which can be identified through direct observation or remote sensing means. It reflects the natural attributes of the land, and its properties mainly depend on natural factors such as climate, topography, soil, hydrology, and geological conditions. Land cover change is a geographical phenomenon in the macro-slow earth system, which reflects the complex interaction between ecological environment change and human activities. Remote sensing large spatio-temporal data has multi-dimensional characteristics such as time, space, and attributes, which can provide protection for discovering and analyzing this geographical phenomenon.
[0036] In some embodiments, the present application can acquire dual-time-phase remote sensing image data, wherein the dual-time-phase remote sensing image data here can be understood as remote sensing images obtained at different times of land cover. The data enhancement processing here can be understood as image enhancement processing on the dual-time-phase remote sensing image.
[0037] After the dual-time-phase remote sensing image data is processed by the data enhancement processing of the embodiments of the present application, a dual-time-phase semantic change detection data set can be obtained, which includes dual-time-phase images and semantic labels of corresponding change regions. Wherein the dual-time-phase images here can be understood as two remote sensing images of the same region observed at different time points by land cover, by comparing the differences between the two images, the change information of land cover can be captured.
[0038] Next, the embodiments of the present application can input the dual-time-phase images and the semantic labels of the corresponding change regions (land cover change regions) into a preset semantic change detection network model considering geosciences attributes, to detect the semantic change of land cover using the preset semantic change detection network model considering geosciences attributes. Wherein the preset semantic change detection network model considering geosciences attributes here can be understood as a pre-set trained semantic change detection network model considering geosciences attributes, which can detect the semantic change of land cover.
[0039] Optionally, in an embodiment of the present application, the dual-time-phase remote sensing image data is subjected to data enhancement processing to obtain a dual-time-phase semantic change detection data set, comprising: performing random histogram equalization processing, random rotation processing and random color jittering processing on the dual-time-phase remote sensing image data to obtain the dual-time-phase semantic change detection data set.
[0040] As can be understood from the related description of other embodiments, the present application can perform data enhancement processing on dual-time-phase remote sensing image data of land cover to obtain a dual-time-phase semantic change detection data set for semantic change detection of land cover.
[0041] In some embodiments, when the present application performs data enhancement processing on dual-time-phase remote sensing image data of land cover, the main but not limited data processing includes random histogram equalization processing, random rotation processing and random color jittering processing, etc., so as to complete image enhancement of the dual-time-phase remote sensing image and obtain dual-time-phase image data.
[0042] The random histogram equalization processing can re-distribute the pixel values of the image by performing non-linear stretching on the image, so that the gray histogram of the image changes from a certain gray interval to a uniform distribution in the entire gray range. This processing aims to improve the generalization ability of the deep learning model.
[0043] The random rotation processing refers to rotating the image at a certain random angle, which can enhance the data of the image and improve the generalization ability of the deep learning model.
[0044] The random color jittering processing can change the color value of the pixel by adding random noise between adjacent pixels of the image, so as to achieve a visual jittering effect.
[0045] Optionally, in an embodiment of the present application, before the dual-time-phase semantic change detection data set is input into the preset semantic change detection network model considering geosciences attributes, it further comprises: collecting an optical remote sensing image pair of the same region in the historical land cover area and different time phases, registering the optical remote sensing image pair to obtain a registered image pair; generating a change sample through the registered image pair, using the change sample to construct a training set and a test set, and using the training set and the test set to construct the preset semantic change detection network model considering geosciences attributes.
[0046] In actual execution process, before the dual-time-phase image data and the semantic label of the corresponding land cover change area are input into a certain semantic change detection network model, a certain semantic change detection network model considering geosciences attributes needs to be constructed. In the construction process of the semantic change detection network model considering geosciences attributes, certain data needs to be trained and tested.
[0047] Based on this, the embodiment of the present application can collect the same region, different time optical remote sensing image pairs in the historical ground surface coverage region, and process them, such as registering the optical remote sensing image pairs to obtain registered image pairs, and generating certain change samples through the remote sensing image pairs, so as to divide the change samples into a training set and a test set, which are respectively used in the training and testing in the construction process of the semantic change detection network model.
[0048] For example, the present application can collect some same region, different time optical remote sensing image pairs in which the ground surface coverage has changed The image pairs are registered to align the pixels in the same geographical position; then the registered image pairs are cropped into a plurality of image blocks of a fixed size, and the image blocks of one time phase and the image blocks of another time phase constitute a change sample; finally, all the change samples are used to construct a two-time phase semantic change detection data set, and the data set is divided into a training set and a test set, so as to train the semantic change detection network model by using the training set, and test the effectiveness of the semantic change detection network model by using the test set.
[0049] Figure 2 The flow chart of the construction of the semantic change detection network model considering geosciences attributes for an embodiment of the present application. As Figure 2 shown:
[0050] Step S201, constructing a semantic change detection data set, including a training set and a test set;
[0051] Step S202, constructing a pre-trained semantic change detection network model considering geosciences attributes;
[0052] Step S203, retraining the pre-trained semantic change detection network model considering geosciences attributes by using the training set;
[0053] Step S204, testing the trained semantic change detection network model considering geosciences attributes;
[0054] Step S205, testing the trained semantic change detection network model considering geosciences attributes by using the test set of the two-time phase remote sensing image pairs;
[0055] Step S206, outputting the test result to obtain the effectiveness of the semantic change detection network model considering geosciences attributes.
[0056] Optionally, in an embodiment of the present application, before the dual-time semantic change detection dataset is input into the preset geoscience attribute considering semantic change detection network model, it further includes: performing data enhancement processing on the training data in the training set to obtain dual-time training images, inputting the dual-time training images and the semantic labels of the corresponding change regions into the pre-trained semantic change detection network model to generate temporal semantic features through the twin semantic encoder in the pre-trained semantic change detection network model; generating implicit semantic change features based on the temporal semantic features to generate a semantic change map of one time phase based on the implicit change detection decoder in the pre-trained semantic change detection network model and the implicit semantic change features, and generate a semantic segmentation map of another time phase except for the one time phase through the semantic decoder in the pre-trained semantic change detection network model and the temporal semantic features; and generating a semantic change training result by masking the semantic segmentation map with the semantic change map to construct the preset semantic change detection network model.
[0057] In some other embodiments, the data in the training set and the test set also need to be subjected to data enhancement processing, i.e. random histogram equalization processing, random rotation processing and random color jittering processing, etc. to perform image enhancement on the remote sensing images (images) to obtain dual-time training images. Then, the dual-time training images and the semantic labels of the corresponding change regions are taken as inputs of the pre-trained geoscience attribute considering semantic change detection network model to retrain the pre-trained semantic change detection network model. Figure 3 The architecture schematic diagram of the geoscience attribute considering semantic change detection network model of an embodiment of the present application is shown in FIG. 1. Figure 3
[0058] The geoscience attribute considering semantic change detection network model in the embodiment of the present application mainly but not limitedly consists of two parts of an encoder structure and a decoder structure. The encoder includes but is not limited to a semantic encoder and a spatio-temporal encoder. The semantic encoder can but is not limited to consist of convolution, regularization and ReLU activation function modules. The spatio-temporal encoder can but is not limited to consist of relative position encoding, self-attention and multi-layer perception modules. The decoder includes but is not limited to a multi-task decoder consisting of an implicit change detection decoder and a semantic decoder, which mainly but not limitedly consists of convolution, up-sampling and ReLU activation function modules. In addition, 6 repeated residual blocks are used for connection in the encoder and the decoder.
[0059] Firstly, the embodiment of the present application can generate temporal semantic features through the basic twin semantic encoder in the pre-trained semantic change detection network model; then abstract category prototypes P C A disentangled semantic encoder is trained by a pixel-to-prototype intra-temporal and prototype-to-prototype inter-temporal contrastive learning strategy to overcome the spatio-temporal heterogeneity of land cover changes and obtain disentangled temporal semantic features
[0060] The disentangled semantic encoder generated by the contrastive learning strategy in the embodiments of the present application can provide accurate semantic class understanding for the semantic segmentation and implicit change detection branches. Unlike traditional methods that indirectly learn the relative distance between different class embeddings by optimizing pixel-level predictions through cross-entropy loss, the contrastive learning strategy directly pulls the same-class features at the pixel level and pushes away different-class features by using class prototypes as anchors for pixel features.
[0061] Then, the embodiments of the present application can concatenate the temporal semantic features as the input of the spatio-temporal correlation converter to generate implicit semantic change features and integrate them into the implicit change detection decoder and the semantic decoder through a certain decoder to realize multi-task decoding, i.e., a semantic change map of one temporal phase is output by the implicit change detection decoder and another semantic segmentation map of another temporal phase output by the mask semantic decoder to generate a semantic change map to obtain the semantic change detection result of the dual temporal images.
[0062] Finally, the embodiments of the present application can realize the decoupling of semantic change detection into two interrelated sub-tasks: semantic segmentation in one temporal phase and implicit change detection with semantic centers in another temporal phase, which suppresses the spatio-temporal heterogeneity of land cover while enhancing the spatio-temporal correlation of land cover, and mutually complements the learned specific information through joint representation learning, realizes the construction of a semantic change detection network model considering geosciences attributes, and completes high-precision semantic change detection.
[0063] Optionally, in an embodiment of the present application, before inputting the dual temporal semantic change detection dataset into the preset semantic change detection network model considering geosciences attributes, it further includes: calculating the semantic segmentation loss and the direction-independent implicit change detection loss of the pre-trained semantic change detection network model; calculating the semantic change detection loss according to the average of the semantic segmentation loss and the direction-independent implicit change detection loss to determine the preset semantic change detection network model considering geosciences attributes through the semantic change detection loss.
[0064] In certain embodiments, the present application can also construct a certain target loss function to calculate the semantic change detection loss of the semantic change detection network model. By minimizing the multi-task loss function and combining the stochastic gradient descent optimization algorithm to iteratively update the model parameters of the semantic segmentation and semantic change detection branches, the final semantic change detection network model is determined.
[0065] For example, the present application can adopt a semantic segmentation loss and a direction-independent implicit change detection loss The average of the two is used to determine the semantic change detection loss The formula can be but is not limited to as follows:
[0066]
[0067] In order to cope with the limited semantic annotation challenge, the On the basis of the cross-entropy loss, the pixel-to-prototype within the time phase and the prototype-to-prototype between the time phases are introduced, and the formula can be but is not limited to as follows:
[0068]
[0069] Wherein, C is the number of semantic categories, y i and p i respectively represent the true value and the predicted probability of the sample belonging to the i-th category, P + and P _ are the positive prototype and the negative prototype of pixel i, e i is the time phase semantic feature of the pixel level, and τ is the temperature parameter.
[0070] Further, the direction-independent implicit change detection loss contains two parts, one is the implicit semantic change loss in the direction from t1 to t2, and the other is the implicit semantic change loss in the direction from t2 to t1, both of which are calculated by cross-entropy, and the formula can be but is not limited to as follows:
[0071]
[0072] Wherein, the weight w i is the proportion of each semantic change category. In y i is the semantic change label of t1, and in y i is the semantic change label of t2.
[0073] So far, the construction of the semantic change detection network model considering the geosciences attribute has been completed. The constructed semantic change detection model considering the geosciences attribute can use the category prototype as the anchor point of the pixel feature, adopt the pixel-to-prototype and prototype-to-prototype comparison learning strategy to suppress the spatio-temporal heterogeneity of the land cover, provide shared and accurate semantic category understanding for semantic segmentation and semantic change detection, simultaneously adopt the spatio-temporal converter to associate the spatio-temporal change dependence and enhance the spatio-temporal correlation of the land cover, which is helpful to realize high-precision “from-to” semantic change detection, and output the “from-to” semantic change result
[0074] In the test phase, the embodiments of the present application can test the constructed semantic change detection network model considering geosciences attributes by using the test set. The semantic change detection method (S-cCDNet) containing the semantic change detection network model considering geosciences attributes in the embodiments of the present application is compared with other deep learning-based semantic change detection methods on the Second data set. The precision indicators of semantic change include overall accuracy OA, intersection over union IoU, separable kappa coefficient Sek, harmonic mean of recall Pscd and precision Rscd, etc. Table 1 is a quantitative result comparison table of S-cCDNet and other comparison methods on the Second data set, which can be expressed as follows:
[0075] Table 1
[0076]
[0077] As can be seen from Table 1, compared with other methods, the semantic change detection method (S-cCDNet) considering geosciences attributes in the embodiments of the present application has achieved good results in each indicator, which can achieve the highest recall of 68.92%, the highest intersection over union of 59.35%, and the highest harmonic mean of 64.99%. These results prove the ability of the semantic change detection method (S-cCDNet) considering geosciences attributes in the embodiments of the present application in semantic and semantic change information mining, Figure 4 is a semantic change detection result diagram of an embodiment of the present application, and part of the visual results are shown in Figure 4 .
[0078] Step S102, input the dual-time semantic change detection data set into the preset semantic change detection network model considering geosciences attributes, so as to output the semantic change position and semantic change category of the dual-time image by using the preset semantic change detection network model considering geosciences attributes.
[0079] The person skilled in the art can understand that the specific geosciences attributes of the land cover will bring mixed effects to the semantic change detection task. Specifically, the spatio-temporal heterogeneity of the land cover will introduce significant intra-class variance, and the spatio-temporal correlation of the land cover will provide a reference for the land cover change.
[0080] As a possible implementation manner, after the semantic change detection network model considering geosciences attributes is constructed, the semantic change of the dual-time image can be detected by using the trained semantic change detection network model considering geosciences attributes.
[0081] In the detection process, the embodiment of the application mainly suppresses the spatio-temporal heterogeneity of the land cover while enhancing the spatio-temporal correlation of the land cover through the trained semantic change detection network model considering the geosciences attributes, and in this case, outputs the high-precision semantic change position and semantic change category of the land cover.
[0082] In step S103, the semantic change result of the land cover is generated according to the semantic change position and the semantic change category.
[0083] In some embodiments, after obtaining the semantic change position and the semantic change category of the land cover, the application can obtain the final semantic change result of the land cover according to the semantic change position and the semantic change category.
[0084] The embodiment of the application can realize dual-temporal semantic change detection by using the semantic change detection network model considering the geosciences attributes, can locate the change area and the semantic category of the change, focuses on the semantic-change of interest, and can realize a high recall rate in semantic change detection.
[0085] According to the semantic change detection method considering the geosciences attributes proposed in the embodiment of the application, the semantic change position and the semantic change category of the dual-temporal remote sensing image data of the land cover and the semantic change detection network model considering the geosciences attributes are generated, and finally the semantic change result of the dual-temporal image is generated. Thus, the semantic change detection task of the land cover is decoupled into two interrelated sub-tasks: semantic segmentation in one time phase and implicit change detection centered on semantics in another time phase, which suppresses the spatio-temporal heterogeneity of the land cover while enhancing the spatio-temporal correlation of the land cover, and the specific information learned by the joint representation learning is mutually complementary, thereby realizing high-precision semantic change detection; moreover, the semantic change detection network model considering the geosciences attributes is constructed based on the process, and the unwound semantic encoder generated by the contrast learning strategy in the semantic change detection network model can provide accurate semantic category understanding for the semantic segmentation and implicit change detection branches, thereby realizing dual-temporal semantic change detection simply and efficiently, accurately locating the change position of the dual-temporal image and identifying the semantic category of the change. Thus, the problems such as insufficient exploration of the geosciences attributes of the ground objects playing a key role in semantic change detection in the related art are solved.
[0086] Secondly, the semantic change detection device considering the geosciences attributes proposed in the embodiment of the application is described with reference to the accompanying drawings.
[0087] Figure 5is a structural schematic diagram of a geoscience attribute considering semantic change detection device according to an embodiment of the present application.
[0088] As shown in Figure 5 The geoscience attribute considering semantic change detection device 10 includes a first processing module 100, a detection module 200, and a first generation module 300.
[0089] The first processing module 100 is configured to obtain double-time-phase remote sensing image data and perform data enhancement processing on the double-time-phase remote sensing image data to obtain a double-time-phase semantic change detection data set, wherein the double-time-phase semantic change detection data set includes double-time-phase images and semantic labels of corresponding change regions.
[0090] The detection module 200 is configured to input the double-time-phase semantic change detection data set into a preset geoscience attribute considering semantic change detection network model to output semantic change positions and semantic change categories of the double-time-phase images by using the preset geoscience attribute considering semantic change detection network model.
[0091] The first generation module 300 is configured to generate a semantic change result of the double-time-phase images according to the semantic change positions and the semantic change categories.
[0092] Optionally, in an embodiment of the present application, the first processing module 100 includes a processing unit configured to perform random histogram equalization processing, random rotation processing, and random color jittering processing on the double-time-phase remote sensing image data to obtain the double-time-phase semantic change detection data set.
[0093] Optionally, in an embodiment of the present application, the geoscience attribute considering semantic change detection device further includes a collection module and a second generation module.
[0094] The collection module is configured to collect optical remote sensing image pairs of the same region and different time phases in a historical land surface cover region before inputting the double-time-phase semantic change detection data set into the preset geoscience attribute considering semantic change detection network model, and perform registration on the optical remote sensing image pairs to obtain registered image pairs.
[0095] The second generation module is configured to generate change samples by using the registered image pairs, construct a training set and a test set by using the change samples, and construct the preset geoscience attribute considering semantic change detection network model by using the training set and the test set.
[0096] Optionally, in an embodiment of the present application, the geoscience attribute considering semantic change detection device further includes a second processing module, a third generation module, and a construction module.
[0097] The second processing module is configured to perform data enhancement processing on training data in the training set before inputting the dual-temporal semantic change detection dataset into the preset semantic change detection network model considering geosciences attributes, to obtain dual-temporal training images, and input the dual-temporal training images and semantic labels of the corresponding change regions into the pre-trained semantic change detection network model, so as to generate temporal semantic features by using a twin semantic encoder in the pre-trained semantic change detection network model.
[0098] The third generating module is configured to generate implicit semantic change features based on the temporal semantic features, to generate a semantic change map of one temporal phase based on an implicit change detection decoder in the pre-trained semantic change detection network model and the implicit semantic change features, and to generate a semantic segmentation map of another temporal phase except the one temporal phase based on a semantic decoder in the pre-trained semantic change detection network model and the temporal semantic features.
[0099] The constructing module is configured to mask the semantic segmentation map by using the semantic change map to generate a semantic change training result, to construct the preset semantic change detection network model considering geosciences attributes.
[0100] Optionally, in an embodiment of the present application, the method further includes a calculating module and a determining module.
[0101] The calculating module is configured to calculate a semantic segmentation loss and a direction-independent implicit change detection loss of the pre-trained semantic change detection network model before inputting the dual-temporal semantic change detection dataset into the preset semantic change detection network model considering geosciences attributes.
[0102] The determining module is configured to calculate a semantic change detection loss according to an average of the semantic segmentation loss and the direction-independent implicit change detection loss, to determine the preset semantic change detection network model considering geosciences attributes by using the semantic change detection loss.
[0103] It should be noted that the foregoing explanation and description of the embodiment of the method for detecting semantic changes considering geosciences attributes are also applicable to the embodiment of the device for detecting semantic changes considering geosciences attributes, and thus will not be described herein again.
[0104] The semantic change detection device considering geosciences attributes provided by the embodiment of the present application can generate the semantic change position and semantic change category of the land cover through the dual-time-phase remote sensing image data of the land cover and the semantic change detection network model considering geosciences attributes, and finally generate the semantic change result of the dual-time-phase image. Thus, the semantic change detection task of the dual-time-phase image is decoupled into two interrelated subtasks: semantic segmentation in one time phase and semantic-centered implicit change detection in another time phase, which suppresses the spatiotemporal heterogeneity of the land cover while enhancing the spatiotemporal correlation of the land cover, and mutually complements the specific information learned through joint representation learning, thereby realizing high-precision semantic change detection. Moreover, the embodiment of the present application constructs the semantic change detection network model considering geosciences attributes based on the process. The unwrapped semantic encoder generated by the contrast learning strategy in the semantic change detection network model can provide accurate semantic category understanding for the semantic segmentation and implicit change detection branches, so that the dual-time-phase semantic change detection can be realized simply and efficiently, the change position of the dual-time-phase image can be accurately located, and the semantic category of the change can be identified. Thus, the problem that the semantic change detection work in the related art is mainly based on a multi-task framework, and the consistency between the semantic segmentation and binary change detection two subtasks is constrained to realize an observable semantic-change detection result, but the exploration of the geosciences attributes of the ground features playing a key role in the semantic change detection is insufficient.
[0105] Figure 6 The structural schematic diagram of the electronic device provided by the embodiment of the present application is provided. The electronic device can include:
[0106] The memory 601, the processor 602, and the computer program stored in the memory 601 and executable on the processor 602.
[0107] The processor 602 implements the semantic change detection method considering geosciences attributes provided in the above embodiments when executing the program.
[0108] Further, the electronic device further includes:
[0109] The communication interface 603 is used for communication between the memory 601 and the processor 602.
[0110] The memory 601 is used to store the computer program executable on the processor 602.
[0111] The memory 601 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.
[0112] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 6 Only one thick line is used to represent the bus in the figure, but it does not mean that there is only one bus or only one type of bus.
[0113] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can complete communication between each other through an internal interface.
[0114] The processor 602 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.
[0115] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the above-mentioned method for detecting semantic change considering geoscientific attributes.
[0116] The embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program can run computer instructions, and the computer instructions are executed by a processor to implement the method for detecting semantic change considering geoscientific attributes provided by the embodiment of the present application.
[0117] In the description of the application, reference to "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. In addition, the usage of "N" means at least two, for example, two, three or the like, unless explicitly stated otherwise.
[0118] Furthermore, the terms "first", "second", or the like, are used merely as a designation of certain elements or features, and do not imply or connote relative importance or a specific order of categorization of the indicated features. Accordingly, features described as "first" or "second" can be explicitly or implicitly included in at least one of the features. In the description of the application, the meaning of "N" is at least two, for example, two, three, etc., unless explicitly specified otherwise.
[0119] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments which can be managed as one or more modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions or steps, and alternate implementations are possible. In some embodiments, the processes and methods described can be executed by one or more apparatuses or devices, either directly or after conversion to another language. Alternate implementations are possible.
[0120] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions stored in a computer readable medium, which can be executed by an instruction execution system, apparatus or device, such as a computer-based system, a processor-based system, or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or a combination of the above. For the purposes of this specification, a "computer readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus or device. The computer readable medium can be a computer readable storage medium or a computer readable signal medium. The computer readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a propagation medium. The computer readable signal medium can include, but is not limited to, a computer readable medium that facilitates transfer of the program from one place to another. A specific example of a computer readable medium is a non-transitory computer-readable storage medium. A specific example of a computer readable signal medium is a source or destination of the computer readable medium. Another specific example of a computer readable signal medium is a computer readable signal travelling through space. Thus, a computer readable medium can take many forms of hardware to carry out the program for use by or in connection with the instruction execution system, apparatus or device.
[0121] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware and in another embodiment, the hardware can be implemented using any or a combination of the following technologies, which are each well known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon an application of data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.
[0122] Those of skill in the art would understand that the steps carried out in the above-mentioned embodiment methods can be carried out by program instructions to relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.
[0123] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.
[0124] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.
Claims
1. A method for detecting semantic change considering geoscientific properties, characterized in that, The method comprises the following steps: acquiring double-time-phase remote sensing image data, and performing data enhancement processing on the double-time-phase remote sensing image data to obtain a double-time-phase semantic change detection data set, wherein the double-time-phase semantic change detection data set comprises double-time-phase images and semantic labels of corresponding change regions of the double-time-phase images; inputting the double-time-phase semantic change detection data set into a preset geology attribute considering semantic change detection network model to output semantic change positions and semantic change categories of the double-time-phase images by using the preset geology attribute considering semantic change detection network model; generating a semantic change result of the double-time-phase images according to the semantic change positions and the semantic change categories; before inputting the double-time-phase semantic change detection data set into the preset geology attribute considering semantic change detection network model, the method further comprises: performing the data enhancement processing on training data in a training set to obtain double-time-phase training images, inputting the double-time-phase training images and the semantic labels of corresponding change regions of the double-time-phase training images into a pre-trained semantic change detection network model to generate time-phase semantic features by using a twin semantic encoder in the pre-trained semantic change detection network model; generating implicit semantic change features based on the time-phase semantic features to generate a semantic change map of one time phase based on an implicit change detection decoder in the pre-trained semantic change detection network model and the implicit semantic change features, and generate a semantic segmentation map of another time phase except the one time phase by using a semantic decoder in the pre-trained semantic change detection network model and the time-phase semantic features; and generating a semantic change training result by using the semantic change map to mask the semantic segmentation map to construct the preset geology attribute considering semantic change detection network model.
2. The method of claim 1, wherein, The data enhancement processing on the double-time-phase remote sensing image data to obtain the double-time-phase semantic change detection data set comprises: performing random histogram equalization processing, random rotation processing and random color dithering processing on the double-time-phase remote sensing image data to obtain the double-time-phase semantic change detection data set.
3. The method of claim 1, wherein, Before inputting the double-time-phase semantic change detection data set into the preset geology attribute considering semantic change detection network model, the method further comprises: collecting an optical remote sensing image pair of the same region and different time phases in a historical land surface coverage region, and performing registration on the optical remote sensing image pair to obtain a registered image pair; generating change samples by using the registered image pair to construct the training set and the test set by using the change samples, and constructing the preset geology attribute considering semantic change detection network model by using the training set and the test set.
4. The method of claim 1, wherein, Before inputting the double-time-phase semantic change detection data set into the preset geology attribute considering semantic change detection network model, the method further comprises: calculating a semantic segmentation loss and a direction-independent implicit change detection loss of the pre-trained semantic change detection network model; A semantic change detection loss is calculated according to an average of the semantic segmentation loss and the direction-independent implicit change detection loss, so as to determine the preset semantic change detection network model considering geosciences attributes through the semantic change detection loss.
5. A device for detecting semantic change in consideration of geoscientific properties, characterized in that, Comprise: The processing module is used for acquiring double-time remote sensing image data, and performing data enhancement processing on the double-time remote sensing image data to obtain the double-time semantic change detection data set, wherein the double-time semantic change detection data set comprises double-time images and semantic labels of corresponding change regions of the double-time images; The detection module is used for inputting the double-time semantic change detection data set into a preset semantic change detection network model considering geosciences attributes, so as to output semantic change positions and semantic change categories of the double-time images by using the preset semantic change detection network model considering geosciences attributes; The generation module is used for generating a semantic change result of the double-time images according to the semantic change positions and the semantic change categories; The second processing module is used for performing the data enhancement processing on training data in a training set to obtain double-time training images before inputting the double-time semantic change detection data set into the preset semantic change detection network model considering geosciences attributes, and inputting the double-time training images and semantic labels of corresponding change regions of the double-time training images into a pre-trained semantic change detection network model, so as to generate temporal semantic features by using a twin semantic encoder in the pre-trained semantic change detection network model; the third generation module is used for generating implicit semantic change features based on the temporal semantic features, generating a semantic change map of one time phase based on an implicit change detection decoder in the pre-trained semantic change detection network model and the implicit semantic change features, and generating a semantic segmentation map of another time phase except the one time phase by using a semantic decoder in the pre-trained semantic change detection network model and the temporal semantic features; and the construction module is used for generating a semantic change training result by using the semantic change map to mask the semantic segmentation map, so as to construct the preset semantic change detection network model considering geosciences attributes.
6. The apparatus of claim 5, wherein, The processing module comprises: The processing unit is used for performing random histogram equalization processing, random rotation processing and random color dithering processing on the double-time remote sensing image data to obtain the double-time semantic change detection data set.
7. An electronic device, comprising: Comprise: A memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the semantic change detection method considering geosciences attributes according to any one of claims 1-4.
8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the semantic change detection method considering geosciences attributes according to any one of claims 1-4.
9. A computer program product comprising a computer program, characterized in that, The computer program is executed to implement the semantic change detection method considering geosciences attributes according to any one of claims 1-4.