Forest type change detection method and device

Abstract

The invention relates to a forest type change detection method and device, and the method comprises the steps: obtaining a sample image data set, and constructing a forest type change detection model; the forest type change detection model comprises an encoder and a decoder; inputting each pair of image blocks into an encoder to obtain a multi-scale depth feature map; inputting the multi-scale depth feature map into a decoder to obtain a forest type change detection result of each pair of image blocks; calculating a loss value of the forest type change detection model according to the forest type change detection result of each pair of image blocks, the label data of each pair of image blocks and a preset loss function; performing iterative training on the forest type change detection model according to the loss value to obtain a trained forest type change detection model; and inputting the high-resolution remote sensing images of the two time phases of the to-be-detected area into the trained forest type change detection model to obtain a forest type change detection result, thereby improving the efficiency and precision of forest type change detection.

Description

Technical Field

[0001] The present application relates to the technical field of forest type monitoring, and in particular to a method and device for detecting forest type changes. Background Art

[0002] Forests are the main body of terrestrial ecosystems and play a vital role in regulating climate change, providing ecosystem services and promoting human social and economic development. According to the internal characteristics, external features and dynamic laws of forest communities, forest types are mainly divided into coniferous forests and broad-leaved forests.

[0003] Forest type change detection refers to analyzing the differences in forest type distribution between different phases in the same area. For example, if the same location is coniferous forest in the previous phase and broad-leaved forest in the later phase, it is considered that a change in forest type has occurred. The purpose of forest type change detection is to detect whether a change in forest type has occurred in the area and to obtain forest type information before and after the change, so as to scientifically analyze the current status and dynamic change laws of forest resources, which is of great significance for the protection, rational use and quality improvement of forest resources.

[0004] In the related technology, high-resolution remote sensing images are manually interpreted visually and ground survey data is referenced to manually outline the areas where forest type changes occur. This method is not only time-consuming and laborious, but also highly subjective, and its accuracy and efficiency are difficult to guarantee. Summary of the invention

[0005] Based on this, the purpose of the present application is to provide a method and device for detecting forest type changes, which have the advantages of high efficiency and high accuracy in detecting forest type changes.

[0006] According to a first aspect of an embodiment of the present application, a forest type change detection method is provided, comprising the following steps:

[0007] Acquire a sample image data set; the sample image data set includes several pairs of image blocks and label data of each pair of image blocks; each pair of image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data of each pair of image blocks includes a forest type label of each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label of each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change state label of each pixel in the high-resolution remote sensing image block of the later phase compared with the corresponding pixel in the high-resolution remote sensing image block of the previous phase;

[0008] Construct a forest type change detection model; the forest type change detection model includes an encoder and a decoder;

[0009] Input each pair of image blocks into the encoder to obtain a multi-scale depth feature map;

[0010] The multi-scale deep feature map is input into the decoder to obtain the forest type change detection result for each pair of image blocks;

[0011] Calculate the loss value of the forest type change detection model based on the forest type change detection results of each pair of image blocks, the label data of each pair of image blocks, and the preset loss function;

[0012] Iteratively training the forest type change detection model according to the loss value until the loss value meets a predetermined training termination condition, thereby obtaining a trained forest type change detection model;

[0013] The high-resolution remote sensing images of the area to be detected in two phases are input into the trained forest type change detection model to obtain the forest type change detection results.

[0014] According to a second aspect of an embodiment of the present application, a forest type change detection device is provided, comprising:

[0015] The data set acquisition module is used to acquire a sample image data set; the sample image data set includes a plurality of pairs of image blocks and label data of each pair of image blocks; each pair of image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data of each pair of image blocks includes a forest type label of each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label of each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change state label of each pixel in the high-resolution remote sensing image block of the later phase compared with the corresponding pixel in the high-resolution remote sensing image block of the previous phase;

[0016] A model building module is used to build a forest type change detection model; the forest type change detection model includes an encoder and a decoder;

[0017] A depth feature map acquisition module is used to input each pair of image blocks into the encoder to obtain a multi-scale depth feature map;

[0018] A depth feature map input module is used to input the multi-scale depth feature map into the decoder to obtain the forest type change detection result for each pair of image blocks;

[0019] A loss value calculation module, used to calculate the loss value of the forest type change detection model according to the forest type change detection results of each pair of image blocks, the label data of each pair of image blocks and a preset loss function;

[0020] A model training module is used to iteratively train the forest type change detection model according to the loss value until the loss value meets a predetermined training termination condition, thereby obtaining a trained forest type change detection model;

[0021] The detection result acquisition module is used to input the high-resolution remote sensing images of the area to be detected in two phases into the trained forest type change detection model to obtain the forest type change detection results.

[0022] The embodiment of the present application obtains a sample image data set; the sample image data set includes a plurality of pairs of image blocks and label data for each pair of image blocks; each pair of image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data for each pair of image blocks includes a forest type label for each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label for each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change status label for each pixel in the high-resolution remote sensing image block of the later phase compared to the corresponding pixel in the high-resolution remote sensing image block of the previous phase; a forest type change detection model is constructed; the forest type change detection model includes an encoder with and decoder; input each pair of image blocks into the encoder to obtain a multi-scale depth feature map; input the multi-scale depth feature map into the decoder to obtain the forest type change detection result of each pair of image blocks; calculate the loss value of the forest type change detection model according to the forest type change detection result of each pair of image blocks, the label data of each pair of image blocks and the preset loss function; according to the loss value, iteratively train the forest type change detection model until the loss value meets the predetermined training termination condition to obtain the trained forest type change detection model; input the high-resolution remote sensing images of the two phases of the area to be detected into the trained forest type change detection model to obtain the forest type change detection result. This application trains the forest type change detection model, and automatically outputs the forest type change detection result based on the trained forest type change detection model. It has the advantages of automation and high precision, can reduce manual workload, and meet the business needs of dynamic monitoring of forest resources.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0024] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a flow chart of a forest type change detection method provided in one embodiment of the present application;

[0026] Figure 2 This is a structural block diagram of a forest type change detection device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0028] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0029] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0030] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0031] In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0032] See also Figure 1 , which is a flow chart of a forest type change detection method provided in an embodiment of the present application. The forest type change detection method provided in an embodiment of the present application comprises the following steps:

[0033] S10: Obtain a sample image dataset; the sample image dataset includes several pairs of image blocks and label data for each pair of image blocks; each pair of image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data for each pair of image blocks includes a forest type label for each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label for each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change status label for each pixel in the high-resolution remote sensing image block of the later phase compared to the corresponding pixel in the high-resolution remote sensing image block of the previous phase.

[0034] Among them, high-resolution remote sensing images refer to remote sensing images with high spatial resolution, clear ground texture and rich spatial detail information, which can be obtained by satellite imaging or drone photography. High-resolution remote sensing image blocks are obtained by dividing high-resolution remote sensing images according to certain spatial dimensions.

[0035] In an embodiment of the present application, high-resolution remote sensing images of the same area in two phases are obtained, and the high-resolution remote sensing images of the two phases are divided into pairs of image blocks, each pair of image blocks includes a high-resolution remote sensing image block of the previous phase and a high-resolution remote sensing image block of the next phase. The forest type of each pixel in the high-resolution remote sensing image block of the previous phase and the high-resolution remote sensing image block of the next phase are respectively marked to obtain the forest type label of each pixel in the high-resolution remote sensing image block of the previous phase and the forest type label of each pixel in the high-resolution remote sensing image block of the next phase. The forest type labels of the high-resolution remote sensing image blocks of the previous and next phases are compared pixel by pixel, and the pixels with the same forest type label of the previous phase and the next phase are marked as unchanged, and the pixels with inconsistent forest type labels of the previous phase and the next phase are marked as changed, so as to obtain the forest type change status label.

[0036] S20: Construct a forest type change detection model; the forest type change detection model includes an encoder and a decoder.

[0037] In an embodiment of the present application, the encoder is used to extract deep semantic features, and the decoder is used to automatically analyze and process the deep semantic features extracted by the encoder, and simultaneously output the forest type change status of the previous and next phases and the forest type information of the previous and next phases.

[0038] S30: Input each pair of image blocks into the encoder to obtain a multi-scale depth feature map.

[0039] In an embodiment of the present application, the encoder can simultaneously receive high-resolution remote sensing image blocks of two phases, extract deep semantic features of the high-resolution remote sensing image blocks of the previous phase, and obtain a corresponding multi-scale depth feature map. The encoder can also extract deep semantic features of the high-resolution remote sensing image blocks of the subsequent phase, and obtain a corresponding multi-scale depth feature map.

[0040] S40: Input the multi-scale depth feature map into the decoder to obtain the forest type change detection result of each pair of image blocks.

[0041] In an embodiment of the present application, a multi-scale depth feature map corresponding to a high-resolution remote sensing image block in a previous phase and a multi-scale depth feature map corresponding to a high-resolution remote sensing image block in a subsequent phase are input into a decoder, and the decoder can predict the forest type of the high-resolution remote sensing image block in the previous phase, the forest type of the high-resolution remote sensing image block in the subsequent phase, and the change status of the forest type between the previous and next phases.

[0042] S50: Calculate the loss value of the forest type change detection model according to the forest type change detection result of each pair of image blocks, the label data of each pair of image blocks and a preset loss function.

[0043] The preset loss functions include but are not limited to the cross entropy loss function and the KL divergence loss function.

[0044] In an embodiment of the present application, the forest type of the high-resolution remote sensing image block in the previous phase, the forest type of the high-resolution remote sensing image block in the later phase, the change status of the forest type between the previous and next phases, the forest type label of the high-resolution remote sensing image block in the previous phase, the forest type label of the high-resolution remote sensing image block in the later phase, and the forest type change status label between the previous and next phases predicted by the decoder are used as inputs to a preset loss function, and the preset loss function outputs the loss value of the forest type change detection model.

[0045] S60: Iteratively train the forest type change detection model according to the loss value until the loss value meets a predetermined training termination condition, thereby obtaining a trained forest type change detection model.

[0046] The predetermined training termination condition may be that the loss value is less than a preset loss threshold, or that the loss value fluctuates within a preset threshold range.

[0047] In an embodiment of the present application, a stochastic gradient descent algorithm is used to iteratively optimize the network weight parameters of the forest type change detection model so that the loss value continues to decrease until the loss value is less than a preset loss threshold. The network weight parameters are saved to obtain a trained forest type change detection model.

[0048] S70: Inputting high-resolution remote sensing images of two phases of the area to be detected into the trained forest type change detection model to obtain forest type change detection results.

[0049] The area to be detected is the area where forest type change detection is required.

[0050] In an embodiment of the present application, high-resolution remote sensing images of the before and after phases are taken of the area to be detected, and the high-resolution remote sensing images of the before and after phases are input into a trained forest type change detection model to obtain the forest type recognition result of the before phase, the forest type recognition result of the after phase and the forest type change status of the before and after phases of the area to be detected.

[0051] The embodiment of the present application is applied, by obtaining a sample image data set; the sample image data set includes several pairs of image blocks and label data for each pair of image blocks; each pair of image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data for each pair of image blocks includes a forest type label for each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label for each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change status label for each pixel in the high-resolution remote sensing image block of the later phase compared to the corresponding pixel in the high-resolution remote sensing image block of the previous phase; a forest type change detection model is constructed; the forest type change detection model includes an encoder and a decoder; input each pair of image blocks into the encoder to obtain a multi-scale depth feature map; input the multi-scale depth feature map into the decoder to obtain the forest type change detection result of each pair of image blocks; calculate the loss value of the forest type change detection model according to the forest type change detection result of each pair of image blocks, the label data of each pair of image blocks and the preset loss function; according to the loss value, iteratively train the forest type change detection model until the loss value meets the predetermined training termination condition to obtain the trained forest type change detection model; input the high-resolution remote sensing images of the two phases of the area to be detected into the trained forest type change detection model to obtain the forest type change detection result. This application trains the forest type change detection model, and automatically outputs the forest type change detection result based on the trained forest type change detection model. It has the advantages of automation and high precision, can reduce manual workload, and meet the business needs of dynamic monitoring of forest resources.

[0052] In an optional embodiment, step S10 includes steps S11 to S13, which are specifically as follows:

[0053] S11: Obtain high-resolution remote sensing images of the same area in two phases.

[0054] In the embodiment of the present application, the high-resolution remote sensing images of the two phases include the high-resolution remote sensing images of the front phase and the high-resolution remote sensing images of the back phase. The high-resolution remote sensing images can be high-resolution remote sensing images of the Gaofen-2 satellite or other satellite high-resolution remote sensing images.

[0055] S12: performing data preprocessing on the high-resolution remote sensing images of the two time phases to obtain preprocessed remote sensing images of the two time phases.

[0056] Among them, data preprocessing includes but is not limited to radiation calibration, atmospheric correction, orthorectification, spatial-spectral fusion and geometric registration.

[0057] In an embodiment of the present application, data preprocessing is performed on the high-resolution remote sensing image of the front phase and the high-resolution remote sensing image of the rear phase respectively to obtain the preprocessed high-resolution remote sensing image of the front phase and the preprocessed high-resolution remote sensing image of the rear phase.

[0058] S13: Cut the pre-processed remote sensing images of the two time phases into pairs of image blocks according to a fixed size to obtain a sample image data set.

[0059] In the embodiment of the present application, the high-resolution remote sensing image of the pre-processed front phase and the high-resolution remote sensing image of the pre-processed back phase are respectively cut according to a fixed size to obtain a pair of image blocks. Among them, the spatial range and the number of bands of the image block of the front phase and the image block of the back phase are exactly the same. For example, the spatial size of the image block is 224×224 pixels.

[0060] By preprocessing and cropping high-resolution remote sensing images of the same area in two phases, a sample image dataset can be automatically and quickly constructed.

[0061] In an optional embodiment, step S12 includes step S121, which is specifically as follows:

[0062] S121: Perform radiometric calibration, atmospheric correction, orthorectification, spatial-spectral fusion, and geometric registration on the high-resolution remote sensing images of the two time phases to obtain the preprocessed remote sensing images of the two time phases.

[0063] In the embodiment of the present application, data preprocessing is performed on the high-resolution remote sensing images of the two phases. Specifically, radiation calibration, atmospheric correction, orthorectification and space-spectrum fusion are performed on the high-resolution remote sensing images of the two phases, so as to correct the radiation distortion and geometric distortion in the remote sensing image imaging process and improve the spatial resolution of the remote sensing image. Then, using a standard unbiased electronic map as the spatial reference, no less than 50 control points are selected, and a rational function model is used to accurately geometrically align the high-resolution remote sensing images of the two phases, so that the registration error between the high-resolution remote sensing images of the two phases and the standard unbiased electronic map is less than 1 pixel, and two high-resolution remote sensing images of the same geographical range but different phases are obtained.

[0064] In an optional embodiment, the encoder includes a first backbone network and a second backbone network. Step S30 includes steps S31 to S32, which are as follows:

[0065] S31: inputting the high-resolution remote sensing image block of the previous phase into the first backbone network to obtain a multi-scale depth feature map of the previous phase;

[0066] S32: Input the high-resolution remote sensing image block of the later phase into the second backbone network to obtain a multi-scale depth feature map of the later phase.

[0067] Among them, the first backbone network and the second backbone network are both Swin Transformer backbone networks.

[0068] In the embodiment of the present application, the encoder is composed of two parallel Swin Transformer backbone networks that share network weight parameters, and the high-resolution remote sensing image block of the previous phase is input into a Swin Transformer backbone network to obtain the multi-scale depth feature map of the previous phase. Specifically, the multi-scale depth feature map of the previous phase is represented as:

[0069] feats1=(feat 1，1 ,feat. 1，2 ,feat. 1，3 ,feat. 1，4 )=ST(img1)

[0070] Among them, img1 represents the high-resolution remote sensing image block of the previous phase.

[0071] The high-resolution remote sensing image block of the later phase is input into another Swin Transformer backbone network to obtain the multi-scale depth feature map of the later phase. The multi-scale depth feature map of the later phase is expressed as:

[0072] feats2=(feat 2，1 ,feat. 2，2 ,feat 2，3 ,feat 2，4 )=ST(img2)

[0073] Among them, img2 represents the high-resolution remote sensing image block of the later phase.

[0074] By using two parallel Swin Transformer backbone networks to extract the deep semantic features of the high-resolution remote sensing image blocks in the previous phase and the high-resolution remote sensing image blocks in the later phase respectively, the generation efficiency and quality of multi-scale deep feature maps can be improved.

[0075] In an optional embodiment, the decoder includes a front phase feature decoding branch, a change detection decoding branch and a rear phase feature decoding branch. Step S40 includes steps S41 to S42, which are as follows:

[0076] S41: Input the multi-scale depth feature map of the previous phase into the feature decoding branch of the previous phase to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase.

[0077] The forest type probability is used to indicate the possibility of belonging to a certain forest type.

[0078] In an embodiment of the present application, through the previous phase feature decoding branch, the probability of each pixel in the high-resolution remote sensing image block of the previous phase belonging to the coniferous forest type, the probability of not belonging to the coniferous forest type, the probability of belonging to the broad-leaved forest type, and the probability of not belonging to the broad-leaved forest type can be predicted.

[0079] S42: Input the post-phase multi-scale depth feature map into the post-phase feature decoding branch to obtain the forest type probability of each pixel in the post-phase high-resolution remote sensing image block.

[0080] In an embodiment of the present application, through the post-phase feature decoding branch, the probability of each pixel in the high-resolution remote sensing image block of the post-phase belonging to the coniferous forest type, the probability of not belonging to the coniferous forest type, the probability of belonging to the broad-leaved forest type, and the probability of not belonging to the broad-leaved forest type can be predicted.

[0081] S43: Input the multi-scale depth feature map of the previous phase and the multi-scale depth feature map of the subsequent phase into the change detection decoding branch to obtain the forest type change probability of each pixel in each pair of image blocks.

[0082] Among them, the probability of forest type change is used to indicate the possibility of changes in forest types between previous and subsequent phases.

[0083] In the embodiment of the present application, the probability of changes in forest types between previous and subsequent phases can be predicted through the change detection decoding branch.

[0084] Through the previous phase feature decoding branch, the change detection decoding branch and the next phase feature decoding branch, the forest type of the high-resolution remote sensing image block in the previous phase, the forest type of the high-resolution remote sensing image block in the next phase and the change status of the forest type in the previous and next phases can be automatically and quickly predicted.

[0085] In an optional embodiment, the front-phase feature decoding branch includes a first feature pyramid, a first fusion module, a first upsampling layer, a first full convolution layer, and a first softmax layer connected in sequence. Step S41 includes steps S411 to S412, which are specifically as follows:

[0086] S411: Input the multi-scale depth feature map of the previous phase into the first feature pyramid, the first fusion module and the first upsampling layer to obtain a first feature map.

[0087] In the embodiment of the present application, after using the first feature pyramid to enhance the semantic information of the multi-scale depth feature map of the previous phase, the multi-scale depth feature map is fused by the first fusion module, and upsampled to the original image block size through the first upsampling layer to obtain the first feature map. Specifically, the first feature map is represented as:

[0088]

[0089] S412: Input the first feature map into the first full convolution layer and the first softmax layer to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase.

[0090] In the embodiment of the present application, the first fully convolutional layer is used to convert the feature dimension of the first feature map into the number of forest types, and then the first Softmax layer is used to determine the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase. Specifically, the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase is expressed as:

[0091]

[0092] Through the cooperation of the first feature pyramid, the first fusion module, the first upsampling layer, the first full convolution layer and the first softmax layer, the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase can be automatically and quickly obtained.

[0093] In an optional embodiment, the post-temporal feature decoding branch includes a second feature pyramid, a second fusion module, a second upsampling layer, a second full convolution layer, and a second softmax layer connected in sequence. Step S42 includes steps S421 to S422, which are specifically as follows:

[0094] S421: Input the posterior phase multi-scale depth feature map into the second feature pyramid, the second fusion module and the second upsampling layer to obtain a second feature map.

[0095] In the embodiment of the present application, after the semantic information of the multi-scale depth feature map of the phase is enhanced by using the second feature pyramid, the multi-scale depth feature map is fused by the second fusion module, and the second upsampling layer is upsampled to the original image block size to obtain the second feature map. Specifically, the second feature map is represented as:

[0096]

[0097] S422: Input the second feature map into the second full convolution layer and the second softmax layer to obtain the forest type probability of each pixel in the high-resolution remote sensing image block in the later phase.

[0098] In the embodiment of the present application, the second fully convolutional layer is used to convert the feature dimension of the second feature map into the number of forest types, and then the second Softmax layer is used to determine the forest type probability of each pixel in the high-resolution remote sensing image block of the later phase. Specifically, the forest type probability of each pixel in the high-resolution remote sensing image block of the later phase is expressed as:

[0099]

[0100] Through the cooperation of the second feature pyramid, the second fusion module, the second upsampling layer, the second full convolution layer and the second softmax layer, the forest type probability of each pixel in the high-resolution remote sensing image block of the later phase can be obtained automatically and quickly.

[0101] In an optional embodiment, the change detection decoding branch includes a spatiotemporal information fusion module, a third feature pyramid, a third fusion module, a third upsampling layer, a third full convolution layer, and a third softmax layer connected in sequence. Step S43 includes steps S431 to S433, which are specifically as follows:

[0102] S431: Input the multi-scale depth feature map of the previous phase and the multi-scale depth feature map of the subsequent phase into the spatiotemporal information fusion module to obtain a third feature map.

[0103] In the embodiment of the present application, the multi-scale depth feature map of the previous phase and the multi-scale depth feature map of the subsequent phase are fused by the spatiotemporal information fusion module to obtain a multi-scale third feature map. Specifically, the third feature map is represented as:

[0104] feats cd =Merge(feats1,feats2)

[0105] S432: Input the third feature map into the third feature pyramid, the third fusion module and the third upsampling layer to obtain a fourth feature map.

[0106] In the embodiment of the present application, after using the third feature pyramid to enhance the semantic information of the multi-scale third feature map, the multi-scale third feature map is fused by the third fusion module, and upsampled to the original image block size through the third upsampling layer to obtain the fourth feature map. Specifically, the fourth feature map is represented as:

[0107]

[0108] S433: Input the fourth feature map into the third full convolution layer and the third softmax layer to obtain the forest type change probability of each pixel in each pair of image blocks.

[0109] In the embodiment of the present application, the third fully convolutional layer is used to change the feature dimension of the fourth feature map to 2, and then the second Softmax layer is used to determine the probability of forest type change of each pixel in each pair of image blocks. Specifically, the probability of forest type change of each pixel in each pair of image blocks is expressed as:

[0110]

[0111] Through the cooperation of the spatiotemporal information fusion module, the third feature pyramid, the third fusion module, the third upsampling layer, the third full convolution layer and the third softmax layer, the probability of forest type change of each pixel in each pair of image blocks can be obtained automatically and quickly.

[0112] In an optional embodiment, the preset loss function includes a first loss function, a second loss function and a third loss function, and step S50 includes steps S51 to S54, which are specifically as follows:

[0113] S51: Inputting the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase and the forest type label of each pixel in the high-resolution remote sensing image block of the previous phase into a first loss function to obtain a first loss value.

[0114] In the embodiment of the present application, the expression of the first loss function is as follows:

[0115]

[0116] Among them, L set1 represents the first loss value, represents the forest type probability of the jth pixel in the high-resolution remote sensing image block of the i-th previous phase, represents the forest type label of the jth pixel in the high-resolution remote sensing image block of the ith previous phase, m represents the total number of high-resolution remote sensing image blocks of the previous phase, and n represents the total number of pixels in the high-resolution remote sensing image block of each previous phase.

[0117] S52: Inputting the forest type probability of each pixel in the high-resolution remote sensing image block in the later phase and the forest type label of each pixel in the high-resolution remote sensing image block in the later phase into the second loss function to obtain a second loss value.

[0118] In the embodiment of the present application, the expression of the second loss function is as follows:

[0119]

[0120] Among them, L seg2 represents the second loss value, represents the forest type probability of the kth pixel in the high-resolution remote sensing image block of the i-th later phase, Represents the forest type label of the kth pixel in the high-resolution remote sensing image block of the i-th later phase.

[0121] S53: Inputting the forest type change probability of each pixel in each pair of image blocks and the forest type change state label of each pair of image blocks into the third loss function to obtain a third loss value.

[0122] In this embodiment of the present application, the expression of the third loss function is as follows:

[0123]

[0124] Among them, L cd represents the third loss value, represents the probability of forest type change of the qth pixel in the high-resolution image block of the ith later phase compared to the qth pixel in the high-resolution image block of the ith later phase, Indicates the forest type change status label of the qth pixel in the high-resolution image block in the i-th later phase compared with the qth pixel in the high-resolution image block in the i-th previous phase.

[0125] S54: Obtain a total loss value according to the first loss value, the second loss value and the third loss value.

[0126] In the embodiment of the present application, the first loss value, the second loss value and the third loss value are summed to obtain a total loss value. Specifically, the calculation formula of the total loss value is as follows:

[0127] L total =L seg1 +L seg2 +L cd

[0128] The forest type change detection model runs based on the Pytorch deep learning framework. When the network training is initialized, the hyperparameters involved in the model training are set, including the learning rate, the number of training iterations, the batch size, the model optimization algorithm, etc. The network weight parameters are continuously optimized iteratively to make the overall loss value L total Preferably, the learning rate is set to 0.0001, the number of training iterations is set to 200, the batch size is set to 8, and the model optimization algorithm is the stochastic gradient descent method.

[0129] Based on the total loss function, the forest type change detection model is iteratively trained to automatically and quickly obtain the trained forest type change detection model.

[0130] The following is an embodiment of the device of the present application, which can be used to execute the content of the method in the embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the content of the method in the embodiment of the present application.

[0131] See also Figure 2 , which shows a schematic diagram of the structure of a forest type change detection device provided in an embodiment of the present application. The forest type change detection device 8 provided in an embodiment of the present application comprises:

[0132] The data set acquisition module 81 is used to acquire a sample image data set; the sample image data set includes a plurality of pairs of image blocks and label data of each pair of image blocks; each pair of image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data of each pair of image blocks includes a forest type label of each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label of each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change state label of each pixel in the high-resolution remote sensing image block of the later phase compared with the corresponding pixel in the high-resolution remote sensing image block of the previous phase;

[0133] A model building module 82 is used to build a forest type change detection model; the forest type change detection model includes an encoder and a decoder;

[0134] A depth feature map acquisition module 83, used for inputting each pair of image blocks into an encoder to obtain a multi-scale depth feature map;

[0135] A depth feature map input module 84 is used to input the multi-scale depth feature map into the decoder to obtain the forest type change detection result of each pair of image blocks;

[0136] A loss value calculation module 85, used to calculate the loss value of the forest type change detection model according to the forest type change detection result of each pair of image blocks, the label data of each pair of image blocks and a preset loss function;

[0137] A model training module 86 is used to iteratively train the forest type change detection model according to the loss value until the loss value satisfies a predetermined training termination condition, thereby obtaining a trained forest type change detection model;

[0138] The detection result acquisition module 87 is used to input the high-resolution remote sensing images of the area to be detected in two phases into the trained forest type change detection model to obtain the forest type change detection result.

[0139] It should be noted that the forest type change detection device provided in the above embodiment only uses the division of the above functional modules as an example when executing the forest type change detection method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the forest type change detection device provided in the above embodiment and the forest type change detection method belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0140] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0141] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting forest type changes, characterized in that: The method comprises the following steps: Acquire a sample image data set; the sample image data set includes a plurality of pairs of image blocks and label data of each pair of the image blocks; each pair of the image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data of each pair of the image blocks includes a forest type label of each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label of each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change state label of each pixel in the high-resolution remote sensing image block of the later phase compared with the corresponding pixel in the high-resolution remote sensing image block of the previous phase; Constructing a forest type change detection model; the forest type change detection model includes an encoder and a decoder; Input each pair of the image blocks into the encoder to obtain a multi-scale depth feature map; Inputting the multi-scale depth feature map into the decoder to obtain a forest type change detection result for each pair of image blocks; Calculating the loss value of the forest type change detection model according to the forest type change detection results of each pair of image blocks, the label data of each pair of image blocks and a preset loss function; Iteratively training the forest type change detection model according to the loss value until the loss value meets a predetermined training termination condition, thereby obtaining a trained forest type change detection model; The high-resolution remote sensing images of the area to be detected in two phases are input into the trained forest type change detection model to obtain the forest type change detection result.

2. The forest type change detection method according to claim 1, characterized in that: The encoder includes a first backbone network and a second backbone network; The step of inputting each pair of the image blocks into the encoder to obtain a multi-scale depth feature map comprises: Inputting the high-resolution remote sensing image block of the previous phase into the first backbone network to obtain a multi-scale depth feature map of the previous phase; The high-resolution remote sensing image block of the later phase is input into the second backbone network to obtain a multi-scale depth feature map of the later phase.

3. The forest type change detection method according to claim 2, characterized in that: The decoder comprises a front phase feature decoding branch, a change detection decoding branch and a rear phase feature decoding branch; The step of inputting the multi-scale depth feature map into the decoder to obtain the forest type change detection result of each pair of image blocks includes: Inputting the multi-scale depth feature map of the previous phase into the feature decoding branch of the previous phase to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase; Inputting the post-phase multi-scale depth feature map into the post-phase feature decoding branch to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the post-phase; The multi-scale depth feature map of the previous phase and the multi-scale depth feature map of the subsequent phase are input into the change detection decoding branch to obtain the forest type change probability of each pixel in each pair of the image blocks.

4. The forest type change detection method according to claim 3, characterized in that: The front-phase feature decoding branch includes a first feature pyramid, a first fusion module, a first upsampling layer, a first full convolution layer, and a first softmax layer connected in sequence; The step of inputting the multi-scale depth feature map of the previous phase into the feature decoding branch of the previous phase to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase includes: Inputting the front-phase multi-scale depth feature map into the first feature pyramid, the first fusion module and the first upsampling layer to obtain a first feature map; The first feature map is input into the first fully convolutional layer and the first softmax layer to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase.

5. The forest type change detection method according to claim 3, characterized in that: The post-temporal feature decoding branch includes a second feature pyramid, a second fusion module, a second upsampling layer, a second full convolution layer and a second softmax layer connected in sequence; The step of inputting the post-phase multi-scale depth feature map into the post-phase feature decoding branch to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the post-phase includes: Inputting the posterior phase multi-scale depth feature map into the second feature pyramid, the second fusion module and the second upsampling layer to obtain a second feature map; The second feature map is input into the second fully convolutional layer and the second softmax layer to obtain the forest type probability of each pixel in the high-resolution remote sensing image block of the later phase.

6. The forest type change detection method according to claim 3, characterized in that: The change detection decoding branch includes a spatiotemporal information fusion module, a third feature pyramid, a third fusion module, a third upsampling layer, a third full convolution layer, and a third softmax layer connected in sequence; The step of inputting the multi-scale depth feature map of the previous phase and the multi-scale depth feature map of the subsequent phase into the change detection decoding branch to obtain the forest type change probability of each pixel in each pair of the image blocks includes: Inputting the front phase multi-scale depth feature map and the back phase multi-scale depth feature map into the spatiotemporal information fusion module to obtain a third feature map; Inputting the third feature map into the third feature pyramid, the third fusion module and the third upsampling layer to obtain a fourth feature map; The fourth feature map is input into the third fully convolutional layer and the third softmax layer to obtain the forest type change probability of each pixel in each pair of image blocks.

7. The forest type change detection method according to claim 3, characterized in that: The preset loss function includes a first loss function, a second loss function and a third loss function; The step of calculating the loss value of the forest type change detection model according to the forest type change detection results of each pair of image blocks, the forest type change state labels of each pair of image blocks and a preset loss function comprises: Inputting the forest type probability of each pixel in the high-resolution remote sensing image block of the previous phase and the forest type label of each pixel in the high-resolution remote sensing image block of the previous phase into the first loss function to obtain a first loss value; Inputting the forest type probability of each pixel in the high-resolution remote sensing image block of the later phase and the forest type label of each pixel in the high-resolution remote sensing image block of the later phase into the second loss function to obtain a second loss value; Inputting the forest type change probability of each pixel in each pair of the image blocks and the forest type change state label of each pair of the image blocks into the third loss function to obtain a third loss value; A total loss value is obtained according to the first loss value, the second loss value and the third loss value.

8. The method for detecting forest type changes according to any one of claims 1 to 7, characterized in that: The step of obtaining a sample image data set includes: Obtain high-resolution remote sensing images of the same area in two phases; Performing data preprocessing on the high-resolution remote sensing images of the two time phases to obtain preprocessed remote sensing images of the two time phases; The preprocessed remote sensing images of the two time phases are cut into paired image blocks according to a fixed size to obtain a sample image data set.

9. The forest type change detection method according to claim 8, characterized in that: The step of performing data preprocessing on the high-resolution remote sensing images of the two time phases to obtain the preprocessed remote sensing images of the two time phases comprises: The high-resolution remote sensing images of the two time phases are subjected to radiometric calibration, atmospheric correction, orthorectification, spatial-spectral fusion and geometric registration to obtain preprocessed remote sensing images of the two time phases.

10. A forest type change detection device, characterized in that: include: A data set acquisition module, used to acquire a sample image data set; the sample image data set includes a plurality of pairs of image blocks and label data of each pair of image blocks; each pair of image blocks includes a high-resolution remote sensing image block of a previous phase and a high-resolution remote sensing image block of a later phase, and the label data of each pair of image blocks includes a forest type label of each pixel in the high-resolution remote sensing image block of the previous phase, a forest type label of each pixel in the high-resolution remote sensing image block of the later phase, and a forest type change state label of each pixel in the high-resolution remote sensing image block of the later phase compared to the corresponding pixel in the high-resolution remote sensing image block of the previous phase; A model building module, used to build a forest type change detection model; the forest type change detection model includes an encoder and a decoder; A depth feature map acquisition module, used for inputting each pair of image blocks into the encoder to obtain a multi-scale depth feature map; A depth feature map input module, used for inputting the multi-scale depth feature map into the decoder to obtain a forest type change detection result for each pair of image blocks; A loss value calculation module, used to calculate the loss value of the forest type change detection model according to the forest type change detection result of each pair of image blocks, the label data of each pair of image blocks and a preset loss function; A model training module, used for iteratively training the forest type change detection model according to the loss value until the loss value satisfies a predetermined training termination condition, thereby obtaining a trained forest type change detection model; The detection result acquisition module is used to input the high-resolution remote sensing images of the area to be detected in two phases into the trained forest type change detection model to obtain the forest type change detection result.

Images