Lightweight remote sensing change detection method based on backpropagation-free learning

Abstract

The application provides a lightweight remote sensing change detection method based on a non-back propagation learning, comprising the following steps: uniformly cutting remote sensing images before and after changes to be inferred to obtain a sub-image set; using a block dimension-based affinity matrix method, selecting changed sub-images before and after changes and unchanged sub-images before and after changes from the sub-image set as training samples of a neural network; using the training samples, fine-tuning the neural network by using a local learning method, assigning a loss function to a convolutional layer to be trained for fine-tuning, keeping parameters of other convolutional layers unchanged, and making the other convolutional layers perform forward propagation to obtain a fine-tuned neural network; inputting the remote sensing images before and after changes to be inferred into the neural network for inference, and binarizing an inference result to obtain a binary change result. The application adopting the above scheme is suitable for heterogeneous remote sensing image change detection when the computing resources are limited.

Description

Technical Field

[0001] This application relates to the field of heterogeneous remote sensing image change detection technology, and in particular to a lightweight remote sensing change detection method and apparatus based on backpropagation-free learning. Background Technology

[0002] Change detection is of great significance in military surveillance, land use planning, and disaster relief. It can promptly detect information on the actual affected areas after disasters such as mudslides, earthquakes, and typhoons and guide rescue efforts.

[0003] In recent years, many scholars have proposed methods for performing change detection tasks using traditional machine learning techniques. Examples include pixel-level homogeneity transformation based on K-nearest neighbors and dictionary-based change detection methods. However, these methods generally have lower detection accuracy than traditional methods and typically require significant computational resources. This further limits the practical deployment and application of such methods. Summary of the Invention

[0004] This application aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the first objective of this application is to propose a lightweight remote sensing change detection method based on backpropagation-free learning, which is applicable to the detection of changes in heterogeneous remote sensing images when computational resources are limited.

[0006] The second objective of this application is to propose a lightweight remote sensing change detection device based on backpropagation-free learning.

[0007] The third objective of this application is to propose a computer device.

[0008] The fourth objective of this application is to provide a non-transitory computer-readable storage medium.

[0009] To achieve the above objectives, the first aspect of this application proposes a lightweight remote sensing change detection method based on backpropagation-free learning, comprising:

[0010] The remote sensing images before and after the changes to be inferred are uniformly divided to obtain a set of sub-images;

[0011] Using a block-dimensional affinity matrix method, we select changed and unchanged sub-images from the sub-image set as training samples for the neural network.

[0012] Using training samples, the neural network is fine-tuned using a local learning method. During fine-tuning, a loss function is assigned to the convolutional layers that need to be trained, while keeping the parameters of other convolutional layers unchanged. This allows the other convolutional layers to perform forward propagation, resulting in the fine-tuned neural network.

[0013] The remote sensing images before and after the changes to be inferred are input into the neural network for inference, and the inference results are binarized to obtain the binary change results.

[0014] Optionally, in one embodiment of this application, a block-dimensional affinity matrix method is used to select changed and unchanged sub-images from the sub-image set as training samples for the neural network, including:

[0015] The distance between each sub-image and other sub-images is calculated on the remote sensing images before and after the change, and is expressed as:

[0016]

[0017] Where the superscript m stands for pre or post, pre and post representing before and after the change, respectively. i For the i-th sub-image;

[0018] The probability of sub-image change is measured using an exponential function of distance, where the expression when using the affinity matrix is:

[0019]

[0020] in, For normalization parameters, use all The maximum value represents:

[0021] ;

[0022] Summing the probabilities of change for each sub-image and normalizing the result, we obtain the probability of change for each sub-image. The summation of the probabilities of change for the i-th sub-image is expressed as:

[0023]

[0024] Where N is the number of segments in the remote sensing image;

[0025] Based on the change probability of each sub-image, several sub-images before and after the change that are most likely to change and several sub-images before and after the change that are least likely to change are selected from the set of sub-images as training samples for the neural network.

[0026] Optionally, in one embodiment of this application, assigning a loss function to the convolutional layer that needs to be trained includes:

[0027] Deep convolutional layers are used as the layers to be trained. For these deep convolutional layers, a contrastive learning loss function is constructed to increase the difference in output features between changing samples and decrease the difference in output features between invariant samples. The contrastive learning loss function is:

[0028]

[0029] Among them, L m This represents the loss function assigned to the m-th convolutional layer that needs to be trained. The output features of the samples that have changed and those that have not changed are processed separately. t1 and t2 are subnetworks that process the image before and after the change, respectively. The output features represent the image before the change. The output features represent the transformed image.

[0030] (x, y, z) represents the output coordinates, W m H m C m This represents the width, length, and number of channels of the output feature;

[0031] Fine-tuning deep convolutional layers includes:

[0032] The gradient of the output feature is calculated using the contrastive learning loss function and the output feature values, and is expressed as:

[0033]

[0034] Where gO is the gradient of the output feature O, and s(·) is the sign function, expressed as:

[0035]

[0036] The gradient of the weights is calculated based on the gradient of the output features, and is expressed as:

[0037]

[0038] in, It is a subnetwork t p The weight gradient of the m-th layer, where * denotes the convolution operation. For subnetwork t p The input of the m-th layer;

[0039] Weights are updated based on weight gradients, expressed as:

[0040]

[0041] Where k represents the number of iterations.

[0042] Optionally, in one embodiment of this application, the remote sensing images before and after the changes to be inferred are input into a neural network for inference, and the inference result is obtained, which is represented as:

[0043]

[0044] I d =D x +U(D x+1 )+...+U(D x+n )

[0045] Among them, C m Let be the number of channels in the m-th convolutional layer, x, y, z be the output coordinates, t1 and t2 be the subnetworks processing the image before and after the transformation, respectively, and pre and post represent the image before and after the transformation, respectively. The output features represent the image before the change. D represents the output features of the transformed image. x+n D x+1 ... D x+n Let x+n be the number of convolutional layers to be trained, and I be the number of convolutional layers in the neural network. d For changing images;

[0046] The binarized result obtained using the Otsu method (maximum inter-class variance) is expressed as follows:

[0047] I b =Otsu(I d ).

[0048] To achieve the above objectives, a second aspect of the present invention provides a lightweight remote sensing change detection device based on backpropagation-free learning, comprising:

[0049] The image segmentation module is used to uniformly segment the remote sensing images before and after the changes to be inferred, and obtain a set of sub-images;

[0050] The sample selection module is used to select, from the set of sub-images, the changed sub-images before and after the change and the unchanged sub-images before and after the change as training samples for the neural network, using a block-dimensional affinity matrix method.

[0051] The neural network fine-tuning module is used to fine-tune the neural network using training samples and a local learning method. During fine-tuning, a loss function is assigned to the convolutional layer that needs to be trained for fine-tuning, while keeping the parameters of other convolutional layers unchanged. This allows other convolutional layers to perform forward propagation, resulting in the fine-tuned neural network.

[0052] The change detection module is used to input the remote sensing images before and after the change to be inferred into the neural network for inference, and to binarize the inference results to obtain the binary change result.

[0053] Optionally, in one embodiment of this application, a block-dimensional affinity matrix method is used to select changed and unchanged sub-images from the sub-image set as training samples for the neural network, including:

[0054] The distance between each sub-image and other sub-images is calculated on the remote sensing images before and after the change, and is expressed as:

[0055]

[0056] Where the superscript m stands for pre or post, pre and post representing before and after the change, respectively, W i For the i-th sub-image;

[0057] The probability of sub-image change is measured using an exponential function of distance, where the expression when using the affinity matrix is:

[0058]

[0059] in, For normalization parameters, use all The maximum value represents:

[0060] ;

[0061] Summing the probabilities of change for each sub-image and normalizing the result, we obtain the probability of change for each sub-image. The summation of the probabilities of change for the i-th sub-image is expressed as:

[0062]

[0063] Where N is the number of segments in the remote sensing image;

[0064] Based on the change probability of each sub-image, several sub-images before and after the change that are most likely to change and several sub-images before and after the change that are least likely to change are selected from the set of sub-images as training samples for the neural network.

[0065] Optionally, in one embodiment of this application, assigning a loss function to the convolutional layer that needs to be trained includes:

[0066] Deep convolutional layers are used as the layers to be trained. For these deep convolutional layers, a contrastive learning loss function is constructed to increase the difference in output features between changing samples and decrease the difference in output features between invariant samples. The contrastive learning loss function is:

[0067]

[0068]

[0069] Among them, L m This represents the loss function assigned to the m-th convolutional layer that needs to be trained. The output features of the samples that have changed and those that have not changed are processed separately. t1 and t2 are subnetworks that process the image before and after the change, respectively. The output features represent the image before the change. The output features represent the transformed image.

[0070] (x, y, z) represents the output coordinates, W m H m C m This represents the width, length, and number of channels of the output feature;

[0071] Fine-tuning deep convolutional layers includes:

[0072] The gradient of the output feature is calculated using the contrastive learning loss function and the output feature values, and is expressed as:

[0073]

[0074] Where gO is the gradient of the output feature O, and s(·) is the sign function, expressed as:

[0075]

[0076] The gradient of the weights is calculated based on the gradient of the output features, and is expressed as:

[0077]

[0078] in, It is a subnetwork t p The weight gradient of the m-th layer, where * denotes the convolution operation. For subnetwork t p The input of the m-th layer;

[0079] Weights are updated based on weight gradients, expressed as:

[0080]

[0081] Where k represents the number of iterations.

[0082] Optionally, in one embodiment of this application, the remote sensing images before and after the changes to be inferred are input into a neural network for inference, and the inference result is obtained, which is represented as:

[0083]

[0084] I d =D x +U(D x+1 )+...+U(D x+n )

[0085] Among them, Cm Let be the number of channels in the m-th convolutional layer, x, y, z be the output coordinates, t1 and t2 be the subnetworks processing the image before and after the transformation, respectively, and pre and post represent the image before and after the transformation, respectively. The output features represent the image before the change. D represents the output features of the transformed image. x+n D x+1 ... D x+n Let x+n be the number of convolutional layers to be trained, and I be the number of convolutional layers in the neural network. d For changing images;

[0086] The binarized result obtained using the Otsu method (maximum inter-class variance) is expressed as follows:

[0087] I b =Otsu(I d ).

[0088] To achieve the above objectives, a third aspect of the present invention provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described lightweight remote sensing change detection method based on backpropagation-free learning.

[0089] To achieve the above objectives, a fourth aspect of the present invention provides a non-transitory computer-readable storage medium that, when instructions in the storage medium are executed by a processor, enables the execution of a lightweight remote sensing change detection method based on backpropagation-free learning.

[0090] The lightweight remote sensing change detection method and apparatus based on backpropagation-free learning in this application firstly uses a block-dimensional affinity matrix method to quickly select changed and invariant samples as training samples for the neural network. Secondly, a local learning method is used to fine-tune the neural network. This local learning-based fine-tuning method assigns a loss function to each convolutional layer that needs training. After forward propagation through this layer, the output of this layer is used to directly calculate the weight gradient, thereby avoiding the computationally intensive backpropagation algorithm. Finally, the entire image is input into the neural network to perform inference operations, and the result is binarized to obtain a binary change result.

[0091] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0092] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0093] Figure 1 This is a flowchart illustrating a lightweight remote sensing change detection method based on backpropagation-free learning provided in Embodiment 1 of this application.

[0094] Figure 2 This is a schematic diagram of the basic network structure of an embodiment of this application;

[0095] Figure 3 Given the existing affinity matrix + S 3 Example image of the detection effect of method N;

[0096] Figure 4 This is an example diagram illustrating the detection effect of an embodiment of this application;

[0097] Figure 5 This is a schematic diagram of a lightweight remote sensing change detection device based on backpropagation-free learning, provided as an embodiment of this application. Detailed Implementation

[0098] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0099] The following description, with reference to the accompanying drawings, describes a lightweight remote sensing change detection method and apparatus based on backpropagation-free learning according to embodiments of this application.

[0100] Figure 1 This is a flowchart illustrating a lightweight remote sensing change detection method based on backpropagation-free learning provided in Embodiment 1 of this application.

[0101] like Figure 1 As shown, this lightweight remote sensing change detection method based on backpropagation-free learning includes the following steps:

[0102] Step 101: The remote sensing images before and after the changes to be inferred are uniformly divided to obtain a set of sub-images;

[0103] In this embodiment, the image segmentation process includes:

[0104] Acquire remote sensing images before and after the change, with dimensions W×H, where W represents the number of rows and H represents the number of columns.

[0105] The size of the smaller pieces is p×p (usually set to 64×64).

[0106] Step 102: Using the block-dimensional affinity matrix method, select the changed and unchanged sub-images before and after the change from the sub-image set as training samples for the neural network.

[0107] In this embodiment, the process of selecting training samples includes:

[0108] Step 1: Divide the two complete heterogeneous remote sensing images into non-overlapping, uniformly sized p×p blocks. Directly calculate the arithmetic mean of each block, denoted as . and Here, `pre` and `post` represent the small blocks derived from the images before and after the change. `i` represents the i-th small block.

[0109] Step 2: Calculate the distance between each small block and other small blocks on both the before and after images:

[0110]

[0111] m represents the image before or after the event. The larger the distance obtained, the weaker the correlation between the two small patches. A smaller correlation indicates a greater possibility of change.

[0112] Step 3: Measure the probability of change based on an exponential function of distance. Here, we use an affinity matrix, expressed as follows:

[0113]

[0114] in, For normalization parameters, use all The maximum value represents:

[0115]

[0116] Then use the formula to sum. The probability of change for each small block is then normalized. In this embodiment, the neural network is fine-tuned using 8 samples that are least likely to change and 2 samples that are most likely to change.

[0117] Step 103: Using training samples, fine-tune the neural network using a local learning method. During fine-tuning, assign a loss function to the convolutional layers that need to be trained and keep the parameters of other convolutional layers unchanged, so that the other convolutional layers perform forward propagation to obtain the fine-tuned neural network.

[0118] In this embodiment, MobileNet is a very lightweight neural network that can effectively extract features. The MobileNet network is used as the base network, such as... Figure 2As shown, this embodiment only fine-tunes the deeper Conv 3-2, Conv4-2, and Conv 5-2 convolutional layers to achieve deep content learning. Simultaneously, during the fine-tuning process, other convolutional layers only perform forward propagation without updating their parameters to achieve lightweighting.

[0119] In this embodiment, the data used for fine-tuning are the selected training samples and the Mobilenet neural network pre-trained using the Cifar10 natural image dataset. The fine-tuning process includes:

[0120] Step 1: Using the training samples selected in the previous module, perform forward propagation from shallow convolutional layers Conv1-1 to DwC3-2.

[0121] Step 2: For each convolutional layer {Convm-2, m = 3, 4, 5} to be fine-tuned, this embodiment assigns a loss function L. m ,like Figure 2 As shown. The loss function used for training is...

[0122]

[0123] Where L m This represents the loss function assigned to {Convm-2, m = 3, 4, 5}. Each L... m Both include two contrastive learning loss functions. and The output features of the changed samples and the invariant samples are processed separately. This represents the feature output of the {Convm-2, m=3,4,5} layers of subnetwork p when the input is a changing sample (v=+) or a constant sample (v=-). (x, y, z) represent the coordinates of the output. Where W... m H m C m This represents the width, length, and number of channels of the output feature. By using a contrastive learning loss function, the differences in output features of changing samples gradually increase during training, while the differences in features of invariant samples gradually decrease, thereby achieving the ability to highlight changing features and hide invariant features.

[0124] Step 3: Calculate the gradient directly using the loss function and the output feature values.

[0125]

[0126] Where gO is the gradient of the output feature O. s(·) is the sign function:

[0127]

[0128] After obtaining the gradient of the output feature through prior calculation, the formula can be used directly:

[0129]

[0130] Calculate the gradient of the weights. This represents the weight gradients of the {Convm-2, m = 3, 4, 5} layers of subnetwork p. * represents the convolution operation. For subnetwork t p The input of the m-th layer. Finally, using the formula:

[0131]

[0132] Update the weights. Here, k represents different iteration numbers. This embodiment uses this method to fine-tune the neural network 1000 times to achieve sufficient fine-tuning.

[0133] Step 104: Input the remote sensing images before and after the changes to be inferred into the neural network for inference, and binarize the inference results to obtain the binary change results.

[0134] In this embodiment, the entire heterogeneous remote sensing image is input into the finely tuned MobileNet neural network for inference and binarization to obtain a binary transformation result. The neural network inference and binarization process includes:

[0135] Step 1: Input the entire heterogeneous remote sensing image into the neural network for inference. Use the formula...

[0136]

[0137] I d =D 3 +U(D 4 )+U(D 5 )

[0138] The change image I is obtained through reasoning. d .in When the input image is I τ Let τ ∈ {pre,post} be the feature output of neural network p when the image is before (pre) or after (post) the change. U is a bilinear upsampling function that upsamples the smaller output D. 4 D 5 Upsampled to the output D of the previous layer 3 They are the same size. Ultimately, this embodiment uses the Otsu's method to obtain the binarization result:

[0139] I b =Otsu(I d )

[0140] The lightweight remote sensing change detection method based on backpropagation-free learning in this application firstly uses a block-dimensional affinity matrix method to quickly select changed and invariant samples as training samples for the neural network. Secondly, a local learning method is used to fine-tune the neural network. This local learning-based fine-tuning method assigns a loss function to each convolutional layer that needs training. After forward propagation through this layer, the output of this layer is used to directly calculate the weight gradient, thus avoiding the computationally intensive backpropagation algorithm. Finally, the entire image is input into the neural network to perform inference operations, and the result is binarized to obtain a binary change result.

[0141] Table 1 shows the existing affinity matrix +S 3 Comparison results between the N method and the lightweight remote sensing change detection method based on backpropagation-free learning in this embodiment.

[0142] method Kappa time <![CDATA[Affinity matrix + S 3 N]]> 0.8028 74s This embodiment 0.8823 21s

[0143] Table 1

[0144] Figure 3 , Figure 4 These represent the affinity matrix of this embodiment and the existing affinity matrix +S, respectively. 3 Example diagram of the detection effect of the N method and the lightweight remote sensing change detection method based on backpropagation-free learning in this embodiment.

[0145] To achieve the above embodiments, this application also proposes a lightweight remote sensing change detection device based on backpropagation-free learning.

[0146] Figure 5 This is a schematic diagram of a lightweight remote sensing change detection device based on backpropagation-free learning, provided as an embodiment of this application.

[0147] like Figure 5 As shown, the lightweight remote sensing change detection device based on backpropagation-free learning includes:

[0148] The image segmentation module is used to uniformly segment the remote sensing images before and after the changes to be inferred, and obtain a set of sub-images;

[0149] The sample selection module is used to select, from the set of sub-images, the changed sub-images before and after the change and the unchanged sub-images before and after the change as training samples for the neural network, using a block-dimensional affinity matrix method.

[0150] The neural network fine-tuning module is used to fine-tune the neural network using training samples and a local learning method. During fine-tuning, a loss function is assigned to the convolutional layer that needs to be trained for fine-tuning, while keeping the parameters of other convolutional layers unchanged. This allows other convolutional layers to perform forward propagation, resulting in the fine-tuned neural network.

[0151] The change detection module is used to input the remote sensing images before and after the change to be inferred into the neural network for inference, and to binarize the inference results to obtain the binary change result.

[0152] Optionally, in one embodiment of this application, a block-dimensional affinity matrix method is used to select changed and unchanged sub-images from the sub-image set as training samples for the neural network, including:

[0153] The distance between each sub-image and other sub-images is calculated on the remote sensing images before and after the change, and is expressed as:

[0154]

[0155] Where the superscript m stands for pre or post, pre and post representing before and after the change, respectively. i For the i-th sub-image;

[0156] The probability of sub-image change is measured using an exponential function of distance, where the expression when using the affinity matrix is:

[0157]

[0158] in, For normalization parameters, use all The maximum value represents:

[0159] ;

[0160] Summing the probabilities of change for each sub-image and normalizing the result, we obtain the probability of change for each sub-image. The summation of the probabilities of change for the i-th sub-image is expressed as:

[0161]

[0162] Where N is the number of segments in the remote sensing image;

[0163] Based on the change probability of each sub-image, several sub-images before and after the change that are most likely to change and several sub-images before and after the change that are least likely to change are selected from the set of sub-images as training samples for the neural network.

[0164] Optionally, in one embodiment of this application, assigning a loss function to the convolutional layer that needs to be trained includes:

[0165] Deep convolutional layers are used as the layers to be trained. For these deep convolutional layers, a contrastive learning loss function is constructed to increase the difference in output features between changing samples and decrease the difference in output features between invariant samples. The contrastive learning loss function is:

[0166]

[0167] Among them, L m This represents the loss function assigned to the m-th convolutional layer that needs to be trained. The output features of the samples that have changed and those that have not changed are processed separately. t1 and t2 are subnetworks that process the image before and after the change, respectively. The output features represent the image before the change. The output features represent the transformed image.

[0168] (x, y, z) represents the output coordinates, W m H m C m This represents the width, length, and number of channels of the output feature;

[0169] Fine-tuning deep convolutional layers includes:

[0170] The gradient of the output feature is calculated using the contrastive learning loss function and the output feature values, and is expressed as:

[0171]

[0172] Where gO is the gradient of the output feature O, and s(·) is the sign function, expressed as:

[0173]

[0174] The gradient of the weights is calculated based on the gradient of the output features, and is expressed as:

[0175]

[0176] in, It is a subnetwork t p The weight gradient of the m-th layer, where * denotes the convolution operation. For subnetwork t p The input of the m-th layer;

[0177] Weights are updated based on weight gradients, expressed as:

[0178]

[0179] Where k represents the number of iterations.

[0180] Optionally, in one embodiment of this application, the remote sensing images before and after the changes to be inferred are input into a neural network for inference, and the inference result is obtained, which is represented as:

[0181]

[0182] I d =D x +U(D x+1 )+...+U(D x+n )

[0183] Among them, C m Let be the number of channels in the m-th convolutional layer, x, y, z be the output coordinates, t1 and t2 be the subnetworks processing the image before and after the transformation, respectively, and pre and post represent the image before and after the transformation, respectively. The output features represent the image before the change. D represents the output features of the transformed image. x+n D x+1 ... D x+n Let x+n be the number of convolutional layers to be trained, and I be the number of convolutional layers in the neural network. d For changing images;

[0184] The binarized result obtained using the Otsu method (maximum inter-class variance) is expressed as follows:

[0185] I b =Otsu(I d ).

[0186] It should be noted that the foregoing explanation of the embodiment of the lightweight remote sensing change detection method based on backpropagation-free learning also applies to the lightweight remote sensing change detection device based on backpropagation-free learning in this embodiment, and will not be repeated here.

[0187] To implement the above embodiments, the present invention also proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the methods described in the above embodiments.

[0188] To implement the above embodiments, the present invention also proposes a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method of the above embodiments.

[0189] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0190] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0191] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0192] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0193] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0194] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0195] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0196] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A lightweight remote sensing change detection method based on backpropagation-free learning, characterized in that, include: The remote sensing images before and after the changes to be inferred are uniformly divided to obtain a set of sub-images; Using a block-dimensional affinity matrix method, sub-images before and after changes and sub-images before and after changes that have not changed are selected from the set of sub-images as training samples for the neural network. Using the training samples, the neural network is fine-tuned using a local learning method. During fine-tuning, a loss function is assigned to the convolutional layers that need to be trained for fine-tuning, while keeping the parameters of other convolutional layers unchanged. This allows the other convolutional layers to perform forward propagation, resulting in the fine-tuned neural network. The remote sensing images before and after the changes to be inferred are input into the neural network for inference, and the inference results are binarized to obtain the binary change results. The loss function assigned to the convolutional layers that need to be trained includes: Deep convolutional layers are used as the convolutional layers to be trained. For these deep convolutional layers, a contrastive learning loss function is constructed to increase the difference in output features of changing samples and decrease the difference in output features of invariant samples. Fine-tuning the deep convolutional layers includes: The gradient of the output feature is calculated using the contrastive learning loss function and the output feature values, and is expressed as: in, , The output features are respectively processed for samples that have changed and samples that have not changed. , These are subnetworks that process the image before and after the change, respectively. Output features gradient, It is a symbolic function, represented as: The gradient of the weights is calculated based on the gradient of the output features, and is expressed as: in, It is a sub-network The Layer weight gradient, This represents the convolution operation. For sub-networks The Layer input; Weights are updated based on weight gradients, expressed as: in, Indicates the number of iterations; The remote sensing images before and after the changes to be inferred are input into the neural network for inference, and the inference result is obtained, which is expressed as: in, For the first The number of channels in a convolutional layer The output coordinates, , To separate the sub-networks for processing the image before and after the transformation, , These represent the state before and after the change, respectively. The output features represent the image before the change. The output features represent the transformed image. , ... For the convolutional layers that need to be trained, This represents the number of convolutional layers in the neural network. For changing images; The binarized result obtained using the Otsu method (maximum inter-class variance) is expressed as follows: 。 2. The method as described in claim 1, characterized in that, The method of using a block-dimensional affinity matrix to select changed and unchanged sub-images from the sub-image set as training samples for the neural network includes: The distance between each sub-image and other sub-images is calculated on the remote sensing images before and after the change, and is expressed as: Among them, superscript for or , , These represent the state before and after the change, respectively. For the first Zhang Zi's image; The probability of sub-image change is measured using an exponential function of distance, where the expression when using the affinity matrix is: in, To normalize the parameters, use all The maximum value represents: ； Summing the probabilities of change for each sub-image and normalizing the sum, we obtain the probability of change for each sub-image, where, for the ... The summation of the probabilities of change in each sub-image is expressed as: in, The number of segments for the remote sensing image; Based on the change probability of each sub-image, several sub-images before and after the change that are most likely to change and several sub-images before and after the change that are least likely to change are selected from the set of sub-images as training samples for the neural network.

3. The method as described in claim 1, characterized in that, The contrastive learning loss function is: in, Representatives are assigned to the first generation that needs to be trained. The loss function of each convolutional layer The output features represent the image before the change. The output features represent the transformed image. This represents the coordinates of the output. , , This represents the width, length, and number of channels of the output feature.

4. A lightweight remote sensing change detection device based on backpropagation-free learning, characterized in that, include: The image segmentation module is used to uniformly segment the remote sensing images before and after the changes to be inferred, and obtain a set of sub-images; The sample selection module is used to select, from the set of sub-images, the changed sub-images before and after the change and the unchanged sub-images before and after the change as training samples for the neural network, using a block-dimensional affinity matrix method. The neural network fine-tuning module is used to fine-tune the neural network using the training samples and a local learning method. During fine-tuning, a loss function is assigned to the convolutional layer that needs to be trained for fine-tuning, while keeping the parameters of other convolutional layers unchanged, so that other convolutional layers perform forward propagation to obtain the fine-tuned neural network. The change detection module is used to input the remote sensing images before and after the change to be inferred into the neural network for inference, and to binarize the inference result to obtain the binary change result; The loss function assigned to the convolutional layers that need to be trained includes: Deep convolutional layers are used as the convolutional layers to be trained. For these deep convolutional layers, a contrastive learning loss function is constructed to increase the difference in output features of changing samples and decrease the difference in output features of invariant samples. Fine-tuning the deep convolutional layers includes: The gradient of the output feature is calculated using the contrastive learning loss function and the output feature values, and is expressed as: in, , The output features are respectively processed for samples that have changed and samples that have not changed. , These are subnetworks that process the image before and after the change, respectively. Output features gradient, It is a symbolic function, represented as: The gradient of the weights is calculated based on the gradient of the output features, and is expressed as: in, It is a sub-network The Layer weight gradient, This represents the convolution operation. For sub-networks The Layer input; Weights are updated based on weight gradients, expressed as: in, Indicates the number of iterations; The remote sensing images before and after the changes to be inferred are input into the neural network for inference, and the inference result is obtained, which is expressed as: in, For the first The number of channels in a convolutional layer The output coordinates, , To separate the sub-networks for processing the image before and after the transformation, , These represent the state before and after the change, respectively. The output features represent the image before the change. The output features represent the transformed image. , ... For the convolutional layers that need to be trained, This represents the number of convolutional layers in the neural network. For changing images; The binarized result obtained using the Otsu method (maximum inter-class variance) is expressed as follows: 。 5. The apparatus as described in claim 4, characterized in that, The method of using a block-dimensional affinity matrix to select changed and unchanged sub-images from the sub-image set as training samples for the neural network includes: The distance between each sub-image and other sub-images is calculated on the remote sensing images before and after the change, and is expressed as: Among them, superscript for or , , These represent the state before and after the change, respectively. For the first Zhang Zi's image; The probability of sub-image change is measured using an exponential function of distance, where the expression when using the affinity matrix is: in, To normalize the parameters, use all The maximum value represents: ； Summing the probabilities of change for each sub-image and normalizing the sum, we obtain the probability of change for each sub-image, where, for the ... The summation of the probabilities of change in each sub-image is expressed as: in, The number of segments for the remote sensing image; Based on the change probability of each sub-image, several sub-images before and after the change that are most likely to change and several sub-images before and after the change that are least likely to change are selected from the set of sub-images as training samples for the neural network.

6. The apparatus as claimed in claim 4, characterized in that, The contrastive learning loss function is: in, Representatives are assigned to the first generation that needs to be trained. The loss function of each convolutional layer The output features represent the image before the change. The output features represent the transformed image. This represents the coordinates of the output. , , This represents the width, length, and number of channels of the output feature.

7. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-3.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-3.

Images