Change detection method and device of remote sensing image, and electronic equipment

Abstract

The application discloses a change detection method and device of remote sensing images and electronic equipment, and relates to the technical field of artificial intelligence, wherein the change detection method comprises the following steps: acquiring remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; performing registration on the first remote sensing image and the second remote sensing image to obtain a first registered image and a second registered image respectively; inputting the first registered image and the second registered image into a preset semantic segmentation model to output a first target image and a second target image; inputting the first target image and the second target image into a preset image detection model to generate a prediction result image; processing the prediction result image to obtain a target detection result image; and determining an image region with changes between the first remote sensing image and the second remote sensing image according to the target detection result image. The application solves the technical problem of low accuracy in detecting changes in large-size remote sensing images in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a remote sensing image change detection method and device and electronic equipment. BACKGROUND

[0002] The rapid change monitoring technology of fine scene focus has important application value in many fields. The technology can quickly and accurately capture the micro-change information of the ground surface, and provides solid data support for environmental monitoring, disaster warning, city planning and land use management.

[0003] In related technologies, the rapid change monitoring method of fine scene focus mainly includes visual analysis method, traditional machine learning method and deep learning method. The visual analysis method relies on manual comparison, which is low in efficiency and difficult to realize large-scale or continuous monitoring. The generalization ability of the model is limited when the traditional machine learning method faces high-dimensional and complex distributed remote sensing data, and the processing efficiency and accuracy cannot meet the requirements of large-area change monitoring. The current deep learning method or mathematical method directly processes the remote sensing image or performs change monitoring after processing by sliding window slicing. However, the remote sensing image has a large size, and direct processing of the remote sensing image or sliding window slicing for remote sensing image processing is prone to large errors or extremely long processing time, which is not suitable for automatic change monitoring of large-size large-area remote sensing images.

[0004] At present, there is no effective solution to the above problems. SUMMARY

[0005] The embodiments of the present application provide a remote sensing image change detection method and device and electronic equipment to at least solve the technical problem of low accuracy in detecting changes in large-size remote sensing images in related technologies.

[0006] According to an aspect of an embodiment of the present application, a remote sensing image change detection method is provided, including: obtaining remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image; inputting the first registered image and the second registered image into a preset semantic segmentation model to output a first target image and a second target image; inputting the first target image and the second target image into a preset image detection model to generate a prediction result image; processing the prediction result image to obtain a target detection result image, and determining an image region with changes between the first remote sensing image and the second remote sensing image according to the target detection result image.

[0007] Further, the step of registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image comprises: using a preset operator to calculate a preset image to generate gradient information of each pixel point on the preset image, and performing convolution calculation on the gradient information to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information at least includes a gradient amplitude and a direction angle; performing Laplace calculation on the target gradient information of each pixel point on the preset image to generate a Laplace response value of each pixel point, and performing peak detection on the Laplace response value of each pixel point and the Laplace response value of all pixel points in the neighborhood of each pixel point to determine a peak feature point of the preset image; generating a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixel points in the neighborhood of the peak feature point, and determining a target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information at least includes position information and appearance feature information; registering the preset image based on the target feature point to obtain a preset registered image, wherein the preset registered image is the first registered image of the first remote sensing image or the second registered image of the second remote sensing image.

[0008] Further, the step of registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image comprises: using a preset operator to calculate a preset image to generate gradient information of each pixel point on the preset image, and performing convolution calculation on the gradient information to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information at least includes a gradient amplitude and a direction angle; performing Laplace calculation on the target gradient information of each pixel point on the preset image to generate a Laplace response value of each pixel point, and performing peak detection on the Laplace response value of each pixel point and the Laplace response value of all pixel points in the neighborhood of each pixel point to determine a peak feature point of the preset image; generating a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixel points in the neighborhood of the peak feature point, and determining a target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information at least includes position information and appearance feature information; registering the preset image based on the target feature point to obtain a preset registered image, wherein the preset registered image is the first registered image of the first remote sensing image or the second registered image of the second remote sensing image.

[0009] Further, before inputting the first registered image and the second registered image into a preset semantic segmentation model respectively to output a first target image and a second target image, the method further comprises: constructing a preset attention structure and a skip connection structure; and constructing the preset semantic segmentation model based on the preset attention structure and the skip connection structure, wherein the preset semantic segmentation model further comprises a Poisson encoder.

[0010] Further, the step of constructing the preset attention structure comprises: constructing a channel attention module, wherein the channel attention module at least comprises a pooling layer, a full connection layer and an intermediate output layer, the pooling layer at least comprises a maximum pooling and an average pooling, and the maximum pooling and the average pooling are in a parallel relationship; constructing a spatial attention module, wherein the spatial attention module at least comprises the pooling layer and an output layer, the pooling layer at least comprises the maximum pooling and the average pooling, and the maximum pooling and the average pooling are in a serial relationship; and constructing the preset attention structure based on the channel attention module and the spatial attention module.

[0011] Further, the step of inputting the first registered image and the second registered image into the preset semantic segmentation model respectively to output the first target image and the second target image comprises: pre-processing the first registered image and the second registered image respectively by using the Poisson encoder to obtain a first processed image and a second processed image; encoding the first processed image and the second processed image respectively to generate a first pulse sequence and a second pulse sequence, and inputting the first pulse sequence and the second pulse sequence into the preset semantic segmentation model respectively to output the first target image and the second target image.

[0012] Further, before inputting the first target image and the second target image into a preset image detection model to generate a prediction result image, the method further comprises: constructing the preset image detection model, wherein the preset image detection model at least comprises a backbone network, an encoder, a decoder and a prediction head, the backbone network is used to extract features of the target image, the encoder is used to obtain spatial relationships between the features, the decoder is used to map the spatial relationships to a pixel space to generate a feature map, and the prediction head is used to generate the prediction result image according to the feature map.

[0013] Further, the first target image and the second target image are input to a preset image detection model to generate a prediction result image, and the step includes: respectively pre-processing the first target image and the second target image to obtain a first target processing image and a second target processing image; respectively performing convolution on the first target processing image and the second target processing image by using the backbone network to generate a feature map of the first target processing image and a feature map of the second target processing image; respectively converting the feature map of the first target processing image and the feature map of the second target processing image to obtain a first semantic feature vector and a second semantic feature vector, and splicing the first semantic feature vector and the second semantic feature vector to obtain a spliced feature vector sequence; inputting the spliced feature vector sequence to the encoder to obtain a spatial relationship between each semantic feature vector in the spliced feature vector sequence; inputting the spatial relationship to the decoder to generate an initial feature map, and inputting the initial feature map to a preset convolutional neural network to generate a target feature map; inputting the target feature map to the prediction head to generate the prediction result image.

[0014] According to another aspect of the embodiment of the present application, a remote sensing image change detection device is also provided, which comprises: an acquisition unit configured to acquire remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; a registration unit configured to register the first remote sensing image and the second remote sensing image to obtain a first registration image of the first remote sensing image and a second registration image of the second remote sensing image; an input unit configured to input the first registration image and the second registration image into a preset semantic segmentation model to output a first target image and a second target image; a generation unit configured to input the first target image and the second target image into a preset image detection model to generate a prediction result image; and a determination unit configured to process the prediction result image to obtain a target detection result image, and determine an image region with a change between the first remote sensing image and the second remote sensing image according to the target detection result image.

[0015] Further, the registration unit comprises: a first calculation module, configured to calculate a preset image by using a preset operator to generate gradient information of each pixel point on the preset image, and perform convolution calculation on the gradient information to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information at least includes a gradient amplitude and a direction angle; a second calculation module, configured to perform Laplace calculation on the target gradient information of each pixel point on the preset image to generate a Laplace response value of each pixel point, and perform peak value detection on the Laplace response value of each pixel point and the Laplace response values of all pixel points in a neighborhood of each pixel point to determine a peak feature point of the preset image; a first generation module, configured to generate a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixel points in a neighborhood of the peak feature point, and determine a target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information at least includes position information and appearance feature information; and a first registration module, configured to register the preset image based on the target feature point to obtain a preset registration image, wherein the preset registration image is the first registration image of the first remote sensing image or the second registration image of the second remote sensing image.

[0016] Further, the first registration module comprises: a first fusion sub-module, configured to fuse the position information and the appearance feature information of each target feature point to obtain an initial feature vector; a first calculation sub-module, configured to calculate each initial feature vector by using a preset attention layer to obtain each target feature vector, wherein each target feature vector is the target feature vector corresponding to the first remote sensing image or the target feature vector corresponding to the second remote sensing image; a first construction sub-module, configured to construct a score matrix based on each target feature vector, expand the score matrix to obtain an expanded matrix, and iterate the expanded matrix until an iteration number reaches a preset iteration threshold to obtain a target score matrix; and a first registration sub-module, configured to determine a pair point set of the target feature point based on the target score matrix, and register the first remote sensing image and the second remote sensing image based on the pair point set to obtain the first registration image of the first remote sensing image and the second registration image of the second remote sensing image.

[0017] Further, the change detection apparatus comprises: a first construction module, configured to construct a preset attention structure and a skip connection structure before inputting the first registered image and the second registered image into a preset semantic segmentation model respectively and outputting a first target image and a second target image; and a second construction module, configured to construct the preset semantic segmentation model based on the preset attention structure and the skip connection structure, wherein the preset semantic segmentation model further comprises a Poisson encoder.

[0018] Further, the first construction module comprises: a first construction submodule, configured to construct a channel attention module, wherein the channel attention module at least comprises a pooling layer, a full connection layer and an intermediate output layer, the pooling layer at least comprises a max-pooling and an average-pooling, and the max-pooling and the average-pooling are in a parallel relationship; a second construction submodule, configured to construct a spatial attention module, wherein the spatial attention module at least comprises the pooling layer and an output layer, the pooling layer at least comprises the max-pooling and the average-pooling, and the max-pooling and the average-pooling are in a serial relationship; and a third construction submodule, configured to construct the preset attention structure based on the channel attention module and the spatial attention module.

[0019] Further, the input unit comprises: a first processing module, configured to pre-process the first registered image and the second registered image respectively by using the Poisson encoder to obtain a first processed image and a second processed image; and a first encoding module, configured to encode the first processed image and the second processed image respectively to generate a first pulse sequence and a second pulse sequence, and input the first pulse sequence and the second pulse sequence into the preset semantic segmentation model to output the first target image and the second target image.

[0020] Further, the change detection apparatus comprises: a third construction module, configured to construct a preset image detection model before inputting the first target image and the second target image into the preset image detection model to generate a prediction result image, wherein the preset image detection model at least comprises a backbone network, an encoder, a decoder and a prediction head, the backbone network is configured to extract features of the target image, the encoder is configured to obtain spatial relationships between the features, the decoder is configured to map the spatial relationships to a pixel space to generate a feature map, and the prediction head is configured to generate the prediction result image according to the feature map.

[0021] Further, the generation unit comprises: a second processing module, configured to respectively pre-process the first target image and the second target image to obtain a first target processing image and a second target processing image; a first convolution module, configured to respectively perform convolution on the first target processing image and the second target processing image by using the backbone network to generate a feature map of the first target processing image and a feature map of the second target processing image; a first conversion module, configured to respectively convert the feature map of the first target processing image and the feature map of the second target processing image to obtain a first semantic feature vector and a second semantic feature vector, and splice the first semantic feature vector and the second semantic feature vector to obtain a spliced feature vector sequence; a first input module, configured to input the spliced feature vector sequence to the encoder to obtain a spatial relationship between each semantic feature vector in the spliced feature vector sequence; a second input module, configured to input the spatial relationship to the decoder to generate an initial feature map, and input the initial feature map to a preset convolutional neural network to generate a target feature map; and a third input module, configured to input the target feature map to the prediction head to generate the prediction result map.

[0022] According to another aspect of the embodiments of the present application, a computer program product is also provided, comprising a non-volatile computer readable storage medium storing a computer program, which, when executed by a processor, implements the change detection method of the remote sensing image.

[0023] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising one or more processors and a memory, the memory being configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the change detection method of the remote sensing image.

[0024] In the present application, remote sensing images at different time points are acquired to obtain a first remote sensing image and a second remote sensing image, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image, the first registered image and the second registered image are respectively input to a preset semantic segmentation model to output a first target image and a second target image, the first target image and the second target image are input to a preset image detection model to generate a prediction result map, the prediction result map is processed to obtain a target detection result map, and the target detection result map is used to determine an image region with changes between the first remote sensing image and the second remote sensing image, thereby solving the technical problem of low accuracy in detecting changes in large-size remote sensing images in the related art.

[0025] In the present application, by acquiring remote sensing images at two time points, the acquired images can be pairwise registered to ensure the spatial consistency of the two images, thereby obtaining a first registered image and a second registered image, and then the registered images are respectively input into a preset semantic segmentation model to accurately extract a specific region, according to which a first target image and a second target image can be obtained, and then the two target images are input into a preset image detection model to generate a prediction result image, which is subsequently processed to obtain a clear target detection result image, accurately marking the change region, which can effectively improve the accuracy and speed of change detection, reduce the interference of non-concerned regions, realize the rapid positioning and monitoring of fine scene changes in large-area remote sensing images, and thus realize the technical effect of improving the accuracy of detecting large-size remote sensing image changes. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0027] Figure 1 is a flowchart of an optional remote sensing image change detection method according to an embodiment of the present application;

[0028] Figure 2 is a schematic diagram of an optional remote sensing image change detection process according to an embodiment of the present application;

[0029] Figure 3 is a schematic diagram of an optional pairing result according to an embodiment of the present application;

[0030] Figure 4 is a structural schematic diagram of an optional preset semantic segmentation model according to an embodiment of the present application;

[0031] Figure 5 is a schematic diagram of an optional preset attention structure according to an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of an optional remote sensing mask image of an airport and port area according to an embodiment of the present application;

[0033] Figure 7 is a schematic diagram of an optional preset image detection model processing process according to an embodiment of the present application;

[0034] Figure 8 is a schematic diagram of an optional remote sensing image change detection device according to an embodiment of the present application;

[0035] Figure 9It is a hardware structure block diagram of an electronic device (or a mobile device) for a change detection method of a remote sensing image according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0038] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) collected and related to the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards in relevant regions, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user selection authorization or refusal. For example, the system and the interface between the related users or institutions are provided, and before obtaining the relevant information, the interface needs to send an acquisition request to the aforementioned user or institution, and after receiving the consent information feedback from the aforementioned user or institution, the relevant information is acquired.

[0039] In the present application, first, the multi-temporal remote sensing image is quickly registered, then the two-stage fine scene change monitoring process of semantic segmentation and change monitoring is constructed, the image mask is obtained through semantic segmentation, so as to narrow the attention area and locate the key area (such as airport and port area), then the image change monitoring is carried out, which not only improves the accuracy of change detection, but also reduces the processing time and resource consumption.

[0040] The present application will be described in detail below in conjunction with various embodiments.

[0041] Embodiment one

[0042] According to an embodiment of the present application, an embodiment of a change detection method of remote sensing images is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0043] Figure 1 is a flowchart of an optional change detection method of remote sensing images according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0044] Step S101, acquiring remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image.

[0045] Optionally, a SAR (Synthetic Aperture Radar) remote sensing image is a kind of remote sensing data that uses radar waves to image ground targets, has all-weather, all-day imaging capability, can penetrate clouds and vegetation, and can obtain accurate information of the ground. Therefore, multi-temporal SAR remote sensing images can be collected.

[0046] In this embodiment, SAR remote sensing images at two different time points are acquired to obtain a first remote sensing image and a second remote sensing image.

[0047] Step S102, registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0048] Optionally, registration is a process of placing remote sensing images at two different time points in the same geographic coordinate system, which ensures that the two images can be accurately compared by eliminating geometric differences caused by satellite attitude, imaging angle or atmospheric conditions, etc.

[0049] In this embodiment, a control point extraction based on SAR-Harris (i.e. corner detection algorithm) space and a control point matching method based on deep learning algorithm can be used for fast registration. Accurate extraction and matching of control points are the key to registration. Through this process, the first registered image of the first remote sensing image and the second registered image of the second remote sensing image that are aligned in spatial coordinates can be obtained, laying a foundation for subsequent change detection.

[0050] Step S103: input the first registration image and the second registration image into a preset semantic segmentation model respectively, and output a first target image and a second target image.

[0051] Optionally, a spiking neural network (SNN) can simulate the working mechanism of biological neurons, and information is transmitted through pulses. This asynchronous and event-driven characteristic makes the SNN more efficient in processing time series data, and can capture changes in the time axis of data. The SNN has a certain robustness to noise (such as coherent spot noise and edge blur of a SAR image), can suppress noise in the SAR image, and improves the performance of SAR image processing. Therefore, the spiking neural network can be used for semantic segmentation of the registration image.

[0052] In this embodiment, by inputting the first registration image and the second registration image into a preset semantic segmentation model (such as an SNN-UNet (Spiking Neural Network-UNet, i.e., a convolutional neural network architecture) network optimized based on a spiking neural network), a mask image of the first registration image and a mask image of the second registration image can be obtained respectively. Then, the first remote sensing image and the second remote sensing image can be cropped according to the mask image of the first registration image and the mask image of the second registration image, to obtain a first target image and a second target image of a specific region (such as an airport, a port, etc.), thereby reducing the range of subsequent change monitoring.

[0053] Step S104: input the first target image and the second target image into a preset image detection model to generate a prediction result map.

[0054] Optionally, a network model based on a BIT (Bi-temporal Image Transformer, i.e., a deep learning model) structure can be used for processing a remote sensing image change detection task. The network combines the powerful feature extraction capability of a convolutional neural network and the context modeling advantage of a Transformer (i.e., a neural network architecture), to achieve efficient identification of high-resolution remote sensing image changes.

[0055] In this embodiment, by inputting the first target image and the second target image into a preset image detection model (such as a network model based on a BIT structure), a prediction result map can be generated, which is a preliminary prediction of the possible change region between the first target image and the second target image.

[0056] Step S105: process the prediction result map to obtain a target detection result map, and determine an image region with changes between the first remote sensing image and the second remote sensing image according to the target detection result map.

[0057] In this embodiment, the prediction result image is post-processed (including applying thresholding, filtering, morphological operations, etc.) to remove noise in the prediction result image, smooth the change area, and generate a final target detection result image (i.e., a binary image). If there is a change between the first remote sensing image and the second remote sensing image, the changed area can be marked as white and the unchanged area can be marked as black in the target detection result image.

[0058] Figure 2 is a schematic diagram of an optional remote sensing image change detection process according to an embodiment of the present invention, such as Figure 2 As shown, SAR remote sensing images of multi-temporal large-area scenes are first collected, and then SAR image registration is performed on the collected SAR remote sensing images. The registered images are then input into a pulse neural network (i.e., a preset semantic segmentation model) to obtain an airport port area mask. Based on the airport port area mask, the airport port area is located to obtain a first target image and a second target image. The first target image and the second target image are then input into a preset image detection model to generate a prediction result map, thereby realizing change detection.

[0059] In summary, by acquiring remote sensing images at different time points and performing pairwise registration on the acquired remote sensing images (i.e., registering the first remote sensing image and the second remote sensing image), a first registered image of the first remote sensing image and a second registered image of the second remote sensing image can be obtained, and then the first registered image and the second registered image are respectively input into the preset semantic segmentation model, and the first target image and the second target image can be output, and the first target image and the second target image can be input into the preset image detection model to generate a prediction result graph, and then the prediction result graph is processed to obtain a target detection result graph, and based on the target detection result graph, the image area where there is a change between the first remote sensing image and the second remote sensing image can be determined, thereby solving the technical problem of low accuracy in detecting changes in large-size remote sensing images in related technologies.

[0060] In order to accurately obtain the preset registration image, in the change detection method of remote sensing images provided in Embodiment One of the present application, a preset operator is used to calculate the preset image, to generate gradient information of each pixel point on the preset image, and to perform convolution calculation on the gradient information, to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information at least includes gradient amplitude and direction angle; Laplace calculation is performed on the target gradient information of each pixel point on the preset image, to generate a Laplace response value of each pixel point, and peak value detection is performed on the Laplace response value of each pixel point and the Laplace response values of all pixel points in the neighborhood of each pixel point, to determine a peak feature point of the preset image; based on the target gradient information of the peak feature point and the target gradient information of all pixel points in the neighborhood of the peak feature point, a gradient statistical histogram is generated, and based on the gradient statistical histogram, a target feature point of the preset image is determined, wherein the target feature point corresponds to target feature point information, and the target feature point information at least includes position information and appearance feature information; based on the target feature point, registration is performed on the preset image, to obtain a preset registration image, wherein the preset registration image is a first registration image of the first remote sensing image or a second registration image of the second remote sensing image.

[0061] Optionally, a preset operator (such as ROEWA (Ratio of Edge Weighted Averages, i.e., an operator for edge detection and feature extraction)) can be used to calculate the preset image (i.e., the first remote sensing image or the second remote sensing image), to generate gradient information of each pixel point on the preset image. Through the ROEWA operator, the problem of coherent speckle noise existing in the SAR image can be overcome, and the edge information of the image can be accurately extracted.

[0062] In the present embodiment, in order to further enhance the effect of feature extraction, convolution calculation can be performed on the gradient information (such as using a Gaussian function to perform convolution calculation), to generate target gradient information (including gradient amplitude and direction angle) of each pixel point on the preset image, and Laplace calculation can be performed on the target gradient information of each pixel point on the preset image, to generate a Laplace response value of each pixel point. The Laplace response value can be subjected to peak value detection with the Laplace response values of all pixel points in the neighborhood (i.e., the surrounding area) of each pixel point, to determine a peak feature point of the preset image.

[0063] In the embodiment, according to the target gradient information of the peak feature point and the target gradient information of all pixel points in the neighborhood of the peak feature point, a gradient statistical histogram (i.e. a visual gradient amplitude and direction angle distribution diagram) can be generated, which is helpful to understand the statistical characteristics of the image features. By using the gradient statistical histogram, the target feature point of the preset image can be further screened and determined. The target feature point not only has clear position information, but also contains rich appearance feature information.

[0064] In the embodiment, the image registration is a process of spatial alignment of remote sensing images from different times, which ensures the consistency of the two images in space. The target feature point can be used to register the preset image to obtain a preset registration image (i.e. a first registration image of the first remote sensing image or a second registration image of the second remote sensing image).

[0065] In order to improve the accuracy of the first registration image of the first remote sensing image and the second registration image of the second remote sensing image, in the change detection method of the remote sensing image provided in Embodiment One of the present application, for each target feature point, the position information and the appearance feature information of the target feature point are fused to obtain an initial feature vector. A preset attention layer is used to calculate each initial feature vector to obtain each target feature vector, wherein each target feature vector is a target feature vector corresponding to the first remote sensing image or a target feature vector corresponding to the second remote sensing image. Based on each target feature vector, a score matrix is constructed, the score matrix is expanded to obtain an expanded matrix, and the expanded matrix is iterated until the number of iterations reaches a preset iteration threshold to obtain a target score matrix. Based on the target score matrix, a pair point set of the target feature point is determined, and based on the pair point set, the first remote sensing image and the second remote sensing image are registered to obtain the first registration image of the first remote sensing image and the second registration image of the second remote sensing image.

[0066] Optionally, in order to uniformly represent the geometric position and visual feature of the feature point, a key point encoder can be used to map the position information and appearance feature information of the target feature point to generate an initial feature vector.

[0067] In the embodiment, a preset attention layer (such as a self-attention layer and a cross-attention layer repeated alternately for L times) is used to calculate each initial feature vector, so that each target feature vector (i.e. a target feature vector corresponding to the first remote sensing image or a target feature vector corresponding to the second remote sensing image) can be obtained, and a score matrix (such as a score matrix of M by N) can be constructed, where M is the number of feature points of the first remote sensing image and N is the number of feature points of the second remote sensing image.

[0068] In this embodiment, incorrect matching may occur due to reasons such as scene changes, image quality differences, occlusion, and perspective differences. Therefore, the score matrix can be expanded (such as adding virtual matching options) to obtain an expanded matrix. In the expanded matrix, each feature point has not only a score for matching with other feature points, but also a score for matching with the virtual matching option. The score for matching with the virtual matching option indicates the probability that the feature point does not match any other actual feature point. If a feature point does not find a suitable matching object in reality, then its matching score with the virtual matching option will be relatively high, and thus it will be marked as unpaired or unmatchable in the final matching decision, thereby more accurately determining the optimal feature point pairing set.

[0069] In this embodiment, the expansion matrix is ​​iterated until the number of iterations reaches a preset iteration threshold (such as T times), and a target score matrix (i.e., an optimal feature point matching matrix) can be obtained. Based on the target score matrix, a pairing point set of the target feature points can be determined, and based on the pairing point set, the first remote sensing image and the second remote sensing image are aligned to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0070] Figure 3 is an optional pairing result diagram according to an embodiment of the present invention, such as Figure 3 As shown in the figure, through image registration, two ASR remote sensing images collected at different time points ( Figure 3 (a) and (b) in Fig. 3 are spatially consistent, that is, any point on the image has the same spatial position in the two ASR images.

[0071] In order to accurately construct a preset semantic segmentation model, in the change detection method of remote sensing images provided in Example 1 of the present application, a preset attention structure and a jump connection structure are constructed; based on the preset attention structure and the jump connection structure, a preset semantic segmentation model is constructed, wherein the preset semantic segmentation model also includes: a Poisson encoder.

[0072] In this embodiment, the preset attention structure (such as CBAM (Convolutional Block Attention Module)) may include a two-dimensional convolution operation, a channel attention mechanism, a spatial attention mechanism, a batch normalization process, and a nonlinear activation function. By introducing the channel attention mechanism and the spatial attention mechanism, the sensitivity and selectivity of the network to image features can be enhanced. The channel attention mechanism can learn the importance of each channel in the feature map, while the spatial attention mechanism can identify the key areas in the image. The combination of the dual attention mechanisms enables the network to focus more on important features while suppressing irrelevant background information.

[0073] In the embodiment, the four times of upsampling and the construction of the skip connection structure can enhance the fusion of low-dimensional features of the feature map, so that the segmentation result is more fine in details such as edge information, and according to the preset attention structure and the skip connection structure, a preset semantic segmentation model can be constructed, the preset semantic segmentation model can further include a Poisson encoder for encoding processing of an output image, and a pulse neuron can be introduced in the preset semantic segmentation model to replace a traditional activation function, so as to significantly improve the accuracy of the segmentation task.

[0074] In the embodiment, in the output stage of the preset semantic segmentation model, the response intensity of different categories can be represented by calculating the pulse firing frequency of the output layer neurons during simulation, and in an ideal case, the correct neurons should fire pulses at the highest frequency, and other neurons remain silent, for this purpose, a cross-entropy loss function based on the pulse firing frequency and the real pixel category can be constructed, and the network state can be reset after each parameter optimization to eliminate the memory effect of the neurons of the preset semantic segmentation model.

[0075] Figure 4 is a structural schematic diagram of an optional preset semantic segmentation model according to an embodiment of the application, as shown in Figure 4 The preset semantic segmentation model includes an input, a CBAM module, a cross-layer connection, and an output, the input is used to input an image into the model, the CBAM module enables the model to pay more attention to key features and regions in the image, and four times of upsampling and the cross-layer connection (i.e., the skip connection structure) are used to enhance the fusion of low-dimensional features of the feature map, so that the segmentation result is more fine in details such as edge information, and the output is used to output the final target image (i.e., the first target image and the second target image).

[0076] In order to improve the accuracy of constructing the preset attention structure, in the remote sensing image change detection method provided in Embodiment One of the application, a channel attention module is constructed, wherein the channel attention module at least includes: a pooling layer, a full connection layer, and an intermediate output layer, the pooling layer at least includes: maximum pooling and average pooling, and the maximum pooling and the average pooling are in a parallel relationship; a spatial attention module is constructed, wherein the spatial attention module at least includes: a pooling layer and an output layer, the pooling layer at least includes: maximum pooling and average pooling, and the maximum pooling and the average pooling are in a serial relationship; and based on the channel attention module and the spatial attention module, a preset attention structure is constructed.

[0077] Optionally, in the channel attention module, in addition to the average pooling, the maximum pooling operation can also be introduced to compress the spatial dimension. In this way, the average and maximum value of the feature map can be pooled in the global range, effectively extracting and aggregating the spatial information. Then, the feature map is processed by a fully connected multilayer perceptron to realize the element-wise summation and merging of the feature map, and the channel attention map can be generated. The channel attention map is multiplied by the original input feature map to realize adaptive optimization of the features. The channel attention mechanism not only reflects the feedback of the average pooling to each pixel point, but also embodies the response of the maximum pooling to the significant feature points in the backpropagation process.

[0078] In the embodiment, the spatial attention module focuses on the depth dimension of the image and aims to identify the key areas in the image. The average pooling and the maximum pooling can be performed on the channel dimension to obtain two single-channel feature maps, and then the two single-channel feature maps are spliced to form an output feature map.

[0079] Figure 5 is a schematic diagram of an optional preset attention structure according to an embodiment of the application, as shown in Figure 5 The upper half of the diagram shows the preset attention structure, including a two-dimensional convolution, a channel attention, a spatial attention, a batch normalization layer and an activation layer. The lower half of the diagram shows the specific structure of the channel attention and the spatial attention. The channel attention processes the conversion from the input feature map to the intermediate output, and the spatial attention processes the mapping from the intermediate output to the final output feature map. The two are cascaded in a serial manner. In the channel attention part, in addition to the average pooling, the maximum pooling is also introduced to compress the spatial dimension. In this way, the average and maximum value of the feature map can be pooled in the global range, effectively extracting and aggregating the spatial information. Then, the element-wise summation and merging of the feature map are realized by a fully connected multilayer perceptron to generate a channel attention map. The attention map is multiplied by the original input feature map to realize adaptive optimization of the features. The channel attention not only reflects the feedback of the average pooling to each pixel point, but also embodies the response of the maximum pooling to the significant feature points in the backpropagation process. The spatial attention part performs the maximum pooling and the average pooling on the channel dimension to obtain two single-channel feature maps, and then splices the two feature maps to form an output feature map.

[0080] In order to accurately obtain the first target image and the second target image, in the remote sensing image change detection method provided in Embodiment One of the present application, a Poisson encoder is used to respectively pre-process the first registered image and the second registered image to obtain a first processed image and a second processed image; the first processed image and the second processed image are respectively encoded to generate a first pulse sequence and a second pulse sequence, and the first pulse sequence and the second pulse sequence are respectively input into a preset semantic segmentation model to output the first target image and the second target image.

[0081] Optionally, the Poisson encoder can pre-process the image, uniformly adjust the image size to 572x572 pixels, and then encode into a pulse sequence to input into the preset semantic segmentation model.

[0082] In the present embodiment, the first registered image and the second registered image are respectively pre-processed by the Poisson encoder to obtain the first processed image and the second processed image, and the first processed image and the second processed image are encoded to generate the first pulse sequence and the second pulse sequence (i.e. each pixel point in the image can emit a pulse according to its gray value with a certain probability), which provides a suitable input format for the preset semantic segmentation model. The first pulse sequence and the second pulse sequence generated are respectively input into the preset semantic segmentation model to obtain the mask image of the remote sensing image of a specific region (such as the mask image of the airport and port area), and then the remote sensing image can be cropped according to the region range where the mask image is located to output the first target image and the second target image.

[0083] It should be noted that the first layer and the layers before it of the preset semantic segmentation model do not change with time when receiving the input image. In order to improve the calculation efficiency, the first layer neurons in the preset semantic segmentation model can be specially processed to be outside the time cycle and do not need to accumulate information over time.

[0084] Figure 6 is a schematic diagram of an optional remote sensing mask image of an airport and port area according to an embodiment of the present application, as shown in Figure 6 The rectangular box is a cropping region set based on the mask image, and the first target image and the second target image can be obtained by cropping the remote sensing image according to the region.

[0085] In order to accurately generate the prediction result image, in the remote sensing image change detection method provided in Embodiment One of the present application, a preset image detection model is constructed, wherein the preset image detection model at least includes: a backbone network, an encoder, a decoder, and a prediction head. The backbone network is used to extract the features of the target image, the encoder is used to obtain the spatial relationship between the features, the decoder is used to map the spatial relationship to the pixel space to generate a feature map, and the prediction head is used to generate the prediction result image according to the feature map.

[0086] Optionally, the overall architecture of the preset image detection model can comprise a backbone network, a BIT module (including an encoder, a decoder) and a prediction head.

[0087] In this embodiment, the backbone network is responsible for extracting high-level semantic features from the input image pair, and then the high-level semantic features are sent to the BIT module, which converts the feature mapping into a compact semantic feature vector through a spatial attention mechanism, and then uses the encoder to model the context of the semantic feature vector, obtain the spatial relationship between the features, and input the spatial relationship into the decoder, which can map the spatial relationship to the pixel space to generate a feature map, and then input the feature map into the prediction head to generate a prediction result map.

[0088] In order to accurately generate a prediction result map, in the change detection method for remote sensing images provided in Embodiment One of the present application, the first target image and the second target image are respectively preprocessed to obtain a first target processed image and a second target processed image; the first target processed image and the second target processed image are respectively convolved using a backbone network to generate a feature map of the first target processed image and a feature map of the second target processed image; the feature map of the first target processed image and the feature map of the second target processed image are respectively converted to obtain a first semantic feature vector and a second semantic feature vector, and the first semantic feature vector and the second semantic feature vector are spliced to obtain a spliced feature vector sequence; the spliced feature vector sequence is input into an encoder to obtain the spatial relationship between each semantic feature vector in the spliced feature vector sequence; the spatial relationship is input into a decoder to generate an initial feature map, and the initial feature map is input into a preset convolutional neural network to generate a target feature map; the target feature map is input into a prediction head to generate a prediction result map.

[0089] Optionally, the first target image and the second target image are respectively preprocessed (such as scaling, cropping and normalization) to obtain a first target processed image and a second target processed image to meet the input requirements of the preset image detection model.

[0090] In this embodiment, the first target processed image and the second target processed image are respectively convolved using a backbone network (such as an improved ResNet (Residual Network, a kind of convolutional neural network architecture) network for extracting deep features in images) to generate a feature map of the first target processed image and a feature map of the second target processed image, and the feature map of the first target processed image and the feature map of the second target processed image can be respectively converted to obtain a first semantic feature vector and a second semantic feature vector, and the first semantic feature vector and the second semantic feature vector can be spliced to obtain a spliced feature vector sequence.

[0091] In this embodiment, the spliced feature vector sequence is input into the encoder, the spatial relationship between each semantic feature vector in the spliced feature vector sequence can be obtained, and then the spatial relationship is input into the decoder. The decoder can map the context-rich semantic feature vector back to the pixel space to generate a refined feature map (i.e., an initial feature map can be generated), which enhances the semantic information of the original feature map.

[0092] In this embodiment, the initial feature map can be input into a preset convolutional neural network (i.e., a shallow CNN (Convolutional Neural Network)). The preset convolutional neural network can generate a change prediction map (i.e., a target feature map) through a series of convolutional layers and up-sampling operations, and can generate a final pixel-level change classification map (i.e., a prediction result map) through an activation function layer (such as a softmax layer or a sigmoid layer).

[0093] Figure 7 is a schematic diagram of an optional preset image detection model processing flow according to an embodiment of the present application, as shown in Figure 7 two cropped remote sensing images (i.e., a first target image and a second target image) are input, high-level semantic features are extracted from the two input images using a backbone network to generate a feature map, and the feature map of the first target processing image and the feature map of the second target processing image can be converted respectively to obtain two semantic tokens (i.e., a first semantic feature vector and a second semantic feature vector), and the first semantic feature vector and the second semantic feature vector can be spliced to obtain a spliced feature vector sequence. The spliced feature vector sequence is input into a Transformer encoder, the spatial relationship between each semantic feature vector in the spliced feature vector sequence can be obtained, and then the spatial relationship is input into two Transformer decoders to generate an initial feature map. The initial feature map can be input into a shallow CNN to generate a change prediction map, and the change prediction map can be post-processed to obtain a final change detection result map. The white area in the change detection result map is a changed area, and the black area is an unchanged area.

[0094] In the embodiment of the present application, by fusing the two-stage fine scene focusing change detection process of semantic segmentation and change monitoring, the change in the SAR remote sensing image can be efficiently and accurately monitored. First, the UNet architecture optimized by the pulse neural network is introduced to obtain a preset semantic segmentation model, which effectively processes the SAR image, improves the accuracy of semantic segmentation, ensures the accurate definition of the attention area, and further optimizes the preset semantic segmentation model by using the preset attention structure, which significantly improves the performance of edge detection and makes the segmentation result more delicate. Then, by using the preset image detection model, the change detection of the segmented key area can accurately capture the differences between two images, reduce the interference of non-attention areas, and significantly improve the efficiency and accuracy of change detection.

[0095] The following will be described in detail in combination with another embodiment.

[0096] Embodiment two

[0097] The change detection device for remote sensing images provided in the embodiment comprises a plurality of implementation units, each of which corresponds to each implementation step in the above embodiment one.

[0098] Figure 8 is a schematic diagram of an optional change detection device for remote sensing images according to the embodiment of the present application, as shown in the figure, the change detection device for remote sensing images can comprise: an acquisition unit 80, a registration unit 81, an input unit 82, a generation unit 83, and a determination unit 84. Figure 8

[0099] The acquisition unit 80 is configured to acquire remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image.

[0100] The registration unit 81 is configured to register the first remote sensing image and the second remote sensing image to obtain a first registration image of the first remote sensing image and a second registration image of the second remote sensing image.

[0101] The input unit 82 is configured to input the first registration image and the second registration image into a preset semantic segmentation model respectively to output a first target image and a second target image.

[0102] The generation unit 83 is configured to input the first target image and the second target image into a preset image detection model to generate a prediction result image.

[0103] The determination unit 84 is configured to process the prediction result image to obtain a target detection result image, and determine an image area with changes between the first remote sensing image and the second remote sensing image according to the target detection result image.

[0104] ​The change detection device for remote sensing images can obtain remote sensing images at different time points through the obtaining unit 80 to obtain a first remote sensing image and a second remote sensing image, perform registration on the first remote sensing image and the second remote sensing image through the registration unit 81 to obtain a first registration image of the first remote sensing image and a second registration image of the second remote sensing image, input the first registration image and the second registration image into a preset semantic segmentation model through the input unit 82 respectively, output a first target image and a second target image, input the first target image and the second target image into a preset image detection model through the generating unit 83 to generate a prediction result image, process the prediction result image through the determining unit 84 to obtain a target detection result image, and determine an image region with changes between the first remote sensing image and the second remote sensing image according to the target detection result image.

[0105] Optionally, the registration unit comprises: a first calculation module configured to calculate a preset image by using a preset operator to generate gradient information of each pixel point on the preset image, and perform convolution calculation on the gradient information to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information at least comprises a gradient amplitude and a direction angle; a second calculation module configured to perform Laplace calculation on the target gradient information of each pixel point on the preset image to generate a Laplace response value of each pixel point, and perform peak value detection on the Laplace response value of each pixel point and the Laplace response values of all pixel points in a neighborhood of each pixel point to determine a peak feature point of the preset image; a first generation module configured to generate a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixel points in a neighborhood of the peak feature point, and determine a target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information at least comprises position information and appearance feature information; and a first registration module configured to perform registration on the preset image based on the target feature point to obtain a preset registration image, wherein the preset registration image is the first registration image of the first remote sensing image or the second registration image of the second remote sensing image.

[0106] Optionally, the first registration module comprises: a first fusion submodule configured to fuse the target feature point position information and the appearance feature information to obtain an initial feature vector for each target feature point; a first calculation submodule configured to calculate each initial feature vector using a preset attention layer to obtain each target feature vector, wherein each target feature vector is a target feature vector corresponding to the first remote sensing image or a target feature vector corresponding to the second remote sensing image; a first construction submodule configured to construct a score matrix based on each target feature vector, expand the score matrix to obtain an expanded matrix, and iterate the expanded matrix until the number of iterations reaches a preset iteration threshold to obtain a target score matrix; and a first registration submodule configured to determine a paired point set of the target feature points based on the target score matrix, and register the first remote sensing image and the second remote sensing image based on the paired point set to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0107] Optionally, the change detection device comprises: a first construction module configured to construct a preset attention structure and a skip connection structure before inputting the first registered image and the second registered image into the preset semantic segmentation model to output the first target image and the second target image; and a second construction module configured to construct the preset semantic segmentation model based on the preset attention structure and the skip connection structure, wherein the preset semantic segmentation model further comprises a Poisson encoder.

[0108] Optionally, the first construction module comprises: a first construction submodule configured to construct a channel attention module, wherein the channel attention module comprises at least a pooling layer, a full connection layer, and an intermediate output layer, the pooling layer comprises at least a maximum pooling and an average pooling, and the maximum pooling and the average pooling are in a parallel relationship; a second construction submodule configured to construct a spatial attention module, wherein the spatial attention module comprises at least a pooling layer and an output layer, the pooling layer comprises at least a maximum pooling and an average pooling, and the maximum pooling and the average pooling are in a serial relationship; and a third construction submodule configured to construct the preset attention structure based on the channel attention module and the spatial attention module.

[0109] Optionally, the input unit comprises: a first processing module configured to pre-process the first registered image and the second registered image using the Poisson encoder to obtain a first processed image and a second processed image; and a first encoding module configured to encode the first processed image and the second processed image to generate a first pulse sequence and a second pulse sequence, and input the first pulse sequence and the second pulse sequence into the preset semantic segmentation model to output the first target image and the second target image.

[0110] Optionally, the change detection apparatus comprises a third construction module configured to construct a preset image detection model before inputting the first target image and the second target image into the preset image detection model to generate the prediction result image, wherein the preset image detection model comprises at least a backbone network, an encoder, a decoder and a prediction head, the backbone network is configured to extract features of the target image, the encoder is configured to obtain spatial relationships between the features, the decoder is configured to map the spatial relationships to a pixel space to generate a feature map, and the prediction head is configured to generate the prediction result image according to the feature map.

[0111] Optionally, the generation unit comprises a second processing module configured to pre-process the first target image and the second target image respectively to obtain a first target processed image and a second target processed image; a first convolution module configured to perform convolution on the first target processed image and the second target processed image respectively by using the backbone network to generate a feature map of the first target processed image and a feature map of the second target processed image; a first conversion module configured to convert the feature map of the first target processed image and the feature map of the second target processed image respectively to obtain a first semantic feature vector and a second semantic feature vector, and splice the first semantic feature vector and the second semantic feature vector to obtain a spliced feature vector sequence; a first input module configured to input the spliced feature vector sequence into the encoder to obtain spatial relationships between each semantic feature vector in the spliced feature vector sequence; a second input module configured to input the spatial relationships into the decoder to generate an initial feature map, and input the initial feature map into a preset convolutional neural network to generate a target feature map; and a third input module configured to input the target feature map into the prediction head to generate the prediction result image.

[0112] The change detection apparatus for remote sensing images described above can further comprise a processor and a memory, and the above-mentioned acquisition unit 80, registration unit 81, input unit 82, generation unit 83, determination unit 84 and the like are stored in the memory as program units, and the corresponding functions are realized by the processor executing the above-mentioned program units stored in the memory.

[0113] The processor described above comprises a core, and the core retrieves the corresponding program units from the memory. The core can be set to one or more, and the prediction result image is processed by adjusting the core parameters to obtain a target detection result image, and the target detection result image is used to determine the image region with changes between the first remote sensing image and the second remote sensing image.

[0114] The memory described above can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0115] According to another aspect of the embodiments of the present application, a computer program product is also provided, which comprises a non-volatile computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the change detection method of a remote sensing image according to any one of the above.

[0116] When the computer program product is executed on the data processing device, the program is adapted to perform the following method steps: obtaining remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image, registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image, inputting the first registered image and the second registered image into a preset semantic segmentation model respectively to output a first target image and a second target image, inputting the first target image and the second target image into a preset image detection model to generate a prediction result image, processing the prediction result image to obtain a target detection result image, and determining an image region with changes between the first remote sensing image and the second remote sensing image according to the target detection result image.

[0117] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises one or more processors and a memory, the memory being configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the change detection method of a remote sensing image according to the above.

[0118] Figure 9 is a hardware structure block diagram of an electronic device (or a mobile device) for a change detection method of a remote sensing image according to an embodiment of the present application. As shown in Figure 9 , the electronic device can include one or more processors (for example, processors 902a, 902b,..., 902n, etc. in Figure 9 , which can include but are not limited to processing devices such as microprocessors MCU or programmable logic devices FPGA, etc.), a memory 904 for storing data. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. Those skilled in the art can understand that Figure 9 , the structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the electronic device can also include more or less components than Figure 9 , or have a different configuration from Figure 9 .

[0119] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0120] The embodiments or examples of the present disclosure are not exhaustive, and are only a part of the embodiments or examples, and are not specific limitations on the protection scope of the present disclosure. Each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily, for example, a scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be exchanged arbitrarily, in addition, the optional ways or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the embodiments or examples can be combined arbitrarily, for example, the steps of different embodiments or examples can be combined arbitrarily, a certain embodiment or example can be combined with the optional ways or optional examples of other embodiments or examples.

[0121] In the above-described embodiments of the present disclosure, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0122] In the several embodiments of the present disclosure, it should be understood that the disclosed technology can be implemented in other ways. Of course, the embodiment described above is only a schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0123] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0124] In addition, each functional unit in each embodiment of the present disclosure can be integrated in a processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0125] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0126] The above is only the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method for detecting changes in remote sensing images, characterized in that: include: Acquiring remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; The first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image; wherein, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image, including: using a preset operator to calculate the preset image to generate gradient information of each pixel point on the preset image, and performing convolution calculation on the gradient information to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information includes at least: gradient amplitude and direction angle; performing Laplace calculation on the target gradient information of each pixel point on the preset image to generate each The method comprises the steps of: calculating a Laplace response value of a pixel point, performing peak detection on the Laplace response value of each pixel point and the Laplace response values ​​of all pixels in the neighborhood of each pixel point, and determining a peak feature point of the preset image; generating a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixels in the neighborhood of the peak feature point, and determining a target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information includes at least position information and appearance feature information; registering the preset image based on the target feature point to obtain a preset registered image, wherein the preset registered image is the first registered image of the first remote sensing image or the second registered image of the second remote sensing image; Inputting the first registered image and the second registered image into a preset semantic segmentation model respectively, and outputting a first target image and a second target image; Inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph; The prediction result map is processed to obtain a target detection result map, and an image region where a change occurs between the first remote sensing image and the second remote sensing image is determined based on the target detection result map.

2. The remote sensing image change detection method according to claim 1, characterized in that: The step of registering the preset image based on the target feature points to obtain a preset registered image includes: For each target feature point, fusing the position information and the appearance feature information of the target feature point to obtain an initial feature vector; Using a preset attention layer to calculate each of the initial feature vectors to obtain each target feature vector, wherein each of the target feature vectors is the target feature vector corresponding to the first remote sensing image or the target feature vector corresponding to the second remote sensing image; Based on each of the target eigenvectors, a score matrix is ​​constructed, the score matrix is ​​expanded to obtain an expanded matrix, and the expanded matrix is ​​iterated until the number of iterations reaches a preset iteration threshold to obtain a target score matrix; Based on the target score matrix, a pairing point set of the target feature point is determined, and based on the pairing point set, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

3. The remote sensing image change detection method according to claim 1, characterized in that: Before respectively inputting the first registered image and the second registered image into a preset semantic segmentation model and outputting a first target image and a second target image, the method further includes: Construct preset attention structure and skip connection structure; Based on the preset attention structure and the jump connection structure, the preset semantic segmentation model is constructed, wherein the preset semantic segmentation model also includes: a Poisson encoder.

4. The remote sensing image change detection method according to claim 3, characterized in that: The steps to build a preset attention structure include: Construct a channel attention module, wherein the channel attention module includes at least: a pooling layer, a fully connected layer, and an intermediate output layer, the pooling layer includes at least: maximum pooling and average pooling, and the maximum pooling and the average pooling are in parallel relationship; Constructing a spatial attention module, wherein the spatial attention module at least includes: the pooling layer and the output layer, the pooling layer at least includes: the maximum pooling and the average pooling, and the maximum pooling and the average pooling are in a series relationship; Based on the channel attention module and the spatial attention module, the preset attention structure is constructed.

5. The remote sensing image change detection method according to claim 3, characterized in that: The steps of inputting the first registered image and the second registered image into a preset semantic segmentation model respectively and outputting a first target image and a second target image include: Preprocessing the first registered image and the second registered image respectively using the Poisson encoder to obtain a first processed image and a second processed image; The first processed image and the second processed image are respectively encoded to generate a first pulse sequence and a second pulse sequence, and the first pulse sequence and the second pulse sequence are respectively input into the preset semantic segmentation model to output the first target image and the second target image.

6. The remote sensing image change detection method according to claim 1, characterized in that: Before inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph, the method further includes: Construct the preset image detection model, wherein the preset image detection model includes at least: a backbone network, an encoder, a decoder, and a prediction head, the backbone network is used to extract the features of the target image, the encoder is used to obtain the spatial relationship between the features of the target image, the decoder is used to map the spatial relationship to the pixel space and generate a feature map, and the prediction head is used to generate the prediction result map based on the feature map.

7. The remote sensing image change detection method according to claim 6, characterized in that: The step of inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph includes: Preprocessing the first target image and the second target image respectively to obtain a first target processed image and a second target processed image; Using the backbone network to perform convolution on the first target processed image and the second target processed image respectively to generate a feature map of the first target processed image and a feature map of the second target processed image; Converting the feature map of the first target processed image and the feature map of the second target processed image to obtain a first semantic feature vector and a second semantic feature vector, and concatenating the first semantic feature vector and the second semantic feature vector to obtain a concatenated feature vector sequence; Inputting the concatenated feature vector sequence into the encoder to obtain the spatial relationship between each semantic feature vector in the concatenated feature vector sequence; Inputting the spatial relationship into the decoder to generate an initial feature map, and inputting the initial feature map into a preset convolutional neural network to generate a target feature map; The target feature map is input into the prediction head to generate the prediction result map.

8. A remote sensing image change detection device, characterized in that: include: an acquisition unit, configured to acquire remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; a registration unit, configured to register the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image; An input unit, configured to input the first registered image and the second registered image into a preset semantic segmentation model respectively, and output a first target image and a second target image; a generating unit, configured to input the first target image and the second target image into a preset image detection model to generate a prediction result image; a determining unit, configured to process the prediction result map to obtain a target detection result map, and determine an image region where a change occurs between the first remote sensing image and the second remote sensing image based on the target detection result map; The registration unit includes: a first calculation module, which is used to use a preset operator to calculate a preset image, generate gradient information of each pixel on the preset image, and perform convolution calculation on the gradient information to generate target gradient information of each pixel on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information includes at least: gradient amplitude and direction angle; a second calculation module, which is used to perform Laplace calculation on the target gradient information of each pixel on the preset image, generate a Laplace response value of each pixel, and perform Laplace calculation on the Laplace response value of each pixel and the Laplace response values ​​of all pixels in the neighborhood of each pixel. Perform peak detection to determine the peak feature point of the preset image; a first generation module is used to generate a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixel points in the neighborhood of the peak feature point, and determine the target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information includes at least: position information and appearance feature information; a first registration module is used to register the preset image based on the target feature point to obtain a preset registered image, wherein the preset registered image is the first registered image of the first remote sensing image or the second registered image of the second remote sensing image.

9. An electronic device, characterized in that: The method comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the remote sensing image change detection method according to any one of claims 1 to 7.

Images