Remote sensing image change detection method and device and electronic equipment

Abstract

The invention discloses a change detection method and device of a remote sensing image and electronic equipment, and relates to the technical field of artificial intelligence, and the change detection method comprises the steps: obtaining remote sensing images at different time points, obtaining a first remote sensing image and a second remote sensing image, carrying out the registration of the first remote sensing image and the second remote sensing image, and obtaining a registration result; respectively obtaining a first registration image and a second registration image, respectively inputting the first registration image and the second registration image into a preset semantic segmentation model, outputting a first target image and a second target image, inputting the first target image and the second target image into a preset image detection model, and generating a prediction result image, and processing the prediction result image to obtain a target detection result image, and determining a changed image area between the first remote sensing image and the second remote sensing image according to the target detection result image. According to the invention, the technical problem of low accuracy of detecting the change of the large-size remote sensing image in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a remote sensing image change detection method and device, and electronic equipment. Background Art

[0002] The rapid change monitoring technology with fine scene focus has important application value in many fields. This technology can quickly and accurately capture surface micro-change information and provide solid data support for environmental monitoring, disaster warning, urban planning and land use management.

[0003] In the related technologies, the rapid change monitoring methods with fine scene focus are mainly divided into: visual analysis methods, traditional machine learning methods, and deep learning methods. Among them, the visual analysis method relies on manual comparison, which is inefficient and difficult to achieve large-scale or continuous monitoring; the traditional machine learning method is limited in the generalization ability of the model when facing high-dimensional and complexly distributed remote sensing data, and the processing efficiency and accuracy cannot meet the requirements of large-area change monitoring; and the current deep learning method or mathematical method directly processes remote sensing images or monitors changes after sliding window slicing, but remote sensing images have a large size. Directly processing remote sensing images or using sliding window slicing to process remote sensing images is prone to large errors or extremely long processing time, which is not suitable for automatic change monitoring scenarios of large-scale large-area remote sensing images.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a remote sensing image change detection method and device, and electronic equipment, so as to at least solve the technical problem of low accuracy in detecting changes in large-size remote sensing images in the related art.

[0006] According to one aspect of an embodiment of the present invention, a change detection method for a remote sensing image is provided, comprising: acquiring remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image; inputting the first registered image and the second registered image into a preset semantic segmentation model, respectively, and outputting a first target image and a second target image; inputting the first target image and the second target image into a preset image detection model to generate a prediction result map; processing the prediction result map to obtain a target detection result map, and determining an image area where there is a change between the first remote sensing image and the second remote sensing image based on the target detection result map.

[0007] Furthermore, the step of registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image includes: using a preset operator to calculate the preset image to generate gradient information of each pixel point on the preset image, and performing convolution calculation on the gradient information to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information includes at least: gradient amplitude and direction angle; performing Laplace calculation on the target gradient information of each pixel point on the preset image to generate a Laplace response value of each pixel point, and performing convolution calculation on the target gradient information of each pixel point on the preset image to generate a Laplace response value of each pixel point The method comprises the steps of: performing peak detection on the Laplace response value and the Laplace response values ​​of all pixels in the neighborhood of each pixel point to determine the peak feature point of the preset image; generating a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixels in the neighborhood of the peak feature point, and determining the target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information at least includes: position information and appearance feature information; and registering the preset image based on the target feature point to obtain a preset registered image, wherein the preset registered image is the first registered image of the first remote sensing image or the second registered image of the second remote sensing image.

[0008] Furthermore, based on the target feature points, the preset image is registered to obtain a preset registered image, comprising: for each target feature point, the position information and the appearance feature information of the target feature point are fused to obtain an initial feature vector; each initial feature vector is calculated using a preset attention layer to obtain each target feature vector, wherein each target feature vector is the target feature vector corresponding to the first remote sensing image or the target feature vector corresponding to the second remote sensing image; based on each target feature vector, a score matrix is ​​constructed, the score matrix is ​​expanded to obtain an expanded matrix, and the expanded matrix is ​​iterated until the number of iterations reaches a preset iteration threshold to obtain a target score matrix; based on the target score matrix, a pairing point set of the target feature point is determined, and based on the pairing point set, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0009] Furthermore, before respectively inputting the first registered image and the second registered image into a preset semantic segmentation model and outputting the first target image and the second target image, it also includes: constructing a preset attention structure and a jump connection structure; based on the preset attention structure and the jump connection structure, constructing the preset semantic segmentation model, wherein the preset semantic segmentation model also includes: a Poisson encoder.

[0010] Furthermore, the step of constructing a preset attention structure includes: constructing a channel attention module, wherein the channel attention module includes at least: a pooling layer, a fully connected layer, and an intermediate output layer, and the pooling layer includes at least: maximum pooling and average pooling, and the maximum pooling and the average pooling are in parallel; constructing a spatial attention module, wherein the spatial attention module includes at least: the pooling layer and an output layer, and the pooling layer includes at least: the maximum pooling and the average pooling, and the maximum pooling and the average pooling are in series; based on the channel attention module and the spatial attention module, the preset attention structure is constructed.

[0011] Furthermore, the steps of respectively inputting the first registered image and the second registered image into a preset semantic segmentation model to output a first target image and a second target image include: using the Poisson encoder to preprocess the first registered image and the second registered image to obtain a first processed image and a second processed image; respectively encoding the first processed image and the second processed image to generate a first pulse sequence and a second pulse sequence, and respectively inputting the first pulse sequence and the second pulse sequence into the preset semantic segmentation model to output the first target image and the second target image.

[0012] Furthermore, before inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph, it also includes: constructing the preset image detection model, wherein the preset image detection model includes at least: a backbone network, an encoder, a decoder, and a prediction head, the backbone network is used to extract features of the target image, the encoder is used to obtain the spatial relationship between the features, the decoder is used to map the spatial relationship to a pixel space to generate a feature graph, and the prediction head is used to generate the prediction result graph based on the feature graph.

[0013] Furthermore, the step of inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph includes: preprocessing the first target image and the second target image respectively to obtain a first target processed image and a second target processed image; using the backbone network to convolve the first target processed image and the second target processed image respectively to generate a feature map of the first target processed image and a feature map of the second target processed image; converting the feature map of the first target processed image and the feature map of the second target processed image respectively to obtain a first semantic feature vector and a second semantic feature vector, and splicing the first semantic feature vector and the second semantic feature vector to obtain a spliced ​​feature vector sequence; inputting the spliced ​​feature vector sequence into the encoder to obtain the spatial relationship between each semantic feature vector in the spliced ​​feature vector sequence; inputting the spatial relationship into the decoder to generate an initial feature map, and inputting the initial feature map into a preset convolutional neural network to generate a target feature map; inputting the target feature map into the prediction head to generate the prediction result graph.

[0014] According to another aspect of an embodiment of the present invention, a remote sensing image change detection device is also provided, including: an acquisition unit, used to acquire remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; a registration unit, used to register the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image; an input unit, used to input the first registered image and the second registered image into a preset semantic segmentation model, respectively, and output a first target image and a second target image; a generation unit, used to input the first target image and the second target image into a preset image detection model to generate a prediction result map; a determination unit, used to process the prediction result map to obtain a target detection result map, and determine, based on the target detection result map, an image area where there is a change between the first remote sensing image and the second remote sensing image.

[0015] Furthermore, the registration unit includes: a first calculation module, which is used to use a preset operator to calculate the preset image, generate gradient information of each pixel on the preset image, and perform convolution calculation on the gradient information to generate target gradient information of each pixel on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information includes at least: gradient amplitude and direction angle; a second calculation module, which is used to perform Laplace calculation on the target gradient information of each pixel on the preset image, generate a Laplace response value of each pixel, and perform Laplace calculation on the Laplace response value of each pixel and the Laplace of all pixels in the neighborhood of each pixel. The response value is used for peak detection to determine the peak feature point of the preset image; a first generation module is used to generate a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixel points in the neighborhood of the peak feature point, and determine the target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information at least includes: position information and appearance feature information; a first registration module is used to register the preset image based on the target feature point to obtain a preset registered image, wherein the preset registered image is the first registered image of the first remote sensing image or the second registered image of the second remote sensing image.

[0016] Furthermore, the first registration module includes: a first fusion submodule, which is used to fuse the position information and the appearance feature information of the target feature point for each target feature point to obtain an initial feature vector; a first calculation submodule, which is used to calculate each of the initial feature vectors using a preset attention layer to obtain each target feature vector, wherein each of the target feature vectors is the target feature vector corresponding to the first remote sensing image or the target feature vector corresponding to the second remote sensing image; a first construction submodule, which is used to construct a score matrix based on each target feature vector, expand the score matrix to obtain an expanded matrix, and iterate the expanded matrix until the number of iterations reaches a preset iteration threshold to obtain a target score matrix; a first registration submodule, which is used to determine the pairing point set of the target feature point based on the target score matrix, and based on the pairing point set, register the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0017] Furthermore, the change detection device includes: a first construction module, used to construct a preset attention structure and a jump connection structure before respectively inputting the first registered image and the second registered image into a preset semantic segmentation model and outputting the first target image and the second target image; a second construction module, used to construct the preset semantic segmentation model based on the preset attention structure and the jump connection structure, wherein the preset semantic segmentation model also includes: a Poisson encoder.

[0018] Furthermore, the first construction module includes: a first construction sub-module, used to construct a channel attention module, wherein the channel attention module includes at least: a pooling layer, a fully connected layer, and an intermediate output layer, and the pooling layer includes at least: maximum pooling and average pooling, and the maximum pooling and the average pooling are in parallel; a second construction sub-module, used to construct a spatial attention module, wherein the spatial attention module includes at least: the pooling layer and an output layer, and the pooling layer includes at least: the maximum pooling and the average pooling, and the maximum pooling and the average pooling are in series; a third construction sub-module, used to construct the preset attention structure based on the channel attention module and the spatial attention module.

[0019] Furthermore, the input unit includes: a first processing module, used to use the Poisson encoder to preprocess the first registered image and the second registered image respectively to obtain a first processed image and a second processed image; a first encoding module, used to encode the first processed image and the second processed image respectively to generate a first pulse sequence and a second pulse sequence, and input the first pulse sequence and the second pulse sequence into the preset semantic segmentation model respectively to output the first target image and the second target image.

[0020] Furthermore, the change detection device includes: a third construction module, used to construct the preset image detection model before inputting the first target image and the second target image into the preset image detection model to generate a prediction result map, wherein the preset image detection model includes at least: a backbone network, an encoder, a decoder, and a prediction head, the backbone network is used to extract the features of the target image, the encoder is used to obtain the spatial relationship between the features, the decoder is used to map the spatial relationship to the pixel space to generate a feature map, and the prediction head is used to generate the prediction result map based on the feature map.

[0021] Furthermore, the generation unit includes: a second processing module, used to pre-process the first target image and the second target image respectively to obtain a first target processed image and a second target processed image; a first convolution module, used to use the backbone network to convolve the first target processed image and the second target processed image respectively to generate a feature map of the first target processed image and a feature map of the second target processed image; a first conversion module, used to convert the feature map of the first target processed image and the feature map of the second target processed image respectively to obtain a first semantic feature vector and a second semantic feature vector, and splice the first semantic feature vector and the second semantic feature vector to obtain a spliced ​​feature vector sequence; a first input module, used to input the spliced ​​feature vector sequence into the encoder to obtain the spatial relationship between each semantic feature vector in the spliced ​​feature vector sequence; a second input module, used to input the spatial relationship into the decoder to generate an initial feature map, and input the initial feature map into a preset convolutional neural network to generate a target feature map; a third input module, used to input the target feature map into the prediction head to generate the prediction result map.

[0022] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image change detection method described in any one of the above is implemented.

[0023] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned remote sensing image change detection methods.

[0024] In the present invention, remote sensing images at different time points are acquired to obtain a first remote sensing image and a second remote sensing image, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image, the first registered image and the second registered image are respectively input into a preset semantic segmentation model, a first target image and a second target image are output, the first target image and the second target image are input into a preset image detection model to generate a prediction result graph, the prediction result graph is processed to obtain a target detection result graph, and based on the target detection result graph, an image region where a change exists between the first remote sensing image and the second remote sensing image is determined, thereby solving the technical problem of low accuracy in detecting changes in large-size remote sensing images in the related art.

[0025] In the present invention, by acquiring remote sensing images at two time points, the acquired images can be registered in pairs to ensure the spatial consistency of the two images, thereby obtaining a first registered image and a second registered image, and then the registered images are respectively input into a preset semantic segmentation model to accurately extract a specific area. According to the specific area, a first target image and a second target image can be obtained, and then the two target images are input into a preset image detection model to generate a prediction result map, and the prediction result map is subsequently processed to obtain a clear target detection result map, which accurately marks the changed area, can effectively improve the accuracy and speed of change detection, reduce interference in non-focus areas, and realize rapid positioning and monitoring of fine scene changes in large-area remote sensing images, thereby achieving a technical effect of improving the accuracy of detecting changes in large-size remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 is a flow chart of an optional remote sensing image change detection method according to an embodiment of the present invention;

[0028] Figure 2 is a schematic diagram of an optional remote sensing image change detection process according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of an optional pairing result according to an embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of the structure of an optional preset semantic segmentation model according to an embodiment of the present invention;

[0031] Figure 5 is a schematic diagram of an optional preset attention structure according to an embodiment of the present invention;

[0032] Figure 6 is a schematic diagram of an optional remote sensing mask image of an airport and port area according to an embodiment of the present invention;

[0033] Figure 7 is a schematic diagram of an optional preset image detection model processing flow according to an embodiment of the present invention;

[0034] Figure 8 is a schematic diagram of an optional remote sensing image change detection device according to an embodiment of the present invention;

[0035] Fig. 9The present invention is a hardware structure block diagram of an electronic device (or mobile device) for a remote sensing image change detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) collected and involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set between the system and the relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0039] In the present invention, firstly, rapid image registration is performed for multi-temporal remote sensing images, and then a two-stage fine scene change monitoring process of semantic segmentation and change monitoring is constructed. The image mask is obtained through semantic segmentation to narrow the area of ​​interest and locate the key areas (such as airports and ports). Then, image change monitoring is performed, which not only improves the accuracy of change detection, but also reduces processing time and resource consumption.

[0040] The present invention is described in detail below in conjunction with various embodiments.

[0041] Embodiment 1

[0042] According to an embodiment of the present invention, an embodiment of a change detection method for a remote sensing image is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] Figure 1 is a flow chart of an optional remote sensing image change detection method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0044] Step S101, acquiring remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image.

[0045] Alternatively, SAR (Synthetic Aperture Radar) remote sensing images are a type of remote sensing data that uses radar waves to image ground targets. They have all-weather and all-day imaging capabilities and can penetrate clouds, fog and vegetation to obtain accurate information on the surface. Therefore, multi-temporal SAR remote sensing images can be collected.

[0046] In this embodiment, by acquiring SAR remote sensing images at two different time points, a first remote sensing image and a second remote sensing image can be obtained.

[0047] Step S102: registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0048] Alternatively, registration is the process of placing two remote sensing images at different time points in the same geographic coordinate system, ensuring that the two images can be accurately compared by eliminating geometric differences caused by satellite attitude, imaging angle or atmospheric conditions.

[0049] In this embodiment, a control point extraction method based on SAR-Harris (i.e., corner detection algorithm) space and a control point matching method based on a deep learning algorithm can be used for rapid registration. Accurate extraction and matching of control points are the key to registration. Through this process, a first registered image of the first remote sensing image aligned in spatial coordinates and a second registered image of the second remote sensing image can be obtained, laying the foundation for subsequent change detection.

[0050] Step S103: input the first registered image and the second registered image into a preset semantic segmentation model respectively, and output a first target image and a second target image.

[0051] Alternatively, a spiking neural network (SNN) can simulate the working mechanism of biological neurons and transmit information through pulses. This asynchronous and event-driven feature makes SNN more efficient in processing time series data and can capture changes in data on the time axis. In addition, SNN has a certain robustness to noise (such as coherent speckle noise and edge blur in SAR images), can suppress noise in SAR images, and improve the performance of SAR image processing. Therefore, a spiking neural network can be used to perform semantic segmentation on the registered images.

[0052] In this embodiment, by inputting the first registered image and the second registered image into a preset semantic segmentation model (such as a SNN-UNet (Spiking Neural Network-UNet, i.e., a convolutional neural network architecture) network optimized based on a spiking neural network, a mask image of the first registered image and a mask image of the second registered image can be obtained respectively, and then the first remote sensing image and the second remote sensing image can be cropped according to the mask image of the first registered image and the mask image of the second registered image to obtain the first target image and the second target image of a specific area (such as an airport, a port, etc.), thereby narrowing the scope of subsequent change monitoring.

[0053] Step S104, inputting the first target image and the second target image into a preset image detection model to generate a prediction result image.

[0054] Optionally, a network model based on the BIT (Bi-temporal Image Transformer, i.e., deep learning model) structure can be used to process remote sensing image change detection tasks. This network achieves efficient recognition of changes in high-resolution remote sensing images by combining the powerful feature extraction capabilities of convolutional neural networks and the context modeling advantages of Transformer (i.e., neural network architecture).

[0055] In this embodiment, the first target image and the second target image are input into a preset image detection model (such as a network model based on a BIT structure) to generate a prediction result map, that is, a preliminary prediction of the possible change area between the first target image and the second target image.

[0056] Step S105, processing the prediction result map to obtain a target detection result map, and determining an image region where there is a change between the first remote sensing image and the second remote sensing image based on the target detection result map.

[0057] In this embodiment, the prediction result image is post-processed (including applying thresholding, filtering, morphological operations, etc.) to remove noise in the prediction result image, smooth the change area, and generate a final target detection result image (i.e., a binary image). If there is a change between the first remote sensing image and the second remote sensing image, the changed area can be marked as white and the unchanged area can be marked as black in the target detection result image.

[0058] Figure 2 is a schematic diagram of an optional remote sensing image change detection process according to an embodiment of the present invention, such as Figure 2 As shown, SAR remote sensing images of multi-temporal large-area scenes are first collected, and then SAR image registration is performed on the collected SAR remote sensing images. The registered images are then input into a pulse neural network (i.e., a preset semantic segmentation model) to obtain an airport port area mask, and based on the airport port area mask, the airport port area is located to obtain a first target image and a second target image. Then, the first target image and the second target image are input into a preset image detection model to generate a prediction result map, thereby realizing change detection.

[0059] In summary, by acquiring remote sensing images at different time points and performing pairwise registration on the acquired remote sensing images (i.e., registering the first remote sensing image and the second remote sensing image), a first registered image of the first remote sensing image and a second registered image of the second remote sensing image can be obtained, and then the first registered image and the second registered image are respectively input into a preset semantic segmentation model, and a first target image and a second target image can be output, and the first target image and the second target image can be input into a preset image detection model to generate a prediction result graph, and then the prediction result graph is processed to obtain a target detection result graph, and based on the target detection result graph, the image area where there is a change between the first remote sensing image and the second remote sensing image can be determined, thereby solving the technical problem of low accuracy in detecting changes in large-size remote sensing images in related technologies.

[0060] In order to accurately obtain a preset registration image, in the remote sensing image change detection method provided in the first embodiment of the present application, a preset operator is used to calculate the preset image to generate gradient information of each pixel point on the preset image, and a convolution calculation is performed on the gradient information to generate target gradient information of each pixel point on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information includes at least: gradient amplitude and direction angle; Laplace calculation is performed on the target gradient information of each pixel point on the preset image to generate a Laplace response value of each pixel point, and the Laplace response value of each pixel point and the neighboring values ​​of each pixel point are calculated. The Laplace response values ​​of all pixels in the domain are peak detected to determine the peak feature points of the preset image; a gradient statistical histogram is generated based on the target gradient information of the peak feature point and the target gradient information of all pixels in the neighborhood of the peak feature point, and the target feature points of the preset image are determined based on the gradient statistical histogram, wherein the target feature points correspond to target feature point information, and the target feature point information at least includes: position information and appearance feature information; based on the target feature points, the preset image is registered to obtain a preset registered image, wherein the preset registered image is a first registered image of the first remote sensing image or a second registered image of the second remote sensing image.

[0061] Optionally, a preset operator (such as ROEWA (Ratio of Edge Weighted Averages, an operator used for edge detection and feature extraction)) can be used to calculate a preset image (i.e., the first remote sensing image or the second remote sensing image) to generate gradient information for each pixel point on the preset image. The ROEWA operator can overcome the coherent speckle noise problem in the SAR image and accurately extract the edge information of the image.

[0062] In this embodiment, in order to further enhance the effect of feature extraction, a convolution calculation can be performed on the gradient information (such as using a Gaussian function to perform a convolution calculation) to generate target gradient information (including gradient amplitude and direction angle) for each pixel point on the preset image, and a Laplace calculation can be performed on the target gradient information for each pixel point on the preset image to generate a Laplace response value for each pixel point. The Laplace response value and the Laplace response values ​​of all pixels in the neighborhood of each pixel point (i.e., the surrounding area) can be peak detected to determine the peak feature point of the preset image.

[0063] In this embodiment, a gradient statistical histogram (i.e., a visual gradient amplitude and direction angle distribution diagram) can be generated based on the target gradient information of the peak feature point and the target gradient information of all pixels in the neighborhood of the peak feature point, which helps to understand the statistical characteristics of image features. The gradient statistical histogram can be used to further screen and determine the target feature points of the preset image. The target feature points not only have clear location information, but also contain rich appearance feature information.

[0064] In this embodiment, image registration is the process of spatially aligning remote sensing images from different times to ensure the spatial consistency of the two images. The preset images can be registered according to the target feature points to obtain a preset registered image (i.e., the first registered image of the first remote sensing image or the second registered image of the second remote sensing image).

[0065] In order to improve the accuracy of obtaining a first registered image of a first remote sensing image and a second registered image of a second remote sensing image, in a remote sensing image change detection method provided in a first embodiment of the present application, for each target feature point, the target feature point position information and the appearance feature information are fused to obtain an initial feature vector; a preset attention layer is used to calculate each initial feature vector to obtain each target feature vector, wherein each target feature vector is a target feature vector corresponding to the first remote sensing image or a target feature vector corresponding to the second remote sensing image; based on each target feature vector, a score matrix is ​​constructed, the score matrix is ​​expanded to obtain an expanded matrix, and the expanded matrix is ​​iterated until the number of iterations reaches a preset iteration threshold to obtain a target score matrix; based on the target score matrix, a pairing point set of the target feature point is determined, and based on the pairing point set, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0066] Optionally, in order to uniformly represent the geometric position and visual features of the feature points, a key point encoder may be used to map the position information and appearance feature information of the target feature points to generate an initial feature vector.

[0067] In this embodiment, a preset attention layer (such as a self-attention layer and a cross-attention layer that are repeated and alternated L times) is used to calculate each initial feature vector, so that each target feature vector (that is, the target feature vector corresponding to the first remote sensing image or the target feature vector corresponding to the second remote sensing image) can be obtained, and a score matrix (such as an M-by-N score matrix) can be constructed, where M is the number of feature points of the first remote sensing image, and N is the number of feature points of the second remote sensing image.

[0068] In this embodiment, since incorrect matching may occur due to scene changes, image quality differences, occlusions, perspective differences, etc., the score matrix can be expanded (such as adding virtual matching options) to obtain an expanded matrix. In the expanded matrix, each feature point has not only a score for matching with other feature points, but also a score for matching with the virtual matching option. The score for matching with the virtual matching option indicates the probability that the feature point does not match any other actual feature point. If a feature point does not find a suitable matching object in reality, then its matching score with the virtual matching option will be relatively high, and thus it will be marked as unpaired or unmatchable in the final matching decision, thereby more accurately determining the optimal feature point pairing set.

[0069] In this embodiment, the expansion matrix is ​​iterated until the number of iterations reaches a preset iteration threshold (such as T times), and a target score matrix (i.e., an optimal feature point matching matrix) can be obtained. According to the target score matrix, a pairing point set of the target feature points can be determined, and according to the pairing point set, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0070] Figure 3 is a schematic diagram of an optional pairing result according to an embodiment of the present invention, such as Figure 3 As shown in Figure 2, through image registration, two ASR remote sensing images collected at different time points ( Figure 3 (a) and (b) in Fig. 3 are consistent in space, that is, any point on the image has the same spatial position in the two ASR images.

[0071] In order to accurately construct a preset semantic segmentation model, in the change detection method of remote sensing images provided in Example 1 of the present application, a preset attention structure and a jump connection structure are constructed; based on the preset attention structure and the jump connection structure, a preset semantic segmentation model is constructed, wherein the preset semantic segmentation model also includes: a Poisson encoder.

[0072] In this embodiment, the preset attention structure (such as CBAM (Convolutional Block Attention Module)) may include a two-dimensional convolution operation, a channel attention mechanism, a spatial attention mechanism, a batch normalization process, and a nonlinear activation function. By introducing the channel attention mechanism and the spatial attention mechanism, the sensitivity and selectivity of the network to image features can be enhanced. The channel attention mechanism can learn the importance of each channel in the feature map, while the spatial attention mechanism can identify the key areas in the image. The combination of the dual attention mechanisms enables the network to focus more on important features while suppressing irrelevant background information.

[0073] In this embodiment, four upsamplings are used and a skip connection structure is constructed to enhance the low-dimensional feature fusion of the feature map, so that the segmentation results are more refined in details such as edge information. According to the preset attention structure and the skip connection structure, a preset semantic segmentation model can be constructed. The preset semantic segmentation model can also include a Poisson encoder for encoding the output image. Pulse neurons can also be introduced into the preset semantic segmentation model to replace the traditional activation function, thereby significantly improving the accuracy of the segmentation task.

[0074] In this embodiment, in the output stage of the preset semantic segmentation model, the response strength of different categories can be represented by calculating the pulse firing frequency of the output layer neurons during the simulation. Ideally, the correct neurons should fire pulses at the highest frequency, while other neurons remain silent. To this end, a cross-entropy loss function based on the pulse firing frequency and the real pixel category can be constructed, and the network state is reset after each parameter optimization to eliminate the memory effect of the neurons of the preset semantic segmentation model.

[0075] Figure 4 is a schematic diagram of the structure of an optional preset semantic segmentation model according to an embodiment of the present invention, such as Figure 4 As shown, the preset semantic segmentation model includes an input, a CBAM module, a cross-layer connection, and an output. The input is used to input the image into the model. The CBAM module enables the model to pay more attention to the key features and areas in the image, and uses four upsampling and cross-layer connections (i.e., a jump connection structure) to enhance the low-dimensional feature fusion of the feature map, so that the segmentation result is more refined in details such as edge information. The output is used to output the final target image (i.e., the first target image and the second target image).

[0076] In order to improve the accuracy of constructing a preset attention structure, in the change detection method of remote sensing images provided in Example 1 of the present application, a channel attention module is constructed, wherein the channel attention module at least includes: a pooling layer, a fully connected layer, and an intermediate output layer, and the pooling layer at least includes: maximum pooling and average pooling, and the maximum pooling and the average pooling are in a parallel relationship; a spatial attention module is constructed, wherein the spatial attention module at least includes: a pooling layer and an output layer, and the pooling layer at least includes: maximum pooling and average pooling, and the maximum pooling and the average pooling are in a series relationship; based on the channel attention module and the spatial attention module, a preset attention structure is constructed.

[0077] Optionally, in the channel attention module, in addition to average pooling, a maximum pooling operation can also be introduced to compress the spatial dimension. In this way, the feature map can be averaged and pooled at the maximum value on a global scale to effectively extract and aggregate spatial information. Subsequently, the feature map is processed by a fully connected multi-layer perceptron to achieve element-by-element summation and merging of the feature map, and a channel attention map can be generated. The channel attention map is multiplied with the original input feature map to achieve adaptive optimization of the features. The channel attention mechanism not only reflects the feedback of average pooling to each pixel, but also reflects the response of maximum pooling to significant feature points during the back propagation process.

[0078] In this embodiment, the spatial attention module focuses on the depth dimension of the image, aims to identify key areas in the image, and can perform average pooling and maximum pooling on the channel dimension to obtain two single-channel feature maps, which are then concatenated to form an output feature map.

[0079] Figure 5 is a schematic diagram of an optional preset attention structure according to an embodiment of the present invention, such as Figure 5 As shown in the figure, the upper part shows the preset attention structure, including two-dimensional convolution, channel attention, spatial attention, batch normalization layer and activation layer, and the lower part shows the specific structure of channel attention and spatial attention. Channel attention processes the conversion from input feature map to intermediate output, while spatial attention processes the mapping from intermediate output to final output feature map, and the two are cascaded in a serial manner. In the channel attention part, in addition to average pooling, maximum pooling is also introduced to compress the spatial dimension. In this way, the feature map can be averaged and pooled at the maximum value on a global scale to effectively extract and aggregate spatial information. Subsequently, through a fully connected multi-layer perceptron, the element-by-element summation and merging of the feature map are realized to generate a channel attention map, which is multiplied with the original input feature map to achieve adaptive optimization of the feature. The channel attention not only reflects the feedback of average pooling to each pixel, but also reflects the response of maximum pooling to significant feature points during the back propagation process. The spatial attention part obtains two single-channel feature maps by performing maximum pooling and average pooling on the channel dimension, and then splices the two feature maps to form the output feature map.

[0080] In order to accurately obtain the first target image and the second target image, in the remote sensing image change detection method provided in Example 1 of the present application, a Poisson encoder is used to preprocess the first registered image and the second registered image, respectively, to obtain a first processed image and a second processed image; the first processed image and the second processed image are encoded, respectively, to generate a first pulse sequence and a second pulse sequence, and the first pulse sequence and the second pulse sequence are respectively input into a preset semantic segmentation model to output the first target image and the second target image.

[0081] Optionally, the Poisson encoder can preprocess the image, uniformly adjust the image size to 572×572 pixels, and then encode it into a pulse sequence to input into a preset semantic segmentation model.

[0082] In this embodiment, a Poisson encoder is used to preprocess the first registered image and the second registered image respectively, so as to obtain a first processed image and a second processed image, and the first processed image and the second processed image are encoded to generate a first pulse sequence and a second pulse sequence (that is, each pixel in the image can emit pulses with a certain probability according to its grayscale value), so as to provide a suitable input format for a preset semantic segmentation model, and then the generated first pulse sequence and the second pulse sequence are respectively input into the preset semantic segmentation model, so as to obtain a mask image of a remote sensing image of a specific area (such as a mask image of an airport and a port area), and then the remote sensing image can be cropped according to the area where the mask image is located, and the first target image and the second target image are output.

[0083] It should be noted that the first layer and the previous layers of the preset semantic segmentation model do not change over time when receiving input images. In order to improve computational efficiency, the first layer neurons can be specially processed in the preset semantic segmentation model, placing them outside the time loop and eliminating the need to accumulate information over time.

[0084] Figure 6 is a schematic diagram of an optional remote sensing mask image of an airport and port area according to an embodiment of the present invention, such as Figure 6 As shown, the rectangular box is a cropping area set based on the mask image. By cropping the remote sensing image according to the area, a first target image and a second target image can be obtained.

[0085] In order to accurately generate a prediction result map, in the remote sensing image change detection method provided in Example 1 of the present application, a preset image detection model is constructed, wherein the preset image detection model includes at least: a backbone network, an encoder, a decoder, and a prediction head. The backbone network is used to extract features of the target image, the encoder is used to obtain the spatial relationship between the features, the decoder is used to map the spatial relationship to the pixel space and generate a feature map. The prediction head is used to generate a prediction result map based on the feature map.

[0086] Optionally, the overall architecture of the preset image detection model may include: a backbone network, a BIT module (including an encoder, a decoder), and a prediction head.

[0087] In this embodiment, the backbone network is responsible for extracting high-level semantic features from the input image pair. Subsequently, the high-level semantic features are fed into the BIT module, which converts the feature map into a compact semantic feature vector through a spatial attention mechanism. The encoder is then used to perform context modeling on the semantic feature vector, obtain the spatial relationship between the features, and input the spatial relationship into the decoder, which can map the spatial relationship to the pixel space to generate a feature map, which is then input into the prediction head to generate a prediction result map.

[0088] In order to accurately generate a prediction result map, in the remote sensing image change detection method provided in Example 1 of the present application, the first target image and the second target image are preprocessed respectively to obtain a first target processed image and a second target processed image; the first target processed image and the second target processed image are convolved respectively using a backbone network to generate a feature map of the first target processed image and a feature map of the second target processed image; the feature map of the first target processed image and the feature map of the second target processed image are converted respectively to obtain a first semantic feature vector and a second semantic feature vector, and the first semantic feature vector and the second semantic feature vector are spliced ​​to obtain a spliced ​​feature vector sequence; the spliced ​​feature vector sequence is input into an encoder to obtain a spatial relationship between each semantic feature vector in the spliced ​​feature vector sequence; the spatial relationship is input into a decoder to generate an initial feature map, and the initial feature map is input into a preset convolutional neural network to generate a target feature map; the target feature map is input into a prediction head to generate a prediction result map.

[0089] Optionally, the first target image and the second target image are preprocessed (such as scaling, cropping and normalization) respectively to obtain a first target processed image and a second target processed image to meet the input requirements of a preset image detection model.

[0090] In this embodiment, a backbone network (such as an improved ResNet (Residual Network, a convolutional neural network architecture) network for extracting deep features in images) is used to convolve the first target processing image and the second target processing image respectively to generate a feature map of the first target processing image and a feature map of the second target processing image, and the feature map of the first target processing image and the feature map of the second target processing image can be converted respectively to obtain a first semantic feature vector and a second semantic feature vector, and the first semantic feature vector and the second semantic feature vector are spliced ​​to obtain a spliced ​​feature vector sequence.

[0091] In this embodiment, the concatenated feature vector sequence is input into the encoder to obtain the spatial relationship between each semantic feature vector in the concatenated feature vector sequence, and then the spatial relationship is input into the decoder. The decoder can map the context-rich semantic feature vector back to the pixel space to generate a refined feature map (i.e., it can generate an initial feature map), thereby enhancing the semantic information of the original feature map.

[0092] In this embodiment, the initial feature map can be input into a preset convolutional neural network (i.e., a shallow CNN (Convolutional Neural Network)). The preset convolutional neural network can generate a change prediction map (i.e., a target feature map) through a series of convolutional layers and upsampling operations, and can generate a final pixel-level change classification map (i.e., a prediction result map) through an activation function layer (such as a softmax layer or a sigmoid layer).

[0093] Figure 7 is a schematic diagram of an optional preset image detection model processing flow according to an embodiment of the present invention, such as Figure 7 As shown, two cropped remote sensing images (i.e., the first target image and the second target image) are input, and a backbone network is used to extract high-level semantic features from the two input images to generate a feature map. The feature map of the first target processed image and the feature map of the second target processed image can be converted respectively to obtain two semantic tokens (i.e., the first semantic feature vector and the second semantic feature vector), and the first semantic feature vector and the second semantic feature vector are concatenated to obtain a concatenated feature vector sequence, and the concatenated feature vector sequence is input into the Transformer encoder to obtain the spatial relationship between each semantic feature vector in the concatenated feature vector sequence, and then the spatial relationship is input into two Transformer decoders to generate an initial feature map, and the initial feature map can be input into a shallow CNN to generate a change prediction map, and the change prediction map is post-processed to obtain the final change detection result map, in which the white area is the area with change, and the black area is the area without change.

[0094] In an embodiment of the present invention, by integrating semantic segmentation and change monitoring into a dual-stage fine scene focus change detection process, changes in SAR remote sensing images can be monitored efficiently and accurately. First, a pulse neural network optimized UNet architecture is introduced to obtain a preset semantic segmentation model, which effectively processes SAR images, improves the accuracy of semantic segmentation, ensures accurate definition of the focus area, and further optimizes the preset semantic segmentation model using a preset attention structure, significantly improves the performance of edge detection, and makes the segmentation result more delicate. Then, a preset image detection model is used to perform change detection on the segmented key areas, which can accurately capture the difference between the two images, reduce interference from non-focus areas, and significantly improve the efficiency and accuracy of change detection.

[0095] The following is a detailed description in conjunction with another embodiment.

[0096] Embodiment 2

[0097] A remote sensing image change detection device provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1.

[0098] Figure 8 is a schematic diagram of an optional remote sensing image change detection device according to an embodiment of the present invention, such as Figure 8 As shown, the remote sensing image change detection device may include: an acquisition unit 80 , a registration unit 81 , an input unit 82 , a generation unit 83 , and a determination unit 84 .

[0099] The acquisition unit 80 is used to acquire remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image;

[0100] A registration unit 81, configured to register the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image;

[0101] An input unit 82, used to input the first registered image and the second registered image into a preset semantic segmentation model respectively, and output a first target image and a second target image;

[0102] A generating unit 83, configured to input the first target image and the second target image into a preset image detection model to generate a prediction result image;

[0103] The determination unit 84 is used to process the prediction result map to obtain a target detection result map, and determine the image region where there is a change between the first remote sensing image and the second remote sensing image based on the target detection result map.

[0104] The above-mentioned remote sensing image change detection device can acquire remote sensing images at different time points through the acquisition unit 80 to obtain a first remote sensing image and a second remote sensing image, align the first remote sensing image and the second remote sensing image through the registration unit 81 to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image, respectively input the first registered image and the second registered image into a preset semantic segmentation model through the input unit 82, output a first target image and a second target image, input the first target image and the second target image into a preset image detection model through the generation unit 83, generate a prediction result map, process the prediction result map through the determination unit 84 to obtain a target detection result map, and determine the image area where there is a change between the first remote sensing image and the second remote sensing image based on the target detection result map.

[0105] Optionally, the registration unit includes: a first calculation module, which is used to use a preset operator to calculate the preset image, generate gradient information of each pixel on the preset image, and perform convolution calculation on the gradient information to generate target gradient information of each pixel on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information includes at least: gradient amplitude and direction angle; a second calculation module, which is used to perform Laplace calculation on the target gradient information of each pixel on the preset image, generate a Laplace response value of each pixel, and perform Laplace calculation on the Laplace response value of each pixel and the Laplace response of all pixels in the neighborhood of each pixel. The invention relates to a method for performing peak detection on a preset image based on a target gradient information of the peak feature point and target gradient information of all pixel points in a neighborhood of the peak feature point to generate a gradient statistical histogram, and determining the target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information includes at least position information and appearance feature information; the first registration module is used to register the preset image based on the target feature point to obtain a preset registered image, wherein the preset registered image is a first registered image of the first remote sensing image or a second registered image of the second remote sensing image.

[0106] Optionally, the first registration module includes: a first fusion submodule, used to fuse the target feature point position information and appearance feature information for each target feature point to obtain an initial feature vector; a first calculation submodule, used to use a preset attention layer to calculate each initial feature vector to obtain each target feature vector, wherein each target feature vector is a target feature vector corresponding to the first remote sensing image or a target feature vector corresponding to the second remote sensing image; a first construction submodule, used to construct a score matrix based on each target feature vector, expand the score matrix to obtain an expanded matrix, and iterate the expanded matrix until the number of iterations reaches a preset iteration threshold to obtain a target score matrix; the first registration submodule, used to determine the pairing point set of the target feature point based on the target score matrix, and based on the pairing point set, align the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

[0107] Optionally, the change detection device includes: a first construction module, used to construct a preset attention structure and a jump connection structure before respectively inputting the first registered image and the second registered image into a preset semantic segmentation model and outputting the first target image and the second target image; a second construction module, used to construct a preset semantic segmentation model based on the preset attention structure and the jump connection structure, wherein the preset semantic segmentation model also includes: a Poisson encoder.

[0108] Optionally, the first building module includes: a first building sub-module, used to build a channel attention module, wherein the channel attention module includes at least: a pooling layer, a fully connected layer, and an intermediate output layer, the pooling layer includes at least: maximum pooling and average pooling, and the maximum pooling and the average pooling are in parallel; a second building sub-module, used to build a spatial attention module, wherein the spatial attention module includes at least: a pooling layer, an output layer, the pooling layer includes at least: maximum pooling and average pooling, and the maximum pooling and the average pooling are in series; a third building sub-module, used to construct a preset attention structure based on the channel attention module and the spatial attention module.

[0109] Optionally, the input unit includes: a first processing module, used to use a Poisson encoder to preprocess the first registered image and the second registered image, respectively, to obtain a first processed image and a second processed image; a first encoding module, used to encode the first processed image and the second processed image, respectively, to generate a first pulse sequence and a second pulse sequence, and respectively input the first pulse sequence and the second pulse sequence into a preset semantic segmentation model, and output a first target image and a second target image.

[0110] Optionally, the change detection device includes: a third construction module, used to construct a preset image detection model before inputting the first target image and the second target image into the preset image detection model to generate a prediction result map, wherein the preset image detection model includes at least: a backbone network, an encoder, a decoder, and a prediction head, the backbone network is used to extract features of the target image, the encoder is used to obtain the spatial relationship between the features, the decoder is used to map the spatial relationship to the pixel space to generate a feature map, and the prediction head is used to generate a prediction result map based on the feature map.

[0111] Optionally, the generation unit includes: a second processing module, used to pre-process the first target image and the second target image respectively to obtain the first target processed image and the second target processed image; a first convolution module, used to use the backbone network to convolve the first target processed image and the second target processed image respectively to generate a feature map of the first target processed image and a feature map of the second target processed image; a first conversion module, used to convert the feature map of the first target processed image and the feature map of the second target processed image respectively to obtain a first semantic feature vector and a second semantic feature vector, and splice the first semantic feature vector and the second semantic feature vector to obtain a spliced ​​feature vector sequence; a first input module, used to input the spliced ​​feature vector sequence into the encoder to obtain the spatial relationship between each semantic feature vector in the spliced ​​feature vector sequence; a second input module, used to input the spatial relationship into the decoder to generate an initial feature map, and input the initial feature map into a preset convolutional neural network to generate a target feature map; a third input module, used to input the target feature map into the prediction head to generate a prediction result map.

[0112] The above-mentioned remote sensing image change detection device may also include a processor and a memory. The above-mentioned acquisition unit 80, registration unit 81, input unit 82, generation unit 83, determination unit 84, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0113] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be provided, and the prediction result map is processed by adjusting kernel parameters to obtain a target detection result map, and the image area where there is a change between the first remote sensing image and the second remote sensing image is determined based on the target detection result map.

[0114] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0115] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, any one of the above-mentioned remote sensing image change detection methods is implemented.

[0116] When the computer program product is executed on a data processing device, it is suitable for executing a program that is initialized with the following method steps: acquiring remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image, registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image, respectively inputting the first registered image and the second registered image into a preset semantic segmentation model, outputting a first target image and a second target image, inputting the first target image and the second target image into a preset image detection model, generating a prediction result map, processing the prediction result map to obtain a target detection result map, and determining, based on the target detection result map, an image area where there is a change between the first remote sensing image and the second remote sensing image.

[0117] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned remote sensing image change detection method.

[0118] Fig. 9 1 is a hardware structure block diagram of an electronic device (or mobile device) for a remote sensing image change detection method according to an embodiment of the present invention. Fig. 9 As shown, the electronic device may include one or more processors (e.g., Fig. 9 The processor 902a, processor 902b, ..., processor 902n, etc., which may include but are not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 904 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Fig. 9 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig. 9 More or fewer components as shown, or with Fig. 9 Different configurations are shown.

[0119] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0120] The embodiments or examples of the present disclosure are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0121] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0122] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0124] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0126] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A change detection method for remote sensing images, characterized in that: include: Acquire remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; Registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image; Inputting the first registered image and the second registered image into a preset semantic segmentation model respectively, and outputting a first target image and a second target image; Inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph; The prediction result map is processed to obtain a target detection result map, and an image region where a change occurs between the first remote sensing image and the second remote sensing image is determined based on the target detection result map.

2. The remote sensing image change detection method according to claim 1, characterized in that: The step of registering the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image comprises: Using a preset operator, calculating a preset image to generate gradient information of each pixel on the preset image, and performing convolution calculation on the gradient information to generate target gradient information of each pixel on the preset image, wherein the preset image is the first remote sensing image or the second remote sensing image, and the target gradient information includes at least: gradient amplitude and direction angle; Performing Laplace calculation on the target gradient information of each pixel on the preset image to generate a Laplace response value of each pixel, and performing peak detection on the Laplace response value of each pixel and the Laplace response values ​​of all pixels in the neighborhood of each pixel to determine a peak feature point of the preset image; Generate a gradient statistical histogram based on the target gradient information of the peak feature point and the target gradient information of all pixel points in the neighborhood of the peak feature point, and determine the target feature point of the preset image based on the gradient statistical histogram, wherein the target feature point corresponds to target feature point information, and the target feature point information includes at least: position information and appearance feature information; Based on the target feature points, the preset images are registered to obtain preset registered images, wherein the preset registered image is the first registered image of the first remote sensing image or the second registered image of the second remote sensing image.

3. The remote sensing image change detection method according to claim 2, characterized in that: The step of registering the preset image based on the target feature points to obtain a preset registered image comprises: For each of the target feature points, the position information and the appearance feature information of the target feature point are fused to obtain an initial feature vector; Using a preset attention layer to calculate each of the initial feature vectors to obtain each target feature vector, wherein each of the target feature vectors is the target feature vector corresponding to the first remote sensing image or the target feature vector corresponding to the second remote sensing image; Based on each of the target feature vectors, a score matrix is ​​constructed, the score matrix is ​​expanded to obtain an expanded matrix, and the expanded matrix is ​​iterated until the number of iterations reaches a preset iteration threshold to obtain a target score matrix; Based on the target score matrix, a pairing point set of the target feature points is determined, and based on the pairing point set, the first remote sensing image and the second remote sensing image are registered to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image.

4. The remote sensing image change detection method according to claim 1, characterized in that: Before respectively inputting the first registered image and the second registered image into a preset semantic segmentation model and outputting a first target image and a second target image, the method further includes: Construct preset attention structures and skip connection structures; Based on the preset attention structure and the jump connection structure, the preset semantic segmentation model is constructed, wherein the preset semantic segmentation model also includes: a Poisson encoder.

5. The remote sensing image change detection method according to claim 4, characterized in that: The steps to build a preset attention structure include: Constructing a channel attention module, wherein the channel attention module at least includes: a pooling layer, a fully connected layer, and an intermediate output layer, and the pooling layer at least includes: maximum pooling and average pooling, and the maximum pooling and the average pooling are in parallel relationship; Constructing a spatial attention module, wherein the spatial attention module at least includes: the pooling layer and the output layer, the pooling layer at least includes: the maximum pooling and the average pooling, and the maximum pooling and the average pooling are in a series relationship; Based on the channel attention module and the spatial attention module, the preset attention structure is constructed.

6. The remote sensing image change detection method according to claim 4, characterized in that: The steps of respectively inputting the first registered image and the second registered image into a preset semantic segmentation model and outputting a first target image and a second target image include: Preprocessing the first registered image and the second registered image respectively by using the Poisson encoder to obtain a first processed image and a second processed image; The first processed image and the second processed image are respectively encoded to generate a first pulse sequence and a second pulse sequence, and the first pulse sequence and the second pulse sequence are respectively input into the preset semantic segmentation model to output the first target image and the second target image.

7. The remote sensing image change detection method according to claim 1, characterized in that: Before inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph, the method further includes: Construct the preset image detection model, wherein the preset image detection model includes at least: a backbone network, an encoder, a decoder, and a prediction head, wherein the backbone network is used to extract the features of the target image, the encoder is used to obtain the spatial relationship between the features, the decoder is used to map the spatial relationship to the pixel space and generate a feature map, and the prediction head is used to generate the prediction result map based on the feature map.

8. The remote sensing image change detection method according to claim 7, characterized in that: The step of inputting the first target image and the second target image into a preset image detection model to generate a prediction result graph includes: Preprocessing the first target image and the second target image respectively to obtain a first target processed image and a second target processed image; Using the backbone network to convolve the first target processed image and the second target processed image respectively, to generate a feature map of the first target processed image and a feature map of the second target processed image; Converting the feature map of the first target processed image and the feature map of the second target processed image respectively to obtain a first semantic feature vector and a second semantic feature vector, and concatenating the first semantic feature vector and the second semantic feature vector to obtain a concatenated feature vector sequence; Inputting the concatenated feature vector sequence into the encoder to obtain the spatial relationship between each semantic feature vector in the concatenated feature vector sequence; Inputting the spatial relationship into the decoder to generate an initial feature map, and inputting the initial feature map into a preset convolutional neural network to generate a target feature map; The target feature map is input into the prediction head to generate the prediction result map.

9. A remote sensing image change detection device, characterized in that: include: An acquisition unit, used for acquiring remote sensing images at different time points to obtain a first remote sensing image and a second remote sensing image; a registration unit, configured to register the first remote sensing image and the second remote sensing image to obtain a first registered image of the first remote sensing image and a second registered image of the second remote sensing image; An input unit, used to input the first registered image and the second registered image into a preset semantic segmentation model respectively, and output a first target image and a second target image; A generating unit, configured to input the first target image and the second target image into a preset image detection model to generate a prediction result graph; The determination unit is used to process the prediction result map to obtain a target detection result map, and determine an image area where there is a change between the first remote sensing image and the second remote sensing image based on the target detection result map.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the remote sensing image change detection method described in any one of claims 1 to 8.

Images