Change analysis method and device based on cross-temporal frequency domain information interaction and readable storage medium thereof

Abstract

The invention provides a change analysis method and device based on cross-temporal frequency domain information interaction and a readable storage medium thereof. The method comprises the following steps: acquiring a dual-temporal remote sensing image and partitioning the dual-temporal remote sensing image; extracting spatial features through a shared encoder; fourier transform is carried out on the double-tense features in a feature domain, high and low frequency components are divided according to a step length r, low frequency information is exchanged, and after inverse transform, splicing is carried out to obtain a fusion feature F; performing same frequency domain interaction on an original image in a data domain, and modeling through a video encoder to obtain a feature E; and finally fusing F and E, and decoding and outputting a change prediction map. According to the method, the high and low frequency information proportion is dynamically regulated and controlled through the frequency domain step length, dual interaction of the data domain and the feature domain is combined, structural difference perception is enhanced, pseudo change interference is suppressed, and change detection precision and robustness are remarkably improved. Experiments show that the method is superior to the prior art in F1 score, intersection-union ratio and other indexes, and is suitable for the fields of smart cities, disaster monitoring and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of time series image processing, and in particular to a change analysis method and device based on cross-temporal frequency domain information interaction and a readable storage medium thereof. Background Art

[0002] Change detection has important applications in engineering fields such as smart city construction, traffic inspection, land use monitoring, and disaster response. It obtains information about ground feature changes through multi-temporal image analysis, providing data support for decision-making. With the improvement of remote sensing image resolution and acquisition frequency, change detection faces higher requirements for accuracy and robustness. Current mainstream methods are mostly based on temporal feature comparison or spatial semantic interaction of deep neural networks, but they have the following limitations: 1. It has insufficient ability to perceive fine-grained changes in complex backgrounds (such as building edges and subtle changes in vegetation) and is easily disturbed by non-structural factors such as lighting and perspective changes; 2. The image frequency domain information is not fully exploited, making it difficult to effectively model the structural change characteristics in time-series images (such as scene structure differences in the low-frequency domain and detail evolution in the high-frequency domain); 3. The cross-temporal feature interaction only stays in the spatial domain or semantic layer, lacking the complementary effect of frequency domain features, resulting in blurred boundaries of change areas and significant pseudo-change noise.

[0003] Therefore, there is an urgent need for a change analysis method, device and readable storage medium based on cross-temporal frequency domain information interaction to solve the problems existing in the prior art. Summary of the Invention

[0004] The embodiments of the present invention provide a change analysis method, device and readable storage medium based on cross-temporal frequency domain information interaction, which address the problems existing in current technologies, such as insufficient accuracy in detecting fine-grained changes in complex scenarios, and lack of cross-temporal interaction modeling of frequency domain information, resulting in weak anti-interference ability and fuzzy expression of change boundaries.

[0005] The core technology of this invention mainly introduces a cross-temporal frequency domain information interaction mechanism in the two-layer architecture of feature domain and data domain, uses the frequency domain step size to dynamically control the ratio of high-frequency and low-frequency components, and exchanges the low-frequency components of dual-temporal images / features to achieve deep evolution of temporal information and enhancement of structural differences, thereby improving the accuracy and robustness of change detection.

[0006] In a first aspect, the present invention provides a change analysis method based on cross-temporal frequency domain information interaction, the method comprising the following steps: S1, obtain a pair of bi-temporal images X1 and X2 of the same region and divide them into a set of sub-blocks {x1, x2} of uniform size; S2, extracting bi-temporal spatial features f1 and f2 through a parameter-sharing image encoder; S3, perform feature domain cross-temporal frequency domain information interaction on the bi-temporal spatial features f1 and f2, and splice them along the time series dimension to obtain the bi-temporal interactive fusion feature F; S4, perform data domain cross-temporal frequency domain information interaction on X1 and X2, and splice them along the time series dimension to obtain a dual-temporal interactive fusion image sequence S; S5. Globally model the bi-temporal interactive fusion image sequence S through a video encoder to obtain the data domain information interaction feature E; S6. Splice the bi-temporal interaction fusion feature F and the data domain information interaction feature E, upsample them through the decoder to generate the change prediction map Y, and splice them into a complete change map according to the original position.

[0007] Furthermore, the feature domain cross-temporal frequency domain information interaction includes: S31. Perform Fourier transform on f1 and f2 to the frequency domain, and divide the low-frequency component and high-frequency component by the frequency domain step r to obtain the high and low frequency spectrum characteristics of each time state. and ; S32. Exchange the bi-temporal low-frequency components at the same step length r to generate interactive feature combinations and ; S33. Perform inverse Fourier transform on the interactive feature combination to restore it to spatial domain enhanced features and ; S34. Splicing spatial domain enhancement features along the channel dimension 、 The fusion feature F is obtained by combining the time series features.

[0008] Furthermore, the image encoder is a CNN backbone network and the video encoder is a 3DSwin-Transformer.

[0009] Furthermore, the frequency domain step size r is positively correlated with the proportion of high-frequency and low-frequency components: the larger r is, the more structural information the low-frequency components contain and the less detail information the high-frequency components contain; and vice versa.

[0010] Furthermore, the frequency domain step length r ranges from 4 to 32.

[0011] Furthermore, the encoder restores the feature map to the original image resolution through multi-level upsampling and convolution operations, and outputs a change prediction map with pixel values of 0 or 1, where "1" indicates that a change has occurred and "0" indicates that no change has occurred.

[0012] Furthermore, the bi-temporal image pairs are remote sensing images or video sequences, and the change detection includes scenes of land feature evolution, disaster response, or urban planning.

[0013] In a second aspect, the present invention provides a change analysis device based on cross-temporal frequency domain information interaction, comprising: The image acquisition module acquires a pair of bi-temporal images X1 and X2 of the same region and divides them into a set of sub-blocks {x1, x2} of uniform size; Feature extraction module, which extracts bi-temporal spatial features f1 and f2 through a parameter-sharing image encoder; The feature domain interaction module performs feature domain cross-temporal frequency domain information interaction on the bi-temporal spatial features f1 and f2, and splices them along the time series dimension to obtain the bi-temporal interaction fusion feature F; The data domain interaction module performs data domain cross-temporal frequency domain information interaction on X1 and X2, and splices them along the time dimension to obtain a dual-temporal interactive fusion image sequence S. The dual-temporal interactive fusion image sequence S is globally modeled through a video encoder to obtain the data domain information interaction feature E. The change detection module splices the bi-temporal interaction fusion feature F and the data domain information interaction feature E, generates a change prediction map Y through upsampling by the decoder, and splices it into a complete change map according to the original position.

[0014] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned change analysis method based on cross-temporal frequency domain information interaction.

[0015] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes the above-mentioned change analysis method based on cross-temporal frequency domain information interaction.

[0016] The main contributions and innovations of the present invention are as follows: 1. Dual-domain frequency-domain collaborative modeling: Through the dual-layer frequency-domain interaction of the feature domain (high-level abstract features) and the data domain (original image), it takes into account both structural difference modeling and input layer change response enhancement, improving the separability and boundary clarity of the changing area; 2. Dynamic frequency domain step size control: Flexibly control the ratio of high- and low-frequency information through multi-level frequency domain step sizes, adaptively balancing scene structure perception and detail feature extraction, and suppressing interference from false changes such as illumination and noise. 3. Cross-temporal low-frequency exchange: This simulates the temporal evolution of bi-temporal features, enhancing the model's ability to model long-term and complex changes in terrain, significantly improving key metrics such as precision and F1 score (for example, in this example, the F1 score increased by 1.16%, with fewer missed and false positives). 4. Strong engineering applicability: It is suitable for multiple scenarios such as high-resolution remote sensing imagery and video surveillance, and its generalization ability is superior to traditional single-domain interaction methods (such as the dual-time transformer STENet).

[0017] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 Flowchart of a change analysis method based on cross-temporal frequency domain information interaction according to an embodiment of the present invention; Figure 2 Flowchart of a process for cross-temporal frequency domain information interaction according to an embodiment of the present invention; Figure 3 is a diagram of detection results according to an embodiment of the present invention; Figure 4 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0020] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0021] Defects of existing technology: 1. Traditional methods rely on spatial or semantic feature comparison and are not sensitive enough to fine-grained changes (such as changes in small objects).

[0022] 2. It is easily affected by interference such as lighting, noise, and changes in viewing angle, resulting in false alarms.

[0023] 3. Lack of full utilization of image frequency domain information (such as low-frequency structure and high-frequency details), resulting in blurred boundaries or insufficient modeling of structural differences.

[0024] Based on this, the present invention solves the problems existing in the prior art by frequency domain transformation and cross-temporal low-frequency component exchange.

[0025] Example 1 The present invention aims to propose a change analysis method based on cross-temporal frequency domain information interaction, specifically, referring to Figure 1-3 , the method comprises the following steps: S1, get the same area at different times (temporal and ), the bi-temporal images are respectively recorded as and To improve the parallel efficiency of subsequent processing, and Divide into several sub-blocks of the same size to form an image block set ; S2, use parameter sharing image encoder (such as CNN backbone network), respectively and Perform feature extraction, capture spatial texture and structural information, and obtain high-level abstract features and ; Parameter sharing here means that the bi-temporal images use the same set of parameters and The encoder (such as CNN) is used to extract features and ensure the consistency of the feature space, which can avoid the deviation introduced by the difference in encoder parameters and focus on the timing change itself.

[0026] S3, yes and Perform cross-temporal frequency domain information interaction in the feature domain and splice along the time series dimension to obtain the dual-temporal interaction fusion feature F.

[0027] In one embodiment, in step S3, the cross-temporal frequency domain information interaction operation is specifically as follows: S31, feature map and Perform Fourier transform to map it from the spatial domain to the frequency domain. Set the frequency domain step size based on the frequency domain amplitude spectrum. (For example, setting from 4 to 32) divides the spectrum into low-frequency components containing structural information and high-frequency components containing edge and detail information. The longer the step size, the more information the low-frequency components contain and the less high-frequency information; conversely, the shorter the step size, the more high-frequency information and the less low-frequency information. Finally, the high and low frequency spectrum characteristics of each temporal state are obtained. and ; S32. Exchange the same step length in the frequency domain The low-frequency components of the next two time states, when the frequency domain step size gradually increases, the low-frequency component information gradually decreases, the high-frequency component information gradually increases, and the low-frequency components after exchange gradually become ineffective, thereby realizing the feature evolution between different time series and obtaining the feature combination after information interaction and ; S33, combining the features after interaction and Restore to the spatial domain through inverse Fourier transform to obtain enhanced features and ; S34, the characteristics and Perform temporal splicing along the channel dimension to obtain the fusion feature F; S4. Input image and Perform the same steps as S3 to achieve more direct data domain cross-temporal frequency domain information interaction. Through the information evolution at the image level, the responsiveness of the changed area at the input layer is enhanced. The images after interaction are spliced along the temporal dimension to obtain the image sequence S. S5, the image sequence Perform global modeling through a video encoder (such as 3D Swin-Transformer) to obtain the data domain information interaction feature E; S6. Interacting features of feature domains The feature E that interacts with the data domain is spliced along the channel dimension and gradually upsampled to the original image resolution through decoder modeling to obtain the change prediction map , where each pixel value indicates whether a change has occurred. A pixel value of 1 indicates a change, and a pixel value of 0 indicates no change. S7. Spatially stitch the predicted change blocks along the original image spatial position to restore them to a complete change map with the same size as the input image.

[0028] For ease of understanding, the following are supplementary explanations of the professional terms or terminology of the present invention: 1. Definition of Fourier Transform: A mathematical transformation that converts signals from the spatial domain (or time domain) to the frequency domain. It decomposes the image into a superposition of sine waves of different frequencies to achieve the mapping of the image from pixel value representation (spatial domain) to frequency component representation (frequency domain).

[0029] Function: In the present invention, it is used to decompose image features or original images into low-frequency components (structural information) and high-frequency components (detail information), providing a basis for cross-temporal frequency domain interaction.

[0030] 2. Frequency Domain and Spatial Domain Frequency domain: The frequency representation of an image, where low frequencies correspond to global structure (such as background and outlines) and high frequencies correspond to local details (such as edges and textures). The frequency domain amplitude spectrum reflects the intensity of different frequency components. This invention uses the "frequency domain step size r" to divide the high and low frequency ranges.

[0031] Spatial domain: The original pixel space of the image directly represents the image content with pixel value distribution, such as bi-temporal images 、 and its sub-tiles.

[0032] 3. Convolutional Neural Network (CNN) Definition: A deep learning model that automatically extracts the spatial features of images through structures such as convolutional layers and pooling layers, and is good at capturing local texture and structural information.

[0033] Function: This invention acts as an "image encoder" (such as Resnet-18) for bi-temporal images. 、 Perform feature extraction and generate high-level abstract features 、 .

[0034] 3D Swin-Transformer definition: A 3D vision model based on the Transformer architecture that processes the spatiotemporal features of videos or multi-frame image sequences through a self-attention mechanism and has global modeling capabilities.

[0035] Function: In this invention, it acts as a "video encoder" to perform temporal modeling on the image sequence S after data domain interaction, capture global context associations across time states, and output data domain features E.

[0036] 5. Inverse Fourier Transform Definition: The inverse process of Fourier transform converts the frequency domain signal back to the spatial domain and restores it to an intuitively understandable image or feature map.

[0037] Function: In the present invention, it is used to combine the features after frequency domain interaction (such as ) is converted back to the spatial domain to generate enhanced features 、 , for subsequent splicing and decoding.

[0038] 6. Definition of cross-temporal frequency domain information interaction: In the frequency domain representation of dual-temporal images (t1 and t2), frequency components at different moments (especially low-frequency structural information) are exchanged or fused to simulate the evolution of temporal features and enhance the structural differences or detail responses in the changed areas.

[0039] Core operation: Under the same frequency domain step length r, exchange the low-frequency components of the two temporal states (for example, replace the low-frequency components of t1 with the low-frequency components of t2); adjust the ratio of high and low frequencies by adjusting the step length r to achieve feature evolution from "structure-dominated" to "detail-dominated".

[0040] 7. Frequency Domain Step Size (r) definition: The frequency domain amplitude spectrum division threshold is used to define the range of low-frequency components and high-frequency components. The value of step size r (such as 4, 8, 16, and 32) determines the amount of structural information contained in the low-frequency components: a larger r value means a wider frequency range covered by the low-frequency components, more structural information contained, and less high-frequency detail; a smaller r value means that the low-frequency components contain only a small amount of global structure, and a higher proportion of high-frequency detail.

[0041] 8. Data Domain and Feature Domain Data domain: At the original image level (such as X1 and X2), the pixel values are directly transformed and interacted in the frequency domain to enhance the change response of the input layer.

[0042] Feature domain: The high-level abstract feature layers (such as f1 and f2) encoded by the neural network perform frequency domain interactions on the semantic-level features to model the temporal structure differences.

[0043] Dual-domain collaboration: The data domain focuses on guiding changes in the input layer, while the feature domain focuses on high-level semantic evolution, jointly improving detection accuracy.

[0044] 9. Video Encoder Definition: A neural network module that processes temporal image data and is good at capturing spatiotemporal dependencies across frames.

[0045] The present invention is applied to globally model the image sequence S after data domain interaction (image after dual temporal frequency domain interaction) and output the feature E that integrates spatiotemporal information, which is different from the "image encoder" that only processes a single frame.

[0046] 10.Decoder definition: A module in deep learning used to restore high-level feature maps to their original size output, usually by gradually restoring spatial details through upsampling (such as interpolation, deconvolution) and convolution operations.

[0047] Function: In the present invention, the splicing features (F+E) of the feature domain and the data domain are decoded to generate a pixel-level change prediction map Y.

[0048] Example 2 Based on the same concept, the present invention selects high-resolution remote sensing images of Guangzhou City, Guangdong Province for experimental verification. The experimental data set is a three-channel RGB image with a spatial resolution of 0.55 meters. Figure 3 As shown in the figure, temporal image 1 is a remote sensing image acquired in 2006, and temporal image 2 is a remote sensing image of the corresponding area in 2019. Figures (a)-(f) show the change comparison of six groups of different ground feature areas. This experiment uses a change analysis method based on cross-temporal frequency domain information interaction proposed in this paper to detect and analyze the evolution process of typical urban ground features. The specific steps are as follows: Step 1: Load high-resolution remote sensing images of Guangzhou City, Guangdong Province in 2006 and 2019, crop the dual-temporal remote sensing images and their corresponding change annotation maps to generate image blocks of size 256×256, and divide them into training, validation, and test sets in proportion.

[0049] Step 2: Pre-train the CNN model (specifically, the Resnet-18 network) in the image encoder and the Video Transformer model (specifically, the 3D Video Swin-Transformer network) in the video encoder on the ImageNet classification dataset. The CNN model and Video Transformer model can also use other suitable networks available in the art.

[0050] Step 3: Use CNN to extract multi-scale features of size (64×64×64, 32×32×64, 16×16×64), perform feature domain cross-temporal frequency domain information interaction, and obtain feature F.

[0051] Step 4: Obtain the image sequence after interaction by performing data domain and cross-temporal frequency domain information interaction on the bi-temporal data , after Video Transformer modeling, multi-scale features E with sizes of (64×64×64, 32×32×64, 16×16×64) are obtained.

[0052] Step 5: The same-size features of the multi-scale features F and E are concatenated along the channel dimension and predicted and decoded by a decoder stacked by ResNet Blocks.

[0053] Step 6: Subtraction, 4x upsampling, and convolution are performed on the multi-scale fused feature maps of the dual-temporal image output, each with a size of 64×64×64, to obtain a change detection map of size 256×256×2. Model training is completed by minimizing the loss between the detection prediction map and the label map using the cross-entropy loss function and the stochastic gradient descent algorithm. This technique is well known to those skilled in the art and will not be detailed here.

[0054] Step 7: Load the trained parameters of this method, obtain the predicted output on the test set, and compare the change detection results with the test set labels. Calculate the precision (the proportion of real changes in the predicted changed area (to avoid false detection)), recall (the proportion of correctly predicted real change areas (to avoid missed detection)), F1 score (a comprehensive evaluation index commonly used in machine learning and computer vision (such as change detection) to measure the balance between the precision and recall of the model, especially for class imbalanced data), intersection over union (IoU, an evaluation index of change detection results, calculating the overlap ratio between the predicted change area and the real change area) and overall accuracy, as shown in Figure 7. Figure 3 As shown in Figure 1, there are six image pairs (a) to (f), which are the results of the change detection diagrams of the proposed method (the present invention method) and the reference method (dual time converter). The detection evaluation results are shown in Table 1: Table 1

[0055] The dual-time converter is a method proposed in the prior art. For reference, see X. Pan, J. Lai, Y. Jin, X.Zhou and J. Zheng, "STENet: A Spatial Selection and Temporal Evolution Network for Change Detection in Remote Sensing Images," in IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1-15, 2024, Art no. 4410915. The method proposed in this invention is the method proposed in Table 1. The detection results of the proposed method surpass the results of the latest dual-time converter method in all aspects. Figure 4 The results show that the proposed method has fewer missed detections (light gray pixels) and false detections (dark gray pixels), as well as higher accuracy (white pixels), which further verifies the effectiveness of the proposed method in the task of change detection in dual-temporal remote sensing images.

[0056] The present invention targets the input data domain and high-level feature domain in the dual-temporal change detection task, performs dual-frequency domain cross-temporal information interaction modeling, and improves the method's accurate perception of structural changes and effective suppression of task-irrelevant interference. By controlling the frequency domain step size, the high- and low-frequency components are guided to retain information to varying degrees. The larger the frequency domain step size, the more low-frequency information that enhances scene structure perception, and the weaker the high-frequency low-frequency information that enhances scene structure perception. Low-frequency components with different information retention levels in dual-temporal images or features are exchanged to achieve effective information flow of dual-temporal information and enhance the modeling capability of changing trends. The dual-layer cross-temporal frequency domains of the data domain and feature domain collaborate to enhance the recognition and edge clarity of the changing area, effectively overcoming the limitations of traditional methods in complex background and fine-grained change detection, and demonstrating stronger cross-scene generalization capabilities and engineering applicability.

[0057] Example 3 Based on the same concept, the present invention also proposes a change analysis device based on cross-temporal frequency domain information interaction, comprising: The image acquisition module acquires a pair of bi-temporal images X1 and X2 of the same region and divides them into a set of sub-blocks {x1, x2} of uniform size; Feature extraction module, which extracts bi-temporal spatial features f1 and f2 through a parameter-sharing image encoder; The feature domain interaction module performs feature domain cross-temporal frequency domain information interaction on the bi-temporal spatial features f1 and f2, and splices them along the time series dimension to obtain the bi-temporal interaction fusion feature F; The data domain interaction module performs data domain cross-temporal frequency domain information interaction on X1 and X2, and splices them along the time dimension to obtain a dual-temporal interactive fusion image sequence S. The dual-temporal interactive fusion image sequence S is globally modeled through a video encoder to obtain the data domain information interaction feature E. The change detection module splices the bi-temporal interaction fusion feature F and the data domain information interaction feature E, generates a change prediction map Y through upsampling by the decoder, and splices it into a complete change map according to the original position.

[0058] Example 4 This embodiment also provides an electronic device, referring to Figure 4 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0059] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0060] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0061] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0062] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the change analysis methods based on cross-temporal frequency domain information interaction in the above embodiments.

[0063] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0064] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0065] The input / output device 408 is used to input or output information.

[0066] Example 5 This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes program code for controlling a process to execute a process. The process includes the change analysis method based on cross-temporal frequency domain information interaction according to embodiment one.

[0067] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0068] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0069] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0070] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A change analysis method based on cross-temporal frequency domain information interaction, characterized in that: The following steps are involved: S1, obtain a pair of bi-temporal images X1 and X2 of the same region and divide them into a set of sub-blocks {x1, x2} of uniform size; S2, extracting bi-temporal spatial features f1 and f2 through a parameter-sharing image encoder; S3, perform feature domain cross-temporal frequency domain information interaction on the bi-temporal spatial features f1 and f2, and splice them along the time series dimension to obtain the bi-temporal interactive fusion feature F; S4, perform data domain cross-temporal frequency domain information interaction on X1 and X2, and splice them along the time series dimension to obtain a dual-temporal interactive fusion image sequence S; S5. Globally model the bi-temporal interactive fusion image sequence S through a video encoder to obtain the data domain information interaction feature E; S6. Splice the bi-temporal interaction fusion feature F and the data domain information interaction feature E, upsample them through the decoder to generate the change prediction map Y, and splice them into a complete change map according to the original position.

2. The change analysis method based on cross-temporal frequency domain information interaction according to claim 1, characterized in that: The feature domain cross-temporal frequency domain information interaction includes: S31. Perform Fourier transform on f1 and f2 to the frequency domain, and divide the low-frequency component and high-frequency component by the frequency domain step r to obtain the high and low frequency spectrum characteristics of each time state. and ; S32. Exchange the bi-temporal low-frequency components at the same step length r to generate interactive feature combinations and ; S33. Perform inverse Fourier transform on the interactive feature combination to restore it to spatial domain enhanced features and ; S34. Splicing spatial domain enhancement features along the channel dimension 、 The fusion feature F is obtained by combining the time series features.

3. The change analysis method based on cross-temporal frequency domain information interaction according to claim 1, characterized in that: The image encoder is a CNN backbone network, and the video encoder is a 3DSwin-Transformer.

4. The change analysis method based on cross-temporal frequency domain information interaction according to claim 2, characterized in that: The frequency domain step length r is positively correlated with the proportion of high-frequency and low-frequency components: the larger r is, the more structural information the low-frequency components contain, and the less detailed information the high-frequency components contain.

5. The change analysis method based on cross-temporal frequency domain information interaction according to claim 2, characterized in that: The frequency domain step length r ranges from 4 to 32.

6. The change analysis method based on cross-temporal frequency domain information interaction according to claim 1, characterized in that: The encoder restores the feature map to the original image resolution through multi-level upsampling and convolution operations, and outputs a change prediction map with pixel values of 0 or 1, where "1" indicates that a change has occurred and "0" indicates that no change has occurred.

7. A change analysis method based on cross-temporal frequency domain information interaction according to any one of claims 1 to 6, characterized in that: The bi-temporal image pair is a remote sensing image or a video sequence, and the change detection includes scenes of land feature evolution, disaster response or urban planning.

8. A change analysis device based on cross-temporal frequency domain information interaction, characterized in that: include: The image acquisition module acquires a pair of bi-temporal images X1 and X2 of the same region and divides them into a set of sub-blocks {x1, x2} of uniform size; Feature extraction module, which extracts bi-temporal spatial features f1 and f2 through a parameter-sharing image encoder; The feature domain interaction module performs feature domain cross-temporal frequency domain information interaction on the bi-temporal spatial features f1 and f2, and splices them along the time series dimension to obtain the bi-temporal interaction fusion feature F; The data domain interaction module performs data domain cross-temporal frequency domain information interaction on X1 and X2, and splices them along the time series dimension to obtain a dual-temporal interactive fusion image sequence S; The dual-temporal interactive fusion image sequence S is globally modeled through the video encoder to obtain the data domain information interaction feature E; The change detection module splices the bi-temporal interaction fusion feature F and the data domain information interaction feature E, generates a change prediction map Y through upsampling by the decoder, and splices it into a complete change map according to the original position.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the change analysis method based on cross-temporal frequency domain information interaction according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes the change analysis method based on cross-temporal frequency domain information interaction according to any one of claims 1 to 7.

Images