Remote sensing image target trend detection methods, devices, equipment and media

Abstract

This invention provides a method for detecting target trends in remote sensing images, applicable to the field of remote sensing image processing and analysis. The method includes: acquiring and preprocessing an input image; separating the target region from the preprocessed image using a semantic segmentation network based on spectral features and prior geographic location information; constructing a cost metric function using optimal transport theory, solving for the optimal transport plan, and obtaining a smoothed feature map of the image; extracting texture and motion features based on the feature map, and fusing them to obtain the spatiotemporal features of the target region; capturing the spatiotemporal dependencies in the feature sequences of multi-temporal remote sensing images using a time series analysis model; constructing a target trend prediction model based on the feature map, spatiotemporal features, and spatiotemporal dependencies; and outputting the trend prediction result using the target trend prediction model. This invention also provides a remote sensing image target trend detection device, equipment, and medium.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and analysis technology, and more specifically to a method, apparatus, device, and medium for detecting target trends in remote sensing images. Background Technology

[0002] Currently, target trend detection in remote sensing images mainly employs methods based on pixel differences, feature extraction and classification, and time series analysis.

[0003] Pixel-based methods identify changes in target areas by comparing pixel values ​​in remote sensing images at different time points. Their advantage lies in their simplicity and directness; however, they are easily affected by environmental factors such as lighting conditions and seasonal changes, leading to a high false detection rate.

[0004] Feature extraction and classification methods first preprocess the image to extract target features such as color, texture, and shape, and then use machine learning or deep learning models for classification and trend prediction. This method has high accuracy but suffers from high computational complexity and a strong dependence on feature selection. Common approaches include image preprocessing (denoising, enhancement), feature extraction (using algorithms such as SIFT and HOG), feature selection, model training (such as SVM and CNN), and trend prediction. The aim is to extract target objects through segmentation processing of multi-temporal remote sensing images and analyze the results based on geometric attributes or spectral texture features. However, this method faces challenges due to varying segmentation results, and the accuracy of trend detection largely depends on the segmentation quality.

[0005] Time series analysis methods treat remote sensing image data as time series data and use statistical models (such as ARIMA and LSTM) to analyze target change trends. While capable of capturing long-term trends, they do not adequately consider the spatial characteristics of image data. The main idea is to overlay multi-temporal data for target object extraction, obtaining target objects with consistent shape, size, and location across multiple temporal images. Then, target trend detection is performed using either a classification-then-detection or detection-then-classification approach. The architecture typically includes three main steps: semantic label data input, classifier selection, and change extraction. Traditional classification-then-detection methods rely on classifying and comparing raw spectral data. While this provides detailed information on change category migration, it is limited by the classification accuracy of a single-period remote sensing image. Single-temporal classification errors propagate to the multi-temporal change detection process, causing error accumulation. In contrast, detection-then-classification methods directly perform interpolation or overlay classification on multi-temporal data to obtain complete detection results.

[0006] However, existing methods have many drawbacks. Time series analysis often ignores the spatial continuity of images, affecting the accuracy of target trend detection; pixel difference methods are easily affected by environmental factors, leading to false alarms; in the target object extraction process, traditional methods rely heavily on accurate registration of multi-temporal data, and because they use image feature parameters from multiple time points simultaneously, improper parameter settings can easily cause over-segmentation or under-segmentation of ground features, resulting in high heterogeneity of extracted target objects, oversampling, and boundary discontinuities, leading to the "distributed gap" problem. In addition, in remote sensing image trend analysis, there is a "spatiotemporal semantic gap" between low-level ground feature changes and high-level target semantic changes. Although traditional pixel-level or feature parameter-based analysis methods can capture specific changes at the ground feature level, they cannot determine the overall classification transformation of high-level targets based on this. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] To address the heterogeneity of targets caused by inconsistent feature parameters during multi-temporal target extraction in existing methods for processing remote sensing images, and to alleviate the "distributed gap" phenomenon, this invention provides a method, apparatus, device, and medium for remote sensing image target trend detection. By leveraging the nonlinear mapping capability of optimal transmission, it establishes hidden connections between low-level ground feature changes and high-level scene semantic changes. Furthermore, it mitigates the "spatiotemporal semantic gap" phenomenon through temporal modeling, ultimately improving the accuracy and robustness of target trend detection and effectively utilizing the continuous spatial information implicit in multiple remote sensing images.

[0009] (II) Technical Solution

[0010] To address the aforementioned technical problems, embodiments of the present invention provide a method, apparatus, device, and medium for detecting target trends in remote sensing images.

[0011] According to a first aspect of the present invention, a method for detecting target trends in remote sensing images is provided, comprising: acquiring an input image, the input image including multiple consecutive temporal remote sensing images; preprocessing the input image to obtain a preprocessed image; separating a target region from the preprocessed image using a semantic segmentation network based on spectral features and prior geographic location information; calculating the pixel-level Wasserstein distance between images using optimal transport theory and the Sinkhorn iterative algorithm, constructing a cost metric function, solving for the optimal transport plan, and obtaining a smoothed feature map of an image based on the optimal transport plan; extracting texture features based on the feature map by combining a local binary mode algorithm and an optical flow method to extract motion features, fusing the texture features and motion features to obtain the spatiotemporal features of the target region; capturing the spatiotemporal dependencies in the feature sequences of multi-temporal remote sensing images using a time series analysis model; constructing a target trend prediction model based on the feature map, spatiotemporal features, and spatiotemporal dependencies; outputting the trend prediction result using the target trend prediction model, determining the trend type by setting a threshold, and plotting a trend map.

[0012] In some exemplary embodiments, the input image is preprocessed, including: using Gaussian filtering to remove image noise and performing masking to reduce the influence of noise and interference areas; using scale-invariant feature transform feature matching method to linearly normalize the image brightness to the same range to ensure brightness consistency between images; and using image registration and orthorectification methods to spatially align multi-temporal images to eliminate illumination and atmospheric differences.

[0013] In some exemplary embodiments, the optimal transport theory is utilized to calculate the pixel-level Wasserstein distance between images based on the Sinkhorn iterative algorithm, construct a cost metric function, solve for the optimal transport plan, and obtain a smoothed feature map of an image based on the optimal transport plan. This includes: using optimal transport theory and the Sinkhorn iterative algorithm to calculate the pixel-level Wasserstein distance between images and constructing a cost metric function for the optimal transport map; obtaining a cost matrix for two images based on the cost metric function, the cost matrix including the correlation calculation results of corresponding pixels in the two images; solving for the optimal transport plan between the two images based on the cost matrix, the optimal transport plan satisfying the optimal solution under specific constraints; and obtaining a smoothed feature map of an image based on the solved optimal transport plan.

[0014] In some exemplary embodiments, the semantic segmentation network is a deep learning network trained on a large amount of labeled remote sensing image data to accurately identify and extract target regions.

[0015] In some exemplary embodiments, based on feature mapping, texture features are extracted using a local binary mode algorithm, and motion features are extracted using optical flow. The texture features and motion features are then fused to obtain the spatiotemporal features of the target region. This includes: based on feature mapping, using a local binary mode algorithm to extract texture features from the target region, generating texture feature descriptors by comparing the grayscale values ​​of pixels with their neighboring pixels to characterize the texture information of the target region; based on feature mapping, using optical flow to extract motion features from the target region, calculating the motion vectors of pixels based on the temporal changes of pixels in the image sequence and the correlation between adjacent frames, thereby obtaining the motion information of the target region; and using feature concatenation or weighted fusion to fuse the texture information and motion information to form the spatiotemporal features of the target region, which can comprehensively reflect the characteristics of the target region in different times and spaces.

[0016] In some exemplary embodiments, a time series analysis model is used to capture the spatiotemporal dependencies in the feature sequences of multi-temporal remote sensing images. This includes: arranging the spatiotemporal features of the multi-temporal remote sensing images into a feature sequence according to time order, and inputting it into a time series analysis model, which includes either a long short-term memory network or a gated recurrent unit; using the time series analysis model to learn the dependencies between different time steps in the feature sequence, so as to capture the change patterns of the target area in the time dimension and the change trends of spatial features over time.

[0017] In some exemplary embodiments, the target trend prediction model outputs trend prediction results, determines the trend type by setting thresholds, and plots a trend graph. This includes: inputting the multi-temporal remote sensing image data to be predicted into the trained target trend prediction model after undergoing the same preprocessing and feature extraction steps as the training data to obtain the target trend prediction results; setting thresholds for different trend types based on actual application needs and historical data statistics; comparing the prediction results with the set thresholds to determine the type of target trend; and plotting a trend graph in a visual manner based on the determination of the target trend type to intuitively display the changing trend of the target area at different times.

[0018] According to a second aspect of the present invention, a remote sensing image target trend detection device is provided, comprising: a first acquisition module for acquiring an input image, the input image including multiple consecutive temporal remote sensing images; a preprocessing module for preprocessing the input image to obtain a preprocessed image; a second acquisition module for separating a target region from the preprocessed image based on spectral features and prior geographic location information using a semantic segmentation network; a third acquisition module for calculating the pixel-level Wasserstein distance between images using optimal transport theory and based on the Sinkhorn iterative algorithm, constructing a cost metric function, solving for the optimal transport plan, and obtaining a smoothed feature map of an image based on the optimal transport plan; a fourth acquisition module for extracting texture features based on the feature map, combining a local binary mode algorithm and an optical flow method to extract motion features, fusing texture features and motion features to obtain the spatiotemporal features of the target region; a fifth acquisition module for capturing the spatiotemporal dependencies in the feature sequences of multi-temporal remote sensing images using a time series analysis model; a model construction module for constructing a target trend prediction model based on the feature map, spatiotemporal features, and spatiotemporal dependencies; and a prediction module for outputting trend prediction results using the target trend prediction model, determining the trend type by setting a threshold, and drawing a trend map.

[0019] According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0020] A fourth aspect of the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0021] (III) Beneficial Effects

[0022] As can be seen from the above technical solutions, the remote sensing image target trend detection method, apparatus, device, and medium provided by the embodiments of the present invention have at least the following beneficial effects:

[0023] (1) By using optimal transport mapping, the heterogeneity of target area features in multiple time phases is solved; the hidden connection between low-level land cover changes and high-level scene semantic changes is established, which captures subtle changes between images more accurately, reduces noise interference, and improves the accuracy of trend detection.

[0024] (2) By extracting spatiotemporal features, the spatial continuity of remote sensing images is effectively utilized, which improves the spatial resolution and reliability of trend prediction.

[0025] (3) By mining spatiotemporal correlations through time-series modeling based on GRU modules, important feature information is retained and irrelevant content is filtered out, thereby alleviating the spatiotemporal semantic gap phenomenon and ultimately improving the accuracy and robustness of target trend detection, effectively utilizing the continuous spatial information hidden in multiple remote sensing images. Attached Figure Description

[0026] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0027] Figure 1 A flowchart illustrating a remote sensing image target trend detection method according to an embodiment of the present invention is shown schematically.

[0028] Figure 2 The diagram illustrates the architecture of a remote sensing image target trend detection method according to an embodiment of the present invention.

[0029] Figure 3 The illustration schematically shows the optimal transmission remote sensing image target trend detection results according to an embodiment of the present invention;

[0030] Figure 4 A remote sensing image target trend detection device according to an embodiment of the present invention is illustrated schematically;

[0031] Figure 5 A block diagram of an electronic device for a remote sensing image target trend detection method according to an embodiment of the present invention is shown schematically. Detailed Implementation

[0032] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0033] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0034] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0035] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0036] Figure 1 A flowchart illustrating a remote sensing image target trend detection method according to an embodiment of the present invention is shown schematically. Figure 2 The diagram illustrates the architecture of a remote sensing image target trend detection method according to an embodiment of the present invention.

[0037] like Figure 1 and Figure 2 As shown, a remote sensing image target trend detection method according to an embodiment of the present invention includes steps S110-S180.

[0038] In step S110, an input image is acquired, which includes multiple consecutive temporal remote sensing images.

[0039] In step S120, the input image is preprocessed to obtain a preprocessed image.

[0040] In embodiments of the present invention, the input image is preprocessed, including: using Gaussian filtering to remove image noise and performing masking to reduce the influence of noise and interference areas; using the Scale-Invariant Feature Transform (SIFT) feature matching method to linearly normalize the image brightness to the same range to ensure brightness consistency between images; and using image registration and orthorectification methods to spatially align multi-temporal images to eliminate illumination and atmospheric differences.

[0041] In step S130, based on spectral features and prior geographical location information, a semantic segmentation network is used to separate the target region from the preprocessed image.

[0042] In embodiments of the present invention, the semantic segmentation network is a deep learning network trained on a large amount of labeled remote sensing image data to accurately identify and extract target regions. Step S130 specifically includes: defining the region of interest using prior information such as geographic location and spectral features; using a threshold-based, morphological, or semantic segmentation network to extract the relevant target region, reducing background interference, focusing on the change information of the target region, thereby improving the accuracy of trend detection and the efficiency of subsequent analysis.

[0043] In step S140, using optimal transport theory, the pixel-level Wasserstein distance between images is calculated based on the Sinkhorn iterative algorithm, a cost metric function is constructed, the optimal transport plan is solved, and a smoothed feature map of an image is obtained based on the plan.

[0044] In an embodiment of the present invention, step S140 may specifically include: using optimal transport theory, calculating the pixel-level Wasserstein distance between images based on the Sinkhorn iterative algorithm, and constructing a cost metric function for the optimal transport mapping; obtaining a cost matrix of two images based on the cost metric function, the cost matrix including the correlation calculation results of corresponding pixels in the two images; solving for the optimal transport plan between the two images based on the cost matrix, the optimal transport plan satisfying the optimal solution under specific constraints; and obtaining a smoothed feature map of an image based on the solved optimal transport plan.

[0045] Introducing optimal transport theory into remote sensing image trend detection precisely quantifies changes between images, improving the accuracy of trend detection. This method constructs the forward transport cost using a Gaussian kernel function and solves the optimal transport scheme through Sinkhorn iteration to achieve information transfer and fusion. This addresses the target heterogeneity caused by inconsistent feature parameters in multi-temporal target extraction, mitigating the distributed gap phenomenon.

[0046] For example, by utilizing optimal transport theory and solving the pixel-level Wasserstein distance between images based on the Sinkhorn iterative algorithm, the distributed gap in the extraction results can be alleviated, and the smooth transition of image features and the precise quantification of changes can be achieved.

[0047] Given remote sensing images at time N Remote sensing images at time N+1 First, construct the cost metric function for the optimal transport mapping, and let... For a regenerating nucleus Hilbert space (RKHS) And its associated Gaussian kernel function for kernel embedding : .in Represent a d-dimensional linear space over the real number field R. Then we can obtain Cost matrix The result contains the dot product of two images:

[0048] (1)

[0049] Based on the cost matrix and The optimal transmission plan between them is denoted as A matrix is ​​defined as the unique solution to the following equation:

[0050] (2)

[0051] (3)

[0052] in, Indicates transmission cost, It is the entropy regularization function. It is responsible for controlling the sparsity of the transmission cost matrix. U is the space of acceptable coupling, and the source domain vector of optimal transmission is given by... If the target domain vector is b, then Constrained by the following conditions:

[0053] (4)

[0054] in, Represents the space of positive real numbers. This represents an n-dimensional vector with elements equal to 1.

[0055] Based on the solved transmission plan get Smoothed feature maps :

[0056] (5)

[0057] in, It is the transmission target vector ( The number of elements in ).

[0058] In step S150, based on feature mapping, texture features are extracted by combining the local binary mode algorithm and motion features are extracted by optical flow method. The texture features and motion features are fused to obtain the spatiotemporal features of the target area, which include color, texture changes and spatial position movement.

[0059] In an embodiment of the present invention, step S150 may specifically include: extracting texture features of the target region based on feature mapping using a local binary mode algorithm, generating a texture feature descriptor by comparing the gray value relationship between a pixel and its neighboring pixels to characterize the texture information of the target region; extracting motion features of the target region based on feature mapping using optical flow, calculating the motion vector of the pixel according to the temporal changes of pixels in the image sequence and the correlation between adjacent frames, thereby obtaining the motion information of the target region; and fusing texture information and motion information by using feature splicing or weighted fusion to form the spatiotemporal features of the target region, which can comprehensively reflect the characteristics of the target region in different times and spaces.

[0060] Based on a multi-head self-attention module, this structure takes multi-dimensional features such as color, texture, and location as input. After optimization through relevant task objectives of trend prediction, it can adaptively fuse and filter key features, fully consider the spatial and temporal continuity of remote sensing images, and enhance the robustness of the model.

[0061] In step S160, a time series analysis model is used to capture the spatiotemporal dependencies in the feature sequences of multi-temporal remote sensing images.

[0062] In an embodiment of the present invention, step S160 may specifically include: forming a feature sequence from the spatiotemporal features of multi-temporal remote sensing images in chronological order, and inputting it into a time series analysis model, wherein the time series analysis model includes either a long short-term memory network or a gated recurrent unit; using the time series analysis model, learning the dependencies between different time steps in the feature sequence, so as to capture the change pattern of the target area in the time dimension and the change trend of spatial features over time.

[0063] In step S170, a target trend prediction model is constructed based on feature mapping, spatiotemporal features, and spatiotemporal dependencies.

[0064] This architecture, based on the GRU model, introduces update and reset gates to control the flow of information. The update gate determines the degree of integration between the current information and historical information, while the reset gate adjusts the influence of the previous state on the current state. This effectively captures the spatiotemporal correlations in image sequences and alleviates the spatiotemporal semantic gap. While maintaining modeling capabilities, this architecture reduces the number of parameters, effectively improving computational efficiency.

[0065] In step S180, the trend prediction result is output using the target trend prediction model, the trend type is determined by setting a threshold, and a trend graph is drawn.

[0066] In an embodiment of the present invention, step S180 may specifically include: inputting the multi-temporal remote sensing image data to be predicted into the trained target trend prediction model after undergoing the same preprocessing and feature extraction steps as the training data to obtain the prediction result of the target trend; setting thresholds for different trend types according to actual application requirements and historical data statistics; comparing the prediction result with the set thresholds to determine the type of target trend; and drawing a trend map in a visual manner based on the determination result of the type of target trend to intuitively display the changing trend of the target area at different times.

[0067] Figure 3 The illustration shows the optimal target trend detection results for transmitted remote sensing images according to an embodiment of the present invention.

[0068] like Figure 3 As shown, Figure 3 From left to right, the images show early remote sensing images of towns in desert areas, images of towns after land reclamation, and a trend map of these changes. The first two satellite images provide a direct comparison of the same area at different times, revealing spatial changes such as urban area shifts and farmland pattern adjustments, and allowing for the estimation of the speed and magnitude of these changes. The third, abstract two-dimensional image more clearly quantifies various types of regional changes. Combined with the prediction results, regional development trends can be determined, such as the economic vitality and potential pressures reflected in continued urban growth, and the agricultural processes and policy effects involved in farmland changes. Furthermore, potential problems can be identified, such as insufficient regional development momentum or land use conflicts, providing important references for regional planning, resource management, and policy formulation, and contributing to sustainable regional development.

[0069] Figure 4 The illustration schematically depicts a remote sensing image target trend detection device according to an embodiment of the present invention.

[0070] like Figure 4 As shown, the remote sensing image target trend detection device 800 of this embodiment includes a first acquisition module 810, a preprocessing module 820, a second acquisition module 830, a third acquisition module 840, a fourth acquisition module 850, a fifth acquisition module 860, a model building module 870, and a prediction module 880.

[0071] The first acquisition module 810 is used to acquire input images, which include multiple consecutive temporal remote sensing images.

[0072] The preprocessing module 820 is used to preprocess the input image to obtain a preprocessed image.

[0073] The second acquisition module 830 is used to separate the target region from the preprocessed image based on spectral features and prior geographical location information using a semantic segmentation network.

[0074] The third acquisition module 840 is used to calculate the pixel-level Wasserstein distance between images using optimal transmission theory and the Sinkhorn iterative algorithm, construct a cost metric function, solve the optimal transmission plan, and obtain a smoothed feature map of an image based on the plan.

[0075] The fourth acquisition module 850 is used to extract texture features based on feature mapping, combined with the local binary mode algorithm and the optical flow method to extract motion features, and to fuse texture features and motion features to obtain the spatiotemporal features of the target region.

[0076] The fifth acquisition module 860 is used to capture the spatiotemporal dependencies in the feature sequences of multi-temporal remote sensing images using a time series analysis model.

[0077] Model building module 870 is used to build target trend prediction models based on feature mapping, spatiotemporal features, and spatiotemporal dependencies.

[0078] The prediction module 880 is used to output trend prediction results using the target trend prediction model, determine the trend type by setting a threshold, and draw a trend graph.

[0079] According to embodiments of the present invention, any and more modules among the first acquisition module 810, preprocessing module 820, second acquisition module 830, third acquisition module 840, fourth acquisition module 850, fifth acquisition module 860, model building module 870, and prediction module 880 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of the present invention, at least one of the first acquisition module 810, preprocessing module 820, second acquisition module 830, third acquisition module 840, fourth acquisition module 850, fifth acquisition module 860, model building module 870, and prediction module 880 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or any other reasonable means of integrating or packaging circuits, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the first acquisition module 810, preprocessing module 820, second acquisition module 830, third acquisition module 840, fourth acquisition module 850, fifth acquisition module 860, model building module 870, and prediction module 880 can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0080] Figure 5 A block diagram of an electronic device for a remote sensing image target trend detection method according to an embodiment of the present invention is shown schematically.

[0081] like Figure 5 As shown, an electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0082] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 902 and / or RAM 903. It should be noted that programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0083] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0084] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0085] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0086] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

Claims

1. A method for detecting target trends in remote sensing images, characterized in that, include: Acquire input images, which include multiple consecutive temporal remote sensing images; The input image is preprocessed to obtain a preprocessed image; Based on spectral features and prior geographical location information, a semantic segmentation network is used to separate the target region from the preprocessed image; Using optimal transport theory, the pixel-level Wasserstein distance between images is calculated based on the Sinkhorn iterative algorithm, and a cost metric function for optimal transport mapping is constructed. Based on the cost metric function, a cost matrix of two images is obtained, which includes the association calculation results of corresponding pixels in the two images. The optimal transmission plan between two images is solved based on the cost matrix, and the optimal transmission plan satisfies the optimal solution under specific constraints; and based on the solved optimal transmission plan, a smoothed feature map of an image is obtained. Based on the feature mapping, texture features are extracted by combining the local binary mode algorithm and motion features are extracted by optical flow method. The texture features and motion features are then fused to obtain the spatiotemporal features of the target region. Time series analysis models are used to capture the spatiotemporal dependencies in feature sequences of multi-temporal remote sensing images; Based on the feature mapping, the spatiotemporal features, and the spatiotemporal dependencies, a target trend prediction model is constructed. The target trend prediction model is used to output trend prediction results. The trend type is determined by setting a threshold, and a trend graph is drawn.

2. The method according to claim 1, characterized in that, The preprocessing of the input image includes: Gaussian filtering is used to remove image noise, and masking is performed to reduce the impact of noise and interference areas. The scale-invariant feature transform feature matching method is used to linearly normalize the image brightness to the same range to ensure brightness consistency between images; and Image registration and orthorectification methods are used to spatially align multi-temporal images to eliminate differences in illumination and atmosphere.

3. The method according to claim 1, characterized in that, The semantic segmentation network is a deep learning network trained on a large amount of labeled remote sensing image data to accurately identify and extract target regions.

4. The method according to claim 1, characterized in that, Based on the feature mapping, texture features are extracted using the local binary pattern algorithm, and motion features are extracted using the optical flow method. The texture features and motion features are then fused to obtain the spatiotemporal features of the target region, including: Based on the feature mapping, the local binary mode algorithm is used to extract texture features from the target region. By comparing the gray value relationship between a pixel and its neighboring pixels, a texture feature descriptor is generated to characterize the texture information of the target region. Based on the feature mapping, the optical flow method is used to extract motion features of the target region. According to the temporal changes of pixels in the image sequence and the correlation between adjacent frames, the motion vector of the pixel is calculated, thereby obtaining the motion information of the target region. The texture information and motion information are fused by feature splicing or weighted fusion to form the spatiotemporal features of the target area, which can comprehensively reflect the characteristics of the target area in different times and spaces.

5. The method according to claim 1, characterized in that, The method of using a time series analysis model to capture the spatiotemporal dependencies in multi-temporal remote sensing image feature sequences includes: The spatiotemporal features of multi-temporal remote sensing images are arranged into a feature sequence according to time order and input into a time series analysis model. The time series analysis model includes either a long short-term memory network or a gated recurrent unit. By using the time series analysis model, the dependencies between different time steps in the feature sequence are learned to capture the changing patterns of the target region in the time dimension and the changing trends of spatial features over time.

6. The method according to claim 1, characterized in that, The step of using the target trend prediction model to output trend prediction results, determining the trend type by setting a threshold, and drawing a trend graph includes: After undergoing the same preprocessing and feature extraction steps as the training data, the multi-temporal remote sensing image data to be predicted is input into the trained target trend prediction model to obtain the prediction result of the target trend. Based on actual application needs and historical data statistics, set thresholds for different trend types; The prediction results are compared with the set thresholds to determine the type of the target trend; Based on the determination of the type of the target trend, a trend chart is drawn in a visual manner to intuitively show the changing trend of the target area at different times.

7. A remote sensing image target trend detection device, characterized in that, The device includes: The first acquisition module is used to acquire input images, which include multiple consecutive temporal remote sensing images; The preprocessing module is used to preprocess the input image to obtain a preprocessed image; The second acquisition module is used to separate the target region from the preprocessed image based on spectral features and prior geographical location information using a semantic segmentation network; The third acquisition module is used to calculate the pixel-level Wasserstein distance between images using optimal transport theory and the Sinkhorn iterative algorithm, and construct a cost metric function for the optimal transport mapping; obtain a cost matrix of two images based on the cost metric function, the cost matrix including the association calculation results of corresponding pixels in the two images; solve for the optimal transport plan between the two images based on the cost matrix, the optimal transport plan satisfying the optimal solution under specific constraints; and obtain a smoothed feature map of an image based on the solved optimal transport plan. The fourth acquisition module is used to extract texture features based on the feature mapping, combine the local binary mode algorithm to extract texture features and the optical flow method to extract motion features, and fuse the texture features and the motion features to obtain the spatiotemporal features of the target region. The fifth acquisition module is used to capture the spatiotemporal dependencies in the feature sequences of multi-temporal remote sensing images using a time series analysis model; The model building module is used to build a target trend prediction model based on the feature mapping, the spatiotemporal features, and the spatiotemporal dependencies. The prediction module is used to output trend prediction results using the target trend prediction model, determine the trend type by setting a threshold, and draw a trend graph.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more computer programs. The one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

Images