Method for detecting cultivated land change based on cross-modal remote sensing data

Abstract

The present application relates to the field of cultivated land change detection, and particularly relates to a cultivated land change detection method based on cross-modal remote sensing data, which comprises the following steps: constructing a cross-modal data pair based on an event pre-semantic map before a cultivated land change event and a remote sensing image after the event; performing multi-scale feature extraction, cross-modal feature fusion and feature enhancement according to the cross-modal data pair to generate multi-scale cross-modal enhanced features, which are used for cultivated land change detection; the method can effectively overcome the dependence on the remote sensing image before the change event, enhance the utilization of non-image data such as cultivated land vector data, land cover maps and land use maps, and improve the detection accuracy and efficiency of the cultivated land change event.

Description

Technical Field

[0001] This invention relates to the field of farmland change detection, and in particular to a method, apparatus, computer equipment, and storage medium for farmland change detection based on cross-modal remote sensing data. Background Technology

[0002] Farmland is a fundamental prerequisite for ensuring national food security, and rapid and accurate monitoring of farmland use is of great significance. Compared with traditional statistical surveys, emerging remote sensing technology has significant advantages such as wide coverage, high monitoring efficiency, and objectivity and accuracy, and has become an important means of dynamically monitoring farmland changes. In particular, remote sensing image change detection methods can accurately identify changes in the covered farmland by comparing differences between remote sensing images from different time periods.

[0003] However, traditional remote sensing image change detection methods mostly rely on pairs of remote sensing images before and after a change event. This paradigm severely restricts the efficiency and application scenarios of farmland change information identification. Specifically, due to limitations in imaging conditions, sensor performance, and revisit cycles, it is difficult to obtain pairs of remote sensing images before and after a change event in the study area. Secondly, existing methods do not adequately explore massive amounts of farmland vector data, significantly reducing the efficiency of utilizing non-image data. Summary of the Invention

[0004] Based on this, the purpose of this invention is to provide a method, apparatus, computer equipment, and storage medium for detecting farmland change based on cross-modal remote sensing data. The invention constructs cross-modal data pairs based on pre-event semantic maps and post-event remote sensing images of farmland change events. Multi-scale feature extraction, cross-modal feature fusion, and feature enhancement are performed on these cross-modal data pairs to generate multi-scale cross-modal enhanced features for farmland change detection. This effectively overcomes the dependence on pre-event remote sensing images, enhances the utilization of farmland vector data, and improves the detection accuracy and efficiency of farmland change events.

[0005] In a first aspect, embodiments of this application provide a method for detecting farmland changes based on cross-modal remote sensing data, comprising the following steps:

[0006] The system obtains a pre-event semantic map, a post-event remote sensing image, and a farmland change detection model for the target area before and after a farmland change event. The pre-event semantic map consists of vector data, land cover maps, and land use maps used to characterize cover categories. The farmland change detection model includes a feature extraction module, a feature fusion module, a feature enhancement module, and a change prediction module.

[0007] The pre-event semantic map and the post-event remote sensing image are input into the feature extraction module for multi-scale feature extraction to obtain depth features of the pre-event semantic map at several scales and depth features of the post-event remote sensing image at several scales.

[0008] The depth features of several scales of the semantic map before the event and the depth features of several scales of the remote sensing image after the event are input into the feature fusion module to perform cross-modal feature fusion to obtain cross-modal fused features of several scales.

[0009] The cross-modal fusion features at several scales are input into the feature enhancement module for feature enhancement to obtain cross-modal enhanced features at several scales;

[0010] The cross-modal enhancement features at several scales are input into the change prediction module to predict the change probability, thereby obtaining change probability maps at several scales. Based on the change probability maps at several scales, the change probability maps are fused and farmland change is detected to obtain the farmland change detection results for the target area.

[0011] Secondly, embodiments of this application provide a farmland change detection device based on cross-modal remote sensing data, comprising:

[0012] The data acquisition module is used to obtain a pre-event semantic map, a post-event remote sensing image, and a farmland change detection model for the target area before and after a farmland change event. The pre-event semantic map consists of vector data, land cover maps, and land use maps used to characterize cover categories. The farmland change detection model includes a feature extraction module, a feature fusion module, a feature enhancement module, and a change prediction module.

[0013] The multi-scale feature extraction module is used to input the pre-event semantic map and the post-event remote sensing image into the feature extraction module for multi-scale feature extraction, so as to obtain the depth features of the pre-event semantic map at several scales and the depth features of the post-event remote sensing image at several scales.

[0014] The cross-modal feature fusion module is used to input feature extraction maps of several scales of the pre-event semantic map and feature extraction maps of several scales of the post-event remote sensing image into the feature fusion module to perform cross-modal feature fusion and obtain cross-modal fused features of several scales.

[0015] A cross-modal feature enhancement module is used to input the cross-modal fused features at several scales into the feature enhancement module for feature enhancement, thereby obtaining cross-modal enhanced features at several scales;

[0016] The farmland change detection module is used to input the cross-modal enhanced features at several scales into the change prediction module to predict the change probability and obtain change probability maps at several scales; and to perform fusion and farmland change detection based on the change probability maps at several scales to obtain the farmland change detection results of the target area.

[0017] Thirdly, embodiments of this application provide a computer device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method for detecting farmland changes based on cross-modal remote sensing data as described in the first aspect.

[0018] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the steps of the farmland change detection method based on cross-modal remote sensing data as described in the first aspect.

[0019] In this application embodiment, a method, apparatus, computer equipment, and storage medium for detecting farmland change based on cross-modal remote sensing data are provided. Cross-modal data pairs are constructed based on pre-event semantic maps and post-event remote sensing images of farmland change events. Multi-scale feature extraction, cross-modal feature fusion, and feature enhancement are performed on the cross-modal data pairs to generate multi-scale cross-modal enhanced features for farmland change detection. This can effectively overcome the dependence on pre-event remote sensing images, enhance the utilization of farmland vector data, and improve the detection accuracy and efficiency of farmland change events.

[0020] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0021] Figure 1 A flowchart illustrating a method for detecting farmland changes based on cross-modal remote sensing data, provided in one embodiment of this application;

[0022] Figure 2 A flowchart illustrating step S2 of a method for detecting farmland changes based on cross-modal remote sensing data, provided in one embodiment of this application;

[0023] Figure 3 A flowchart illustrating step S3 of a method for detecting farmland changes based on cross-modal remote sensing data, provided in one embodiment of this application;

[0024] Figure 4 A flowchart illustrating step S4 of a method for detecting farmland changes based on cross-modal remote sensing data, provided in one embodiment of this application;

[0025] Figure 5A flowchart illustrating step S5 of a method for detecting farmland changes based on cross-modal remote sensing data, provided in one embodiment of this application;

[0026] Figure 6 A flowchart illustrating step S6 of a method for detecting farmland changes based on cross-modal remote sensing data, provided in another embodiment of this application;

[0027] Figure 7 A flowchart illustrating step S6 of the method for detecting farmland changes based on cross-modal remote sensing data, provided in yet another embodiment of this application;

[0028] Figure 8 A schematic diagram of the structure of a farmland change detection device based on cross-modal remote sensing data provided in one embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0031] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0033] Please see Figure 1 , Figure 1 The following is a flowchart illustrating a method for detecting farmland change based on cross-modal remote sensing data, provided in one embodiment of this application. The method includes the following steps:

[0034] S1: Obtain the pre-event semantic map, post-event remote sensing imagery, and farmland change detection model of the target area before and after the farmland change event.

[0035] The execution subject of the farmland change detection method based on cross-modal remote sensing data is the detection equipment (hereinafter referred to as the detection equipment) of the farmland change detection method based on cross-modal remote sensing data. In an optional embodiment, the detection equipment can be a computer device, a server, or a server cluster composed of multiple computer devices.

[0036] In this embodiment, the detection device obtains a pre-event semantic map, a post-event remote sensing image, and a farmland change detection model for the target area before and after a farmland change event. The pre-event semantic map consists of vector data, land cover maps, and land use maps used to characterize cover categories. The farmland change detection model includes a feature extraction module, a feature fusion module, a feature enhancement module, and a change prediction module.

[0037] S2: Input the pre-event semantic map and the post-event remote sensing image into the feature extraction module for multi-scale feature extraction to obtain depth features of the pre-event semantic map at several scales and depth features of the post-event remote sensing image at several scales.

[0038] In this embodiment, the detection device inputs the pre-event semantic map and the post-event remote sensing image into the feature extraction module for multi-scale feature extraction, thereby obtaining depth features of several scales of the pre-event semantic map and depth features of several scales of the post-event remote sensing image.

[0039] Please see Figure 2 , Figure 2 The flowchart of step S2 in the method for detecting farmland change based on cross-modal remote sensing data provided in one embodiment of this application includes steps S21 to S22, as follows:

[0040] S21: Perform multi-scale feature extraction on the pre-event semantic map according to the preset semantic map encoder to obtain depth features of the pre-event semantic map at several scales.

[0041] Considering that the semantic map only represents the classification value of land cover categories, in this embodiment, the detection device can use four convolutional blocks as different levels of encoding layers of the semantic map encoder. The detection device uses the semantic map encoder to perform multi-scale feature extraction on the pre-event semantic map to obtain depth features at four scales: 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the spatial size of the pre-event semantic map.

[0042] S22: Perform multi-scale feature extraction on the post-event remote sensing image according to the preset remote sensing image encoder to obtain depth features of the post-event remote sensing image at several scales.

[0043] Considering that remote sensing images contain rich spectral and texture information, in this embodiment, the detection device can use the SegFormer network as a remote sensing image encoder. The detection device uses the remote sensing image encoder to perform multi-scale feature extraction on the post-event remote sensing image to obtain depth features at four scales: 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the spatial size of the post-event remote sensing image.

[0044] It should be noted that the semantic map encoder and remote sensing image encoder mentioned in this embodiment are not limited to a specific model.

[0045] S3: Input the depth features of several scales of the semantic map before the event and the depth features of several scales of the remote sensing image after the event into the feature fusion module to perform cross-modal feature fusion and obtain cross-modal fused features of several scales.

[0046] In this embodiment, the detection device inputs several scales of depth features from the pre-event semantic map and several scales of depth features from the post-event remote sensing image into the feature fusion module to perform cross-modal feature fusion, thereby obtaining cross-modal fused features at several scales.

[0047] Please see Figure 3 , Figure 3 The flowchart of step S3 in the farmland change detection method based on cross-modal remote sensing data provided in one embodiment of this application includes steps S31 to S32, as follows:

[0048] S31: Channel stitching is performed based on the depth features of the pre-event semantic map and the depth features of the post-event remote sensing image at the same scale to obtain semantic map-remote sensing image temporal features at several scales.

[0049] In this embodiment, the detection device performs channel stitching based on the depth features of the pre-event semantic map and the depth features of the post-event remote sensing image at the same scale to obtain semantic map-remote sensing image temporal features at several scales.

[0050] S32: Aggregate spatial-temporal neighborhood information based on the temporal features of the semantic map-remote sensing image at several scales to obtain the cross-modal fusion features at several scales.

[0051] In this embodiment, the detection device aggregates spatial-temporal neighborhood information based on the temporal features of the semantic map-remote sensing image at several scales to obtain the cross-modal fusion features at several scales.

[0052] Specifically, the detection equipment uses a first three-dimensional convolutional network ( - - - ), second three-dimensional convolutional network ( - - - ) and two-dimensional convolutional networks ( - - - By aggregating neighborhood information in the spatial and temporal dimensions of the temporal features of the semantic map-remote sensing image at several scales, the neighborhood information of cross-modal data can be fully aggregated in the spatial and temporal dimensions, thereby more effectively modeling the change relationship between the semantic map and the remote sensing image, capturing the difference information between different time phases, and obtaining the cross-modal fusion features at several scales.

[0053] S4: Input the cross-modal fusion features at several scales into the feature enhancement module for feature enhancement to obtain cross-modal enhanced features at several scales.

[0054] In this embodiment, the detection device inputs the cross-modal fusion features at several scales into the feature enhancement module for feature enhancement to obtain cross-modal enhanced features at several scales;

[0055] Please see Figure 4 , Figure 4 The flowchart of step S4 in the farmland change detection method based on cross-modal remote sensing data provided in one embodiment of this application includes steps S41 to S42, as follows:

[0056] S41: Using a channel attention mechanism, adaptive weight calculation is performed on the cross-modal fusion features at several scales to obtain adaptive weight parameters at several scales; the cross-modal fusion features at the same scale are multiplied by the adaptive weight parameters to obtain channel-weighted fusion features at several scales; the cross-modal fusion features at the same scale are residually connected with the channel-weighted fusion features to obtain cross-modal channel enhancement features at several scales.

[0057] In this embodiment, the detection device employs a channel attention mechanism to adaptively calculate the weights of the cross-modal fusion features at several scales, thereby obtaining adaptive weight parameters at several scales to enhance the model's perception of interest channels.

[0058] The detection device multiplies the cross-modal fusion features at the same scale with the adaptive weight parameters to obtain channel-weighted fusion features at several scales. It then performs residual connections between the cross-modal fusion features at the same scale and the channel-weighted fusion features to obtain cross-modal channel-enhanced features at several scales. By using residual connections, the original information is preserved, thereby improving the accuracy of the enhanced features.

[0059] S42: Employ a global self-attention mechanism to perform global self-attention weighting on cross-modal channel enhancement features at several scales to obtain globally weighted features at several scales; perform residual connection between the cross-modal channel enhancement features at the same scale and the globally weighted features to obtain cross-modal enhancement features at several scales.

[0060] In this embodiment, the detection device employs a global self-attention mechanism. Based on a preset global weighted feature extraction algorithm, it performs global self-attention weighting on cross-modal channel enhancement features at several scales to obtain globally weighted features at several scales. The global weighted feature extraction algorithm is as follows:

[0061]

[0062]

[0063] In the formula, For globally weighted features, To query features, Key features, For value characteristics, For dimension parameters, It is the transpose symbol. To enhance features across modal channels, For normalized exponential functions, For the first convolution function, For the second convolution function, This is the third convolution function.

[0064] The detection device performs residual connections between the cross-modal channel enhancement features at the same scale and the global weighted features to obtain cross-modal enhancement features at several scales. This captures long-range dependencies in the spatial dimension to improve the model's attention to changing regions and preserves the original information through residual connections, further enhancing the robustness of the enhanced features.

[0065] S5: Input the cross-modal enhancement features at several scales into the change prediction module to predict the change probability and obtain change probability maps at several scales; perform fusion and farmland change detection based on the change probability maps at several scales to obtain the farmland change detection results of the target area.

[0066] In this embodiment, the detection device inputs the cross-modal enhancement features at several scales into the change prediction module to predict the change probability and obtain a change probability map at several scales.

[0067] Specifically, the change prediction module employs a change output head, which is composed of a multilayer sensing mechanism. The detection device uses the change output head to predict the change probability of the cross-modal enhancement features at several scales, thereby obtaining change probability maps at several scales.

[0068] The detection equipment fuses and detects farmland changes based on the change probability maps at several scales to obtain farmland change detection results for the target area. It constructs cross-modal data pairs based on the pre-event semantic map and post-event remote sensing images before and after the farmland change event. Based on the cross-modal data pairs, it performs multi-scale feature extraction, cross-modal feature fusion, and feature enhancement to generate multi-scale cross-modal enhanced features for farmland change detection. This can effectively overcome the dependence on pre-event remote sensing images, enhance the utilization of farmland vector data, and improve the detection accuracy and efficiency of farmland change events.

[0069] Please see Figure 5 , Figure 5 The flowchart of step S5 in the farmland change detection method based on cross-modal remote sensing data provided in one embodiment of this application includes steps S51 to S52, as follows:

[0070] S51: Upsample the change probability maps at several scales to obtain upsampled change probability maps at several scales; stitch the upsampled change probability maps at several scales along the channel dimension to obtain a multi-scale change probability stitched map.

[0071] In this embodiment, the detection device upsamples the change probability map at several scales to obtain the upsampled change probability map at several scales, and restores it to the input image size in the spatial dimension.

[0072] The detection device stitches together the upsampled probability maps of several scales along the channel dimension to obtain a multi-scale probability map.

[0073] S52: The multi-scale change probability stitched image is input into a preset two-dimensional convolutional network and processed sequentially through a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation function layer, and a Sigmoid normalization function to obtain a fused change probability prediction image; the fused change probability prediction image is binarized, and farmland change detection is performed based on the pixel values ​​in the fused change probability prediction image and a preset change threshold to obtain a pixel-level binary change image.

[0074] In this embodiment, the detection device stitches together the upsampled probability maps of several scales along the channel dimension to obtain a multi-scale probability stitched map. The multi-scale probability stitched map is then input into a preset two-dimensional convolutional network and processed sequentially through a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation function layer, and a Sigmoid normalization function to obtain a fused probability prediction map.

[0075] The detection device performs binarization processing on the fused change probability prediction map, and performs farmland change detection based on the pixel values ​​in the fused change probability prediction map and a preset change threshold to obtain a pixel-level binary change map.

[0076] Specifically, for each pixel in the fused change probability prediction map, if the pixel value is greater than the change threshold, the pixel is determined to be "farmland changed"; conversely, if the pixel value is less than or equal to the change threshold, the pixel is determined to be "farmland unchanged". The change threshold can be set to 0.5. This method can explicitly avoid dependence on remote sensing imagery prior to the change event, enhance the utilization efficiency of non-image semantic map data, and improve the accuracy and efficiency of farmland change event identification.

[0077] In an optional embodiment, step S6 is further included: training the farmland change detection model. See also... Figure 6 , Figure 6 The flowchart of step S6 in the method for detecting farmland change based on cross-modal remote sensing data provided in another embodiment of this application includes steps S61 to S63 as follows:

[0078] S61: Obtain several first sample data groups and corresponding label data of the sample region.

[0079] In this embodiment, the detection device obtains several first sample data groups and corresponding label data of the sample area, wherein the first sample data group includes a semantic map before the event and a remote sensing image after the event.

[0080] S62: Input several sets of first sample data into the farmland change detection model to be trained to detect farmland changes and obtain the corresponding fused change probability prediction map of several sets of first sample data.

[0081] In this embodiment, the detection device inputs several first sample data sets into the cultivated land change detection model to be trained to detect cultivated land changes and obtains the corresponding fused change probability prediction map of several first sample data sets.

[0082] S63: Based on the corresponding fusion change probability prediction map, label data, and preset joint loss function of several first sample data groups, obtain several joint loss values, and train the farmland change detection model to be trained based on the several joint loss values.

[0083] The joint loss function is:

[0084]

[0085] In the formula, For the joint loss value, For label data, This is a prediction graph for the probability of fusion changes.

[0086] In this embodiment, the detection device obtains several joint loss values ​​based on the corresponding fusion change probability prediction maps, label data, and preset joint loss functions of several first sample data groups, and trains the farmland change detection model to be trained based on the several joint loss values.

[0087] Please see Figure 7 , Figure 7 The flowchart of step S6 in the method for detecting farmland change based on cross-modal remote sensing data provided in another embodiment of this application also includes steps S64 to S68 as follows:

[0088] S64: Obtain the second sample data set for several covered categories of the sample region in the current iteration, along with the corresponding label data.

[0089] In this embodiment, the detection device obtains a second sample data set of several coverage categories of the sample area in the current iteration and corresponding label data, wherein the second sample data set includes a semantic map before the event and remote sensing imagery after the event.

[0090] S65: Based on the label data of the second sample data group of several coverage categories in the current iteration, perform a common masking on the corresponding pre-event semantic map and post-event remote sensing image to obtain the semantic labels of several coverage categories in the current iteration; downsample the semantic labels of several coverage categories to obtain the semantic feature vectors of several coverage categories in the current iteration.

[0091] In this embodiment, the detection device performs a joint masking on the corresponding pre-event semantic map and post-event remote sensing image based on the label data of the second sample data group of several coverage categories in the current iteration, and obtains the semantic labels of several coverage categories in the current iteration.

[0092] The detection device downsamples the semantic labels of several coverage categories to obtain semantic feature vectors of several coverage categories in the current iteration.

[0093] S66: Based on the semantic feature vectors of several covered categories in the current iteration, update the corresponding prototype feature vectors of the covered categories in the preset dynamic memory of covered category prototype features to obtain the updated dynamic memory of covered category prototype features for the current iteration.

[0094] In this embodiment, the detection device updates the corresponding prototype feature vectors of the covered categories in the preset dynamic memory of covered category prototype features based on the semantic feature vectors of several covered categories in the current iteration, so as to obtain the updated dynamic memory of covered category prototype features for the current iteration.

[0095] S67: Randomly sample the semantic feature vectors of several covered categories in the current iteration, and use the randomly sampled semantic feature vectors as query vectors to obtain the query vectors of several covered categories in the current iteration; obtain the global semantic alignment loss value based on the query vectors of several covered categories in the current iteration, the prototype feature vectors of several covered categories in the updated prototype feature dynamic memory of the covered categories in the current iteration, and the preset global semantic alignment loss function.

[0096] In this embodiment, the detection device randomly samples the semantic feature vectors of several covered categories in the current iteration, and uses the randomly sampled semantic feature vectors as query vectors to obtain query vectors for several covered categories in the current iteration.

[0097] The detection device obtains a global semantic alignment loss value based on the query vectors of several coverage categories in the current iteration, the prototype feature vectors of several coverage categories in the updated prototype feature dynamic memory of the current iteration, and a preset global semantic alignment loss function. The global semantic alignment loss function is as follows:

[0098]

[0099] In the formula, This represents the global semantic alignment loss value. The set of covered categories in the current iteration. Indicates the first Coverage categories, For the first Query vectors covering each category, For the first Prototype feature vectors covering each category, Indicates the relationship with the first Different coverage categories In order to be with the first Prototype feature vectors for each coverage category with different coverage categories. In order to be with the first A set of different coverage categories. This is the temperature coefficient.

[0100] S68: Train the farmland change detection model to be trained based on several of the joint loss values ​​and the global semantic alignment loss value.

[0101] In this embodiment, the detection device trains the farmland change detection model to be trained based on several joint loss values ​​and the global semantic alignment loss value. Under the guidance of change priors, the semantic consistency of the coverage categories is constrained, thereby improving the accuracy of the farmland change detection model in detecting farmland changes.

[0102] Please refer to Figure 8 , Figure 8 This is a schematic diagram of a farmland change detection device based on cross-modal remote sensing data according to an embodiment of this application. This device can be implemented in whole or in part through software, hardware, or a combination of both. The farmland change detection device 8 includes:

[0103] The data acquisition module 81 is used to acquire a pre-event semantic map, a post-event remote sensing image, and a farmland change detection model for the target area before and after a farmland change event. The pre-event semantic map consists of vector data, a land cover map, and a land use map used to characterize the cover category. The farmland change detection model includes a feature extraction module, a feature fusion module, a feature enhancement module, and a change prediction module.

[0104] The multi-scale feature extraction module 82 is used to input the pre-event semantic map and the post-event remote sensing image into the feature extraction module for multi-scale feature extraction, so as to obtain the depth features of the pre-event semantic map at several scales and the depth features of the post-event remote sensing image at several scales.

[0105] The cross-modal feature fusion module 83 is used to input the depth features of several scales of the semantic map before the event and the depth features of several scales of the remote sensing image after the event into the feature fusion module to perform cross-modal feature fusion and obtain cross-modal fused features of several scales.

[0106] The cross-modal feature enhancement module 84 is used to input the cross-modal fused features at several scales into the feature enhancement module for feature enhancement, thereby obtaining cross-modal enhanced features at several scales;

[0107] The farmland change detection module 85 is used to input the cross-modal enhanced features at several scales into the change prediction module to predict the change probability and obtain change probability maps at several scales; and to perform fusion and farmland change detection based on the change probability maps at several scales to obtain the farmland change detection results of the target area.

[0108] In this embodiment, a data acquisition module obtains a pre-event semantic map, a post-event remote sensing image, and a farmland change detection model for the target area before and after a farmland change event. The pre-event semantic map consists of vector data, a land cover map, and a land use map used to characterize land cover categories. The farmland change detection model includes a feature extraction module, a feature fusion module, a feature enhancement module, and a change prediction module. A multi-scale feature extraction module inputs the pre-event semantic map and the post-event remote sensing image into the feature extraction module for multi-scale feature extraction, obtaining depth features at several scales of the pre-event semantic map and the post-event remote sensing image. A cross-modal feature fusion module... The depth features at several scales of the pre-event semantic map and the depth features at several scales of the post-event remote sensing image are input into the feature fusion module for cross-modal feature fusion to obtain cross-modal fused features at several scales. The cross-modal feature enhancement module then inputs these cross-modal fused features at several scales into the feature enhancement module for feature enhancement to obtain cross-modal enhanced features at several scales. The farmland change detection module inputs these cross-modal enhanced features at several scales into the change prediction module for change probability prediction to obtain change probability maps at several scales. Based on the change probability maps at several scales, fusion and farmland change detection are performed to obtain the farmland change detection results for the target area. Based on the semantic maps before and after the farmland change event, and the remote sensing images after the event, cross-modal data pairs are constructed. Multi-scale feature extraction, cross-modal feature fusion, and feature enhancement are performed on the cross-modal data pairs to generate multi-scale cross-modal enhanced features for farmland change detection. This can effectively overcome the dependence on remote sensing images before the change event, enhance the utilization of farmland vector data, and improve the detection accuracy and efficiency of farmland change events.

[0109] Please refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. The computer device 9 includes: a processor 91, a memory 92, and a computer program 93 stored in the memory 92 and executable on the processor 91; the computer device can store multiple instructions, which are adapted to be loaded and executed by the processor 91. Figures 1 to 7 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figures 1 to 7The specific details of the illustrated embodiments will not be elaborated here.

[0110] The processor 91 may include one or more processing cores. The processor 91 connects to various parts of the server using various interfaces and lines, and executes various functions and processes data of the farmland change detection device 6 based on cross-modal remote sensing data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 92, and by calling data from the memory 92. Optionally, the processor 91 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 91 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 91 and may be implemented as a separate chip.

[0111] The memory 92 may include random access memory (RAM) or read-only memory. Optionally, the memory 92 may include a non-transitory computer-readable storage medium. The memory 92 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 92 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing operation types, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 92 may also be at least one storage device located remotely from the aforementioned processor 91.

[0112] This application embodiment also provides a storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described above. Figures 1 to 7 For specific steps and execution processes of the illustrated embodiment, please refer to [link / reference]. Figures 1 to 7The specific details of the illustrated embodiments will not be elaborated here.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0114] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the algorithm. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0116] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0119] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0120] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.

Claims

1. A method for detecting farmland change based on cross-modal remote sensing data, characterized in that, Includes the following steps: The system obtains a pre-event semantic map, a post-event remote sensing image, and a farmland change detection model for the target area before and after a farmland change event. The pre-event semantic map consists of vector data, land cover maps, and land use maps used to characterize cover categories. The farmland change detection model includes a feature extraction module, a feature fusion module, a feature enhancement module, and a change prediction module. The pre-event semantic map and the post-event remote sensing image are input into the feature extraction module for multi-scale feature extraction to obtain depth features of the pre-event semantic map at several scales and depth features of the post-event remote sensing image at several scales. The depth features of several scales of the semantic map before the event and the depth features of several scales of the remote sensing image after the event are input into the feature fusion module to perform cross-modal feature fusion to obtain cross-modal fused features of several scales. The cross-modal fusion features at several scales are input into the feature enhancement module. A channel attention mechanism is used to adaptively calculate the weights of the cross-modal fusion features at several scales to obtain adaptive weight parameters at several scales. The cross-modal fusion features at the same scale are multiplied by the adaptive weight parameters to obtain channel-weighted fusion features at several scales. The cross-modal fusion features at the same scale are residually connected with the channel-weighted fusion features to obtain cross-modal channel enhancement features at several scales. A global self-attention mechanism is employed to perform global self-attention weighting on cross-modal channel enhancement features at several scales to obtain globally weighted features at several scales; the cross-modal channel enhancement features at the same scale are then residually connected with the globally weighted features to obtain cross-modal enhancement features at several scales. The cross-modal enhancement features at several scales are input into the change prediction module to predict the change probability, thereby obtaining change probability maps at several scales. Based on the change probability maps at several scales, the change probability maps at several scales are upsampled to obtain upsampled change probability maps at several scales. The upsampled change probability maps at several scales are then stitched together along the channel dimension to obtain a multi-scale change probability stitched map. The multi-scale change probability mosaic is input into a preset two-dimensional convolutional network and processed sequentially through a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation function layer, and a Sigmoid normalization function to obtain a fused change probability prediction map. The fused change probability prediction map is then binarized, and farmland change detection is performed based on the pixel values ​​in the fused change probability prediction map and a preset change threshold to obtain a pixel-level binary change map, thus obtaining the farmland change detection result for the target area.

2. The method for detecting farmland change based on cross-modal remote sensing data according to claim 1, characterized in that, The step of inputting the pre-event semantic map and the post-event remote sensing image into the feature extraction module for multi-scale feature extraction to obtain depth features at several scales of the pre-event semantic map and the post-event remote sensing image includes the following steps: The pre-event semantic map is subjected to multi-scale feature extraction based on a preset semantic map encoder to obtain depth features of the pre-event semantic map at several scales. The post-event remote sensing image is subjected to multi-scale feature extraction based on a preset remote sensing image encoder to obtain depth features of the post-event remote sensing image at several scales.

3. The method for detecting farmland change based on cross-modal remote sensing data according to claim 1, characterized in that, The step of inputting depth features at several scales from the pre-event semantic map and depth features at several scales from the post-event remote sensing image into the feature fusion module for cross-modal feature fusion to obtain cross-modal fused features at several scales includes the following steps: Channel stitching is performed on the depth features of the pre-event semantic map and the post-event remote sensing image at the same scale to obtain semantic map-remote sensing image temporal features at several scales. Based on the temporal features of the semantic map-remote sensing image at several scales, spatial-temporal neighborhood information is aggregated to obtain the cross-modal fusion features at several scales.

4. The method for detecting farmland change based on cross-modal remote sensing data according to claim 1, characterized in that, The method also includes the step of training the farmland change detection model. The training of the farmland change detection model includes the following steps: Several first sample data groups and corresponding label data of the sample area are obtained, wherein the first sample data group includes a semantic map before the event and a remote sensing image after the event. Several sets of first sample data are input into the farmland change detection model to be trained to detect farmland changes and obtain the corresponding fused change probability prediction map of several sets of first sample data. Based on the fused change probability prediction maps, label data, and a preset joint loss function corresponding to several first sample data sets, several joint loss values ​​are obtained. The farmland change detection model to be trained is then trained based on these joint loss values, wherein the joint loss function is: In the formula, For the joint loss value, For label data, This is a prediction graph for the probability of fusion changes.

5. The method for detecting farmland change based on cross-modal remote sensing data according to claim 4, characterized in that, The training of the farmland change detection model further includes the following steps: Obtain a second set of sample data for several coverage categories of the sample region in the current iteration, along with corresponding label data. The second set of sample data includes a semantic map before the event and remote sensing imagery after the event. Based on the label data of the second sample data group of several coverage categories in the current iteration, a common mask is applied to the corresponding pre-event semantic map and post-event remote sensing image to obtain the semantic labels of several coverage categories in the current iteration; the semantic labels of several coverage categories are downsampled to obtain the semantic feature vectors of several coverage categories in the current iteration. Based on the semantic feature vectors of several covered categories in the current iteration, the corresponding prototype feature vectors of the covered categories in the preset dynamic memory of covered category prototype features are updated to obtain the updated dynamic memory of covered category prototype features for the current iteration. The semantic feature vectors of several covered categories in the current iteration are randomly sampled, and the randomly sampled semantic feature vectors are used as query vectors to obtain query vectors for several covered categories in the current iteration. Based on the query vectors of several covered categories in the current iteration, the prototype feature vectors of several covered categories in the updated prototype feature dynamic memory of the covered categories in the current iteration, and a preset global semantic alignment loss function, a global semantic alignment loss value is obtained, wherein the global semantic alignment loss function is: In the formula, This represents the global semantic alignment loss value. The set of covered categories in the current iteration. Indicates the first Coverage categories, For the first Query vectors covering each category, For the first Prototype feature vectors covering each category, Indicates the relationship with the first Different coverage categories In order to be with the first Prototype feature vectors for each coverage category with different coverage categories. In order to be with the first A set of different coverage categories. Temperature coefficient; The farmland change detection model to be trained is trained based on several joint loss values ​​and the global semantic alignment loss value.

6. A farmland change detection device based on cross-modal remote sensing data, characterized in that, include: The data acquisition module is used to obtain a pre-event semantic map, a post-event remote sensing image, and a farmland change detection model for the target area before and after a farmland change event. The pre-event semantic map consists of vector data, land cover maps, and land use maps used to characterize cover categories. The farmland change detection model includes a feature extraction module, a feature fusion module, a feature enhancement module, and a change prediction module. The multi-scale feature extraction module is used to input the pre-event semantic map and the post-event remote sensing image into the feature extraction module for multi-scale feature extraction, so as to obtain the depth features of the pre-event semantic map at several scales and the depth features of the post-event remote sensing image at several scales. The cross-modal feature fusion module is used to input the depth features of several scales of the semantic map before the event and the depth features of several scales of the remote sensing image after the event into the feature fusion module to perform cross-modal feature fusion and obtain cross-modal fused features of several scales. A cross-modal feature enhancement module is used to input the cross-modal fusion features at several scales into the feature enhancement module, and to perform adaptive weight calculation on the cross-modal fusion features at several scales using a channel attention mechanism to obtain adaptive weight parameters at several scales; the cross-modal fusion features at the same scale are multiplied by the adaptive weight parameters to obtain channel-weighted fusion features at several scales; the cross-modal fusion features at the same scale are residually connected with the channel-weighted fusion features to obtain cross-modal channel enhancement features at several scales. A global self-attention mechanism is employed to perform global self-attention weighting on cross-modal channel enhancement features at several scales to obtain globally weighted features at several scales; the cross-modal channel enhancement features at the same scale are then residually connected with the globally weighted features to obtain cross-modal enhancement features at several scales. The farmland change detection module is used to input the cross-modal enhanced features at several scales into the change prediction module to predict the change probability and obtain change probability maps at several scales; based on the change probability maps at several scales, the change probability maps at several scales are upsampled to obtain upsampled change probability maps at several scales; and the upsampled change probability maps at several scales are stitched together in the channel dimension to obtain a multi-scale change probability stitched map. The multi-scale change probability mosaic is input into a preset two-dimensional convolutional network and processed sequentially through a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation function layer, and a Sigmoid normalization function to obtain a fused change probability prediction map. The fused change probability prediction map is then binarized, and farmland change detection is performed based on the pixel values ​​in the fused change probability prediction map and a preset change threshold to obtain a pixel-level binary change map, thus obtaining the farmland change detection result for the target area.

7. A computer device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for detecting farmland change based on cross-modal remote sensing data as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for detecting farmland changes based on cross-modal remote sensing data as described in any one of claims 1 to 5.

Images