Cultivated land recognition method and device based on multi-temporal feature fusion

Abstract

The present application relates to the field of artificial intelligence, and provides a cultivated land recognition method and device based on multi-temporal feature fusion, which can construct a time sequence sample set according to remote sensing image data and cultivated land vector graph spot data, solve the problems of inconsistent spatial positions of different time phase images, distorted ground object reflection information and cloud pollution; construct a high-dimensional feature tensor of each sampling point according to the spectral characteristics and spatial texture characteristics of each sampling point, integrate the multi-temporal and multi-modal features, and comprehensively capture the spectral response characteristics and spatial structure information of cultivated land; construct an initial network model including a spatial feature extraction branch, a time sequence feature extraction branch, an attention mechanism module, a feature pyramid network and a feature output layer, solve the problems of serious single time phase feature confusion and insufficient spatio-temporal feature fusion in the cultivated land recognition process; construct a cultivated land priori knowledge graph and a knowledge constraint loss, and improve the accuracy and explainability of cultivated land recognition.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for identifying cultivated land based on multi-temporal feature fusion. Background Technology

[0002] Arable land resources are the foundation of national food security. Existing methods for identifying arable land can be mainly divided into the following categories: methods based on manual visual interpretation, which rely on experienced interpreters to delineate high-resolution images. This method has high accuracy but extremely low efficiency and long cycle, and cannot meet the needs of large-scale, high-frequency monitoring; methods based on traditional machine learning, such as using algorithms like Support Vector Machine (SVM) and Random Forest (RF) to extract spectral and texture features from single-phase images for classification; semantic segmentation methods based on deep learning to automatically extract high-level semantic features from images; and methods based on time series analysis, which identify arable land by analyzing the changing patterns of crop growth cycles.

[0003] Despite the achievements of the above methods, accurate identification of arable land in complex scenarios still faces the following challenges: Relying on single-phase imagery makes it difficult to distinguish between cultivated land and grassland, woodland and wasteland, resulting in numerous instances of "different objects with the same spectrum" and "different spectra with the same object." Existing time-series analysis methods focus on changes in the temporal dimension of cultivated land, neglecting the spatial texture and shape characteristics of land features. This leads to blurred boundaries in the identification results, making it easy to miss small plots. Currently, there is a lack of a mechanism that can efficiently and adaptively fuse spatial texture with phenological time-series features. Furthermore, existing deep learning models are data-driven black-box models, relying entirely on sample training. They fail to transform prior knowledge such as the phenological patterns of cultivated land and the spatial adjacency relationships of land features into constraints for the model, resulting in poor generalization ability and decreased identification performance in areas with cloud or fog interference or insufficient samples. Summary of the Invention

[0004] In view of the above, it is necessary to provide a method and device for farmland identification based on multi-temporal feature fusion, which aims to solve the problems of serious confusion of single-temporal features, insufficient fusion of spatiotemporal features and lack of physical knowledge constraints in the farmland identification process.

[0005] A method for identifying cultivated land based on multi-temporal feature fusion, the method comprising: Collect remote sensing image data and ground-measured cultivated land vector plot data, and construct a time-series sample set including cultivated land and non-cultivated land labels based on the remote sensing image data and the cultivated land vector plot data; Random sampling is performed on the cultivated land area and non-cultivated land area of ​​each time series sample in the time series sample set to obtain multiple sampling points; Extract the spectral and spatial texture features of each sampling point, and construct a high-dimensional feature tensor for each sampling point based on the spectral and spatial texture features of each sampling point; Construct an initial network model; wherein the initial network model includes a spatial feature extraction branch, a temporal feature extraction branch, an attention mechanism module, a feature pyramid network, and a feature output layer; Construct a prior knowledge graph of arable land, and construct a knowledge constraint loss based on the prior knowledge graph of arable land; Based on the knowledge constraint loss, the initial network model is trained using the high-dimensional feature tensor of each sampling point to obtain the farmland identification model. In response to the farmland identification instruction based on the target remote sensing image, the target remote sensing image is processed using the farmland identification model to obtain the farmland identification result of the target remote sensing image.

[0006] A farmland identification device based on multi-temporal feature fusion, the farmland identification device comprising: The construction unit is used to collect remote sensing image data and ground-measured cultivated land vector plot data, and to construct a time-series sample set including cultivated land and non-cultivated land labels based on the remote sensing image data and the cultivated land vector plot data. The sampling unit is used to randomly sample the cultivated land area and non-cultivated land area of ​​each time series sample in the time series sample set to obtain multiple sampling points; The construction unit is also used to extract the spectral features and spatial texture features of each sampling point, and to construct a high-dimensional feature tensor for each sampling point based on the spectral features and spatial texture features of each sampling point. The building unit is also used to build an initial network model; wherein the initial network model includes a spatial feature extraction branch, a temporal feature extraction branch, an attention mechanism module, a feature pyramid network, and a feature output layer; The construction unit is also used to construct a prior knowledge graph of arable land and to construct a knowledge constraint loss based on the prior knowledge graph of arable land. The training unit is used to train the initial network model based on the knowledge constraint loss and the high-dimensional feature tensor of each sampling point to obtain the farmland identification model. The processing unit is configured to respond to a farmland identification instruction based on a target remote sensing image, process the target remote sensing image using the farmland identification model, and obtain the farmland identification result of the target remote sensing image.

[0007] A computer device, the computer device comprising: A memory for storing at least one instruction; and a processor for executing the instructions stored in the memory to implement the farmland identification method based on multi-temporal feature fusion.

[0008] A computer-readable storage medium storing at least one instruction, which is executed by a processor in a computer device to implement the farmland identification method based on multi-temporal feature fusion.

[0009] As can be seen from the above technical solutions, this invention can construct a temporal sample set including labels for cultivated land and non-cultivated land based on remote sensing image data and cultivated land vector patch data, solving the problems of inconsistent spatial locations of images at different time phases, distortion of ground object reflection information, and cloud pollution; it constructs a high-dimensional feature tensor for each sampling point based on the spectral and spatial texture features of each sampling point, integrating multi-temporal and multi-modal features to comprehensively capture the spectral response characteristics and spatial structure information of cultivated land; it constructs an initial network model including spatial feature extraction branches, temporal feature extraction branches, attention mechanism modules, feature pyramid networks, and feature output layers, solving the problems of severe single-temporal feature confusion and insufficient spatiotemporal feature fusion in the cultivated land identification process; and it constructs a prior knowledge graph of cultivated land and knowledge constraint loss, improving the accuracy and interpretability of cultivated land identification. Attached Figure Description

[0010] Figure 1 This is a flowchart of a preferred embodiment of the farmland identification method based on multi-temporal feature fusion of the present invention.

[0011] Figure 2 This is a functional block diagram of a preferred embodiment of the farmland identification device based on multi-temporal feature fusion of the present invention.

[0012] Figure 3 This is a schematic diagram of the structure of a computer device that implements a preferred embodiment of the farmland identification method based on multi-temporal feature fusion according to the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the farmland identification method based on multi-temporal feature fusion of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0015] The farmland identification method based on multi-temporal feature fusion is applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0016] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0017] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0018] The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0019] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0020] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0021] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0022] S10: Collect remote sensing image data and ground-measured cultivated land vector plot data, and construct a time-series sample set including cultivated land and non-cultivated land labels based on the remote sensing image data and the cultivated land vector plot data.

[0023] In this embodiment, the remote sensing image data may include satellite remote sensing data, which may include four bands: red, green, blue, and near-infrared (NIR), with a resolution of 0.75m.

[0024] The time interval for the remote sensing image data can be 10 days, 16 days, etc.

[0025] In this embodiment, constructing a time-series sample set including labels for cultivated land and non-cultivated land based on the remote sensing image data and the cultivated land vector plot data includes: The remote sensing image data is converted to the target geographic coordinate system and adjusted to the target spatial resolution to perform spatiotemporal registration on the remote sensing image data to obtain the first data. The pixel values ​​of the first data are converted into surface reflectance (SR) to perform radiometric calibration and atmospheric correction on the first data, thus obtaining the second data; Based on the cloud detection algorithm and the least squares smoothing filter (Savitzky-Golay, SG), invalid pixel regions in the second data are identified and removed to perform cloud masking on the second data to obtain the third data; The cultivated land area and non-cultivated land area in the third data are marked based on the cultivated land vector plot data; The third data, which is combined with annotations, is used to obtain the time-series sample set.

[0026] By performing spatiotemporal registration, it can be ensured that images taken at different times are perfectly aligned in spatial position.

[0027] Among these methods, radiometric calibration and atmospheric correction can eliminate the influence of atmospheric scattering and absorption on images.

[0028] By performing cloud masking, invalid pixel areas can be eliminated, thus preventing cloud contamination from affecting subsequent recognition results.

[0029] For example, the cloud detection algorithm may include thresholding, machine learning algorithms, etc. The target spatial resolution can be 0.75m. High-resolution data of 0.75m combined with a precise preprocessing process can clearly present the subtle texture and boundary features of farmland, providing data support for subsequent accurate identification.

[0030] The time-series sample set includes different time phases and different regions.

[0031] Among them, real remote sensing time-series data are often missing due to cloud cover. Therefore, algorithms such as linear interpolation and spatiotemporal kriging can be used to supplement the remote sensing image data through interpolation or reconstruction methods to ensure temporal continuity.

[0032] The above embodiments address issues such as inconsistent spatial locations in images from different time periods, distortion of ground feature reflection information, and cloud contamination, providing a high-quality, standardized data foundation for subsequent feature extraction and model training. The labeled time-series sample set covers image features of cultivated land at different growth stages, meeting the model's need for multi-time-series data.

[0033] S11, Randomly sample the cultivated land area and non-cultivated land area of ​​each time series sample in the time series sample set to obtain multiple sampling points.

[0034] In this embodiment, the sampling points need to be evenly distributed to cover the entire image area to ensure the representativeness of feature extraction.

[0035] S12, extract the spectral features and spatial texture features of each sampling point, and construct a high-dimensional feature tensor for each sampling point based on the spectral features and spatial texture features of each sampling point.

[0036] In this embodiment, the step of extracting the spectral features and spatial texture features of each sampling point, and constructing a high-dimensional feature tensor for each sampling point based on the spectral features and spatial texture features of each sampling point includes: The Normalized Difference Vegetation Index (NDVI), Modified Normalized Difference Water Index (MNDWI), and Bare Soil Index (BSI) were calculated for each sampling point. Based on the NDVI, MNDWI, and BSI values ​​of each sampling point, a sequence was constructed to obtain the spectral characteristics of each sampling point. The entropy of each sampling point is calculated using the Gray Level Co-occurrence Matrix (GLCM) as the spatial texture feature of each sampling point; The spectral features and spatial texture features of each sampling point are accumulated in the time dimension to obtain the high-dimensional feature tensor of each sampling point; The dimension of the high-dimensional feature tensor for each sampling point is: time step × number of feature channels × target spatial resolution; The number of feature channels is the sum of the number of spectral feature channels and the number of spatial texture feature channels.

[0037] The reflectance of each sampling point across all composite phases (red, green, blue, and near-infrared bands) can be extracted first, and the normalized vegetation index, normalized differential water index, and bare soil index of each sampling point can be calculated based on the reflectance of each sampling point across all composite phases.

[0038] The larger the entropy value, the more complex the image texture.

[0039] The target spatial resolution can be 1024×1024 pixels.

[0040] The time step refers to the number of time-series images. For example, when there are 4 time phases, the time step is 4.

[0041] When there are 4 reflectivity bands, 3 exponents, and 1 entropy value, for a total of 8 channels, the number of characteristic channels is 8.

[0042] In the above embodiments, spectral features and spatial texture features are extracted as multimodal features, which comprehensively capture the spectral response characteristics and spatial structure information of cultivated land, providing rich feature support for distinguishing cultivated land from non-cultivated land.

[0043] Furthermore, the introduction of various spectral indices can effectively enhance the distinction between cultivated land and different land features such as vegetation, water bodies, and bare soil. For example, a higher NDVI value indicates higher vegetation cover, and the NDVI value of cultivated land during the crop growing season is significantly higher than that of bare soil and built-up areas.

[0044] In addition, the construction of high-dimensional feature tensors can integrate features from multiple time series and multiple modalities, providing structured data input for feature learning of subsequent models.

[0045] S13, Construct an initial network model; wherein the initial network model includes a spatial feature extraction branch, a temporal feature extraction branch, an attention mechanism module, a Feature Pyramid Network (FPN), and a feature output layer.

[0046] In this embodiment, the dual-branch network structure focuses on the extraction of spatial features and temporal features respectively, which can fully explore the texture and shape features of cultivated land in the spatial dimension and the phenological change features in the temporal dimension (such as the cyclical changes of crop sowing, growth and harvesting).

[0047] The introduction of the attention mechanism enables adaptive weighted fusion of spatial and temporal features, automatically adjusting the weights based on the importance of different features to farmland identification. For example, during the crop growing season, temporal features (such as phenological changes) have higher weights; during the non-growing season, spatial features (such as field shape and texture) have higher weights, thereby improving the effectiveness of feature fusion.

[0048] By effectively fusing features at different scales through the feature pyramid network, the problem of missed detection in small plots is solved, the spatial resolution and boundary accuracy of farmland identification are improved, and the preliminary classification probability map can clearly show the distribution range of farmland.

[0049] S14, construct a prior knowledge graph of arable land, and construct a knowledge constraint loss based on the prior knowledge graph of arable land.

[0050] In this embodiment, constructing the prior knowledge graph of arable land includes: Determine entity nodes based on the type of target features; Each entity node is connected to form an edge according to phenological rules and spatial adjacency rules; The prior knowledge graph of cultivated land is generated based on the entity nodes and the edges.

[0051] The entity nodes may include major land features such as farmland, forest land, water bodies, and buildings.

[0052] The phenological rules may include: the NDVI curve of cultivated land exhibits a "double-peak" or "single-peak" shape, and the NDVI value drops sharply in specific harvest months (e.g., the wheat harvest season in northern China is June-July, and the rice harvest season is September-October). For example, in winter wheat planting areas, the NDVI curve shows an upward trend in spring (March-May) (growing season), and drops sharply in June-July during the harvest season, forming a "single-peak" shape; in double-cropping rice planting areas, the NDVI curve rises in May-July (early rice growing season) and August-October (late rice growing season), respectively, and drops sharply in the middle of the harvest season, forming a "double-peak" shape.

[0053] The spatial adjacency rules may include: farmland is often adjacent to irrigation ditches and roads, and is rarely completely surrounded by deep water areas; the boundary between farmland and large bodies of water (such as lakes and rivers) should be clear, the probability of farmland being adjacent to roads is high, but the probability of direct adjacency to densely built-up areas is low; farmland pixels should form continuous areas in space, and isolated small patches (such as areas less than 0.01 hectares) are likely to be misclassified.

[0054] The aforementioned prior knowledge graph of arable land presents prior knowledge such as phenological patterns and spatial adjacency relationships of arable land in a structured manner, making up for the shortcomings of the "black box" characteristics of traditional deep learning models and providing clear logical constraints for model training.

[0055] This embodiment constructs a prior knowledge graph of arable land and maps it to semantic constraint rules introduced into the loss function. This forces the model to follow the objective laws of "crop growth cycle" and "geospatial distribution" during the training process, thereby improving the robustness of the model under noise interference and the logical rationality of the recognition results. This solves the problem of traditional deep learning models lacking geoscientific logical constraints.

[0056] In this embodiment, the step of constructing the knowledge constraint loss based on the prior knowledge graph of cultivated land includes: Obtain the temporal feature vector output by the temporal feature extraction branch, and obtain the predefined standard phenological feature center of cultivated land; Calculate the similarity between the temporal feature vector and the standard cultivated land phenological feature center; The difference between 1 and the similarity is calculated to construct the phenological feature constraint loss; wherein, the phenological feature constraint loss is used to transform the phenological rule into a feature space distance constraint; The timing smoothing constraint loss is calculated using the following formula: ; in, The time series smoothing constraint loss is represented by T; the total number of time steps in the time series sample set is represented by t; This represents the probability diagram of cultivated land classification at time step t+1. A probability diagram representing the classification of cultivated land at time step t; This indicates the indicator function that, when an abnormal jump that violates the rules of the cultivated land prior knowledge graph is detected, Set to 1 when no abnormal jumps violating the rules of the cultivated land prior knowledge graph are detected. Set to 0; Obtain the first weight corresponding to the phenological feature constraint loss and the second weight corresponding to the temporal smoothing constraint loss; The knowledge constraint loss is obtained by weighting the first weight, the second weight, the phenological feature constraint loss, and the temporal smoothing constraint loss.

[0057] Cosine similarity can be used as the similarity between the temporal feature vector and the standard phenological feature center of cultivated land. The closer the cosine similarity is to 1, the better the temporal features extracted by the network match the standard phenological features, and the smaller the loss value.

[0058] By introducing the temporal smoothing constraint loss, unreasonable rapid semantic jumps can be penalized. For example, the NDVI value of cultivated land should not show a sudden and large increase or decrease in adjacent time steps (outside of harvesting period). If such a situation occurs, a higher loss value is given.

[0059] The sum of the first weight and the second weight is 1. The first weight and the second weight are used to balance the importance of phenological characteristic constraints and temporal smoothing constraints.

[0060] The abnormal jumps can be identified by combining statistical methods (such as sliding window standard deviation) or by introducing a learnable anomaly detection module.

[0061] By introducing the aforementioned knowledge constraint loss, the subsequent model must adhere to the objective laws of arable land during training, effectively reducing misclassifications caused by factors such as sample noise and cloud / fog interference, and improving the model's robustness in complex scenarios. Furthermore, the temporal smoothing constraint penalizes unreasonable land cover type jumps, ensuring temporal consistency of recognition results and avoiding unrealistic classifications such as arable land suddenly changing to buildings or water bodies in adjacent time steps.

[0062] S15, based on the knowledge constraint loss, the initial network model is trained using the high-dimensional feature tensor of each sampling point to obtain the farmland identification model.

[0063] In this embodiment, the step of training the initial network model using the high-dimensional feature tensor of each sampling point based on the knowledge constraint loss to obtain the farmland identification model includes: Obtain the classification prediction cross-entropy loss, which measures the similarity between the true labels and the model predictions. Obtain a balance coefficient for adjusting the weights of the classification prediction cross-entropy loss and the knowledge constraint loss; Construct a total loss function based on the balance coefficient, the classification prediction cross-entropy loss, and the knowledge constraint loss; With the goal of minimizing the total loss function, the initial network model is trained based on the high-dimensional feature tensor of each sampling point; When the total loss function no longer decreases, the currently obtained model is determined as the farmland identification model.

[0064] For example, the true label for arable land can be 1, and the true label for non-arable land can be 0.

[0065] The configuration of the balance coefficients ensures that the model can accurately fit the sample data while also adhering to prior knowledge rules.

[0066] In the above embodiments, the total loss function combines classification loss and knowledge constraint loss, enabling the model to learn the features of sample data while following the objective laws of cultivated land, effectively improving the model's generalization ability, and maintaining high recognition accuracy even in areas with cloud and fog interference or insufficient samples.

[0067] In this embodiment, the high-dimensional feature tensor of each sampling point can be divided into a training set, a validation set, and a test set according to a certain ratio (e.g., 7:2:1). The initial network model's weight matrix, bias vector, and other parameters are initialized, and training hyperparameters are set, including the learning rate (e.g., 0.001), the number of iterations (e.g., 100 rounds), and the batch size (e.g., 32). Furthermore, stochastic gradient descent or adaptive moment estimation optimization algorithms are used to minimize the total loss function, and the model parameters are continuously updated through backpropagation. During each iteration, the model performance is evaluated using the validation set (e.g., calculating accuracy, recall, and F1 score). When the validation set performance no longer improves for several consecutive rounds (e.g., 10 rounds), training is stopped, and the optimal model parameters are saved.

[0068] The final model also needs to meet the farmland-specific indicators such as ensuring the integrity of the plots and the consistency of the boundaries.

[0069] Reasonable dataset partitioning and hyperparameter settings ensure the stability and effectiveness of model training, avoiding overfitting or underfitting problems.

[0070] S16, in response to the farmland identification instruction based on the target remote sensing image, the target remote sensing image is processed using the farmland identification model to obtain the farmland identification result of the target remote sensing image.

[0071] In this embodiment, the farmland identification command can be automatically triggered when the target remote sensing image is detected to be uploaded to a designated platform.

[0072] In this embodiment, processing the target remote sensing image using the farmland identification model to obtain the farmland identification result of the target remote sensing image includes: Construct a high-dimensional feature tensor for each time step corresponding to the target remote sensing image; In the spatial feature extraction branch, convolution operations are used to perform independent convolution operations on the high-dimensional feature tensor at each time step to obtain the target spatial feature map; In the temporal feature extraction branch, a bidirectional convolutional long short-term memory (Bi-ConvLSTM) network is used to capture the dependencies between the high-dimensional feature tensors of each time step and the time sequence, thereby obtaining the target temporal feature vector. In the attention mechanism module, the target temporal feature vector is expanded into a target feature map with the same number of channels as the target spatial feature map through a convolutional layer, and the target feature map is upsampled to the same size as the target spatial feature map using bilinear interpolation to obtain the target temporal feature map; The target spatial feature map and the target temporal feature map are added element by element to obtain preliminary fused features; The preliminary fusion features are processed by global average pooling to generate channel descriptors; The channel descriptors are input into a fully connected network to generate channel weights; The preliminary fusion features are reweighted based on the channel weights to obtain the target fusion features; Using the feature pyramid network as a decoder, the target fusion features and the target spatial feature map are fused through lateral connections, and the images are gradually upsampled to restore the original resolution of the target remote sensing image to obtain intermediate features; The intermediate features are input into the feature output layer to obtain the classification probability map of cultivated land and non-cultivated land for each pixel in the target remote sensing image, which is used as the cultivated land identification result. The feature output layer includes a convolutional normalization layer, a rectified linear unit (ReLU) function, and a normalized exponential function (Softmax function).

[0073] In the process of performing independent convolution operations, different sizes of convolution kernels (such as 3×3 and 5×5) and the number of convolution layers can be set to gradually extract low-level spatial features (such as edges and textures) and high-level spatial features (such as field shapes and spatial distribution patterns) of cultivated land.

[0074] The last hidden state of the bidirectional convolutional long short-term memory network is the target temporal feature vector.

[0075] Specifically, the target temporal feature vector can be expanded into a target feature map with the same number of channels as the target spatial feature map through a 1×1 convolutional layer.

[0076] The channel descriptor can be input into a two-layer fully connected network (with the hidden layer using the ReLU activation function) to generate the channel weights.

[0077] The farmland identification result is a pixel-level probability map of farmland and non-farmland classification in the area to be tested, with a probability value range of [0,1]. The closer the probability value is to 1, the greater the probability that the pixel is farmland. For example, a probability threshold (such as 0.5) can be configured to classify pixels with a probability value greater than the threshold as farmland and pixels with a probability value less than or equal to the threshold as non-farmland, ultimately generating a farmland distribution map and obtaining the farmland identification result.

[0078] This embodiment constructs a multimodal feature tensor that includes spectrum, spatial texture, and phenological time series, and uses an attention mechanism to adaptively weight and fuse features of different modalities. This effectively distinguishes land features with similar spectra but different growth patterns at the feature level, solving the problem of easy confusion between cultivated land and non-cultivated land in complex scenarios.

[0079] In this embodiment, after obtaining the farmland identification results from the target remote sensing image, the identification results can be optimized by combining morphological post-processing (such as erosion and dilation operations) with farmland continuity rules. Specifically, erosion operations can remove small noise points (such as isolated small patches of farmland) from the identification results, and dilation operations can fill in small voids within the farmland area, ensuring the continuity and integrity of the farmland area and further improving the practicality of the farmland distribution map.

[0080] The combination of morphological post-processing and farmland continuity rules eliminates noise and foreign object areas in the identification results, solves the problems of fragmentation and irregular boundaries in farmland identification results, and improves the completeness, accuracy and practicality of the final farmland distribution map, which can provide reliable data support for farmland supervision, food security assessment and other work.

[0081] As can be seen from the above technical solutions, this invention can construct a temporal sample set including labels for cultivated land and non-cultivated land based on remote sensing image data and cultivated land vector patch data, solving the problems of inconsistent spatial locations of images at different time phases, distortion of ground object reflection information, and cloud pollution; it constructs a high-dimensional feature tensor for each sampling point based on the spectral and spatial texture features of each sampling point, integrating multi-temporal and multi-modal features to comprehensively capture the spectral response characteristics and spatial structure information of cultivated land; it constructs an initial network model including spatial feature extraction branches, temporal feature extraction branches, attention mechanism modules, feature pyramid networks, and feature output layers, solving the problems of severe single-temporal feature confusion and insufficient spatiotemporal feature fusion in the cultivated land identification process; and it constructs a prior knowledge graph of cultivated land and knowledge constraint loss, improving the accuracy and interpretability of cultivated land identification.

[0082] like Figure 2 The diagram shown is a functional block diagram of a preferred embodiment of the farmland identification device based on multi-temporal feature fusion of the present invention. The farmland identification device 11 based on multi-temporal feature fusion includes a construction unit 110, a sampling unit 111, a training unit 112, and a processing unit 113. The module / unit referred to in this invention is a series of computer program segments that can be executed by a processor and perform a fixed function, and are stored in memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0083] The construction unit 110 is used to collect remote sensing image data and ground-measured cultivated land vector plot data, and to construct a time-series sample set including cultivated land and non-cultivated land labels based on the remote sensing image data and the cultivated land vector plot data. The sampling unit 111 is used to randomly sample the cultivated land area and non-cultivated land area of ​​each time series sample in the time series sample set to obtain multiple sampling points; The construction unit 110 is also used to extract the spectral features and spatial texture features of each sampling point, and to construct a high-dimensional feature tensor for each sampling point based on the spectral features and spatial texture features of each sampling point. The construction unit 110 is also used to construct an initial network model; wherein the initial network model includes a spatial feature extraction branch, a temporal feature extraction branch, an attention mechanism module, a feature pyramid network, and a feature output layer; The construction unit 110 is also used to construct a prior knowledge graph of arable land and construct a knowledge constraint loss based on the prior knowledge graph of arable land. The training unit 112 is used to train the initial network model based on the knowledge constraint loss and the high-dimensional feature tensor of each sampling point to obtain the farmland identification model. The processing unit 113 is used to process the target remote sensing image using the farmland identification model in response to the farmland identification instruction based on the target remote sensing image, so as to obtain the farmland identification result of the target remote sensing image.

[0084] As can be seen from the above technical solutions, this invention can construct a temporal sample set including labels for cultivated land and non-cultivated land based on remote sensing image data and cultivated land vector patch data, solving the problems of inconsistent spatial locations of images at different time phases, distortion of ground object reflection information, and cloud pollution; it constructs a high-dimensional feature tensor for each sampling point based on the spectral and spatial texture features of each sampling point, integrating multi-temporal and multi-modal features to comprehensively capture the spectral response characteristics and spatial structure information of cultivated land; it constructs an initial network model including spatial feature extraction branches, temporal feature extraction branches, attention mechanism modules, feature pyramid networks, and feature output layers, solving the problems of severe single-temporal feature confusion and insufficient spatiotemporal feature fusion in the cultivated land identification process; and it constructs a prior knowledge graph of cultivated land and knowledge constraint loss, improving the accuracy and interpretability of cultivated land identification.

[0085] like Figure 3 The diagram shown is a schematic diagram of the structure of a computer device that implements a preferred embodiment of the farmland identification method based on multi-temporal feature fusion according to the present invention.

[0086] The computer device 1 may include a memory 12, a processor 13, and a bus (the arrow in the figure represents the bus), and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a farmland identification program based on multi-temporal feature fusion.

[0087] Those skilled in the art will understand that the schematic diagram is merely an example of computer device 1 and does not constitute a limitation on computer device 1. Computer device 1 can be either a bus topology or a star topology. Computer device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, computer device 1 may also include input / output devices, network access devices, etc.

[0088] It should be noted that the computer device 1 described is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0089] The memory 12 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a portable hard drive of the computer device 1. In other embodiments, the memory 12 can be an external storage device of the computer device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device 1. Furthermore, the memory 12 can include both internal and external storage units of the computer device 1. The memory 12 can be used not only to store application software and various types of data installed on the computer device 1, such as the code of a farmland identification program based on multi-time-series feature fusion, but also to temporarily store data that has been output or will be output.

[0090] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the computer device 1, connecting various components of the computer device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing a farmland identification program based on multi-time-series feature fusion) and calls data stored in the memory 12 to perform various functions of the computer device 1 and process data.

[0091] The processor 13 executes the operating system of the computer device 1 and various installed application programs. The processor 13 executes these application programs to implement the steps in the various embodiments of the farmland identification method based on multi-temporal feature fusion described above, for example... Figure 1 The steps are shown.

[0092] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a construction unit 110, a sampling unit 111, a training unit 112, and a processing unit 113.

[0093] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the farmland identification method based on multi-temporal feature fusion described in the various embodiments of this invention.

[0094] If the modules / units integrated in the computer device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0095] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0096] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0097] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0098] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The bus is represented by only one straight line, but this does not mean that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0099] Although not shown, the computer device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The computer device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0100] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the computer device 1 and other computer devices.

[0101] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the computer device 1 and to display a visual user interface.

[0102] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0103] It will be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0104] Combination Figure 1 The memory 12 in the computer device 1 stores multiple instructions to implement a farmland identification method based on multi-temporal feature fusion, and the processor 13 can execute the multiple instructions to achieve the following: Collect remote sensing image data and ground-measured cultivated land vector plot data, and construct a time-series sample set including cultivated land and non-cultivated land labels based on the remote sensing image data and the cultivated land vector plot data; Random sampling is performed on the cultivated land area and non-cultivated land area of ​​each time series sample in the time series sample set to obtain multiple sampling points; Extract the spectral and spatial texture features of each sampling point, and construct a high-dimensional feature tensor for each sampling point based on the spectral and spatial texture features of each sampling point; Construct an initial network model; wherein the initial network model includes a spatial feature extraction branch, a temporal feature extraction branch, an attention mechanism module, a feature pyramid network, and a feature output layer; Construct a prior knowledge graph of arable land, and construct a knowledge constraint loss based on the prior knowledge graph of arable land; Based on the knowledge constraint loss, the initial network model is trained using the high-dimensional feature tensor of each sampling point to obtain the farmland identification model. In response to the farmland identification instruction based on the target remote sensing image, the target remote sensing image is processed using the farmland identification model to obtain the farmland identification result of the target remote sensing image.

[0105] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0106] It should be noted that all the data involved in this case was legally obtained.

[0107] If any AI models, software tools, or components not belonging to this company appear in the embodiments of this invention, they are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this invention has been obtained by an entity authorized (with the knowledge and consent) or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0108] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0109] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0110] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0112] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0113] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0114] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this invention can also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for identifying cultivated land based on multi-temporal feature fusion, characterized in that, The farmland identification method based on multi-temporal feature fusion includes: Collect remote sensing image data and ground-measured cultivated land vector plot data, and construct a time-series sample set including cultivated land and non-cultivated land labels based on the remote sensing image data and the cultivated land vector plot data; Random sampling is performed on the cultivated land area and non-cultivated land area of ​​each time series sample in the time series sample set to obtain multiple sampling points; Extract the spectral and spatial texture features of each sampling point, and construct a high-dimensional feature tensor for each sampling point based on the spectral and spatial texture features of each sampling point; Construct an initial network model; wherein the initial network model includes a spatial feature extraction branch, a temporal feature extraction branch, an attention mechanism module, a feature pyramid network, and a feature output layer; Construct a prior knowledge graph of arable land, and construct a knowledge constraint loss based on the prior knowledge graph of arable land; Based on the knowledge constraint loss, the initial network model is trained using the high-dimensional feature tensor of each sampling point to obtain the farmland identification model. In response to the farmland identification instruction based on the target remote sensing image, the target remote sensing image is processed using the farmland identification model to obtain the farmland identification result of the target remote sensing image.

2. The method for identifying cultivated land based on multi-temporal feature fusion as described in claim 1, characterized in that, The construction of a time-series sample set including labels for cultivated land and non-cultivated land based on the remote sensing image data and the cultivated land vector plot data includes: The remote sensing image data is converted to the target geographic coordinate system and adjusted to the target spatial resolution to perform spatiotemporal registration on the remote sensing image data to obtain the first data. The pixel values ​​of the first data are converted into surface reflectance to perform radiometric calibration and atmospheric correction on the first data, thus obtaining the second data; Based on the cloud detection algorithm and the least squares smoothing filter, invalid pixel regions in the second data are identified and removed to perform cloud masking processing on the second data to obtain the third data; The cultivated land area and non-cultivated land area in the third data are marked based on the cultivated land vector plot data; The third data, which is combined with annotations, is used to obtain the time-series sample set.

3. The method for identifying cultivated land based on multi-temporal feature fusion as described in claim 2, characterized in that, The process of extracting the spectral and spatial texture features of each sampling point, and constructing a high-dimensional feature tensor for each sampling point based on these features, includes: The normalized vegetation index, normalized differential water index, and bare soil index were calculated for each sampling point. A sequence was constructed based on the normalized vegetation index, normalized differential water index, and bare soil index for each sampling point to obtain the spectral characteristics of each sampling point. The entropy of each sampling point is calculated using the gray-level co-occurrence matrix as the spatial texture feature of each sampling point; The spectral features and spatial texture features of each sampling point are accumulated in the time dimension to obtain the high-dimensional feature tensor of each sampling point; The dimension of the high-dimensional feature tensor for each sampling point is: time step × number of feature channels × target spatial resolution; The number of feature channels is the sum of the number of spectral feature channels and the number of spatial texture feature channels.

4. The method for identifying cultivated land based on multi-temporal feature fusion as described in claim 1, characterized in that, The construction of the prior knowledge graph of arable land includes: Determine entity nodes based on the type of target features; Each entity node is connected to form an edge according to phenological rules and spatial adjacency rules; The prior knowledge graph of cultivated land is generated based on the entity nodes and the edges.

5. The method for identifying cultivated land based on multi-temporal feature fusion as described in claim 4, characterized in that, The loss for constructing knowledge constraints based on the prior knowledge graph of cultivated land includes: Obtain the temporal feature vector output by the temporal feature extraction branch, and obtain the predefined standard phenological feature center of cultivated land; Calculate the similarity between the temporal feature vector and the standard cultivated land phenological feature center; The difference between 1 and the similarity is calculated to construct the phenological feature constraint loss; wherein, the phenological feature constraint loss is used to transform the phenological rule into a feature space distance constraint; The timing smoothing constraint loss is calculated using the following formula: ； in, The time series smoothing constraint loss is represented by T; the total number of time steps in the time series sample set is represented by t; This represents the probability diagram of cultivated land classification at time step t+1. A probability diagram representing the classification of cultivated land at time step t; This indicates the indicator function that, when an abnormal jump that violates the rules of the cultivated land prior knowledge graph is detected, Set to 1 when no abnormal jumps violating the rules of the cultivated land prior knowledge graph are detected. Set to 0; Obtain the first weight corresponding to the phenological feature constraint loss and the second weight corresponding to the temporal smoothing constraint loss; The knowledge constraint loss is obtained by weighting the first weight, the second weight, the phenological feature constraint loss, and the temporal smoothing constraint loss.

6. The method for identifying cultivated land based on multi-temporal feature fusion as described in claim 1, characterized in that, The process of training the initial network model using the high-dimensional feature tensor of each sampling point, based on the knowledge-constrained loss, to obtain the farmland identification model includes: Obtain the classification prediction cross-entropy loss, which measures the similarity between the true labels and the model predictions. Obtain a balance coefficient for adjusting the weights of the classification prediction cross-entropy loss and the knowledge constraint loss; Construct a total loss function based on the balance coefficient, the classification prediction cross-entropy loss, and the knowledge constraint loss; With the goal of minimizing the total loss function, the initial network model is trained based on the high-dimensional feature tensor of each sampling point; When the total loss function no longer decreases, the currently obtained model is determined as the farmland identification model.

7. The method for identifying cultivated land based on multi-temporal feature fusion as described in claim 1, characterized in that, The process of using the farmland identification model to process the target remote sensing image to obtain the farmland identification result of the target remote sensing image includes: Construct a high-dimensional feature tensor for each time step corresponding to the target remote sensing image; In the spatial feature extraction branch, convolution operations are used to perform independent convolution operations on the high-dimensional feature tensor at each time step to obtain the target spatial feature map; In the temporal feature extraction branch, a bidirectional convolutional long short-term memory network is used to capture the dependency relationship between the high-dimensional feature tensors of each time step before and after the time sequence, so as to obtain the target temporal feature vector; In the attention mechanism module, the target temporal feature vector is expanded into a target feature map with the same number of channels as the target spatial feature map through a convolutional layer, and the target feature map is upsampled to the same size as the target spatial feature map using bilinear interpolation to obtain the target temporal feature map; The target spatial feature map and the target temporal feature map are added element by element to obtain preliminary fused features; The preliminary fusion features are processed by global average pooling to generate channel descriptors; The channel descriptors are input into a fully connected network to generate channel weights; The preliminary fusion features are reweighted based on the channel weights to obtain the target fusion features; Using the feature pyramid network as a decoder, the target fusion features and the target spatial feature map are fused through lateral connections, and the images are gradually upsampled to restore the original resolution of the target remote sensing image to obtain intermediate features; The intermediate features are input into the feature output layer to obtain the classification probability map of cultivated land and non-cultivated land for each pixel in the target remote sensing image, which is used as the cultivated land identification result. The feature output layer includes a convolutional normalization layer, a linear rectified function, and a normalized exponential function.

8. A farmland identification device based on multi-temporal feature fusion, characterized in that, The farmland identification device based on multi-temporal feature fusion includes: The construction unit is used to collect remote sensing image data and ground-measured cultivated land vector plot data, and to construct a time-series sample set including cultivated land and non-cultivated land labels based on the remote sensing image data and the cultivated land vector plot data. The sampling unit is used to randomly sample the cultivated land area and non-cultivated land area of ​​each time series sample in the time series sample set to obtain multiple sampling points; The construction unit is also used to extract the spectral features and spatial texture features of each sampling point, and to construct a high-dimensional feature tensor for each sampling point based on the spectral features and spatial texture features of each sampling point. The building unit is also used to build an initial network model; wherein the initial network model includes a spatial feature extraction branch, a temporal feature extraction branch, an attention mechanism module, a feature pyramid network, and a feature output layer; The construction unit is also used to construct a prior knowledge graph of arable land and to construct a knowledge constraint loss based on the prior knowledge graph of arable land. The training unit is used to train the initial network model based on the knowledge constraint loss and the high-dimensional feature tensor of each sampling point to obtain the farmland identification model. The processing unit is configured to respond to a farmland identification instruction based on a target remote sensing image, process the target remote sensing image using the farmland identification model, and obtain the farmland identification result of the target remote sensing image.

Images