Method and device for identifying cultivated land non-agriculturalization based on prior knowledge and feature fusion
By integrating prior information and characteristics in the identification of non-agriculturalization of cultivated land, the problem of low generalization ability in the existing technology is solved, and the accurate identification of diverse non-agriculturalization types is achieved, and the efficiency of farmland monitoring is improved.
Patent Information
- Application Number
- CN202411137447.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2025-04-22
- Estimated Expiration
- 2044-08-19
AI Technical Summary
The prior art fails to effectively utilize prior information in the identification of non-agriculturalization of cultivated land, resulting in low generalization ability of identification and difficulty in identifying diverse non-agriculturalization types.
The recognition method based on the fusion of prior and feature is adopted, and the resolution generalization, scale feature generation and prior information matching of the cultivated land image to be identified is carried out, spatial features, semantic features and scale features are fused, mask features are generated and decoded to identify the non-agriculturalized types of cultivated land.
The generalization ability of arable land is improved, and the diverse types of non-agriculturalization can be more accurately identified, which enhances the monitoring and processing capabilities of arable land changes.
Smart Images
Figure CN119181015B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of agricultural land identification, and in particular to a method and device for identifying non-agricultural land based on prior and feature fusion. Background Art
[0002] Cultivated land is an important land resource, and the change in the amount of cultivated land is directly related to the maintenance of the dynamic balance of cultivated land and the sustainable development of agriculture. In order to timely grasp the status quo and changes of cultivated land and deal with non-agricultural cultivated land in a timely manner, it is indispensable to monitor and identify non-agricultural cultivated land.
[0003] In the past, the monitoring and identification of non-agriculturalization of cultivated land mainly relied on a combination of manual visual interpretation and field investigation, which had low timeliness and could not timely discover and stop the "non-agriculturalization" of cultivated land. Existing deep learning methods rely on mining information in remote sensing images to realize the identification of non-agriculturalization of cultivated land, ignoring the influence of prior information such as spatial characteristics and temporal characteristics. At the same time, there are various types of cultivated land in my country, and there are also many types of non-agriculturalization. Existing deep learning methods usually have difficulty in identifying various types of non-agriculturalization. Therefore, how to enhance the generalization ability of non-agriculturalization identification of cultivated land is still an urgent problem to be solved. Summary of the invention
[0004] The present application provides a method and device for identifying non-agricultural land of cultivated land based on prior information and feature fusion, so as to solve the technical problem that the prior art does not consider the influence of prior information on the identification of non-agricultural land of cultivated land, resulting in low generalization ability of the identification of non-agricultural land of cultivated land.
[0005] According to a first aspect of the implementation of the present application, a method for identifying non-agriculturalization of cultivated land based on prior knowledge and feature fusion is provided, comprising:
[0006] Generalize the resolution of the cultivated land image to be identified to obtain a first generalized image;
[0007] Generating a scale feature on the first generalized image to obtain a first scale feature;
[0008] Corresponding to the first scale feature, a first spatial feature and a first semantic feature are matched in preset prior information to obtain; and, a fusion mask is generated for the first scale feature, the first spatial feature and the first semantic feature to obtain a first mask feature;
[0009] Mask decoding is performed on the first mask feature to obtain a first category probability; and, based on the first category probability, a farmland non-agricultural type corresponding to the farmland image to be identified is obtained.
[0010] The present application first performs resolution generalization on the cultivated land image to be identified to obtain a first generalized image, which can improve the generalization ability in terms of resolution, and then performs scale feature generation on the first generalized image to obtain a first scale feature, which can obtain the image's scale feature, and then obtain the first spatial feature and the first semantic feature by corresponding matching in the preset prior information, which can obtain the image's spatial and semantic features based on the prior information, and fuse them to obtain the first mask feature, and then perform mask decoding on the first mask feature and identify the corresponding cultivated land non-agricultural type, taking into account the influence of multiple features on the identification, and improving the generalization ability of cultivated land non-agricultural identification.
[0011] In certain embodiments of the present application, generalizing the resolution of the cultivated land image to be identified to obtain a first generalized image specifically includes:
[0012] Characterizing the farmland image to be identified to obtain a first image feature; and obtaining a second image feature based on the first image feature and a multi-head self-attention mechanism;
[0013] fusing the first image feature with the second image feature to obtain a third image feature;
[0014] According to the third image feature, a fourth image feature is obtained based on a multi-layer perceptron;
[0015] Performing image embedding on the third image feature to obtain a fifth image feature;
[0016] The fourth image feature and the fifth image feature are fused to obtain a first generalized image.
[0017] The present application first characterizes the cultivated land image to be identified to obtain a first image feature, and obtains a second image feature based on a multi-head self-attention mechanism, and then fuses the first image feature and the second image feature to obtain a third image feature, and then based on the third image feature, obtains a fourth image feature by a multi-layer perceptron, performs image embedding to obtain a fifth image feature, and fuses the fourth image feature and the fifth image feature to obtain a first generalized image. Through the multi-layer perceptron and image embedding, the generalization ability of the image in terms of resolution is improved.
[0018] In certain embodiments of the present application, the performing image embedding on the third image feature to obtain the fifth image feature specifically includes:
[0019] According to common sense of multiple resolution domains, the third image feature is domain-commonly embedded to obtain multiple first embedded features;
[0020] Performing domain embedding enhancement on the plurality of first embedding features respectively to obtain a plurality of second embedding features;
[0021] The multiple second embedded features are concatenated to obtain a fifth image feature.
[0022] This application first performs domain-common embedding on the third image feature according to the common sense of multiple resolution domains to obtain multiple first embedded features, then performs domain embedding enhancement on the multiple first embedded features respectively, and then splices them to obtain the fifth image feature. Through domain-common embedding and domain embedding enhancement, the domain adaptation capability is improved, thereby improving the generalization capability of the image in terms of resolution.
[0023] In some embodiments of the present application, the generating of scale features on the first generalized image to obtain the first scale features specifically includes:
[0024] Convolving the first generalized image according to a plurality of preset scales to obtain a plurality of first-scale embedded features;
[0025] Based on a feature pyramid network, the plurality of first-scale embedded features are fused to obtain a first embedded image;
[0026] Foreground prediction is performed on the first embedded image and optimized through binary cross entropy loss to obtain a first scale feature.
[0027] The present application first convolves the first generalized image according to multiple preset scales to obtain multiple first-scale embedded features, which can obtain the features of the image at multiple scales, and then fuses them based on the feature pyramid network to obtain the first embedded image, and performs foreground prediction and binary cross entropy loss optimization to obtain the first-scale features, which can optimize the fusion of features at multiple scales to the first-scale features, thereby improving the generalization ability of multiple-scale recognition.
[0028] In certain embodiments of the present application, the generation of the preset prior information specifically includes:
[0029] Binarize the historical cultivated land spatial change data to obtain the prior spatial characteristics;
[0030] Integrate historical cultivated land time series change data, historical remote sensing time series data and preset prior image features, and extract the model based on the preset time series features to obtain the prior time series features;
[0031] According to the historical cultivated land phenological change data, a phenological characteristic curve is obtained; and, by combining the preset prior semantic features and the phenological characteristic curve, a priori phenological features are obtained;
[0032] Fusing the prior temporal features and the prior phenological features to obtain prior semantic features;
[0033] The prior spatial features and the prior semantic features are combined to obtain preset prior information.
[0034] The present application first performs a binarization transformation on the historical cultivated land spatial change data to obtain a priori spatial features, which can simplify and convert the spatial change data and reduce the spatial feature dimension; then integrates the historical cultivated land time series change data, the historical remote sensing time series data and the preset prior image features, and obtains the prior time series features according to the preset time series feature extraction model, which can extract the time series change data and integrate the prior image features, thereby reducing the difficulty of subsequent matching and recognition; then obtains the prior phenological features according to the historical cultivated land phenological change data and the preset prior semantic features, which can extract the phenological change data and integrate the prior semantic features, thereby reducing the difficulty of subsequent matching and recognition; then integrates the prior time series features and the prior phenological features to obtain the prior semantic features, and combines the prior semantic features and the prior spatial features to obtain the preset prior information, which can provide the corresponding matching semantic features and spatial features when recognizing cultivated land images, thereby improving the generalization ability of recognizing the non-agriculturalization of cultivated land.
[0035] In some embodiments of the present application, performing mask decoding on the first mask feature to obtain a first category probability specifically includes:
[0036] Based on the multi-head self-attention mechanism, obtaining a first decoding feature according to the first mask feature;
[0037] Based on a preset first category decoding method, obtaining a second decoding feature according to the first decoding feature;
[0038] The second decoding feature is normalized, and a first category probability is obtained based on a multilayer perceptron and the first category decoding method.
[0039] This application first obtains the first decoding feature based on the multi-head self-attention mechanism, and obtains the second decoding feature based on the first category decoding method, and then normalizes and obtains the first category probability based on the multi-layer perceptron and the first category decoding method. It can identify different categories of cultivated land non-agriculturalization and obtain corresponding category probabilities, thereby improving the generalization ability of cultivated land non-agriculturalization identification based on the type of cultivated land non-agriculturalization.
[0040] In some embodiments of the present application, the first category decoding method specifically includes:
[0041] Downsample and activate the input to be decoded to obtain the first intermediate feature;
[0042] Downsampling and activating the first intermediate feature to obtain a second intermediate feature;
[0043] Upsampling, activating, and upsampling the second intermediate feature to obtain a third intermediate feature;
[0044] The third intermediate feature is combined with the input to be decoded to obtain a decoded output.
[0045] The present application first downsamples and activates the input to be decoded to obtain the first intermediate feature, which can map the input to be decoded to a low dimension, reducing the amount of calculation and the amount of parameters, and then downsamples and activates to obtain the second intermediate feature, which can map the first intermediate feature to the target dimension to ensure the compatibility of subsequent operations; then upsamples, activates and upsamples the second intermediate feature to obtain the third intermediate feature, which can remap the second intermediate feature to the initial dimension to ensure network consistency, and fuses the input to be decoded and the third intermediate feature to obtain the decoded output, which can integrate features and enhance expression capabilities.
[0046] According to a second aspect of the implementation of the present application, there is provided a device for identifying non-agriculturalization of cultivated land based on prior knowledge and feature fusion, comprising an image generalization module, a feature generation module, a feature fusion module and a decoding and recognition module;
[0047] The image generalization module is used to perform resolution generalization on the cultivated land image to be identified to obtain a first generalized image;
[0048] The feature generation module is used to generate a scale feature for the first generalized image to obtain a first scale feature;
[0049] The feature fusion module is used to obtain a first spatial feature and a first semantic feature by matching the first scale feature in the preset prior information; and to generate a fusion mask for the first scale feature, the first spatial feature and the first semantic feature to obtain a first mask feature;
[0050] The decoding and identification module is used to perform mask decoding on the first mask feature to obtain a first category probability; and, based on the first category probability, obtain the non-agricultural type of cultivated land corresponding to the cultivated land image to be identified.
[0051] In certain embodiments of the present application, the image generalization module includes an image feature acquisition submodule, an image feature fusion submodule, a multi-layer perception submodule, an image embedding submodule and an image generalization submodule;
[0052] The image feature acquisition submodule is used to characterize the cultivated land image to be identified to obtain a first image feature; and, based on the first image feature, obtain a second image feature based on a multi-head self-attention mechanism;
[0053] The image feature fusion submodule is used to fuse the first image feature with the second image feature to obtain a third image feature;
[0054] The multi-layer perception submodule is used to obtain a fourth image feature based on a multi-layer perceptron according to the third image feature;
[0055] The image embedding submodule is used to perform image embedding on the third image feature to obtain a fifth image feature;
[0056] The image generalization submodule is used to fuse the fourth image feature and the fifth image feature to obtain a first generalized image.
[0057] In certain embodiments of the present application, the image embedding submodule includes a domain common embedding unit, a domain embedding enhancement unit, and a feature stitching unit;
[0058] The domain common embedding unit is used to perform domain common embedding on the third image feature according to multiple resolution domain common senses to obtain multiple first embedded features;
[0059] The domain embedding enhancement unit is used to perform domain embedding enhancement on the plurality of first embedding features respectively to obtain a plurality of second embedding features;
[0060] The feature stitching unit is used to stitch the multiple second embedded features to obtain a fifth image feature.
[0061] In certain embodiments of the present application, the feature generation module includes a scale convolution submodule, an embedding fusion submodule, and a prediction optimization submodule;
[0062] The scale convolution submodule is used to convolve the first generalized image according to a plurality of preset scales to obtain a plurality of first-scale embedded features;
[0063] The embedding fusion submodule is used to fuse the multiple first-scale embedding features based on a feature pyramid network to obtain a first embedded image;
[0064] The prediction optimization submodule is used to perform foreground prediction on the first embedded image and optimize it through binary cross entropy loss to obtain a first scale feature.
[0065] In certain embodiments of the present application, a priori generation module is also included; the priori generation module includes a spatial feature acquisition submodule, a temporal feature acquisition submodule, a phenological feature acquisition submodule, a semantic feature acquisition submodule and a priori information acquisition submodule;
[0066] The spatial feature acquisition submodule is used to perform a binary transformation on the historical cultivated land spatial change data to obtain a priori spatial features;
[0067] The time series feature acquisition submodule is used to fuse historical cultivated land time series change data, historical remote sensing time series data and preset prior image features, and extract the prior time series features according to the preset time series feature extraction model to obtain the prior time series features;
[0068] The phenological characteristic acquisition submodule is used to obtain a phenological characteristic curve based on historical cultivated land phenological change data; and to obtain a priori phenological characteristics by combining preset a priori semantic features and the phenological characteristic curve;
[0069] The semantic feature acquisition submodule is used to fuse the prior temporal features and the prior phenological features to obtain the prior semantic features;
[0070] The prior information acquisition submodule is used to combine the prior spatial features and the prior semantic features to obtain preset prior information.
[0071] In certain embodiments of the present application, the decoding identification module includes a first decoding submodule, a second decoding submodule, and a third decoding submodule;
[0072] The first decoding submodule is used to obtain a first decoding feature according to the first mask feature based on a multi-head self-attention mechanism;
[0073] The second decoding submodule is used to obtain a second decoding feature according to the first decoding feature based on a preset first category decoding method;
[0074] The third decoding submodule is used to normalize the second decoding feature and obtain a first category probability based on a multilayer perceptron and the first category decoding method.
[0075] In some embodiments of the present application, the first category decoding method specifically includes:
[0076] Downsample and activate the input to be decoded to obtain the first intermediate feature;
[0077] Downsampling and activating the first intermediate feature to obtain a second intermediate feature;
[0078] Upsampling, activating, and upsampling the second intermediate feature to obtain a third intermediate feature;
[0079] The third intermediate feature is combined with the input to be decoded to obtain a decoded output.
[0080] The present application first performs resolution generalization on the cultivated land image to be identified to obtain a first generalized image, which can improve the generalization ability in terms of resolution, and then performs scale feature generation on the first generalized image to obtain a first scale feature, which can obtain the image's scale feature, and then obtain the first spatial feature and the first semantic feature by corresponding matching in the preset prior information, which can obtain the image's spatial and semantic features based on the prior information, and fuse them to obtain the first mask feature, and then perform mask decoding on the first mask feature and identify the corresponding cultivated land non-agricultural type, taking into account the influence of multiple features on recognition and improving the generalization ability of recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0081] Figure 1 : A flow chart of a method for identifying non-agriculturalization of cultivated land based on prior knowledge and feature fusion shown in certain embodiments of the present application;
[0082] Figure 2 : A model structure diagram of a generalized model shown in certain embodiments of the present application;
[0083] Figure 3 : A schematic diagram of a feature generation process shown in certain embodiments of the present application;
[0084] Figure 4 : A model structure diagram of a decoding model shown in certain embodiments of the present application;
[0085] Figure 5 : A model structure diagram of a category adapter shown in certain embodiments of the present application;
[0086] Figure 6 : A module structure diagram of a device for identifying non-agriculturalization of cultivated land based on prior and feature fusion shown in certain embodiments of the present application. DETAILED DESCRIPTION
[0087] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of the present application, and should not be construed as limitations on the embodiments of the present application. Based on the embodiments shown in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0088] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, unless otherwise clearly and specifically defined, "multiple" and "several" mean two or more.
[0089] In the existing technology, the monitoring and identification of farmland non-agriculturalization relies on manual identification and field investigation, which has low timeliness and is difficult to detect farmland non-agriculturalization in time and stop the behavior of farmland non-agriculturalization; secondly, it relies on mining and analyzing satellite remote sensing images through deep learning technology to realize the identification of farmland non-agriculturalization. This method ignores the influence of prior information such as spatial characteristics and temporal characteristics of farmland non-agriculturalization, so the generalization ability is not high. At the same time, the diversity of farmland types leads to the diversity of non-agriculturalization types. The existing technology is difficult to identify different types of farmland non-agriculturalization and has low generalization ability. Therefore, how to enhance the generalization ability of farmland non-agriculturalization identification is still an urgent problem to be solved.
[0090] Based on the above technical background, please refer to Figure 1 The embodiment of the present application provides a method for identifying non-agricultural land based on prior knowledge and feature fusion, including steps S101 to S104, each of which is as follows:
[0091] Step S101: generalizing the resolution of the cultivated land image to be identified to obtain a first generalized image.
[0092] In certain embodiments of the present application, generalizing the resolution of the cultivated land image to be identified to obtain a first generalized image specifically includes:
[0093] Characterizing the farmland image to be identified to obtain a first image feature; and obtaining a second image feature based on the first image feature and a multi-head self-attention mechanism;
[0094] fusing the first image feature with the second image feature to obtain a third image feature;
[0095] According to the third image feature, a fourth image feature is obtained based on a multi-layer perceptron;
[0096] Performing image embedding on the third image feature to obtain a fifth image feature;
[0097] The fourth image feature and the fifth image feature are fused to obtain a first generalized image.
[0098] The present application first characterizes the cultivated land image to be identified to obtain a first image feature, and obtains a second image feature based on a multi-head self-attention mechanism, and then fuses the first image feature and the second image feature to obtain a third image feature, and then based on the third image feature, obtains a fourth image feature by a multi-layer perceptron, performs image embedding to obtain a fifth image feature, and fuses the fourth image feature and the fifth image feature to obtain a first generalized image. Through the multi-layer perceptron and image embedding, the generalization ability of the image in terms of resolution is improved.
[0099] In certain embodiments of the present application, the performing image embedding on the third image feature to obtain the fifth image feature specifically includes:
[0100] According to common sense of multiple resolution domains, the third image feature is domain-commonly embedded to obtain multiple first embedded features;
[0101] Performing domain embedding enhancement on the plurality of first embedding features respectively to obtain a plurality of second embedding features;
[0102] The multiple second embedded features are concatenated to obtain a fifth image feature.
[0103] This application first performs domain-common embedding on the third image feature according to the common sense of multiple resolution domains to obtain multiple first embedded features, then performs domain embedding enhancement on the multiple first embedded features respectively, and then splices them to obtain the fifth image feature. Through domain-common embedding and domain embedding enhancement, the domain adaptation capability is improved, thereby improving the generalization capability of the image in terms of resolution.
[0104] In some optional embodiments of this application, see Figure 2 , which is a model structure of a generalized model relied upon by the present application when performing resolution generalization on the cultivated land image to be identified to obtain a first generalized image;
[0105] Among them, Norm is normalization processing, Multi-Head Attention is multi-head self-attention mechanism, MLP is multi-layer perceptron, and Domain Bypass is domain bypass image embedding processing.
[0106] Exemplarily, the implementation steps in the generalized model structure shown are specifically:
[0107] The input features are normalized and the second image features are obtained through a multi-head self-attention mechanism;
[0108] fusing the second image feature with the input feature to obtain a third image feature;
[0109] Normalizing the third image feature and obtaining a fourth image feature through a multi-layer perceptron;
[0110] Performing domain bypass image embedding on the third image feature to obtain a fifth image feature;
[0111] The fourth image feature and the fifth image feature are fused to obtain a first generalized image.
[0112] Specifically, the third image feature is subjected to domain bypass image embedding to obtain the fifth image feature, specifically:
[0113] Based on the K-resolution domain common sense, a learnable domain common embedding w com , performing domain common embedding on the third image feature h to obtain K first embedding features;
[0114] Based on learnable domain-specific embedding w k , respectively perform domain embedding enhancement on the K first embedding features to obtain multiple second embedding features;
[0115] The multiple second embedded features are concatenated to obtain a fifth image feature.
[0116] More specifically, on this basis, the first generalized image is:
[0117]
[0118] in, is the first generalized image, w k is the kth domain-specific embedding, w com is the domain common embedding, MLP(·) is the output of the multi-layer perceptron, and h is the third image feature.
[0119] Domain bypass image embedding is considered because the embedding of the image in the residual can be enhanced through domain common embedding and domain specific embedding, thereby improving the domain adaptation ability and thus improving the generalization ability of the image in terms of resolution.
[0120] Step S102: generating a scale feature for the first generalized image to obtain a first scale feature.
[0121] In certain embodiments of this application, see Figure 3 , is a feature generation process for generating a first scale feature by performing scale feature generation on the first generalized image in the present application, specifically including:
[0122] Convolving the first generalized image according to a plurality of preset scales to obtain a plurality of first-scale embedded features;
[0123] Based on a feature pyramid network, the plurality of first-scale embedded features are fused to obtain a first embedded image;
[0124] Foreground prediction is performed on the first embedded image and optimized through binary cross entropy loss to obtain a first scale feature.
[0125] Specifically, the foreground prediction is performed on the first embedded image, and optimized by binary cross entropy loss to obtain the first scale feature, which is specifically:
[0126] Using a 1×1 convolution head to perform foreground prediction on the first embedded image to obtain a foreground segmentation feature;
[0127] The foreground segmentation feature is optimized by binary cross entropy loss to obtain a first scale feature;
[0128] Among them, the binary cross entropy loss is specifically:
[0129]
[0130] Among them, y n is the nth label of the true value of the foreground prediction, N is the total number of labels of the foreground segmentation feature, σ(·) is the foreground probability prediction function, and g n The mask generated for the foreground prediction.
[0131] Consider fusing multiple first-scale embedded features according to a feature pyramid network to generate a first embedded image, which can integrate multi-level features and improve the perception ability of the first embedded image, thereby improving the generalization ability of recognizing multiple scales.
[0132] The present application first convolves the first generalized image according to multiple preset scales to obtain multiple first-scale embedded features, which can obtain the features of the image at multiple scales, and then fuses them based on the feature pyramid network to obtain the first embedded image, and performs foreground prediction and binary cross entropy loss optimization to obtain the first-scale features, which can optimize the fusion of features at multiple scales to the first-scale features, thereby improving the generalization ability of multiple-scale recognition.
[0133] In certain embodiments of the present application, the generation of the preset prior information specifically includes:
[0134] Binarize the historical cultivated land spatial change data to obtain the prior spatial characteristics;
[0135] Integrate historical cultivated land time series change data, historical remote sensing time series data and preset prior image features, and extract the model based on the preset time series features to obtain the prior time series features;
[0136] According to the historical cultivated land phenological change data, a phenological characteristic curve is obtained; and, by combining the preset prior semantic features and the phenological characteristic curve, a priori phenological features are obtained;
[0137] Fusing the prior temporal features and the prior phenological features to obtain prior semantic features;
[0138] The prior spatial features and the prior semantic features are combined to obtain preset prior information.
[0139] The present application first performs a binarization transformation on the historical cultivated land spatial change data to obtain a priori spatial features, which can simplify and convert the spatial change data and reduce the spatial feature dimension; then integrates the historical cultivated land time series change data, the historical remote sensing time series data and the preset prior image features, and obtains the prior time series features according to the preset time series feature extraction model, which can extract the time series change data and integrate the prior image features, thereby reducing the difficulty of subsequent matching and recognition; then obtains the prior phenological features according to the historical cultivated land phenological change data and the preset prior semantic features, which can extract the phenological change data and integrate the prior semantic features, thereby reducing the difficulty of subsequent matching and recognition; then integrates the prior time series features and the prior phenological features to obtain the prior semantic features, and combines the prior semantic features and the prior spatial features to obtain the preset prior information, which can provide the corresponding matching semantic features and spatial features when recognizing cultivated land images, thereby improving the generalization ability of recognizing the non-agriculturalization of cultivated land.
[0140] Step S103: corresponding to the first scale feature, matching a first spatial feature and a first semantic feature in preset prior information; and performing fusion mask generation on the first scale feature, the first spatial feature and the first semantic feature to obtain a first mask feature.
[0141] In certain optional embodiments of the present application, the preferred scheme of the model generated by the fusion mask is the SAM model.
[0142] Step S104: performing mask decoding on the first mask feature to obtain a first category probability; and obtaining the non-agricultural type of cultivated land corresponding to the cultivated land image to be identified according to the first category probability.
[0143] In some embodiments of the present application, performing mask decoding on the first mask feature to obtain a first category probability specifically includes:
[0144] Based on the multi-head self-attention mechanism, obtaining a first decoding feature according to the first mask feature;
[0145] Based on a preset first category decoding method, obtaining a second decoding feature according to the first decoding feature;
[0146] The second decoding feature is normalized, and a first category probability is obtained based on a multilayer perceptron and the first category decoding method.
[0147] In some optional embodiments of this application, see Figure 4 , is a model structure of a decoding model relied upon by the present application when performing mask decoding on the first mask feature to obtain a first category probability;
[0148] Among them, Norm is normalization processing, Prompt is the output prompt, Image Embedding is the first mask feature, Multi-Head Attention is the multi-head self-attention mechanism, MLP is the multi-layer perceptron, and Adapter is the category adapter.
[0149] Exemplarily, the implementation steps in the illustrated decoding model structure are specifically as follows:
[0150] The normalized output prompt is combined with the first mask feature, and a first decoding feature is obtained through a multi-head self-attention mechanism;
[0151] Inputting the first decoding feature into a category adapter, and obtaining a second decoding feature based on a first category decoding method;
[0152] The second decoded feature is normalized and passed through a multi-layer perceptron and a category adapter to obtain the first category probability.
[0153] More specifically, the output prompt is:
[0154]
[0155] Among them, P i To output the prompt, MLP UP (·), MLP DOWN (·) are the output of upsampling and downsampling respectively, ReLU (·) is the activation output, F i is the intermediate space and semantic feature information, F j , W j is the j-th type feature and its corresponding weight, and N is the total number of type features.
[0156] This application first obtains the first decoding feature based on the multi-head self-attention mechanism, and obtains the second decoding feature based on the first category decoding method, and then normalizes and obtains the first category probability based on the multi-layer perceptron and the first category decoding method. It can identify different categories of cultivated land non-agriculturalization and obtain corresponding category probabilities, thereby improving the generalization ability of cultivated land non-agriculturalization identification based on the type of cultivated land non-agriculturalization.
[0157] In some optional embodiments of this application, see Figure 5 , is a category adapter model structure, where Down FC is the downsampling process, ReLU is the activation operation, UP FC is the upsampling process, and FC represents the fully connected operation.
[0158] In some embodiments of the present application, the first category decoding method specifically includes:
[0159] Downsample and activate the input to be decoded to obtain the first intermediate feature;
[0160] Downsampling and activating the first intermediate feature to obtain a second intermediate feature;
[0161] Upsampling, activating, and upsampling the second intermediate feature to obtain a third intermediate feature;
[0162] The third intermediate feature is combined with the input to be decoded to obtain a decoded output.
[0163] The present application first downsamples and activates the input to be decoded to obtain the first intermediate feature, which can map the input to be decoded to a low dimension, reducing the amount of calculation and the amount of parameters, and then downsamples and activates to obtain the second intermediate feature, which can map the first intermediate feature to the target dimension to ensure the compatibility of subsequent operations; then upsamples, activates and upsamples the second intermediate feature to obtain the third intermediate feature, which can remap the second intermediate feature to the initial dimension to ensure network consistency, and fuses the input to be decoded and the third intermediate feature to obtain the decoded output, which can integrate features and enhance expression capabilities.
[0164] Compared with the prior art, the present application first generalizes the resolution of the cultivated land image to be identified to obtain a first generalized image, which can improve the generalization ability in terms of resolution, and then generates scale features for the first generalized image to obtain a first scale feature, which can obtain the scale features of the image, and then obtain the first spatial feature and the first semantic feature by corresponding matching in the preset prior information, and can obtain the spatial and semantic features of the image based on the prior information, and fuse them to obtain the first mask feature, and then perform mask decoding on the first mask feature and identify the corresponding cultivated land non-agricultural type, taking into account the influence of multiple features on recognition, and improving the generalization ability of recognition.
[0165] Corresponding to the previous method, see Figure 6 , the embodiment of the present application provides a device for identifying non-agriculturalization of cultivated land based on prior and feature fusion, including an image generalization module 610, a feature generation module 620, a feature fusion module 630 and a decoding and recognition module 640;
[0166] The image generalization module 610 is used to perform resolution generalization on the cultivated land image to be identified to obtain a first generalized image;
[0167] The feature generation module 620 is used to generate a scale feature for the first generalized image to obtain a first scale feature;
[0168] The feature fusion module 630 is used to obtain a first spatial feature and a first semantic feature by matching the first scale feature in the preset prior information; and to generate a fusion mask for the first scale feature, the first spatial feature and the first semantic feature to obtain a first mask feature;
[0169] The decoding and identification module 640 is used to perform mask decoding on the first mask feature to obtain a first category probability; and, based on the first category probability, obtain the non-agricultural type of cultivated land corresponding to the cultivated land image to be identified.
[0170] In certain embodiments of the present application, the image generalization module 610 includes an image feature acquisition submodule 611, an image feature fusion submodule 612, a multi-layer perception submodule 613, an image embedding submodule 614, and an image generalization submodule 615;
[0171] The image feature acquisition submodule 611 is used to characterize the cultivated land image to be identified to obtain a first image feature; and obtain a second image feature based on the first image feature and a multi-head self-attention mechanism;
[0172] The image feature fusion submodule 612 is used to fuse the first image feature with the second image feature to obtain a third image feature;
[0173] The multi-layer perception submodule 613 is used to obtain a fourth image feature based on a multi-layer perceptron according to the third image feature;
[0174] The image embedding submodule 614 is used to perform image embedding on the third image feature to obtain a fifth image feature;
[0175] The image generalization submodule 615 is used to fuse the fourth image feature and the fifth image feature to obtain a first generalized image.
[0176] In certain embodiments of the present application, the image embedding submodule 614 includes a domain common embedding unit, a domain embedding enhancement unit, and a feature stitching unit;
[0177] The domain common embedding unit is used to perform domain common embedding on the third image feature according to multiple resolution domain common senses to obtain multiple first embedded features;
[0178] The domain embedding enhancement unit is used to perform domain embedding enhancement on the plurality of first embedding features respectively to obtain a plurality of second embedding features;
[0179] The feature stitching unit is used to stitch the multiple second embedded features to obtain a fifth image feature.
[0180] In certain embodiments of the present application, the feature generation module 620 includes a scale convolution submodule 621, an embedding fusion submodule 622, and a prediction optimization submodule 623;
[0181] The scale convolution submodule 621 is used to convolve the first generalized image according to a plurality of preset scales to obtain a plurality of first-scale embedded features;
[0182] The embedding fusion submodule 622 is used to fuse the multiple first-scale embedding features based on a feature pyramid network to obtain a first embedded image;
[0183] The prediction optimization submodule 623 is used to perform foreground prediction on the first embedded image and optimize it through binary cross entropy loss to obtain a first scale feature.
[0184] In certain embodiments of the present application, a priori generation module is also included; the priori generation module includes a spatial feature acquisition submodule, a temporal feature acquisition submodule, a phenological feature acquisition submodule, a semantic feature acquisition submodule and a priori information acquisition submodule;
[0185] The spatial feature acquisition submodule is used to perform a binary transformation on the historical cultivated land spatial change data to obtain a priori spatial features;
[0186] The time series feature acquisition submodule is used to fuse historical cultivated land time series change data, historical remote sensing time series data and preset prior image features, and extract the prior time series features according to the preset time series feature extraction model to obtain the prior time series features;
[0187] The phenological characteristic acquisition submodule is used to obtain a phenological characteristic curve based on historical cultivated land phenological change data; and to obtain a priori phenological characteristics by combining preset a priori semantic features and the phenological characteristic curve;
[0188] The semantic feature acquisition submodule is used to fuse the prior temporal features and the prior phenological features to obtain the prior semantic features;
[0189] The prior information acquisition submodule is used to combine the prior spatial features and the prior semantic features to obtain preset prior information.
[0190] In certain embodiments of the present application, the decoding identification module 640 includes a first decoding submodule 641, a second decoding submodule 642, and a third decoding submodule 643;
[0191] The first decoding submodule 641 is used to obtain a first decoding feature according to the first mask feature based on a multi-head self-attention mechanism;
[0192] The second decoding submodule 642 is configured to obtain a second decoding feature according to the first decoding feature based on a preset first category decoding method;
[0193] The third decoding submodule 643 is used to normalize the second decoding feature and obtain a first category probability based on a multi-layer perceptron and the first category decoding method.
[0194] In some embodiments of the present application, the first category decoding method specifically includes:
[0195] Downsample and activate the input to be decoded to obtain the first intermediate feature;
[0196] Downsampling and activating the first intermediate feature to obtain a second intermediate feature;
[0197] Upsampling, activating, and upsampling the second intermediate feature to obtain a third intermediate feature;
[0198] The third intermediate feature is combined with the input to be decoded to obtain a decoded output.
[0199] The present application first performs resolution generalization on the cultivated land image to be identified to obtain a first generalized image, which can improve the generalization ability in terms of resolution, and then performs scale feature generation on the first generalized image to obtain a first scale feature, which can obtain the image's scale feature, and then obtain the first spatial feature and the first semantic feature by corresponding matching in the preset prior information, which can obtain the image's spatial and semantic features based on the prior information, and fuse them to obtain the first mask feature, and then perform mask decoding on the first mask feature and identify the corresponding cultivated land non-agricultural type, taking into account the influence of multiple features on recognition and improving the generalization ability of recognition.
[0200] Adaptively, the embodiments of the present application also provide a computer device and a computer-readable storage medium.
[0201] The computer device comprises: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor;
[0202] Among them, when the processor executes the computer program, a method for identifying non-agriculturalization of cultivated land based on prior and feature fusion of the present application is implemented.
[0203] The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for a processor to load to execute a method for identifying non-agriculturalization of cultivated land based on prior knowledge and feature fusion according to the present application.
[0204] The above are some implementation methods of the present application, and the purpose, technical scheme and beneficial effects of the present application are further described in detail. It should be clear that the above-mentioned partial implementation methods of the present application cannot be understood as limiting the present application. It is particularly pointed out that for those skilled in the art, any changes, modifications, equivalent substitutions and variations made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A method for identifying cultivated land non-agriculturalization based on prior and feature fusion, characterized in that: include: Generalize the resolution of the cultivated land image to be identified to obtain a first generalized image; Generating a scale feature on the first generalized image to obtain a first scale feature; Corresponding to the first scale feature, a first spatial feature and a first semantic feature are matched in preset prior information to obtain; and, a fusion mask is generated for the first scale feature, the first spatial feature and the first semantic feature to obtain a first mask feature; Performing mask decoding on the first mask feature to obtain a first category probability; and obtaining the non-agricultural type of cultivated land corresponding to the cultivated land image to be identified according to the first category probability; The generation of the preset prior information specifically includes: performing a binarization transformation on the historical cultivated land spatial change data to obtain a priori spatial features; fusing the historical cultivated land time series change data, the historical remote sensing time series data and the preset prior image features, and extracting the model according to the preset time series features to obtain the prior time series features; obtaining the phenological characteristic curve according to the historical cultivated land phenological change data; and obtaining the prior phenological characteristics by combining the preset prior semantic features and the phenological characteristic curve; fusing the prior time series features and the prior phenological features to obtain the prior semantic features; and combining the prior spatial features and the prior semantic features to obtain the preset prior information.
2. The method for identifying non-agricultural land based on prior knowledge and feature fusion according to claim 1, characterized in that: The step of generalizing the resolution of the cultivated land image to be identified to obtain a first generalized image specifically includes: Characterizing the farmland image to be identified to obtain a first image feature; and obtaining a second image feature based on the first image feature and a multi-head self-attention mechanism; fusing the first image feature with the second image feature to obtain a third image feature; According to the third image feature, a fourth image feature is obtained based on a multi-layer perceptron; Performing image embedding on the third image feature to obtain a fifth image feature; The fourth image feature and the fifth image feature are fused to obtain a first generalized image.
3. The method for identifying non-agricultural land based on prior knowledge and feature fusion according to claim 2 is characterized in that: The performing image embedding on the third image feature to obtain the fifth image feature specifically includes: According to common sense of multiple resolution domains, the third image feature is domain-commonly embedded to obtain multiple first embedded features; Performing domain embedding enhancement on the plurality of first embedding features respectively to obtain a plurality of second embedding features; The multiple second embedded features are concatenated to obtain a fifth image feature.
4. The method for identifying cultivated land non-agriculturalization based on prior knowledge and feature fusion according to claim 1, characterized in that: The generating a scale feature of the first generalized image to obtain a first scale feature specifically includes: Convolving the first generalized image according to a plurality of preset scales to obtain a plurality of first-scale embedded features; Based on a feature pyramid network, the plurality of first-scale embedded features are fused to obtain a first embedded image; Foreground prediction is performed on the first embedded image and optimized through binary cross entropy loss to obtain a first scale feature.
5. The method for identifying non-agricultural land based on prior knowledge and feature fusion according to claim 1, characterized in that: The performing mask decoding on the first mask feature to obtain a first category probability specifically includes: Based on the multi-head self-attention mechanism, obtaining a first decoding feature according to the first mask feature; Based on a preset first category decoding method, obtaining a second decoding feature according to the first decoding feature; The second decoding feature is normalized, and a first category probability is obtained based on a multilayer perceptron and the first category decoding method.
6. The method for identifying non-agricultural land based on prior knowledge and feature fusion according to claim 5 is characterized in that: The first category decoding method specifically includes: Downsample and activate the input to be decoded to obtain the first intermediate feature; Downsampling and activating the first intermediate feature to obtain a second intermediate feature; Upsampling, activating, and upsampling the second intermediate feature to obtain a third intermediate feature; The third intermediate feature is combined with the input to be decoded to obtain a decoded output.
7. A device for identifying non-agriculturalization of cultivated land based on prior and feature fusion, characterized in that: It includes image generalization module, feature generation module, feature fusion module and decoding recognition module; The image generalization module is used to perform resolution generalization on the cultivated land image to be identified to obtain a first generalized image; The feature generation module is used to generate a scale feature for the first generalized image to obtain a first scale feature; The feature fusion module is used to obtain a first spatial feature and a first semantic feature by matching the first scale feature in the preset prior information; and to generate a fusion mask for the first scale feature, the first spatial feature and the first semantic feature to obtain a first mask feature; The decoding and identification module is used to perform mask decoding on the first mask feature to obtain a first category probability; and obtain the non-agricultural type of cultivated land corresponding to the cultivated land image to be identified according to the first category probability; The generation of the preset prior information specifically includes: performing a binarization transformation on the historical cultivated land spatial change data to obtain a priori spatial features; fusing the historical cultivated land time series change data, the historical remote sensing time series data and the preset prior image features, and extracting the model according to the preset time series features to obtain the prior time series features; obtaining the phenological characteristic curve according to the historical cultivated land phenological change data; and obtaining the prior phenological characteristics by combining the preset prior semantic features and the phenological characteristic curve; fusing the prior time series features and the prior phenological features to obtain the prior semantic features; and combining the prior spatial features and the prior semantic features to obtain the preset prior information.
8. The device for identifying non-agriculturalization of cultivated land based on prior knowledge and feature fusion according to claim 7, characterized in that: The image generalization module includes an image feature acquisition submodule, an image feature fusion submodule, a multi-layer perception submodule, an image embedding submodule and an image generalization submodule; The image feature acquisition submodule is used to characterize the cultivated land image to be identified to obtain a first image feature; and, based on the first image feature, obtain a second image feature based on a multi-head self-attention mechanism; The image feature fusion submodule is used to fuse the first image feature with the second image feature to obtain a third image feature; The multi-layer perception submodule is used to obtain a fourth image feature based on a multi-layer perceptron according to the third image feature; The image embedding submodule is used to perform image embedding on the third image feature to obtain a fifth image feature; The image generalization submodule is used to fuse the fourth image feature and the fifth image feature to obtain a first generalized image.
9. The device for identifying non-agriculturalization of cultivated land based on prior knowledge and feature fusion according to claim 8, characterized in that: The image embedding submodule includes a domain common embedding unit, a domain embedding enhancement unit and a feature splicing unit; The domain common embedding unit is used to perform domain common embedding on the third image feature according to multiple resolution domain common senses to obtain multiple first embedded features; The domain embedding enhancement unit is used to perform domain embedding enhancement on the plurality of first embedding features respectively to obtain a plurality of second embedding features; The feature stitching unit is used to stitch the multiple second embedded features to obtain a fifth image feature.