Multi-crop classification extraction method, system and device based on satellite remote sensing image and storage medium
Features are extracted through Fourier transform noise reduction and multi-head attention mechanism, combined with the binary classification cross-entropy loss function and feature weight adjustment, the problem of insufficient accuracy in remote sensing crop classification is solved, and the accuracy of multi-crop classification is improved.
Patent Information
- Application Number
- CN202411336481.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2025-01-10
AI Technical Summary
There are insufficient feature extraction and insufficient model accuracy in the existing remote sensing crop classification and extraction based on deep learning, resulting in the loss of classification accuracy and it is difficult to effectively improve the classification and extraction accuracy of multi-crop classification.
Fourier transform is used to perform high-frequency noise reduction, and the sample category balance weight is assigned to multi-crops based on the binary classification cross entropy loss function, and the multi-head attention mechanism is used to extract spectral features, vegetation features and texture features, and multi-crops classification extraction model is trained. And adjust the feature weights according to the classification results to improve the model accuracy.
It effectively reduces the classification accuracy loss caused by insufficient feature extraction and insufficient model accuracy, and improves the accuracy of multi-crop classification extraction.
Smart Images
Figure CN119295962A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of multi-crop classification extraction, and in particular to a multi-crop classification extraction method, system, equipment and storage medium based on satellite remote sensing images. Background Art
[0002] With the continuous development of technologies such as big data, remote sensing, spatial information and the Internet of Things, remote sensing images taken by optical remote sensing satellites can be used to select and analyze various spectral information and spatial information characteristics of ground objects in the images, so as to obtain the corresponding match between the image and the actual ground objects, thereby realizing the classification of ground objects in remote sensing images. The use of remote sensing technology can objectively, accurately and timely provide various information on the ecological environment and crop growth of crops. It is an important source of field data for precision agriculture. The fine classification of crops is of great significance for crop growth monitoring, yield estimation, disaster assessment, etc. It is also an important basis for the rational allocation of resources and precise fertilization in the agricultural production process.
[0003] In the research of classification and extraction of remote sensing crops, there are two main breakthroughs. First, using the characteristics of the crop itself, such as spectral characteristics and texture characteristics, as additional information to fuse multi-source information in order to achieve better results in classification and extraction; second, relying on the most efficient and economical acquisition of crop panchromatic remote sensing images, and achieving the best results by comparing or improving convolutional neural networks. At present, in the classification and extraction of remote sensing crops based on deep learning, there is still a loss of classification accuracy due to insufficient feature extraction and insufficient model accuracy.
[0004] Therefore, how to provide a multi-crop classification extraction method, system, equipment and storage medium based on satellite remote sensing images that can effectively reduce the loss of classification accuracy due to insufficient feature extraction and insufficient model accuracy and effectively improve the accuracy of multi-crop classification extraction is an urgent problem that needs to be solved by technical personnel in this field. Summary of the invention
[0005] In view of this, the present invention proposes a multi-crop classification extraction method, system, device and storage medium based on satellite remote sensing images.
[0006] In order to achieve the above object, the present invention adopts the following technical solution:
[0007] A multi-crop classification extraction method based on satellite remote sensing images, comprising:
[0008] Step 1: Obtain satellite remote sensing sample images containing multiple crops and perform high-frequency noise reduction based on Fourier transform;
[0009] Step 2: Based on the binary cross entropy loss function, sample category balance weights are assigned to multiple crops, and a multi-head attention mechanism is used to perform feature extraction and train a multi-crop classification extraction model; the extracted features include: spectral features, vegetation features, and texture features;
[0010] Step 3: Based on the difference between the multi-crop classification extraction result and the sample label, change the corresponding weights of the extracted features in the multi-crop classification extraction model until the accuracy threshold is met, input the multi-crop satellite remote sensing image to be classified and extracted, and obtain the multi-crop classification extraction result.
[0011] Optionally, in step 1, after obtaining the satellite remote sensing sample image containing multiple crops, it also includes: declouding, orthorectification, radiation calibration, atmospheric correction, and image mosaicking and cropping.
[0012] Optionally, in step 1, high frequency noise reduction is performed based on Fourier transform, specifically:
[0013] The satellite remote sensing sample images are subjected to high-frequency noise reduction processing through a first preset number of sequentially connected processing layers; wherein each processing layer comprises: a dual convolution network for downsampling the input image, a fast Fourier transform module for frequency domain filtering the input image after the downsampling processing, a fast inverse Fourier transform module for restoring the image obtained by multiplying the input image after the frequency domain filtering processing with the weight tensor determined by the training of the multi-crop classification extraction model, and an output module for adding and outputting the input image after the downsampling processing and the image after the restoration processing; wherein the input image of each processing layer is the output of the upper processing layer, and the output of the last processing layer is the image of the satellite remote sensing sample image after the high-frequency noise reduction processing.
[0014] Optionally, in step 2, based on the binary cross entropy loss function, sample category balance weights are assigned to multiple crops, specifically:
[0015] The binary cross entropy loss function is transformed as follows:
[0016] f(p,y)=-lgp t ;
[0017]
[0018] Among them, p is the probability that the sample is predicted to be positive; y is the label of the sample, the positive class is 1 and the negative class is 0;
[0019] Assign sample class balance weights to each class as follows:
[0020] f(p t )=-α t(1-p t ) γ lgp t ;
[0021] Among them, α t is the weight coefficient; (1-p t ) γ is the adjustment coefficient; γ is the adjustable focusing parameter.
[0022] Optionally, in step 2, spectral features include: red band, green band, blue band, near infrared band, and shortwave infrared band of remote sensing images; vegetation features include: normalized vegetation index, ratio vegetation index, and enhanced vegetation index; texture features include: median, covariance, and entropy.
[0023] Optionally, in step 2, a multi-head attention mechanism is used to train feature extraction and multi-crop classification extraction models, specifically:
[0024] The multi-head attention mechanism is used to extract the features of red band, green band, blue band, near-infrared band, normalized vegetation index, ratio vegetation index, enhanced vegetation index, median, covariance and entropy from the images after high-frequency noise reduction.
[0025] A second preset number of sequentially connected Swin-Transformer modules are used to decode the feature image extracted using the multi-head attention mechanism, specifically: the Swin-Transformer module is used to decode the connected image of the up-sampled input image and the image after high-frequency noise reduction processing; wherein the input image of each Swin-Transformer module is the output image of the previous Swin-Transformer module.
[0026] Optionally, after decoding the feature image extracted using the multi-head attention mechanism using a second preset number of sequentially connected Swin-Transformer modules, it also includes: upsampling the output image of the last Swin-Transformer module to obtain an image of the same size as the multi-crop satellite remote sensing image to be classified and extracted.
[0027] The present invention also provides a multi-crop classification extraction system based on satellite remote sensing images using a multi-crop classification extraction method based on satellite remote sensing images, comprising:
[0028] High-frequency noise reduction module: used to obtain satellite remote sensing sample images containing multiple crops and perform high-frequency noise reduction based on Fourier transform;
[0029] Multi-crop classification extraction model training module: Based on the binary cross entropy loss function, sample category balance weights are assigned to multiple crops, and a multi-head attention mechanism is used to perform feature extraction and multi-crop classification extraction model training; the extracted features include: spectral features, vegetation features, and texture features;
[0030] Multi-crop classification extraction result acquisition module: Based on the difference between the multi-crop classification extraction result and the sample label, the corresponding weights of the extracted features in the multi-crop classification extraction model are changed until the accuracy threshold is met, and the multi-crop satellite remote sensing images to be classified and extracted are input to obtain the multi-crop classification extraction results.
[0031] The present invention also provides an electronic device, comprising:
[0032] Memory for storing computer programs;
[0033] The processor is used to implement the steps of a multi-crop classification and extraction method based on satellite remote sensing images when executing a computer program.
[0034] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a multi-crop classification extraction method based on satellite remote sensing images are implemented.
[0035] It can be seen from the above technical solutions that compared with the prior art, the present invention proposes a multi-crop classification extraction method, system, device and storage medium based on satellite remote sensing images. The present invention performs high-frequency noise reduction based on Fourier transform, and based on the high-frequency noise reduction image, uses a binary cross entropy loss function to assign sample category balance weights to multiple crops, and uses a multi-head attention mechanism to extract spectral features, vegetation features and texture features to train a multi-crop classification extraction model, and based on the difference between the multi-crop classification extraction results and the sample labels, changes the corresponding weights of the extracted features in the multi-crop classification extraction model, further improves the model accuracy, and achieves an effective reduction in the classification accuracy loss caused by insufficient feature extraction and insufficient model accuracy in the existing remote sensing crop classification extraction based on deep learning, and effectively improves the multi-crop classification extraction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0037] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0039] Embodiment 1:
[0040] Embodiment 1 of the present invention discloses a method for extracting multiple crops by classification based on satellite remote sensing images. Figure 1 As shown, including:
[0041] Step 1: Obtain satellite remote sensing sample images containing multiple crops and perform high-frequency noise reduction based on Fourier transform.
[0042] After obtaining satellite remote sensing sample images containing multiple crops, the process also includes: declouding, orthorectification, radiation calibration, atmospheric correction, and image mosaicking and cropping.
[0043] In view of the noise generated in the remote sensing imaging process, the present invention proposes a method for high-frequency noise reduction based on Fourier transform, specifically:
[0044] The satellite remote sensing sample image is subjected to high-frequency noise reduction processing through a first preset number of sequentially connected processing layers; wherein each processing layer comprises: a dual convolution network for downsampling the input image, a fast Fourier transform module for frequency domain filtering the input image after the downsampling processing, wherein multiple fast Fourier transform operations can effectively suppress high-frequency noise in the imaging process, a fast inverse Fourier transform module for restoring the image obtained by multiplying the input image after the frequency domain filtering processing with the weight tensor determined by the training of the multi-crop classification extraction model, and an output module for adding and outputting the input image after the downsampling processing and the image after the restoration processing, wherein the important features can be effectively retained by adding the input image after the downsampling processing and the image after the Fourier transform restoration as the output of the processing layer; wherein the input image of each processing layer is the output of the upper processing layer, and the output of the last processing layer is the image of the satellite remote sensing sample image after the high-frequency noise reduction processing.
[0045] Step 2: Based on the binary cross entropy loss function, sample category balance weights are assigned to multiple crops, and the multi-head attention mechanism is used to perform feature extraction and train the multi-crop classification extraction model; the extracted features include: spectral features, vegetation features, and texture features.
[0046] Based on the imbalanced number of pixels in each category in multi-crop remote sensing images (among which background pixels (other categories) are the largest, while building category pixels are the least, with a very large difference), after the imbalanced samples are input into the network model, due to the excessive proportion of a certain category, with the back propagation of the network, the focus is only on a certain category, the feature extraction of a certain category is significantly enhanced, while the feature extraction of other categories is weakened, affecting the overall segmentation accuracy. The present invention proposes a method based on a binary cross entropy loss function to allocate sample category balance weights for multiple crops to balance the negative impact of the imbalance of positive and negative samples in the network, specifically:
[0047] The binary cross entropy loss function is transformed as follows:
[0048] f(p,y)=-lgp t ;
[0049]
[0050] Among them, p is the probability that the sample is predicted to be positive; y is the label of the sample, the positive class is 1 and the negative class is 0;
[0051] Assign sample class balance weights to each class as follows:
[0052] fp t =-α t lgp t
[0053] Among them, α t is the weight coefficient; since the error loss of easy-to-classify samples accounts for most of the overall loss, in order to make the network focus on training difficult samples, the adjustment coefficient is increased again to reduce the weight of easy-to-classify samples, then:
[0054] f(p t )=-α t (1-p t ) γ lgp t ;
[0055] Among them, α t is the weight coefficient; (1-p t ) γ is the adjustment coefficient; γ is the adjustable focusing parameter.
[0056] Aiming at the problem of insufficient feature extraction in the existing remote sensing crop classification extraction model, the present invention proposes feature extraction for spectral features, vegetation features and texture features, wherein the spectral features include: red band, green band, blue band, near infrared band and short-wave infrared band of remote sensing images; the vegetation features include: normalized vegetation index, ratio vegetation index and enhanced vegetation index; the texture features include: median, covariance and entropy.
[0057] The normalized vegetation index is as follows:
[0058]
[0059] Among them, NDIV is the normalized difference vegetation index, NIR is the reflectance value of the near-infrared band, and R is the reflectance value of the red band.
[0060] The ratio vegetation index is as follows:
[0061] RVI = NIR / R;
[0062] Among them, RVI is the ratio vegetation index.
[0063] Enhanced vegetation index, as follows:
[0064]
[0065] Among them, EVI is the enhanced vegetation index; B is the reflectance value of the blue band.
[0066] Entropy, as follows:
[0067]
[0068] Among them, Entropy is entropy; P i is the proportion of pixels with gray value i in the remote sensing image. The multi-head attention mechanism is used to train the feature extraction and multi-crop classification extraction model, specifically:
[0069] The multi-head attention mechanism is used to extract the features of red band, green band, blue band, near-infrared band, normalized vegetation index, ratio vegetation index, enhanced vegetation index, median, covariance and entropy from the images after high-frequency noise reduction.
[0070] A second preset number of sequentially connected Swin-Transformer modules are used to decode the feature image extracted using the multi-head attention mechanism, specifically: the Swin-Transformer module is used to decode the connected image of the up-sampled input image and the image after high-frequency noise reduction processing; wherein the input image of each Swin-Transformer module is the output image of the previous Swin-Transformer module.
[0071] After decoding the feature image extracted by the multi-head attention mechanism using a second preset number of sequentially connected Swin-Transformer modules, the method also includes: upsampling the output image of the last Swin-Transformer module to obtain an image with the same size as the multi-crop satellite remote sensing image to be classified and extracted.
[0072] Step 3: Based on the difference between the multi-crop classification extraction result and the sample label, change the corresponding weights of the extracted features in the multi-crop classification extraction model until the accuracy threshold is met, input the multi-crop satellite remote sensing image to be classified and extracted, and obtain the multi-crop classification extraction result.
[0073] Embodiment 2:
[0074] Embodiment 2 of the present invention discloses a multi-crop classification extraction system based on satellite remote sensing images using a multi-crop classification extraction method based on satellite remote sensing images, comprising:
[0075] High-frequency noise reduction module: used to obtain satellite remote sensing sample images containing multiple crops and perform high-frequency noise reduction based on Fourier transform;
[0076] Multi-crop classification extraction model training module: Based on the binary cross entropy loss function, sample category balance weights are assigned to multiple crops, and a multi-head attention mechanism is used to perform feature extraction and multi-crop classification extraction model training; the extracted features include: spectral features, vegetation features, and texture features;
[0077] Multi-crop classification extraction result acquisition module: Based on the difference between the multi-crop classification extraction result and the sample label, the corresponding weights of the extracted features in the multi-crop classification extraction model are changed until the accuracy threshold is met, and the multi-crop satellite remote sensing images to be classified and extracted are input to obtain the multi-crop classification extraction results.
[0078] Embodiment 3:
[0079] Embodiment 3 of the present invention discloses an electronic device, including:
[0080] Memory for storing computer programs;
[0081] The processor is used to implement the steps of a multi-crop classification and extraction method based on satellite remote sensing images when executing a computer program.
[0082] Embodiment 4:
[0083] Embodiment 4 of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a multi-crop classification extraction method based on satellite remote sensing images are implemented.
[0084] The embodiment of the present invention discloses a method, system, device and storage medium for multi-crop classification extraction based on satellite remote sensing images. The present invention performs high-frequency noise reduction based on Fourier transform, and based on the high-frequency noise reduction image, uses a binary classification cross entropy loss function to allocate sample category balance weights for multiple crops, uses a multi-head attention mechanism to extract spectral features, vegetation features and texture features to train a multi-crop classification extraction model, and based on the difference between the multi-crop classification extraction result and the sample label, changes the corresponding weights of the extracted features in the multi-crop classification extraction model, further improves the model accuracy, and achieves an effective reduction in the classification accuracy loss caused by insufficient feature extraction and insufficient model accuracy in the existing remote sensing crop classification extraction based on deep learning, and effectively improves the multi-crop classification extraction accuracy.
[0085] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0086] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A multi-crop classification extraction method based on satellite remote sensing images, characterized in that: include: Step 1: Obtain satellite remote sensing sample images containing multiple crops and perform high-frequency noise reduction based on Fourier transform; Step 2: Based on the binary cross entropy loss function, sample category balance weights are assigned to multiple crops, and a multi-head attention mechanism is used to perform feature extraction and train a multi-crop classification extraction model; wherein the extracted features include: spectral features, vegetation features, and texture features; Step 3: Based on the difference between the multi-crop classification extraction result and the sample label, the corresponding weights of the extracted features in the multi-crop classification extraction model are changed until the accuracy threshold is met, and the multi-crop satellite remote sensing image to be classified and extracted is input to obtain the multi-crop classification extraction result.
2. The method for extracting multiple crops by classification based on satellite remote sensing images according to claim 1, characterized in that: In step 1, after obtaining the satellite remote sensing sample image containing multiple crops, it also includes: declouding, orthorectification, radiation calibration, atmospheric correction, and image mosaicking and cropping.
3. The method for extracting multiple crops by classification based on satellite remote sensing images according to claim 1, characterized in that: In step 1, high-frequency noise reduction is performed based on Fourier transform, specifically: The satellite remote sensing sample image is subjected to high-frequency noise reduction processing through a first preset number of sequentially connected processing layers; wherein each processing layer includes: a dual convolutional network for downsampling the input image, a fast Fourier transform module for frequency domain filtering the input image after the downsampling processing, a fast inverse Fourier transform module for restoring the image obtained by multiplying the input image after the frequency domain filtering processing with the weight tensor determined by the training of the multi-crop classification extraction model, and an output module for adding and outputting the input image after the downsampling processing and the image after the restoration processing; wherein the input image of each processing layer is the output of the upper processing layer, and the output of the last processing layer is the image of the satellite remote sensing sample image after the high-frequency noise reduction processing.
4. The method for extracting multiple crops by classification based on satellite remote sensing images according to claim 1, characterized in that: In step 2, based on the binary cross entropy loss function, sample class balance weights are assigned to multiple crops, specifically: The binary cross entropy loss function is transformed as follows: f(p,y)=-lgp t ; Among them, p is the probability that the sample is predicted to be positive; y is the label of the sample, the positive class is 1 and the negative class is 0; Assign sample class balance weights to each class as follows: f(p t )=-α t (1-p t ) γ air t ; Among them, α t is the weight coefficient; (1-p t ) γ is the adjustment coefficient; γ is the adjustable focusing parameter.
5. The method for extracting multiple crops by classification based on satellite remote sensing images according to claim 1, characterized in that: In step 2, the spectral characteristics include: red band, green band, blue band, near infrared band, and short-wave infrared band of the remote sensing image; the vegetation characteristics include: normalized vegetation index, ratio vegetation index, and enhanced vegetation index; the texture characteristics include: median, covariance, and entropy.
6. The method for extracting multiple crops by classification based on satellite remote sensing images according to claim 5, characterized in that: In step 2, the multi-head attention mechanism is used to train the feature extraction and multi-crop classification extraction model, specifically: Using a multi-head attention mechanism to extract features of the red band, green band, blue band, near infrared band, normalized vegetation index, ratio vegetation index, enhanced vegetation index, median, covariance and entropy from the image after high-frequency noise reduction processing; A second preset number of sequentially connected Swin-Transformer modules are used to decode the feature image extracted using the multi-head attention mechanism, specifically: a Swin-Transformer module is used to decode the connected image of the upsampled input image and the image after the high-frequency noise reduction process; wherein the input image of each Swin-Transformer module is the output image of the previous Swin-Transformer module.
7. The method for extracting multiple crops by classification based on satellite remote sensing images according to claim 6, characterized in that: After decoding the feature image extracted by the multi-head attention mechanism using a second preset number of sequentially connected Swin-Transformer modules, the method further includes: upsampling the output image of the last Swin-Transformer module to obtain an image of the same size as the multi-crop satellite remote sensing image to be classified and extracted.
8. A system for extracting multiple crops based on satellite remote sensing images using the method for extracting multiple crops based on satellite remote sensing images according to any one of claims 1 to 7, characterized in that: include: High-frequency noise reduction module: used to obtain satellite remote sensing sample images containing multiple crops and perform high-frequency noise reduction based on Fourier transform; Multi-crop classification extraction model training module: Based on the binary cross entropy loss function, sample category balance weights are assigned to multiple crops, and a multi-head attention mechanism is used to perform feature extraction and multi-crop classification extraction model training; wherein the extracted features include: spectral features, vegetation features, and texture features; Multi-crop classification extraction result acquisition module: based on the difference between the multi-crop classification extraction result and the sample label, the corresponding weights of the features extracted in the multi-crop classification extraction model are changed until the accuracy threshold is met, and the multi-crop satellite remote sensing images to be classified and extracted are input to obtain the multi-crop classification extraction results.
9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of a multi-crop classification extraction method based on satellite remote sensing images as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the multi-crop classification extraction method based on satellite remote sensing images as described in any one of claims 1 to 7 are implemented.