Multi-convolution residual network crop classification method and system based on multi-source satellite remote sensing

Abstract

The invention belongs to the technical field of remote sensing image crop classification, and particularly discloses a multi-convolution residual network crop classification method and system based on multi-source satellite remote sensing, and the method comprises the steps: obtaining crop image data; constructing a multi-convolution residual network model of stacked multi-layer convolution operation, causal convolution, expansion convolution and residual connection; and inputting the crop image data into the multi-convolution residual network model, and outputting to obtain a crop classification result. The satellite image pixel-by-pixel classification method solves the problem of poor classification effect in traditional satellite image pixel-by-pixel classification.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image crop classification technology, specifically relating to a crop classification method and system based on multi-source satellite remote sensing multi-convolutional residual networks. Background Technology

[0002] Remote sensing classification and area extraction of crops are the primary steps in agricultural remote sensing monitoring of crops, and the foundation for extracting crop planting information and area, providing basic information for crop growth status, yield, and disaster monitoring. Crop growth is influenced by multiple factors such as temperature and soil conditions, resulting in both commonalities and differences in their growth patterns. Due to varying light sensitivities among different crops, coupled with differences in planting structure and scale, they exhibit unique spectral and spatial characteristics in remote sensing imagery. Therefore, analyzing the differences in characteristics among crops in remote sensing imagery allows for effective identification and classification of crop types.

[0003] In remote sensing imagery-based crop classification research, a common method is to use single-phase remote sensing imagery. This method utilizes a single satellite image as the data source for crop classification. Existing studies on crop classification and zoning using satellite remote sensing imagery are mostly based on single-period, single-satellite imagery, which is fast and efficient, but also has some drawbacks. When the crop planting structure within a region is complex, different crops may exhibit similar spectral characteristics, creating a "same spectrum, different objects" phenomenon, making it difficult for a single remote sensing image to effectively distinguish them, thus affecting the accuracy of crop classification. Furthermore, when using only a single remote sensing image for crop classification and identification, it is difficult to determine the optimal time for classification. The lack of standardized and unified remote sensing image selection criteria may ultimately increase the uncertainty in crop classification accuracy. From extracting crop information by fusing different satellite data to employing multi-sensor data fusion technology, researchers are constantly exploring various methods to improve the accuracy and reliability of crop classification. However, multi-source data fusion also faces problems such as heavy computational burden and poor real-time performance, especially the difficulty of fusing data from different sensors.

[0004] The selection of remote sensing data sources and classification methods will directly affect the extraction accuracy. The comprehensive application of multi-source data fusion and deep learning algorithms will greatly improve the accuracy of crop classification and zoning extraction. However, the following problems also exist: (1) There are defects in using a single period and a single satellite image for crop classification. When the crop planting structure is complex, there may be a situation of "different objects with the same spectrum", that is, the spectral characteristics of multiple crops are similar and it is difficult to distinguish them effectively. This limits the accuracy of crop classification extracted by a single remote sensing image. Multi-source remote sensing data has advantages in improving classification accuracy, but it faces the problem of heavy computational burden. (2) The problem of selecting the best classification period: For crop classification using only one or two periods of remote sensing images, it is impossible to select the best classification period for crops. Due to the lack of standardized discrimination criteria, remote sensing images may increase uncertainty factors during the screening process, thereby affecting the final crop classification accuracy. Summary of the Invention

[0005] The purpose of this invention is to solve the problem of poor classification performance in traditional satellite image pixel-by-pixel classification, and to propose a crop classification method and system based on multi-source satellite remote sensing multi-convolutional residual networks.

[0006] The technical solution of the present invention is as follows: Firstly, a crop classification method based on multi-source satellite remote sensing multi-convolutional residual networks, comprising the following steps: Acquire crop image data; Construct a multi-convolutional residual network model; Crop image data is input into a multi-convolutional residual network model, and crop classification results are output.

[0007] Preferably, the crop image data includes radar VV polarization data, radar VH polarization data, radar VV / VH polarization data, images of eight bands from B2 to B8A of Sentinel-2 satellite imagery, and vegetation indices.

[0008] Preferably, the eight bands of the Sentinel 2 satellite imagery from B2 to B8A are: blue light band imagery, green light band imagery, red light band imagery, near-infrared red edge band imagery, near-infrared band imagery, shortwave infrared 1 band imagery, shortwave infrared 2 band imagery, and mid-infrared band imagery.

[0009] Preferably, the vegetation indices include: normalized difference vegetation index, normalized difference red-edge vegetation index, optimized soil-regulated vegetation index, greenness normalized vegetation index, chlorophyll index, ratio vegetation index, enhanced vegetation index, and red-edge normalized vegetation index.

[0010] Preferably, the multi-convolutional residual network model includes multiple stacked convolutional residual modules; Each of the convolutional residual modules includes a residual connection unit, and a dilated convolutional layer, a causal convolutional layer, a ReLU layer, and a Dropout layer connected in sequence; The input of the residual connection unit and the input of the dilated convolutional layer are both inputs of the convolutional residual module. The output of the residual connection unit is connected to the output of the Dropout layer, and the output of the Dropout layer is the output of the convolutional residual module. The ReLU layer is used to introduce nonlinear characteristics into the multi-convolutional residual network model and alleviate the gradient vanishing problem; The Dropout layer is used to prevent overfitting of multi-convolutional residual network models.

[0011] Preferably, the causal convolutional layer ensures that the convolutional kernel accesses the current and previous time steps by padding the left side of the input crop image data with a preset number of zeros; the kernel size of the causal convolutional layer is 3.

[0012] Preferably, the dilated convolutional layer introduces an exponentially increasing dilation rate between the convolutional kernels, allowing each convolutional operation to cover a larger range of input.

[0013] Preferably, the construction of the multi-convolutional residual network model includes the following steps: Acquire crop image data and field category vector boundary files; Use the class field to label the polygon features in the field vector boundary file with category labels; Treat each polygon feature as an independent sample and extract image features within its range; A sample dataset was constructed based on crop image data and extracted image features; The sample dataset is input into the multi-convolutional residual network model for training, resulting in a trained multi-convolutional residual network model.

[0014] The beneficial effects of this invention are: 1. This invention achieves efficient feature extraction and long-distance dependency modeling of crop image data by stacking multiple layers of convolution operations, causal convolution, dilated convolution, and residual connections.

[0015] 2. By setting the kernel size of the causal convolutional layer to 3, the basic local feature extraction capability of the multi-convolutional residual network model for crop image data is guaranteed.

[0016] 3. By employing an exponentially increasing dilation rate in dilated convolutional layers, a larger receptive field can be achieved in multi-convolutional residual network models, which helps to capture contextual information in crop image data while reducing computational overhead.

[0017] 4. The residual connection unit adds the input features directly to the output of the convolutional layer through skip connections, which can effectively alleviate the gradient vanishing problem when extracting crop image features in multi-convolutional residual network models and promote the effective transfer of crop image features in multi-convolutional residual network models.

[0018] In a second aspect, a crop classification system based on multi-source satellite remote sensing and multi-convolutional residual networks is provided, the system comprising a processor for executing the crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks as described in the first aspect.

[0019] Thirdly, a computer-readable storage medium stores computer instructions, and in response to a computer reading the computer instructions in the storage medium, the computer executes the crop classification method based on multi-source satellite remote sensing multi-convolutional residual networks as described in the first aspect. Attached Figure Description

[0020] Figure 1 The diagram shows a flowchart of a crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks.

[0021] Figure 2 The diagram shows the overall architecture of a multi-convolutional residual network model.

[0022] Figure 3 The image shows a schematic diagram of crop classification results for a certain region in 2024. Detailed Implementation

[0023] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the drawings are merely exemplary and are intended to illustrate the principles and spirit of the invention, and are not intended to limit the scope of the invention.

[0024] Example 1: like Figure 1 As shown, a crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks includes the following steps: S1. Acquire crop image data; S2. Construct a multi-convolutional residual network model; S3. Input crop image data into a multi-convolutional residual network model and output the crop classification results.

[0025] In this embodiment, the crop image data includes radar VV polarization data, radar VH polarization data, radar VV / VH polarization data, images of eight bands from B2 to B8A of Sentinel-2 satellite imagery, and vegetation indices.

[0026] The imagery of the eight bands B2-B8A from the Sentinel-2 satellite is as follows: B2 (Blue Light Band Image): Wavelength 490nm, resolution 10 meters, used to detect the reflectivity of water and vegetation; B3 (Green band image): wavelength 560nm, resolution 10 meters, used for vegetation health monitoring; B4 (red band image): wavelength 665nm, resolution 10m, used for vegetation and soil analysis; B5 (Near-infrared red-edge band image): wavelength 700nm, resolution 20m, used for vegetation moisture and density analysis; B6 (Near-infrared band image): wavelength 740nm, resolution 20m, used for vegetation and soil analysis; B7 (Shortwave Infrared Band 1 Image): Wavelength 890nm, resolution 20m, used for cloud and atmospheric aerosol detection; B8 (Shortwave Infrared Band 2 Image): Wavelength 940nm, resolution 20m, used for cloud and atmospheric aerosol detection; B8A (mid-infrared image): wavelength 1250nm, resolution 60m, used for vegetation and soil analysis.

[0027] The vegetation indices include: Normalized Difference Vegetation Index (NDVI), Normalized Difference Red Edge Vegetation Index (NDRE), Optimized Soil Regulation Vegetation Index (OSAVI), Normalized Greenness Vegetation Index (GNDVI), Chlorophyll Index (LCI), Ratio Vegetation Index (RVI), Enhanced Vegetation Index (EVI), and Normalized Red Edge Vegetation Index (RDVI).

[0028] In this embodiment, a Multi-Convolutional Residual Network (MCRN) model is proposed. This innovative model, based on Convolutional Neural Networks (CNNs), processes one-dimensional data through causal convolution and dilated convolution. The MCRN model comprises multiple stacked convolutional residual modules. Each of the convolutional residual modules includes a residual connection unit, and a dilated convolutional layer, a causal convolutional layer, a ReLU layer, and a Dropout layer connected in sequence; The input of the residual connection unit and the input of the dilated convolutional layer are both inputs of the convolutional residual module. The output of the residual connection unit is connected to the output of the Dropout layer, and the output of the Dropout layer is the output of the convolutional residual module. The ReLU layer is used to introduce nonlinear characteristics into the multi-convolutional residual network model and alleviate the gradient vanishing problem; The Dropout layer is used to prevent overfitting of multi-convolutional residual network models.

[0029] The causal convolutional layer pads the left side of the input crop image data with a preset number of zeros to ensure that the convolutional kernel accesses the current and previous time steps, thereby ensuring that the output does not depend on future information. The kernel size of the causal convolutional layer is 3 to ensure basic local feature extraction capability, and the stride is set to 1 to maintain the time dimension of the feature map. In addition, to keep the input and output lengths consistent, the padding size is set to 1 to compensate for the information offset caused by the causal structure.

[0030] The kernel size of the dilated convolutional layer is set to 3 to balance local feature extraction and global receptive field expansion. The dilated convolutional layer introduces an exponentially increasing dilation rate between the kernels, i.e., the dilation rate is 1 for the first layer and 2 for the second layer, so that each convolution operation covers a larger range of input.

[0031] To effectively alleviate the gradient vanishing problem in multi-convolutional residual network (MCRN) models when extracting crop image features, and to promote the effective transfer of crop image features within MCRN models, this invention proposes an MCRN model that introduces residual connections. Residual connections, through skip connections, directly add input features to the output of convolutional layers, thereby providing an identity mapping path during training, improving gradient flow, enhancing the network's optimization capabilities, and accelerating convergence. Experimental results show that the MCRN model, combining causal convolution, dilated convolution, and residual structures, can effectively model multi-level feature representations of data, exhibiting superior classification accuracy and generalization ability in pixel-by-pixel classification tasks. The overall architecture of the MCRN model is as follows: Figure 2 As shown.

[0032] The multi-convolutional residual network model proposed in this invention has a simple structure and is easy to train when extracting crop image features. It can efficiently process crop image data in parallel and avoid the gradient vanishing or exploding problems that are common in traditional networks when processing crop image data.

[0033] In this embodiment, constructing the multi-convolutional residual network model includes the following steps: Obtain crop image data and field category vector boundary files; the crop image data comes from the GEE (Google Earth Engine) platform, including 7 image data from April to July, and a field category vector boundary file collected in a certain city; The class field is used to label the polygon features in the field vector boundary file with category labels ranging from 1 to 4, representing cotton, jujube trees, corn, and others, respectively. Treat each polygon feature as an independent sample and extract image features within its range; Based on crop image data and extracted image features, a sample dataset was constructed. To ensure the generalization ability of the multi-convolutional residual network model, the dataset was randomly divided into a training set and a test set in an 8:2 ratio, with 80% of the data used for model training and 20% of the data used for model testing and validation. The sample dataset is input into the multi-convolutional residual network model for training, resulting in a trained multi-convolutional residual network model.

[0034] During the training of the multi-convolutional residual network model, comparative experiments were conducted using 1D-CNN, LSTM, and GRU to select the optimal classification model. Model performance was comprehensively evaluated using multiple metrics, including overall classification accuracy, precision, recall, F1 score, and Kappa coefficient. Accuracy measures the overall correctness of the model's classification; precision reflects the accuracy of the model's predictions for each class; recall evaluates the model's ability to identify each class; the F1 score is the harmonic mean of precision and recall, comprehensively reflecting the model's classification performance; and the Kappa coefficient assesses the consistency between the model's classification results and random classification, further validating the model's reliability. Furthermore, for each class, precision, recall, and F1 score were calculated separately to analyze the differences in model performance across different classes. Experimental results are shown in Table 1 and... Figure 3 As shown, the results are a comparative evaluation of the performance of multiple models and the crop classification results of a certain region in 2024. It can be seen that the multi-convolutional residual network model proposed in this invention has better performance in crop classification.

[0035] Table 1 Model Performance Evaluation

[0036] Example 2: Based on Example 1, this embodiment of the invention provides a crop classification system based on multi-source satellite remote sensing multi-convolutional residual network, which is used to configure and execute a crop classification method based on multi-source satellite remote sensing multi-convolutional residual network in Example 1.

[0037] In this embodiment, the system may be an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor. The processor executes the program to implement some or all of the steps of the crop classification method based on multi-source satellite remote sensing and multi-convolutional residual network as described in Embodiment 1.

[0038] In this embodiment, the electronic device may include: a processor, a memory, a bus, and a communication interface. The processor, the communication interface, and the memory are connected via the bus. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes some or all of the steps of the crop classification method based on multi-source satellite remote sensing provided in Embodiment 1 of this application.

[0039] The system in this embodiment of the invention may also be a computer-readable storage medium storing a computer program that, when executed, implements some or all of the steps of the crop classification method based on multi-source satellite remote sensing and multi-convolutional residual network as described in Embodiment 1.

[0040] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0041] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0042] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0043] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0044] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0045] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks, characterized in that, The following steps are involved: Acquire crop image data; Construct a multi-convolutional residual network model; Crop image data is input into a multi-convolutional residual network model, and crop classification results are output.

2. The crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks according to claim 1, characterized in that, The crop image data includes radar VV polarization data, radar VH polarization data, radar VV / VH polarization data, images of eight bands from B2 to B8A of Sentinel-2 satellite imagery, and vegetation indices.

3. The crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks according to claim 2, characterized in that, The eight bands of the Sentinel-2 satellite imagery from B2 to B8A are: blue band imagery, green band imagery, red band imagery, near-infrared red edge band imagery, near-infrared band imagery, shortwave infrared 1 band imagery, shortwave infrared 2 band imagery, and mid-infrared band imagery.

4. The crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks according to claim 2, characterized in that, The vegetation indices include: Normalized Difference Vegetation Index, Normalized Difference Red-Edge Vegetation Index, Soil-Optimized Adjustment Vegetation Index, Greenness Normalized Vegetation Index, Chlorophyll Index, Ratio Vegetation Index, Enhanced Vegetation Index, and Red-Edge Normalized Vegetation Index.

5. The crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks according to claim 1, characterized in that, The multi-convolutional residual network model includes multiple stacked convolutional residual modules; Each of the convolutional residual modules includes a residual connection unit, and a dilated convolutional layer, a causal convolutional layer, a ReLU layer, and a Dropout layer connected in sequence; The input of the residual connection unit and the input of the dilated convolutional layer are both inputs of the convolutional residual module. The output of the residual connection unit is connected to the output of the Dropout layer, and the output of the Dropout layer is the output of the convolutional residual module. The ReLU layer is used to introduce nonlinear characteristics into the multi-convolutional residual network model and alleviate the gradient vanishing problem; The Dropout layer is used to prevent overfitting of multi-convolutional residual network models.

6. The crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks according to claim 5, characterized in that, The causal convolutional layer ensures that the convolutional kernel accesses the current and previous time steps by padding the left side of the input crop image data with a preset number of zeros; the kernel size of the causal convolutional layer is 3.

7. The crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks according to claim 5, characterized in that, The dilated convolutional layer introduces an exponentially increasing dilation rate between the convolutional kernels, allowing each convolutional operation to cover a larger range of the input.

8. The crop classification method based on multi-source satellite remote sensing and multi-convolutional residual networks according to claim 1, characterized in that, The construction of the multi-convolutional residual network model includes the following steps: Acquire crop image data and field category vector boundary files; Use the class field to label the polygon features in the field vector boundary file with category labels; Treat each polygon feature as an independent sample and extract image features within its range; A sample dataset was constructed based on crop image data and extracted image features; The sample dataset is input into the multi-convolutional residual network model for training, resulting in a trained multi-convolutional residual network model.

9. A crop classification system based on multi-source satellite remote sensing and multi-convolutional residual networks, characterized in that, The system includes a processor for executing the crop classification method based on multi-source satellite remote sensing multi-convolutional residual networks according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions. In response to the computer reading the computer instructions in the storage medium, the computer executes the crop classification method based on multi-source satellite remote sensing multi-convolutional residual network as described in any one of claims 1-8.

Images