Remote sensing image crop classification method based on TransUnet

Abstract

The invention discloses a remote sensing image crop classification method based on TransUnet, and relates to the technical field of remote sensing image processing. The method comprises the following steps: embedding a collaborative gating enhanced convolution module in a TransUnet network, embedding an attention coordination mechanism in tiger jump connection, and embedding a reverse probability edge optimization module in the TransUnet network; an embedded cooperative gating enhanced convolution module reduces the interference of background noise on feature extraction through the cooperative effect of self-attention convolution and gating convolution; the attention coordination mechanism enhances the distinguishing ability of similar crops by modeling cross-channel and spatial feature association; the reverse probability edge optimization module is used for balancing the learning deviation of the improved TransUnet network to each category and avoiding the boundary segmentation fuzzy problem of small sample crops due to insufficient feature learning; the crop remote sensing image is segmented through the improved TransUnet network, and a pixel-level classification result is obtained. The method improves the segmentation precision of the crop remote sensing image.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, and in particular to a crop classification method for remote sensing images based on TransUnet. Background Technology

[0002] Agriculture is the cornerstone of human survival and development, and food security is a global strategic issue. Precision agriculture, as the core direction of modern agricultural transformation and upgrading, relies on digital technology and scientific management to optimize resource input, improve production capacity and quality, and reduce environmental impact, making it a key path to solving the food security problem.

[0003] Data-driven approaches are at the core of precision agriculture, and remote sensing technology, with its ability to acquire information over large areas, across multiple time periods, and in multiple dimensions, has become a key support for data collection and analysis in precision agriculture. The scientific nature of its classification and application directly determines the effectiveness of precision agriculture implementation. Through the collaborative application of multi-source equipment such as satellites, drones, and ground sensors, remote sensing technology can capture the spatiotemporal dynamic characteristics of crops and soil in real time, providing a scientific basis for decisions such as precise farmland management and efficient resource allocation. It is a key technological support for the implementation of precision agriculture and is of great significance for promoting the large-scale and standardized application of precision agriculture.

[0004] In existing technologies, the standard Transformer model has been introduced into the field of crop classification for remote sensing images. This type of model, through a self-attention mechanism, can directly model the dependency relationship between any two pixels in the input sequence, thereby efficiently capturing global features. However, the standard Transformer model suffers from insufficient global context and local detail features when processing remote sensing crop images, further leading to lower classification accuracy. Summary of the Invention

[0005] Therefore, it is necessary to provide a crop classification method based on TransUnet for remote sensing images to address the aforementioned technical problems. This method improves the segmentation accuracy of crop remote sensing images.

[0006] The following technical solution is adopted in this specification: This specification provides a crop classification method based on TransUnet remote sensing imagery, including: Acquire remote sensing images of crops; An improved TransUnet network was obtained. The improvements to the TransUnet network include embedding a cooperative gated augmentation convolutional module after the third convolutional neural network in the traditional TransUnet network; embedding a coordinated attention mechanism (CA) in the skip connections between the encoder and decoder; and embedding an inverse probabilistic edge optimization module between the second upsampling module and the second feature concatenation module. The embedded cooperative gated augmentation convolutional module is used to reduce the interference of background noise on feature extraction through the synergistic effect of self-attention convolution and gated convolution. CA is used to enhance the distinguishability of similar crops by modeling cross-channel and spatial feature associations. The inverse probabilistic edge optimization module is used to balance the learning bias of the improved TransUnet network towards each category and avoid the problem of blurred boundary segmentation due to insufficient feature learning in small sample crops. The improved TransUnet network adopts a feature map transfer method based on spiral connections. The improved TransUnet network was used to segment crop remote sensing images, resulting in pixel-level classification results.

[0007] Optionally, a traditional TransUnet network includes an encoder and a decoder; the encoder includes a first convolutional neural network, a second convolutional neural network, a third convolutional neural network, a linear projection layer, multiple Transformer layers, downsampling, and a first set of 3×3 convolutions and the ReLU activation function, which are connected in series. The decoder includes a first upsampling module, a first feature concatenation module, a second set of 3×3 convolutions and ReLU, a second upsampling module, a second feature concatenation module, a third set of 3×3 convolutions and ReLU, a third upsampling module, a third feature concatenation module, a fourth set of 3×3 convolutions and ReLU, a fourth upsampling module, a fifth set of 3×3 convolutions and ReLU, and skip connections between the encoder's convolutional neural network and the decoder's feature concatenation module; The improved TransUnet network also includes a first reverse spiral reconstruction module, a second reverse spiral reconstruction module, and a third reverse spiral reconstruction module; The first reverse spiral reconstruction module is embedded in the first feature stitching module of the decoder; the second reverse spiral reconstruction module is embedded in the second feature stitching module of the decoder; the third reverse spiral reconstruction module is embedded in the third feature stitching module of the decoder; the reverse spiral reconstruction module is used to reconstruct the image during the decoding stage; The embedded collaborative gated augmentation convolution module includes a first 1×1 convolution, Conv, a first self-attention convolution DSA, a second self-attention convolution DSA, a first gated convolution GC, a second gated convolution GC, an adder, and a second 1×1 convolution; Conv includes a 3×3 convolution and ReLU; CA includes a residual connection module, X average pooling, Y average pooling, concatenation + convolution, batch normalization + nonlinear transformation, third and fourth 1×1 convolutions, first Sigmoid function, second Sigmoid function, and reweighting module.

[0008] Optionally, the crop remote sensing image is segmented using an improved TransUnet network to obtain pixel-level classification results, specifically including: The crop remote sensing image is extracted at a 1 / 2 scale, a 1 / 4 scale, and a 1 / 8 scale by sequentially passing through the first convolutional neural network, the second convolutional neural network, and the third convolutional neural network. The 1 / 8 scaled feature map is input into the collaborative gating enhancement module to obtain the first hidden feature, and the first hidden feature is linearly projected to obtain the projected feature; The projected features are input into multiple Transformer layers for processing. The global long-range dependencies of the features are captured through a self-attention mechanism to obtain the second hidden features. The second hidden feature is then passed through downsampling and the first set of 3×3 convolutions and the ReLU activation function in sequence to obtain the encoded feature; Based on the encoded features, the feature maps after CA processing at 1 / 2 scale, 1 / 4 scale, and 1 / 8 scale, multiple convolution operations, upsampling, and feature extraction operations are performed to obtain the decoded features. The decoded features are processed by a segmentation head to obtain pixel-level classification results for crop remote sensing images.

[0009] Optionally, the 1 / 8 scaled feature map is input into the collaborative gating enhancement module to obtain the first hidden feature, which specifically includes: The 1 / 8 scaled feature map is processed by the first 1×1 convolution to obtain the initial features; The initial features are input into Conv, the first DSA and the second DSA for convolution processing; The output of Conv and the output of the first DSA are input together into the first GC for feature extraction to obtain the first gated convolutional feature. The first gated convolutional feature and the output of the second DSA are both input into the second GC for convolution processing to obtain the second gated convolutional feature; The output of Conv, the first gated convolution feature, and the second gated convolution feature are fused and then processed by a second 1×1 convolution to obtain the first hidden feature.

[0010] Optionally, based on the encoded features, the CA-processed 1 / 2 scaled feature map, the CA-processed 1 / 4 scaled feature map, and the CA-processed 1 / 8 scaled feature map, multiple convolution operations, upsampling, and feature extraction operations are performed to obtain the decoded features, specifically including: The encoded features are processed through the first upsampling module to obtain the first upsampled features; The CA-processed 1 / 8 scaled feature map is then processed by the first reverse spiral reconstruction module to extract the index features, thus obtaining the first index features. The first feature concatenation module concatenates the first index feature with the first upsampled feature to obtain the first concatenated feature. The first concatenated feature is processed through a second set of 3×3 convolutions and ReLU, and a second upsampling module to obtain the second upsampled feature; The second upsampled feature is processed by the reverse probability edge optimization module to obtain the reverse probability edge optimized feature; The CA-processed 1 / 4 scaled feature map is then processed by the second reverse spiral reconstruction module to extract index features, resulting in the second index features. The second feature concatenation module concatenates the second index feature with the reverse probability edge optimization feature to obtain the second concatenated feature. The second concatenated feature is processed through a third set of 3×3 convolutions and ReLU, and a third upsampling module to obtain the third upsampling feature; The CA-processed 1 / 2 scaled feature map is then processed by the third reverse spiral reconstruction module to extract the index features, resulting in the third index features. The third feature concatenation module concatenates the third index feature with the third upsampled feature to obtain the third concatenated feature. The third concatenated feature is processed through the fourth group of 3×3 convolutions and ReLU, the fourth upsampling module, and the fifth group of 3×3 convolutions and ReLU to obtain the decoded feature.

[0011] Optionally, the second upsampled feature is processed by the inverse probabilistic edge optimization module to obtain the inverse probabilistic edge optimized feature, specifically including: The second upsampled feature is processed by the sigmoid activation function to obtain a probability map; the sigmoid activation function is used to emphasize the edge regions of the crop remote sensing image. The probability map is calculated in reverse to obtain the inverse probability map; the inverse probability map is used to distinguish the background and crop area in crop remote sensing images. The inverse probability map is multiplied element-wise with the CA-processed 1 / 4 scaled feature map to generate a new feature map; The new feature map and the second upsampled feature are fused to obtain the fused feature; the fused feature includes contextual information and fine edge location information. The fused features are processed by two convolutional layers to obtain the reverse probability edge optimization features.

[0012] Optionally, the CA-processed 1 / 2 scaled feature map is subjected to index feature extraction via a third reverse spiral reconstruction module to obtain the third index feature, which specifically includes: Calculate multiple reverse spiral coordinates based on different rotation angles; Convert multiple reverse spiral coordinates into multiple reverse Cartesian coordinates, and then define the multiple reverse Cartesian coordinates as multiple indices; Extract the third index features of multiple indexes from the feature map after CA processing at a 1 / 2 scaling ratio.

[0013] Optionally, the extraction process of the 1 / 8 scaling feature map after CA processing specifically includes: By performing residual connections on the 1 / 8 scaled feature map, residual connection features are obtained. The residual connection features are input into X-average pooling and Y-average pooling respectively to obtain X-average pooling features and Y-average pooling features; The nonlinear features are obtained by sequentially concatenating, convolving, batch normalizing, and performing nonlinear transformation on the X-average pooling features and Y-average pooling features. After inputting the nonlinear features into the third and fourth 1×1 convolutions for convolution processing, we obtain the X convolution features and the Y convolution features. The X convolutional features are input into the first Sigmoid function to obtain the first Sigmoid features; The Y convolutional features are input into the second Sigmoid function to obtain the second Sigmoid features; The first Sigmoid feature, the second Sigmoid feature, and the residual connection feature are weighted and fused through a reweighting module to obtain a feature map with a 1 / 8 scaling ratio after CA processing.

[0014] Optionally, feature maps at 1 / 2 scaling ratio, 1 / 4 scaling ratio, and 1 / 8 scaling ratio of the crop remote sensing image are extracted sequentially through a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network, specifically including: For any convolutional neural network layer, local features of the input data are extracted through the convolutional neural network to obtain the original feature map including basic scale information; wherein, the input data of the first convolutional neural network is the object remote sensing image, the input data of the second convolutional neural network is the output of the first convolutional neural network, and the input data of the third convolutional neural network is the output of the second convolutional neural network. The original feature map is divided into multiple regions. Starting from the center of the original feature map, multiple polar coordinates are calculated based on the spiral path formula, and the multiple polar coordinates are converted into multiple Cartesian coordinates respectively. Feature sampling and integration are performed by gradually expanding outward along multiple Cartesian coordinates to obtain a scaled feature map carrying information about the center and surrounding spatial structure.

[0015] Optionally, the loss function of the improved TransUnet network is: ; ; ; in, The loss value for the improved TransUnet network, The value of binary cross-entropy loss (BCEloss) is... This is the Dice loss value. For the pixels in the sample label i The value, Pixels in the segmentation prediction results of the improved TransUnet network i The value, N The number of samples.

[0016] This specification provides a crop classification device based on TransUnet remote sensing images, including: The acquisition module is used to acquire remote sensing images of crops; The acquisition module is used to acquire the improved TransUnet network. The improvements to the TransUnet network include embedding a cooperative gated augmentation convolutional module after the third convolutional neural network in the traditional TransUnet network; embedding a coordinated attention mechanism (CA) in the skip connections between the encoder and decoder; and embedding an inverse probabilistic edge optimization module between the second upsampling module and the second feature concatenation module. The embedded cooperative gated augmentation convolutional module reduces the interference of background noise on feature extraction through the synergistic effect of self-attention convolution and gated convolution. CA is used to enhance the distinguishability of similar crops by modeling cross-channel and spatial feature associations. The inverse probabilistic edge optimization module balances the learning bias of the improved TransUnet network towards each category and avoids the problem of blurred boundary segmentation due to insufficient feature learning in small sample crops. The improved TransUnet network adopts a feature map transfer method based on spiral connections. The classification module is used to segment crop remote sensing images using an improved TransUnet network to obtain pixel-level classification results for the crop remote sensing images.

[0017] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described TransUnet-based remote sensing image crop classification method.

[0018] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described TransUnet-based remote sensing image crop classification method.

[0019] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: The TransUnet-based remote sensing image crop classification method provided in this specification improves the TransUnet network by embedding a cooperatively gated enhanced convolutional module after the third convolutional neural network in the traditional TransUnet network, embedding a coordinated attention mechanism (CA) in the skip connections between the encoder and decoder, and embedding an inverse probabilistic edge optimization module between the second upsampling module and the second feature stitching module. The embedded cooperatively gated enhanced convolutional module reduces background noise interference to feature extraction through the synergistic effect of self-attention convolution and gated convolution. CA enhances the distinguishability of similar crops by modeling cross-channel and spatial feature associations. The inverse probabilistic edge optimization module balances the learning bias of the improved TransUnet network towards each category and avoids the problem of blurred boundary segmentation due to insufficient feature learning in small sample crops. The improved TransUnet network adopts a feature map transfer method based on spiral connections. By segmenting crop remote sensing images using the improved TransUnet network, this method improves the segmentation accuracy of crop remote sensing images.

[0020] Among them, the collaborative gated enhanced convolution module uses a three-stage process of fine local feature capture, global context modeling, and gated feature selection to filter out background noise and improve the purity of crop feature expression; CA enhances the ability to capture inter-class differences and alleviates classification confusion by modeling cross-channel-spatial feature association; and the reverse probabilistic edge optimization module dynamically strengthens the learning bias of the small sample crop boundary attention weight balancing model and improves the segmentation accuracy of small sample boundaries. Based on the collaborative design of the collaborative gated enhanced convolution module, CA, and reverse probabilistic edge optimization module, this invention achieves efficient capture and selection of local features and global context features, while optimizing the boundary segmentation effect of small sample crops, and comprehensively improves the segmentation accuracy of crop remote sensing images. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This document provides a flowchart illustrating a crop classification method based on TransUnet for remote sensing images. Figure 2 A schematic diagram of the improved TransUnet network provided by the present invention; Figure 3 A schematic diagram of the collaborative gated enhanced convolutional module LGCG-Conv provided by the present invention; Figure 4 This is a schematic diagram of the reverse probabilistic edge optimization (RP-EO) provided by the present invention; Figure 5 The CA structure diagram provided by this invention; Figure 6 An overview diagram of crop labels provided by this invention; Figure 7 The visualization results comparison chart provided by this invention; Figure 8 A schematic diagram of a crop classification device based on TransUnet remote sensing images provided in this specification; Figure 9 This is a schematic diagram of a computer device for implementing a crop classification method based on TransUnet remote sensing images, as provided in this specification. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0024] Devices such as desktop computers, servers, and laptops are capable of executing the solutions described in this manual. For ease of explanation, the following description will focus on servers as the primary execution method.

[0025] Agriculture, as the cornerstone of human survival and development, not only occupies a central position in the global economic landscape but also directly relates to the global strategic issue of food security. Despite a global agricultural output increase of approximately 17% over the past three decades and a significant improvement in food production capacity, nearly 821 million people worldwide still face food shortages. The United Nations predicts that the global population will reach 9.9 billion by 2050, requiring a further 70% increase in global food production to meet the resulting food demands. The dual pressures of population expansion and escalating food demand compel agricultural production to transform towards higher efficiency, precision, and sustainability while ensuring sufficient output.

[0026] Precision agriculture, as a core direction for the transformation and upgrading of modern agriculture, demonstrates significant advantages in optimizing resource input, improving crop yield and quality, and reducing environmental impact by integrating digital technologies and scientific management methods, providing an effective path to solving the food security problem. Data-driven approaches are a core characteristic of precision agriculture, and remote sensing technology, with its ability to acquire information over a wide area, across multiple time periods, and in multiple dimensions, has become a key support for data collection and analysis in precision agriculture. Through the collaborative application of multi-source equipment such as satellites, drones, and ground sensors, remote sensing technology can capture the spatiotemporal dynamic changes in crop morphology, physiological state, and soil physicochemical properties in real time, providing a scientific basis for decisions such as precise farmland management and efficient resource allocation. The depth and breadth of its application directly affect the effectiveness of precision agriculture implementation.

[0027] Crop extraction and classification, as a fundamental core task of agricultural remote sensing, is a prerequisite for accurate crop area calculation, real-time monitoring of growth dynamics, scientific yield prediction, and optimization of planting structure. It has significant practical implications for improving the precision of agricultural production management. my country's agricultural production areas generally exhibit typical characteristics such as scattered land use types, severe landscape fragmentation, and complex crop planting structures. This is particularly true in major grain-producing areas like Northeast China, where mixed crop cultivation and overlapping plot boundaries significantly increase the difficulty of crop classification. In recent years, the continuous enrichment of remote sensing data resources has provided support for solving this problem: high temporal resolution MODIS data and medium spatial resolution Landsat series data have been widely used in large-scale crop monitoring; while Sentinel-1 and Sentinel-2 data, with their advantages of 10-meter high spatial resolution and a 12-day revisit period, provide high-quality data support for refined crop classification at the regional scale, effectively promoting improvements in classification accuracy and efficiency.

[0028] Supported by multi-source remote sensing data, the deep integration of deep learning technology and agricultural remote sensing has become a research hotspot. Semantic segmentation methods in computer vision are particularly key to driving continuous breakthroughs in crop classification performance. Researchers successfully achieved accurate differentiation between land cover and crop types in a certain region using U-Net and its simplified encoder-decoder structure. Their proposed Multi-Scale Fully Convolutional Network (MSFCN), integrating multi-scale feature extraction and attention mechanisms, demonstrated its superior classification performance on complex land cover datasets. For crop classification needs in a specific region, researchers combined spatial attention mechanisms with U-Net's multi-scale feature fusion, significantly improving the accuracy of crop classification in the Harbin area. To further optimize model practicality, researchers introduced vegetation indices and a lightweight backbone network, enhancing the engineering application potential of the DeepLab V3+ model while maintaining accuracy. Recently, the introduction of Transformer technology has provided a new path to overcome the limitations of local feature extraction in convolutional networks. Researchers' proposed CNN-Transformer coupled network and their developed Transformer-enhanced Swin-UNet both demonstrate excellent global feature modeling capabilities in crop segmentation tasks.

[0029] Research on remote sensing image crop classification technology based on Convolutional Neural Networks (CNNs) has been conducted. CNNs, with their powerful local feature extraction capabilities, have become a core technology in the field of remote sensing image segmentation, providing an effective solution for accurate crop identification and classification in agricultural scenarios. They are a key support for promoting the transformation of precision agriculture from experience-based management to data-driven approaches. Fully Convolutional Networks (FCNs) optimize the structure of traditional CNNs, achieving semantic segmentation with pixel-level output. This overcomes the limitation of traditional methods in ignoring spatial details in remote sensing crop classification, laying the foundation for accurate division of crop and non-crop areas in high-resolution images and directly adapting to the needs of plot-level management in agricultural production.

[0030] Addressing practical challenges in agricultural remote sensing, such as fragmented plots, complex crop morphologies, and dynamic changes in growth stages, the U-Net encoder-decoder architecture demonstrates significant advantages. While initially applied to medical image segmentation, its encoder's ability to extract deep semantic features and its decoder's ability to recover detailed information perfectly aligns with the needs of remote sensing crop segmentation—distinguishing between different crop types while preserving fine structures like plot boundaries—and has thus been widely adapted for agricultural scenarios. The researchers' proposed ResUNet, by integrating residual connections, dilated convolutions, and pyramid scene parsing pooling modules, effectively solves the gradient vanishing problem in crop feature extraction from high-resolution aerial images, significantly improving the accuracy of crop classification in farmland areas and providing technical support for large-scale crop area calculations, planting structure surveys, and other agricultural statistical work. To further optimize the targeted extraction of crop features, the combination of attention mechanisms and CNNs has become a research hotspot. Agricultural remote sensing images often exhibit a mixture of weeds, bare soil, and crop spectra; attention-based models can highlight key crop features through adaptive weighting. The researchers proposed a Squeeze-and-Excitation Block (SE) to capture dependencies between feature channels through a squeeze-excitation operation, reducing interference from background noise such as soil and shadows. The Dual Attention Network (DANet) integrates spatial and channel attention, simultaneously focusing on the spatial distribution characteristics and spectral response differences of crops when distinguishing between different crop types with similar spectra. This enhances the model's adaptability to complex planting scenarios and provides an effective means for accurately identifying mixed-crop farmland. In summary, CNNs and their improved models, by accurately capturing local crop details and flexibly expanding the receptive field, have become the mainstream technology for remote sensing crop classification, providing sophisticated technical support for precision farmland management.

[0031] The application of attention mechanisms in remote sensing crop classification involves dynamically allocating feature weights to enhance relevant information and suppress redundant interference. Existing research, through the design or improvement of attention modules combined with different remote sensing data sources, has become a key technical path to improve model classification performance. Researchers have used a dual attention module (DAM) for Sentinel-2 crop classification, leveraging the spectral and spatial features of the Sentinel-2 dataset to extract information-rich deep features. Another researcher combined a convolutional block attention module (CBAM) with the geographical information of crops and embedded it into a CNN model to mitigate the impact of geographical heterogeneity, suppress unnecessary information, and improve the model's spectral and spatial attention, resulting in superior performance in farmland crop classification tasks. Finally, researchers proposed a spectral-spatial scale attention network (S3ANet) for accurate H2 image-based crop classification. This method adaptively weights each channel, pixel, and scale of the feature map, mitigating intra-class spectral variability, spatial heterogeneity, and scale differences in crop plots, significantly improving classification performance. The above research shows that the application of attention mechanisms is deeply adapted to the core issues of crop classification, which can significantly improve the model's adaptability to complex farmland scenarios.

[0032] Current technologies primarily revolve around two major paradigms: traditional machine learning and deep learning. Traditional methods, such as Support Vector Machines (SVM) and Random Forests (RF), rely on manually designed features like spectral, textural, and vegetation indices. While these methods perform reasonably well in specific scenarios, their generalization ability and efficiency become bottlenecks when dealing with high-dimensional, heterogeneous remote sensing data. In recent years, deep learning methods, especially Convolutional Neural Networks (CNNs) and Transformer models, have significantly improved classification accuracy. CNNs, represented by U-Net and its variants, effectively extract local spatial features and multi-scale contextual information through encoder-decoder structures and skip connections, and are widely used in semantic segmentation tasks. However, the local receptive field mechanism of CNNs inherently limits their ability to capture long-range dependencies and makes it difficult to fully utilize global contextual information, which is particularly prominent in farmland scenarios with complex crop types and blurred plot boundaries.

[0033] Real-world classification tasks in complex farmland scenarios generally face three types of problems: First, non-crop information in the background creates noise interference, reducing the purity of crop feature extraction; second, inter-class similarity exists in crop spectral features, leading to confusion in classification decisions; and third, significant sample imbalance occurs, with a low proportion of small-area crop samples, causing the model to learn biased towards the dominant class and resulting in blurred boundary segmentation for small-sample crops. To address these problems, this invention proposes an improved remote sensing crop classification model, FCR-TransUNet, based on TransUNet. First, a Local-Global Co-Gated Enhanced Convolution Module (LGCG-Conv) is introduced. This module filters out background noise and improves the purity of crop feature representation through a three-stage process of "fine-grained local feature capture, global context modeling, and gated feature selection." Second, a Co-Attention (CA) mechanism is embedded. By modeling cross-channel and cross-space feature associations, the model's ability to capture inter-class differences is enhanced, mitigating classification confusion caused by crop class similarity. Finally, a Reverse Probability Edge Optimization Module (RP-EO) is designed. This module dynamically strengthens the boundary attention weights for small-sample crops, balancing the model's learning bias towards crops with different sample sizes and improving the boundary segmentation accuracy for small-sample crops.

[0034] The improved TransUNet network proposed in this invention aims to improve classification accuracy by achieving efficient fusion of global and local features through structural innovation. While maintaining TransUNet's ability to capture global features, it effectively compensates for its shortcomings in local detail feature extraction, noise suppression, and class balance, thereby more accurately handling crop classification tasks in complex farmland scenarios.

[0035] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0036] Figure 1 This is a flowchart illustrating a TransUnet-based remote sensing image crop classification method described in this specification, which specifically includes the following steps: S101: Acquire remote sensing images of crops.

[0037] Remote sensing images of crops in the study area were collected by using drones equipped with remote sensing imaging equipment.

[0038] S102: Obtain the improved TransUnet network; the improvements to the improved TransUnet network include embedding a cooperative gated augmentation convolutional module after the third convolutional neural network of the traditional TransUnet network, embedding a coordinated attention mechanism (CA) in the skip connections between the encoder and decoder, and embedding an inverse probabilistic edge optimization module between the second upsampling module and the second feature concatenation module; the embedded cooperative gated augmentation convolutional module is used to reduce the interference of background noise on feature extraction through the synergistic effect of self-attention convolution and gated convolution; CA is used to enhance the distinguishability of similar crops by modeling cross-channel and spatial feature association; the inverse probabilistic edge optimization module is used to balance the learning bias of the improved TransUnet network for each category and avoid the problem of blurred boundary segmentation due to insufficient feature learning in small sample crops; the improved TransUnet network adopts a feature map transfer method based on spiral connections.

[0039] In an exemplary embodiment, a conventional TransUnet network includes an encoder and a decoder; the encoder includes a first convolutional neural network, a second convolutional neural network, a third convolutional neural network, a linear projection layer, multiple Transformer layers, downsampling, and a first set of 3×3 convolutions and the ReLU activation function, which are sequentially connected in series.

[0040] The decoder includes a first upsampling module, a first feature concatenation module, a second set of 3×3 convolutions and ReLU, a second upsampling module, a second feature concatenation module, a third set of 3×3 convolutions and ReLU, a third upsampling module, a third feature concatenation module, a fourth set of 3×3 convolutions and ReLU, a fourth upsampling module, a fifth set of 3×3 convolutions and ReLU, and skip connections between the encoder's convolutional neural network and the decoder's feature concatenation module.

[0041] The improved TransUnet network also includes a first inverse spiral reconstruction module, a second inverse spiral reconstruction module, and a third inverse spiral reconstruction module; the first inverse spiral reconstruction module is embedded in the first feature stitching module of the decoder; the second inverse spiral reconstruction module is embedded in the second feature stitching module of the decoder; and the third inverse spiral reconstruction module is embedded in the third feature stitching module of the decoder; the inverse spiral reconstruction modules are used to reconstruct the image during the decoding stage.

[0042] CA includes a residual connection module, X average pooling, Y average pooling, concatenation + convolution, batch normalization + nonlinear transformation, third and fourth 1×1 convolutions, first Sigmoid function, second Sigmoid function, and reweighting module.

[0043] Figure 2This is a schematic diagram of the improved TransUnet network provided by the present invention. The improved TransUnet network is a network built on a U-shaped architecture, such as... Figure 2 As shown, Figure 2 ① in the text represents the collaborative gated enhancement convolution module LGCG-Conv, which reduces the interference of background noise on feature extraction through the synergistic effect of self-attention convolution and gated convolution. Figure 2 ② in the text represents the Coordinating Attention (CA) mechanism. CA models the correlation between cross-channel and spatial features, enhancing the ability to distinguish similar crops. Figure 2 In the diagram, ③ represents the reverse probability edge optimization module, which is used to balance the model's learning bias towards each category and avoid problems such as blurred boundary segmentation due to insufficient feature learning in small sample crops. Figure 2 The small cylinders in the diagram represent the reverse spiral reconfiguration module. Figure 2 The dashed arrow in the image represents downsampling. Figure 2 The thick arrows in the diagram represent 3×3 convolution and spiral connection sampling. Figure 2 The blue arrows in the diagram represent 3×3 convolutions and ReLU. Figure 2 The green arrow in the image represents upsampling. Figure 2 The yellow arrow in the image represents the split head.

[0044] The improved TransUNet network comprises three processes: encoding, skip connections, and decoding. The encoding process first extracts local features using a convolutional encoder. A feature enhancement module is then embedded into the third encoding process to optimize feature extraction. The resulting convolutional feature maps are transformed into sequences and input into the Transformer encoder to capture global information. Before concatenating the features extracted by the encoder with those from the decoder, CA-coordinated attention is used to enhance the modeling of global contextual information. The output of the Transformer encoder is then fused with the convolutional features to form the encoded representation. The decoding process is consistent with the standard U-Net steps, constructing the segmentation result step-by-step. Each step includes two convolutions and one upsampling. In the second decoding process, inverse enhancement is used to highlight boundary details and balance inter-class learning bias.

[0045] In an exemplary embodiment, the embedded collaborative gated augmentation convolution module includes a first 1×1 convolution, Conv, a first self-attention convolution DSA, a second self-attention convolution DSA, a first gated convolution GC, a second gated convolution GC, an adder, and a second 1×1 convolution; Conv includes a 3×3 convolution and ReLU.

[0046] Specifically, improvements are made to the existing TransUnet model by introducing Local-Global Collaborative Enhanced Gated Convolution (LGCG-Conv) during feature extraction to reduce the interference of background noise on the purity of feature extraction. To address the issue of similarity between crop classes, a CA coordinated attention mechanism is further incorporated before feature splicing to model cross-channel and spatial feature associations and enhance the ability to distinguish similar crops. Figure 3 A schematic diagram of the collaborative gating enhanced convolution module LGCG-Conv provided by the present invention is shown below. Figure 3 As shown, Figure 3 The first 1×1 convolution in the array is a 1×1 convolution. Figure 3 Conv in the middle is a 3×3 convolution and ReLU. Figure 3 The first DSA in the diagram is the first self-attention convolutional module. Figure 3 The second DSA in the text is the second self-attention convolutional module. Figure 3 The first GC in the process is the first gated convolution. Figure 3 The second GC in the text refers to the second gated convolution. Figure 3 The first 1×1 convolution in the code is used to perform a 1×1 convolution on the input data of LGCG-Conv. Figure 3 The second 1×1 convolution in Conv is used to perform a 1×1 convolution on the features obtained by fusing the output of the 3×3 convolution, the first gated convolution feature, and the second gated convolution feature. Both the first and second 1×1 convolutions include 1×1 convolution and ReLU. Conv includes 3×3 convolution and ReLU. The + sign indicates concatenation.

[0047] LGCG-Conv consists of two parts: self-attention convolution and gated convolution, as follows: Figure 3 As shown, the initial features are first obtained through a 1×1 convolution. These features are then input into a 3×3 convolution and two self-attention convolutional modules. Next, the output of the 3×3 convolution and the output of the first self-attention convolution module are combined into the first gated convolution for feature extraction. Subsequently, the features obtained from the gated convolution are fed into the second gated convolution along with the output of the second self-attention convolution module to further enhance the classification features. Finally, the initial features obtained from the 3×3 convolution are fused with the output features from the two gated convolutions to serve as the final output of the LGCG-Conv module.

[0048] Decoupled Self-Attention (DSA) evolved from the Transformer multi-head self-attention framework. Its goal is to combine local spatial information with global contextual information to enhance the model's ability to perceive details and global features. Traditional Transformer self-attention quantifies the correlation between locations through dot product operations, significantly improving the capture of global context. However, in high-resolution scenes, it often ignores local spatial relationships, resulting in somewhat coarse feature representations. Decoupled Self-Attention, by introducing convolution operations, encodes local spatial information into feature vectors as the embedding part of the self-attention mechanism, compensating for the shortcomings of pure self-attention in capturing spatial relationships and details, thereby improving the modeling ability for complex scenes.

[0049] Specifically, DSA will take the feature map from the previous layer. x As input, this feature map contains rich spatial and channel information. x After performing multiple convolution operations, a query is generated for self-attention. q ),key( k ) and value ( v Convolutional embeddings, through repeated convolutional embeddings, can better preserve local spatial structure and outperform traditional linear embeddings in terms of parameter and computational cost, thereby enhancing the ability to perceive spatial information. In particular, dilated convolutions expand the receptive field by introducing a dilation rate, acquiring broader contextual information while maintaining the output feature scale, thus enhancing the ability to capture large-scale information.

[0050] To perform self-attention computation, the query vector q With key vector k The original spatial feature map is reshaped into a two-dimensional matrix, with each location corresponding to a row vector. The shape of the matrix is ​​denoted as ( H × W , C ).in, C For the number of channels, H × W This represents the spatial resolution of the original spatial feature map. After reshaping, [the following steps are performed] respectively. q and k Performing matrix multiplication yields a matrix of size ( H × W , H × WThe similarity matrix is ​​given by [reference to a similarity matrix], where any element represents the strength of the correlation between two spatial locations. To achieve dynamic modeling of positional relationships, Softmax normalization is applied to this similarity matrix to obtain an attention weight distribution reflecting positional correlation. This distribution reveals the strength of the relationship between different locations, prompting the model to make fuller use of global contextual information. Value vector v It also undergoes the same processing and is combined with attention weights to achieve weighting. The final output features retain local details while incorporating global information, thereby improving expressive power. The above operations can be defined as formulas (1) to (5):

[0051] (1); in, reshape After the reshaping, M This represents the features after the query vector has been reshaped. q This is the query vector.

[0052] (2); in, N This represents the features after the key vector has been reshaped. k This is the key vector.

[0053] (3); in, T The features represent the reshaped value vector. v It is a value vector.

[0054] (4); in, Representing the i The position is the first j The influence of each position Representative position i The query vector, Representative position j The key vector, n = h × w Indicates the number of pixels.

[0055] (5); in, Represents the output of DSA. B This is a location-related attention diagram.

[0056] Will T and BThe features are multiplied to obtain an optimized feature map, which is then reconstructed to produce the DSA output. To improve numerical stability, the output feature map is normalized. ReLU activation is then applied to the output feature map to enhance the model's nonlinear expressive power.

[0057] Gated convolution (GC) is a convolution operation that incorporates a gating mechanism, aiming to improve the model's ability to filter and integrate different features. This method accepts two input feature maps: one with a large receptive field and the other with a small receptive field. Each of these inputs is subjected to its own convolution operation to generate corresponding gated maps. Then, these two gated maps are multiplied element-wise with the main feature to obtain the final output feature. The calculation process can be expressed as formulas (6)-(8):

[0058] (6); (7); (Gate) (8); in, W g and W f These are embedding matrices for different convolutional projections. F high and F low This indicates two inputs. Gate For your attention, see the diagram. yes s Shape function, Map all values ​​to the range between 0 and 1. F For feature embedding, This indicates that ReLU is activated.

[0059] To address the imbalanced sample problem, the reverse probabilistic edge optimization (RP-EO Module, RP-EO) is used in the decoding stage to calculate the predicted probability distribution of small sample crop regions in real time and dynamically generate edge attention weight maps. This is done to specifically enhance the feature learning weights of small sample crop boundary regions, thereby balancing the model's learning bias towards each category and avoiding problems such as blurred boundary segmentation and missed detections in small-area crops due to insufficient feature learning.

[0060] Figure 4 This is a schematic diagram of the reverse probabilistic edge optimization (RP-EO) provided by the present invention, as shown below. Figure 4 As shown, the input of the remote sensing image is X ∈R C×H×W ,in, C For the number of channels, H ×W Let be the spatial resolution of the image. Therefore, the goal of remote sensing image classification is to predict the corresponding size. H × W Pixel-by-pixel semantic tag mapping.

[0061] The Inverse Probabilistic Edge Optimization Module (RP-EO Module) effectively combines semantic information from high-level feature maps with detailed information from low-level feature maps, thereby improving the edge extraction accuracy of crop regions. Its structure is as follows: Figure 4 As shown, firstly, the second upsampled feature is received. To emphasize the edge regions, the second upsampled feature is first applied. A probability map is generated by processing the data using a sigmoid activation function. Then, the probability map is processed by reverse calculation to distinguish between background and crop areas; the reverse probability map... Used to emphasize non-edge areas, thereby helping to identify and filter background information; inverted probability maps Feature map after CA processing at a 1 / 4 scaling ratio Perform element-wise multiplication to generate new feature maps This involves combining important edge information from higher layers with specific details from lower layers to enhance spatial detail. The resulting new feature map is then processed... Second upsampling features By performing fusion, fusion characteristics are obtained. It preserves both contextual information and fine-grained edge location information. This involves fusing features. Two convolutional layers are performed to integrate information and adjust the filter size to obtain inverse probability edge optimization features. In the last layer, a 1×1 convolutional operation is used to generate the final segmentation mask. The operation process is as follows: Equation (9)-Equation (14):

[0062] (9); (10); (11); (12); (13); Mask (14).

[0063] in, This is the second upsampling feature. It is the sigmoid activation function. This is a probability graph. This is a low-level feature map. This represents element-wise multiplication. For the new feature map, As a feature of fusion, It is a two-layer convolution process. Mask For segmentation mask.

[0064] The Coordination Attention Mechanism (CA) embeds positional information into channel attention, capturing the positional dependencies of features by combining attention mechanisms in different horizontal and vertical directions. Figure 5 The CA structure diagram provided by this invention is as follows: Figure 5 As shown, CA includes a residual connection module, X average pooling, Y average pooling, concatenation + convolution, batch normalization + nonlinear transformation, third 1×1 convolution and fourth 1×1 convolution, first sigmoid function, second sigmoid function and reweighting module.

[0065] First, input the feature map X. （H*W*C） Pooling operations are used to generate one-dimensional features along the horizontal and vertical directions. The one-dimensional feature along the horizontal direction is given by formula (15):

[0066] (15); in, Indicates a channel C At height h Aggregate value of horizontal features of location, j It is a width index. W Input feature map x width, ( h,j () is the input feature map x In the passage c ,high h ,width i Pixel value at a specific spatial location.

[0067] The one-dimensional feature in the vertical direction is given by formula (16): (16); in, Indicates the width Aggregate value of vertical features of location. j It is a height index. Input feature map x In the passage c ,high j ,width w Pixel values ​​at a specific spatial location H Input feature map xThe height.

[0068] The feature maps in the horizontal and vertical directions are concatenated in the spatial dimension to obtain a new feature representation. The feature is then divided into two tensors, and two independent 1×1 convolutions are used to convert these two tensors to have the same number of channels as the original input X. The operation process is shown in formula (17).

[0069] (17); in, , , , f h and f w It is a feature f The two tensors are split, and F1 is a shared 1×1 convolution operation designed to integrate the results in the horizontal and vertical directions. Z h ,Z w [] indicates splicing in the spatial dimension. δ It is a non-linear activation function.

[0070] Finally, the final output is generated by combining the input feature map X and the generated attention weights, which are given by formulas (18) and (19): (18); in, g h This represents the generated horizontal attention weights. , f h and f w It is a feature f The two tensors of the split, This is a 1×1 convolution operation.

[0071] (19); in, g w This represents the generated vertical attention weights. , This is a 1×1 convolution operation.

[0072] Output feature map Y In the passage c ,high i ,width j The final pixel value at the position is formula (20): ( j (20); in, To output feature map Y In the passage c ,high i ,width j The final pixel value at the location, For the input feature map X in the channel c ,high i ,width j The original pixel value at the location, Horizontal attention weight map g h In the passage c ,high i The weight value at the position, ( j (This is a vertical attention weight map) g w In the passage c ,width j The weight value at the location.

[0073] S103: The crop remote sensing image is segmented using an improved TransUnet network to obtain pixel-level classification results of the crop remote sensing image.

[0074] In an exemplary embodiment, a crop remote sensing image is segmented using an improved TransUnet network to obtain pixel-level classification results. Specifically, this includes: sequentially extracting 1 / 2 scale feature maps, 1 / 4 scale feature maps, and 1 / 8 scale feature maps from the crop remote sensing image using a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network; inputting the 1 / 8 scale feature map into a collaborative gating enhancement module to obtain a first hidden feature, and then linearly projecting the first hidden feature to obtain a projected feature; inputting the projected feature into a multi-layer Transformer layer for processing, capturing the global long-range dependencies of the features through a self-attention mechanism to obtain a second hidden feature; sequentially passing the second hidden feature through downsampling and a first set of 3×3 convolutions and the ReLU activation function to obtain encoded features; performing multiple convolution operations, upsampling, and feature extraction operations based on the encoded feature, the CA-processed 1 / 2 scale feature map, the CA-processed 1 / 4 scale feature map, and the CA-processed 1 / 8 scale feature map to obtain decoded features; and processing the decoded features using a segmentation head to obtain pixel-level classification results for the crop remote sensing image.

[0075] In an exemplary embodiment, the 1 / 8 scaled feature map is input to the collaborative gating enhancement module to obtain the first hidden feature. Specifically, this includes: processing the 1 / 8 scaled feature map through a first 1×1 convolution to obtain initial features; inputting the initial features into Conv, the first DSA, and the second DSA for convolution processing; inputting the output of Conv and the output of the first DSA into the first GC for feature extraction to obtain the first gated convolution feature; inputting the first gated convolution feature and the output of the second DSA into the second GC for convolution processing to obtain the second gated convolution feature; and fusing the output of Conv, the first gated convolution feature, and the second gated convolution feature and then processing them through a second 1×1 convolution to obtain the first hidden feature.

[0076] In an exemplary embodiment, based on encoded features, CA-processed 1 / 2 scaled feature maps, CA-processed 1 / 4 scaled feature maps, and CA-processed 1 / 8 scaled feature maps, multiple convolution operations, upsampling, and feature extraction operations are performed to obtain decoded features. Specifically, this includes: passing the encoded features through a first upsampling module to obtain first upsampled features; passing the CA-processed 1 / 8 scaled feature maps through a first inverse spiral reconstruction module to extract index features to obtain first index features; concatenating the first index features with the first upsampled features through a first feature concatenation module to obtain first concatenated features; passing the first concatenated features through a second set of 3×3 convolutions and ReLU, and a second upsampling module to obtain second upsampled features; and processing the second upsampled features through an inverse probability edge optimization module to obtain inverse features. The algorithm optimizes features towards probabilistic edges; it then extracts index features from the CA-processed 1 / 4 scaled feature map using a second inverse spiral reconstruction module to obtain second index features; it then concatenates these second index features with the inverse probabilistic edge optimized features using a second feature concatenation module to obtain second concatenated features; finally, it passes these second concatenated features through a third set of 3×3 convolutions and ReLU, and a third upsampling module to obtain third upsampled features; it then extracts index features from the CA-processed 1 / 2 scaled feature map using a third inverse spiral reconstruction module to obtain third index features; it then concatenates these third index features with the third upsampled features using a third feature concatenation module to obtain third concatenated features; and finally, it passes these third concatenated features through a fourth set of 3×3 convolutions and ReLU, a fourth upsampling module, and a fifth set of 3×3 convolutions and ReLU to obtain decoded features.

[0077] In an exemplary embodiment, the second upsampled feature is processed by the inverse probabilistic edge optimization module to obtain the inverse probabilistic edge optimized feature. Specifically, this includes: processing the second upsampled feature with a sigmoid activation function to obtain a probability map; the sigmoid activation function is used to emphasize the edge regions of the crop remote sensing image; performing inverse calculation on the probability map to obtain an inverse probability map; the inverse probability map is used to distinguish between the background and crop regions of the crop remote sensing image; multiplying the inverse probability map element-wise with the CA-processed 1 / 4 scaling feature map to generate a new feature map; fusing the new feature map with the second upsampled feature to obtain a fused feature; the fused feature includes contextual information and fine edge location information; and performing two convolutional processing on the fused feature to obtain the inverse probabilistic edge optimized feature.

[0078] In an exemplary embodiment, the CA-processed 1 / 2 scaling feature map is processed by a third inverse spiral reconstruction module to extract index features, resulting in third index features. Specifically, this includes: calculating multiple inverse spiral coordinates based on different rotation angles; converting the multiple inverse spiral coordinates into multiple inverse Cartesian coordinates and determining the multiple inverse Cartesian coordinates as multiple indices; and extracting the third index features corresponding to the positions of the multiple indices from the CA-processed 1 / 2 scaling feature map.

[0079] In an exemplary embodiment, the extraction process of the CA-processed 1 / 8 scaling feature map specifically includes: passing the 1 / 8 scaling feature map through residual concatenation to obtain residual concatenation features; inputting the residual concatenation features into X-average pooling and Y-average pooling respectively to obtain X-average pooling features and Y-average pooling features; sequentially concatenating, convolving, batch normalizing, and performing nonlinear transformation on the X-average pooling features and Y-average pooling features to obtain nonlinear features; inputting the nonlinear features into a third 1×1 convolution and a fourth 1×1 convolution respectively to obtain X-convolution features and Y-convolution features; inputting the X-convolution features into a first Sigmoid function to obtain a first Sigmoid feature; inputting the Y-convolution features into a second Sigmoid function to obtain a second Sigmoid feature; and weighting and fusing the first Sigmoid feature, the second Sigmoid feature, and the residual concatenation features through a reweighting module to obtain the CA-processed 1 / 8 scaling feature map.

[0080] In an exemplary embodiment, 1 / 2 scale feature maps, 1 / 4 scale feature maps, and 1 / 8 scale feature maps of crop remote sensing images are extracted sequentially through a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network. Specifically, this includes: for any convolutional neural network layer, extracting local features of the input data through the convolutional neural network to obtain an original feature map including basic scale information; wherein, the input data of the first convolutional neural network is the crop remote sensing image, the input data of the second convolutional neural network is the output of the first convolutional neural network, and the input data of the third convolutional neural network is the output of the second convolutional neural network; dividing the original feature map into multiple regions, starting from the center of the original feature map, calculating multiple polar coordinates based on the spiral path formula, and converting the multiple polar coordinates into multiple Cartesian coordinates respectively; and gradually expanding outward along the multiple Cartesian coordinates to perform feature sampling and integration to obtain a scaled feature map carrying center and surrounding spatial structure information.

[0081] Specifically, to address the limitations of traditional networks in feature extraction and fusion, this invention proposes a feature map transfer method based on spiral connections. This method, through spatial and sequential reconstruction, enables feature maps to be connected in a spiral manner at each stage, emphasizing information acquisition from the feature map center to its periphery. In each encoder layer, this invention divides the feature map into multiple regions and connects these regions using a spiral approach. This process starts from the center of the feature map and progressively expands feature extraction outwards along a spiral path.

[0082] This invention calculates the spiral path based on polar coordinates, and generates the spiral coordinates using formula (21): (twenty one); in, It is the radius. It's the rotation angle. and It is a constant that determines the shape of the spiral.

[0083] Converting polar coordinates to Cartesian coordinates is given by formulas (22) and (23): (twenty two); (twenty three); in,( x , y ) are the coordinates in the image plane.

[0084] To fully acquire feature information, the present invention will The range is set to 0 to 2π n ,in, n To determine the required number of spiral turns, increase the angle increment continuously. ΔθCalculate the corresponding (in the feature map) x , y (Coordinates) and use these coordinates for feature sampling.

[0085] Coordinates generated by the spiral path x , y This invention can locate and sample feature values ​​in a feature map. For each calculated coordinate, the pixel value at the corresponding position is extracted from the feature map to form a new feature map. The new feature map contains structural information from the center to the periphery, allowing the model to gradually move away from the center to capture the features of surrounding pixels, while optimizing the acquisition of global information.

[0086] In the encoder stage, this invention utilizes the generated spiral index to extract features and integrate information. The encoder consists of multiple convolutional layers, and the feature maps in each convolutional layer are extracted through a spiral path, which can effectively capture features at different scales and establish a good connection between local and global information.

[0087] In the decoding stage, this invention employs a reverse spiral index to effectively reconstruct the image. This invention defines the reverse spiral coordinates using a formula similar to that used in the encoding stage, as formula (24):

[0088] (twenty four); in, And 𝑏' are constants suitable for the decoding stage.

[0089] These polar coordinates are then converted to Cartesian coordinates to form the pixel positions required for reconstruction, as shown in formulas (25) and (26): (25); (26); This invention uses reverse spiral coordinate position ( , Extract the corresponding features from the feature map obtained from the encoder, and fuse them with the feature map of the current layer of the decoder to form formula (27): (27); in, These are features extracted from the encoder. It comes directly from the skip connection feature.

[0090] TransUNet improves upon the shortcomings of traditional TransUNet in feature extraction and fusion by introducing a spiral connection design, enabling the network to more effectively integrate local and global information in classification tasks. The Segmented Recursive Structure (SRS) addresses a limitation of TransUNet's deep feature fusion mechanism, primarily its relatively linear feature processing. This approach can lead to the loss of important local information. To mitigate this issue, this invention introduces a Segmented Recursive Structure (SRS) in the decoding part of TransUNet. This structure aims to enhance the decoder's detail recovery capabilities by dynamically combining features of different granularities from upper layers.

[0091] In traditional decoders, feature maps are typically processed only once after upsampling and concatenation with corresponding feature maps from the encoder. This approach is relatively simple in terms of information transmission, limiting the model's ability to deepen feature extraction. To address this challenge, this invention proposes a novel recursive learning mechanism that enables the decoder to achieve richer feature fusion in feature processing.

[0092] The recursive learning module consists of a small network composed of multiple convolutional layers. Each convolutional module takes the current feature map as input, processes it, and uses its output as the input of the next recursive module, thus forming a continuous flow of information.

[0093] Each convolutional module processes the feature map through multiple convolutional layers, performing feature transformation and extraction, while using the ReLU activation function to increase non-linearity.

[0094] By cascading multiple recursive learning modules, a progressive feature extraction process is formed. Each module receives a feature map from the previous module, completes feature extraction, and then passes the result to the next module. Through the recursive learning mechanism, each recursion gradually updates the feature map, promoting feature refinement and fusion, thereby significantly improving the grasp of details. By repeatedly processing and updating feature maps, SRS can form a "short-term memory" phenomenon, allowing the model to refine features multiple times in a short period of time. By enhancing feature fusion between layers, the decoder can acquire richer information at each layer, promoting the recovery of details. Information from different levels can be effectively fused through recursive modules, thereby improving the overall performance of the decoder.

[0095] In one exemplary embodiment, the present invention selects a farm dataset to train the improved TransUnet network.

[0096] 1. A dataset from a certain farm The study area of ​​a certain farm covers 330 square kilometers and is located in a temperate monsoon climate zone, characterized by warm, rainy summers and cold, dry winters with heavy snowfall. The province has fertile soil and suitable climate conditions, and its per capita grain output ranks among the highest in the country. The main grain crops grown include soybeans, corn, and rice. During their growth and development, these crops are influenced by various environmental factors such as temperature, humidity, soil conditions, and climate, exhibiting unique seasonal variations. The farm region's superior agricultural conditions and diverse crop varieties make it an ideal validation scenario for assessing the practical performance and robustness of the proposed classification model.

[0097] Figure 6 The crop label image provided by this invention, such as Figure 6 As shown, (a) label represents the label overview, (b) corn represents corn, (c) soya represents soybean, and (d) paddy represents rice.

[0098] Sentinel-2 is a satellite mission conducted by the European Space Agency. Sentinel-2 consists of two satellites, Sentinel-2A and Sentinel-2B, which simultaneously acquire data from 13 different spectral bands. This data is used to study various Earth surface features, such as land cover, vegetation growth, and water resource management. Google Earth Engine (GEE) is a powerful geospatial analytics platform provided by Google, specifically designed for the processing and analysis of large-scale remote sensing data. GEE provides researchers, developers, and engineers worldwide with a cloud-based environment for storing, processing, and analyzing geospatial data, particularly remote sensing imagery and satellite data. Using GEE, we downloaded Sentinel-2 images from a farm in May, July, and August. These images were then stitched together in ArcGIS. Bands 12, 12, and 12 were selected from the May images; bands 7, 6, and 3 from the July images; and bands 8, 4, and 3 from the August images. Labels for three crops (corn, soybeans, and rice) were manually added to the images to obtain vector files, such as... Figure 6 As shown, there are a total of 5,526 sample units in the labeled study area, including 2,610 corn samples, 1,453 soybean samples and 1,463 rice samples.

[0099] This invention selected images from August for training and testing because August is the peak growth period for crops in the region. During this stage, leaf greenness and canopy coverage are high, and the morphological and spectral characteristics of different crops are most distinct, allowing the model to learn more efficiently and achieve higher classification accuracy. Rainfall is relatively stable during this period, with less cloud and fog interference, resulting in high image quality and facilitating more accurate capture of crop details. This invention applied sliding window cropping to both image and label files, with a window size of 256×256 and a stride (overlap) of 0.1. Data augmentation was performed on both image and label data, including cropping, rotation, and flipping, such as... Figure 4 As shown. This approach exposes the model to a wider sample distribution and greater real-world diversity, thereby enhancing its robustness to unseen samples and reducing the risk of overfitting. The dataset is divided into training, validation, and test sets in a 6:2:2 ratio.

[0100] The BCE loss function is widely used in many tasks, including image segmentation. BCE effectively measures the difference between the predicted classification and the true label for each pixel, promoting the model to learn more accurate pixel-level classification results. Its core idea is to predict the degree of deviation from the true value in logarithmic form. Based on the concept of cross-entropy in information theory, it expresses the degree of matching between the model's predicted probability distribution and the true label, reflecting the model's uncertainty. This helps achieve accurate image classification. The Dice loss function plays a significant role in image classification. It is an index based on the Dice coefficient, used to measure the degree of overlap between the predicted result and the true label. It is particularly suitable for class imbalance scenarios in remote sensing image classification tasks, mitigating the impact of class imbalance, which helps improve segmentation performance. Compared to traditional pixel-level losses, Dice loss is easier to obtain stable gradients. It can also suppress local overfitting by emphasizing the overall overlap effect, improving the model's generalization ability. During training, a combination of BCE and Dice loss functions is used. This combination can alleviate the gradient imbalance problem that may be caused by a single loss function, improving training stability and convergence speed.

[0101] The combination of the Binary Cross-Entropy Loss (BCE) and Dice loss functions can effectively leverage their respective advantages. BCE promotes accurate classification of each pixel, optimizing model performance in a fine-grained manner. Dice helps improve the continuity and integrity of boundaries by emphasizing overlap. Dice is robust to small targets and a few classes, while BCE complements the overall pixel classification performance; the combination of the two can better address the imbalance problem. This combination can alleviate the gradient imbalance problem that may be caused by a single loss, improving training stability and convergence speed. The loss function of the improved TransUnet network is Equation (28):

[0102] (28); in, The loss value for the improved TransUnet network, This is the BCE loss value. , This is the Dice loss value. , For the pixels in the sample label i The value, Pixels in the segmentation prediction results of the improved TransUnet network i The value, N The number of samples.

[0103] In an exemplary embodiment, experiments were conducted on a farm dataset to verify the effectiveness of FCR-TransUNet. Experimental results show that on this farm dataset, the FCR-TransUNet model achieves a Mean Intersection over Union (MIoU) of 92.2%, an F1 score of 92.7%, and precision and recall of 93%. Compared with seven other models, including YOLO11-seg, DeepLabV3+, and SAM2-UNet, FCR-TransUNet demonstrates the best performance. Specifically, its MIoU is 1.8% higher than the best-performing comparative model, SAM2-UNet, and 2.9% higher than the benchmark TransUNet model. Ablation experiments further validate the synergistic effectiveness and architectural design rationality of the LGCG-Conv, CA, and RP-EO modules. In summary, FCR-TransUNet effectively solves the problems of background interference, inter-class confusion, and sample imbalance in complex farmland remote sensing crop classification, providing an efficient and reliable crop classification technology solution for the field of precision agriculture. It has important practical significance for promoting intelligent agricultural production management and ensuring food security.

[0104] In one exemplary embodiment, the research area selected for this invention is located on a farm, with geographical coordinates ranging from 132.5°E to 133.0°E and 47.5°N to 48.0°N, covering an area of ​​330 km². This region has a temperate monsoon climate, characterized by warm, rainy summers and cold, dry winters with abundant snowfall. The research area has fertile soil and suitable climatic conditions, and its per capita grain output ranks among the highest in the country. The main grain crops grown include soybeans, corn, and rice. During their growth and development, these crops are influenced by various environmental factors such as temperature, humidity, soil conditions, and climate, exhibiting unique seasonal variations. The Heilongjiang Friendship Farm, located in this region, possesses superior agricultural production conditions and a diverse range of crops, making it an important verification object for evaluating the effectiveness and robustness of the classification model in a real-world scenario.

[0105] This invention selected August images of the study area for training and testing. August is the peak growing season for crops in the study area, when the morphological and spectral characteristics of different crops show the most significant differences, allowing the model to learn more efficiently and thus improving classification accuracy. Rainfall is relatively stable, with less cloud and fog interference, resulting in high image clarity and more accurate capture of crop details. Sliding window cropping was applied to both the images and label files, with a size of 256*256 and a duplication rate of 0.1. Various data augmentation techniques were applied, such as cropping, rotation, and flipping. Figure 7 As shown, this exposes the model to a richer sample distribution, more closely resembling the diversity of real-world scenarios. This improves the diversity of the dataset and its robustness to unseen samples, while reducing the risk of overfitting. The ratio of the training, validation, and test datasets is 6:2:2.

[0106] In this invention, four common evaluation metrics are used: Mean Intersection over Union (MIoU), Precision, Recall, and F1 Score to comprehensively evaluate the performance of the remote sensing image crop classification model.

[0107] These metrics can quantify the model's performance in crop classification tasks from multiple perspectives, especially in complex remote sensing images, providing a more accurate and reliable performance evaluation. MIoU is a commonly used evaluation metric in remote sensing image segmentation and classification tasks, designed to measure the degree of overlap between predicted results and actual labels. In crop classification tasks, MIoU calculates the Intersection over Union (IoU) ratio for each class and obtains the average, comprehensively reflecting the model's classification performance across multiple crop categories, making it particularly suitable for handling multi-class tasks and imbalanced data.

[0108] The formula for calculating MIoU is formula (29): (29); Precision measures the proportion of actual results in which a model predicts a crop category that truly belongs to that category. High precision means that the model can effectively reduce false positives and avoid misclassifying non-target areas as crop categories. In remote sensing image classification tasks, precision is crucial for ensuring that the model accurately identifies crop regions. The formula for calculating precision is formula (30):

[0109] (30); Recall evaluates the proportion of samples that actually belong to a certain crop category that the model can correctly predict. A higher recall indicates that the model can capture more real crop areas. In remote sensing images, recall is particularly critical because it reflects the model's ability to cover the target area, especially in scenes with complex terrain or significant occlusion. The formula for calculating recall is Equation (31):

[0110] (31); The F1 score is the harmonic mean of precision and recall, which comprehensively considers the balance between the two. In crop classification tasks, the F1 score achieves a good balance between precision and recall, especially when classes are imbalanced, and can effectively evaluate the overall classification performance of the model. The formula for calculating the F1 score is formula (32):

[0111] F1=2 (32); This invention selects seven mainstream semantic segmentation models—PSPNet, UNet, FPN, YOLO11-seg, DeeplabV3+, SAM2-UNet, and TransUNet—as benchmarks to compare and verify the reliability of the FCR-TransUNet method. The comparative experimental results are shown in Table 1. As an early classic model, PSPNet's overall classification performance is relatively limited, with MIoU not reaching 80%, and both precision and recall being low. It lacks the ability to model multi-dimensional features of crops and background in farmland scenes, making it difficult to meet the need for accurate crop type identification. U-Net's classification performance is significantly improved compared to PSPNet, with substantial increases in MIoU and recall, demonstrating the natural advantage of the encoder-decoder architecture in small-sample classification. However, it lacks effective modeling of global semantic associations, making it prone to feature confusion in multi-crop mixed planting scenarios. FPN further outperforms U-Net in MIoU and F1 score, especially in recall, reflecting its stronger coverage of crop features at different scales. However, this model struggles to effectively suppress background noise in complex farmland environments, leading to significant false positives and impacting the reliability of classification results. YOLO11-seg achieves a precision of 90.3%, demonstrating strong crop classification accuracy; its 91.6% recall reflects effective coverage of large crop areas, making it suitable for crop classification scenarios with high accuracy requirements, but its MIoU is only 87.2%. DeepLabV3+, with its combination of dilated convolution and multi-scale pooling, exhibits stronger feature adaptation capabilities in complex farmland scenarios, with high MIoU, F1 score, and recall. SAM2-UNet and TransUNet, as advanced models incorporating Transformer, further improve classification performance. SAM2-UNet achieves an MIoU of 90.4%, showing a good balance between precision and recall. TransUNet's mIoU is 89.3%, indicating that its adaptability to complex farmland environments still needs improvement. The proposed FCR-TransUNet in this study achieves significant breakthroughs in all core metrics: precision reaches 93.1%, recall improves to 93.4%, F1 score and MIoU reach 92.7% and 92.2% respectively, exceeding the best-performing comparative model SAM2-UNet by more than 2%. This demonstrates the superiority and reliability of the FCR-TransUNet method in crop classification tasks.

[0112] Table 1 Comparison of Indicators of Different Models Figure 7 The visualization results comparison chart provided by this invention, such as Figure 7As shown, this invention provides a visual comparison of the crop classification results of various models to intuitively present the advantages of the FCR-TransUNet method. While PSPNet can achieve relatively accurate classification of most contiguous farmlands, it is significantly affected by background noise. In areas where crops meet the background, it struggles to filter redundant information, leading to blurred boundaries and misclassification. Furthermore, its ability to distinguish similar crops is insufficient, and in areas with scattered, small-sample crops, imbalanced samples result in significant missed classifications. U-Net outperforms PSPNet, especially in small-area crop classification, but background interference remains unresolved, and non-crop information is easily misclassified as crop features. Moreover, the boundaries of small-sample crops remain blurred due to learning bias, limiting its performance in areas with complex planting structures. FPN shows progress in multi-scale feature processing, reducing missed classifications of small-area crops, but background noise still interferes with feature extraction, leading to the misclassification of non-crops as target categories in some areas, and there is still room for improvement in boundary details. YOLO11-seg accurately identifies the core areas of contiguous crops, but small-sample crops, due to their low sample size, are often missed as a whole, making it difficult to meet the needs of fine-grained classification. DeepLab V3+ performs excellently in separating crops from the background, with fine detail processing. However, due to insufficient learning weights, the boundary segmentation of small sample crops in extremely small areas remains blurry. SAM2-UNet has high classification precision, reducing misclassification and missed detections, but it still extracts redundant features in areas with severe background interference; a small number of scattered small sample crops still have slight missed detections. TransUNet performs stably in large-area crop classification, but non-crop details in the background still interfere with feature extraction; insufficient global feature modeling for similar crops leads to fragmented misclassifications; and the boundary blurring problem of small sample crops has not been completely resolved due to learning bias. In contrast, FCR-TransUNet clearly distinguishes crops from the background in complex farmland scenarios, accurately classifies similar crops, and has clear and sharp outlines for small plots. It completely overcomes the inherent limitations of other models in terms of background interference, inter-class similarity, and sample imbalance, and its overall performance is more in line with the actual needs of accurate classification in complex agricultural environments.

[0113] To verify the specific contributions of the three proposed enhancement modules to model performance, ablation experiments were conducted using TransUNet as the base network. The results are shown in Table 2. The ablation experiment results show that the synergistic effect of the three modules achieves the optimal improvement in model performance: when only the feature enhancement module is added, the improvement in background noise filtering increases MIoU by 0.9% compared to the base TransUNet; when the CA collaborative attention mechanism is introduced alone, the optimization of similar crop classification accuracy increases MIoU by 1.2%; and when only the reverse attention module is added, the improvement in small sample crop boundary segmentation increases MIoU by 1.6%. When all three modules work together, FCR-TransUNet achieves the highest MIoU (92.2%), exceeding 2.9% compared to the base TransUNet, and significantly outperforming the effects of a single module or any combination of two modules. This fully demonstrates the complementarity and necessity of each module in addressing background interference, inter-class similarity, and sample imbalance, verifying the rationality and effectiveness of the multi-module collaborative design.

[0114] Table 2 Ablation Experiment Results As shown in Table 2, these results fully verify the synergistic effect of the three modules LGCG-Conv, CA, and RP-EO. LGCG-Conv strengthens the expression of crop features, CA improves the feature discrimination of similar categories, and RP-EO balances the feature learning bias of small sample categories. This enables the model to be optimized in the three core capabilities of feature modeling, small sample recognition, and background anti-interference. It is more suitable for the actual farmland scenario of "scattered land, complex background, and diverse planting structure", and provides more reliable technical support for crop classification in precision agriculture.

[0115] The results of this invention show that the four models exhibit significant performance differences in remote sensing crop classification tasks. FCR-TransUNet achieves the best classification results on a farm dataset—achieving a MIoU of 92.2% and an F1 score of 92.7%. This result highlights the unique role of FCR-TransUNet's three core modules in addressing the challenges of complex farmland classification: the LGCG-Conv module filters out background noise through a three-stage "local-global-gating" process; the CA collaborative attention mechanism enhances the capture of inter-class differential features; and the RP-EO module balances sample learning bias. These three modules work together to achieve integrated optimization of "noise suppression, inter-class differentiation, and sample balancing," which aligns with researchers' conclusion that "multi-feature synergy is superior to single-feature synergy."

[0116] Further analysis of the features and modules reveals that the RP-EO module is key to FCR-TransUNet's accurate classification of crops in small sample areas. In small sample regions, its boundary segmentation accuracy is 12% higher than the original TransUNet. This is attributed to the module dynamically strengthening the boundary weights of small samples through inverse probability calculation, preventing learning bias due to low sample proportions (<5%). Secondly, the CA collaborative attention mechanism, through cross-channel-spatial feature association, reduces the classification confusion rate of spectrally similar crops to 4.5%, far lower than DeepLabV3+'s 11.2%. This aligns with researchers' conclusion that "association between red-edge bands and spatial features can improve crop discrimination ability," as the CA mechanism precisely strengthens the coupling relationship between the barley red-edge band (705-740nm) and plant spatial features, forming a unique "spectral-spatial" identification marker. The LGCG-Conv module effectively filters weeds and bare-field noise from a farm dataset, improving crop feature extraction purity by 8%. However, its suppression of spectral similarity interference is weaker than the CA module, indicating that different modules need to target specific classification pain points.

[0117] From the perspective of classification performance differences among different crops and scenarios, the classification accuracy of soybeans and corn in the Friendship Farm dataset (F1 score > 94%) is significantly higher than that of small-area samples (F1 score 90.5%). This difference stems from two factors: First, soybeans and corn have sufficient sample sizes (> 500 samples each), allowing for more thorough model training, while rice has only 86 samples, making it prone to "incomplete feature learning." Second, the plant type differences between rice and corn (rice is upright, corn is bushy) are more significant, making it easier for the CA mechanism to capture their spatial feature differences. However, the leaf texture and spectral curve similarity of rice reach 82%, making it difficult for existing feature combinations to completely distinguish them. In addition, the classification accuracy (MIoU 95.3%) of non-crop categories (such as irrigation ditches and bare land in Friendship Farm) is generally higher than that of crop categories. This is consistent with the researchers' finding that "non-crop spectral features are more unique"—the near-infrared reflectance of ditches and bare land (< 20%) is much lower than that of crops (> 60%), and the LGCG-Conv module can easily filter out such backgrounds without complex feature associations.

[0118] Despite the significant progress made in this invention, three limitations remain: First, the model relies on "high-resolution UAV data (0.1-1m)," and its performance on Sentinel-2 satellite data (10m resolution) has not yet been verified—the low resolution of satellite data may lead to a decrease in the local feature capture capability of the LGCG-Conv module, making it difficult for the CA mechanism to extract fine-grained "spectral-spatial" associations; second, the experiment only covers three types of crops: soybean, corn, and rice, and does not involve economic crops such as cotton and rapeseed. The leaf structure of these crops (such as the palmate leaves of cotton) differs greatly from that of gramineous crops, which may affect the feature association effect of the CA module; finally, the computational cost of FCR-TransUNet is 18% higher than that of the original TransUNet, mainly due to the cross-dimensional computation of the CA module. In large-scale farmland classification of more than 100km², the real-time processing delay may exceed 30s, which is difficult to meet the "real-time decision-making" requirements of precision agriculture.

[0119] To address the three core challenges of remote sensing crop classification in complex farmland scenarios—namely, noise interference from non-crop information in the background, classification confusion due to inter-class similarity in crop spectral features, and blurred segmentation of small crop boundaries caused by sample imbalance—this invention proposes an improved FCR-TransUNet model based on TransUNet. This model incorporates a Local-Global Cooperative Gated Enhanced Convolutional Module (LGCG-Conv) (which uses a three-stage process of fine-grained local feature capture, global context modeling, and gated feature selection to filter out background noise and improve the purity of crop feature representation), embeds a CA collaborative attention mechanism (which enhances the ability to capture inter-class differences and alleviates classification confusion through cross-channel-spatial feature association modeling), and designs a Reverse Probabilistic Edge Optimization Module (RP-EO). (By dynamically strengthening the attention weights of small-sample crop boundaries to balance model learning bias and improve the segmentation accuracy of small-sample boundaries), a targeted solution was developed. To verify the effectiveness and applicability of the model, experiments were conducted on the Heilongjiang Friendship Farm dataset. The results showed that on this farm dataset, FCR-TransUNet achieved an average intersection-over-union (MIoU) ratio of 92.2%, an F1 score of 92.7%, and precision and recall of 93%. This represents a 1.8% improvement over SAM2-UNet, the best-performing model among seven comparison models including YOLO11-seg, DeepLabV3+, and SAM2-UNet, and a 2.9% improvement over the benchmark TransUNet. Ablation experiments also verified the synergistic effectiveness of the three modules and the rationality of the architecture design. In summary, FCR-TransUNet effectively solves the core problem of remote sensing crop classification in complex farmland, providing an efficient and reliable crop classification technology solution for precision agriculture, and has significant practical implications for promoting intelligent agricultural production management and ensuring food security.

[0120] When applying the TransUnet-based remote sensing image crop classification method provided in this manual, it is not necessary to consider... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this manual does not impose any restrictions on it.

[0121] The above describes one or more embodiments of a TransUnet-based remote sensing image crop classification method. Based on the same concept, this specification also provides a corresponding TransUnet-based remote sensing image crop classification device, such as... Figure 8 As shown.

[0122] Figure 8 A schematic diagram of a TransUnet-based remote sensing image crop classification device provided in this specification includes: The acquisition module 801 is used to acquire remote sensing images of crops.

[0123] The acquisition module 802 is used to acquire the improved TransUnet network. The improvements to the improved TransUnet network include embedding a cooperative gated enhancement convolutional module after the third convolutional neural network of the traditional TransUnet network, embedding a coordinated attention mechanism (CA) in the skip connections between the encoder and decoder, and embedding an inverse probabilistic edge optimization module between the second upsampling module and the second feature concatenation module. The embedded cooperative gated enhancement convolutional module is used to reduce the interference of background noise on feature extraction through the synergistic effect of self-attention convolution and gated convolution. CA is used to enhance the ability to distinguish similar crops by modeling cross-channel and spatial feature associations. The inverse probabilistic edge optimization module is used to balance the learning bias of the improved TransUnet network for each category and avoid the problem of blurred boundary segmentation due to insufficient feature learning in small sample crops. The improved TransUnet network adopts a feature map transfer method based on spiral connections.

[0124] The classification module 803 is used to segment crop remote sensing images using an improved TransUnet network to obtain pixel-level classification results for crop remote sensing images.

[0125] Specific limitations regarding the TransUnet-based remote sensing image crop classification device can be found in the above description of the limitations of the TransUnet-based remote sensing image crop classification method, and will not be repeated here. Each module in the aforementioned TransUnet-based remote sensing image crop classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0126] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A crop classification method based on TransUnet remote sensing images is provided.

[0127] This instruction manual also provides Figure 9 The schematic diagram of the computer device shown is as follows: Figure 9 At the hardware level, the computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 A crop classification method based on TransUnet remote sensing images is provided.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A crop classification method based on TransUnet remote sensing images, characterized in that, include: Acquire remote sensing images of crops; An improved TransUnet network is obtained. The improvements include embedding a cooperative gated augmentation convolutional module after the third convolutional neural network in the traditional TransUnet network; embedding a coordinated attention mechanism (CA) in the skip connections between the encoder and decoder; and embedding an inverse probabilistic edge optimization module between the second upsampling module and the second feature concatenation module. The embedded cooperative gated augmentation convolutional module is used to reduce the interference of background noise on feature extraction through the synergistic effect of self-attention convolution and gated convolution. The CA is used to enhance the distinguishability of similar crops by modeling cross-channel and spatial feature associations. The inverse probabilistic edge optimization module is used to balance the learning bias of the improved TransUnet network for each category and avoid the boundary segmentation blurring problem caused by insufficient feature learning in small sample crops. The improved TransUnet network adopts a feature map transfer method based on spiral connections. The crop remote sensing image is segmented using an improved TransUnet network to obtain pixel-level classification results for the crop remote sensing image.

2. The crop classification method based on TransUnet remote sensing images as described in claim 1, characterized in that, The conventional TransUnet network includes an encoder and a decoder; the encoder includes a first convolutional neural network, a second convolutional neural network, a third convolutional neural network, a linear projection layer, multiple Transformer layers, downsampling, and a first set of 3×3 convolutions and the ReLU activation function, which are connected in series. The decoder includes a first upsampling module, a first feature concatenation module, a second set of 3×3 convolutions and ReLU, a second upsampling module, a second feature concatenation module, a third set of 3×3 convolutions and ReLU, a third upsampling module, a third feature concatenation module, a fourth set of 3×3 convolutions and ReLU, a fourth upsampling module, a fifth set of 3×3 convolutions and ReLU, and skip connections between the encoder's convolutional neural network and the decoder's feature concatenation module; The improved TransUnet network also includes a first reverse spiral reconstruction module, a second reverse spiral reconstruction module, and a third reverse spiral reconstruction module; The first reverse spiral reconstruction module is embedded in the first feature stitching module of the decoder; the second reverse spiral reconstruction module is embedded in the second feature stitching module of the decoder; the third reverse spiral reconstruction module is embedded in the third feature stitching module of the decoder; the reverse spiral reconstruction module is used to reconstruct the image during the decoding stage; The embedded collaborative gated augmented convolution module includes a first 1×1 convolution, Conv, a first self-attention convolution DSA, a second self-attention convolution DSA, a first gated convolution GC, a second gated convolution GC, an adder, and a second 1×1 convolution; Conv includes a 3×3 convolution and ReLU; The CA includes a residual connection module, X average pooling, Y average pooling, concatenation + convolution, batch normalization + nonlinear transformation, a third 1×1 convolution and a fourth 1×1 convolution, a first Sigmoid function, a second Sigmoid function, and a reweighting module.

3. The crop classification method based on TransUnet remote sensing images as described in claim 2, characterized in that, The segmentation of the crop remote sensing image using an improved TransUnet network to obtain pixel-level classification results of the crop remote sensing image specifically includes: The crop remote sensing image is extracted at a 1 / 2 scale, a 1 / 4 scale, and a 1 / 8 scale by sequentially passing through the first convolutional neural network, the second convolutional neural network, and the third convolutional neural network. The 1 / 8 scaled feature map is input into the collaborative gating enhancement module to obtain the first hidden feature, and the first hidden feature is linearly projected to obtain the projected feature; The projected features are input into a multi-layer Transformer layer for processing. The global long-distance dependencies of the features are captured through a self-attention mechanism to obtain the second hidden feature. The second hidden feature is sequentially passed through downsampling and the first set of 3×3 convolutions and the ReLU activation function to obtain the encoded feature; Based on the encoded features, the CA-processed 1 / 2 scaled feature map, the CA-processed 1 / 4 scaled feature map, and the CA-processed 1 / 8 scaled feature map, multiple convolution operations, upsampling, and feature extraction operations are performed to obtain the decoded features. The decoded features are processed by a segmentation head to obtain pixel-level classification results of crop remote sensing images.

4. The crop classification method based on TransUnet remote sensing images as described in claim 3, characterized in that, The step of inputting the 1 / 8 scaled feature map into the collaborative gating enhancement module to obtain the first hidden feature specifically includes: The 1 / 8 scaled feature map is processed by the first 1×1 convolution to obtain the initial features; The initial features are respectively input into Conv, the first DSA and the second DSA for convolution processing; The output of Conv and the output of the first DSA are input together into the first GC for feature extraction to obtain the first gated convolutional feature. The first gated convolutional feature and the output of the second DSA are both input into the second GC for convolution processing to obtain the second gated convolutional feature; The output of Conv, the first gated convolutional feature, and the second gated convolutional feature are fused and then processed through a second 1×1 convolution to obtain the first hidden feature.

5. The crop classification method based on TransUnet remote sensing images as described in claim 3, characterized in that, The process of performing multiple convolution operations, upsampling, and feature extraction operations based on the encoded features, CA-processed 1 / 2 scaled feature maps, CA-processed 1 / 4 scaled feature maps, and CA-processed 1 / 8 scaled feature maps to obtain decoded features specifically includes: The encoded features are processed by the first upsampling module to obtain the first upsampled features; The CA-processed 1 / 8 scaled feature map is then processed by the first reverse spiral reconstruction module to extract the index features, thus obtaining the first index features. The first index feature and the first upsampled feature are concatenated by the first feature concatenation module to obtain the first concatenated feature. The first concatenated feature is processed by a second set of 3×3 convolutions and ReLU, and a second upsampling module to obtain the second upsampled feature; The second upsampled feature is processed by the reverse probability edge optimization module to obtain the reverse probability edge optimized feature; The CA-processed 1 / 4 scaled feature map is then processed by the second reverse spiral reconstruction module to extract index features, resulting in the second index features. The second feature is obtained by concatenating the second index feature with the reverse probability edge optimization feature through the second feature concatenation module. The second concatenated feature is processed through a third set of 3×3 convolutions and ReLU, and a third upsampling module to obtain the third upsampling feature; The CA-processed 1 / 2 scaled feature map is then processed by the third reverse spiral reconstruction module to extract the index features, resulting in the third index features. The third index feature and the third upsampled feature are concatenated by the third feature concatenation module to obtain the third concatenated feature. The third concatenated feature is processed through a fourth group of 3×3 convolutions and ReLU, a fourth upsampling module, and a fifth group of 3×3 convolutions and ReLU to obtain the decoded feature.

6. The crop classification method based on TransUnet remote sensing images as described in claim 5, characterized in that, The process of processing the second upsampled feature through the inverse probabilistic edge optimization module to obtain the inverse probabilistic edge optimized feature specifically includes: The second upsampled feature is processed by the sigmoid activation function to obtain a probability map; the sigmoid activation function is used to emphasize the edge regions of the crop remote sensing image. The probability map is reverse-calculated to obtain a reverse probability map; the reverse probability map is used to distinguish the background and crop area in a crop remote sensing image. The inverse probability map is multiplied element-wise with the CA-processed 1 / 4 scaled feature map to generate a new feature map; The new feature map and the second upsampled feature are fused to obtain a fused feature; the fused feature includes contextual information and fine edge location information. The fused features are processed by two convolutional layers to obtain the reverse probability edge optimization features.

7. The crop classification method based on TransUnet remote sensing images as described in claim 5, characterized in that, The step of extracting the index features from the CA-processed 1 / 2 scaled feature map using the third reverse spiral reconstruction module to obtain the third index features specifically includes: Calculate multiple reverse spiral coordinates based on different rotation angles; Convert multiple reverse spiral coordinates into multiple reverse Cartesian coordinates, and then define the multiple reverse Cartesian coordinates as multiple indices; Extract the third index features of multiple indexes from the feature map after CA processing at a 1 / 2 scaling ratio.

8. The crop classification method based on TransUnet remote sensing images as described in claim 3, characterized in that, The extraction process of the feature map after CA processing at a 1 / 8 scaling ratio specifically includes: By performing residual connections on the 1 / 8 scaled feature map, residual connection features are obtained. The residual connection features are input into X-average pooling and Y-average pooling respectively to obtain X-average pooling features and Y-average pooling features; The X-mean pooling feature and the Y-mean pooling feature are sequentially concatenated, convolved, batch normalized, and nonlinearly transformed to obtain the nonlinear feature. After inputting the nonlinear features into the third and fourth 1×1 convolutions for convolution processing, X convolution features and Y convolution features are obtained. The X convolutional features are input into the first Sigmoid function to obtain the first Sigmoid features; The Y convolutional features are input into the second Sigmoid function to obtain the second Sigmoid features; The first Sigmoid feature, the second Sigmoid feature, and the residual connection feature are weighted and fused by a reweighting module to obtain a 1 / 8 scaled feature map after CA processing.

9. The crop classification method based on TransUnet remote sensing images as described in claim 3, characterized in that, The step of sequentially extracting 1 / 2 scale feature maps, 1 / 4 scale feature maps, and 1 / 8 scale feature maps of crop remote sensing images through a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network specifically includes: For any convolutional neural network layer, local features of the input data are extracted through the convolutional neural network to obtain the original feature map including basic scale information; wherein, the input data of the first convolutional neural network is the object remote sensing image, the input data of the second convolutional neural network is the output of the first convolutional neural network, and the input data of the third convolutional neural network is the output of the second convolutional neural network. The original feature map is divided into multiple regions. Starting from the center of the original feature map, multiple polar coordinates are calculated based on the spiral path formula, and the multiple polar coordinates are converted into multiple Cartesian coordinates respectively. Feature sampling and integration are performed by gradually expanding outward along multiple Cartesian coordinates to obtain a scaled feature map carrying information about the center and surrounding spatial structure.

10. The crop classification method based on TransUnet remote sensing images as described in claim 3, characterized in that, The loss function of the improved TransUnet network is: ； ； ； in, The loss value for the improved TransUnet network, The value of binary cross-entropy loss (BCEloss) is... This is the Dice loss value. For the pixels in the sample label i The value, Pixels in the segmentation prediction results of the improved TransUnet network i The value, N The number of samples.

Images