A multi-class crop cultivation area classification method fusing attention mechanism and capsule layer

Abstract

The application discloses a multi-class crop cultivation area classification method fusing an attention mechanism and a capsule layer. The application relates to the technical field of fine classification of multi-class crop cultivation areas in complex agricultural scenes, and aims to solve the problem of insufficient learning and representation ability of a convolutional network for hyperspectral image features when fine classification of multi-class crop cultivation areas is performed. The method fuses a lightweight convolution, spatial and channel attention, and a capsule module in a coding and decoding structure, so that the extraction and representation ability of a network for spatial and spectral features of a hyperspectral remote sensing image is enhanced. Meanwhile, a joint classification and reconstruction loss is used to improve the classification generalization performance of the model, so that high-precision and fine classification of complex cultivation areas is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fine classification of multi-crop cultivation areas in complex agricultural scenes, and particularly relates to a multi-crop cultivation area classification method fusing an attention mechanism and a capsule layer. BACKGROUND

[0002] Efficient and accurate fine classification of multi-crop cultivation areas is a key link to ensure food security and promote sustainable agricultural development. However, in multi-crop agricultural landscapes, due to land fragmentation, diverse crop types, and complex intercropping patterns, combined with soil background differences and weed interference, the spatial and spectral characteristics of ground objects are highly complex, increasing intra-class differences and reducing inter-class separability, thus posing significant challenges to fine classification of cultivation areas.

[0003] Hyperspectral images can accurately depict the spectral characteristics of ground objects, providing unique advantages for fine classification in complex agricultural scenes. In recent years, with the development of deep learning technology, hyperspectral image classification has made significant progress in the field of agricultural remote sensing. Existing models include Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Deep Belief Network (DBN), etc. Among them, CNN performs outstandingly in spatial feature extraction, and its variants such as 1D-CNN, 2D-CNN, and 3D-CNN have achieved good classification results. However, RNN and DBN models need to expand spatial structured images into one-dimensional vectors as input, which easily causes the loss of key joint spectral information. At the same time, traditional CNN is based on scalar neurons, which has limitations in complex feature representation.

[0004] To break through the above bottlenecks, some scholars have proposed the concept of capsules and developed capsule networks. This method models hierarchical spatial relationships through dynamic routing mechanisms and vectorized neurons, and can represent the instantiation parameters of object features. Although existing research has explored the application of capsule networks in hyperspectral image classification, their performance in complex agricultural landscapes is still limited. Specifically, traditional capsule networks are not designed for agricultural scenes, making it difficult to effectively model spatial and spectral features under the conditions of mixed pixels, multiple crop types, and fragmented fields, resulting in insufficient discrimination and classification generalization performance of the network in complex environments. SUMMARY

[0005] The present application aims to provide a multi-crop cultivation area classification method fusing an attention mechanism and a capsule layer to achieve high-precision and fine classification of cultivation areas in complex agricultural landscapes.

[0006] The application provides a multi-class crop cultivation area classification method fusing an attention mechanism and a capsule layer, comprising the following steps:

[0007] Step one: obtaining hyperspectral images of multi-class crop cultivation areas, performing principal component analysis dimension reduction on the hyperspectral images, and performing image blocking to obtain image blocks;

[0008] Step two: performing spatial attention weighting operation on the image blocks;

[0009] Step three: constructing a lightweight convolution module based on the output result of step two;

[0010] Step four: performing channel attention weighting operation on the output result of step three;

[0011] Step five: constructing a capsule layer based on the output result of step four, wherein the capsule layer comprises primary capsules, dynamic routing and digital capsules, vector-based neurons represent capsules, and the modulus and direction of the vector are used to represent the probability and pose attribute of the instance; the primary capsules convert features into vector-form capsules through convolution operation, the digital capsules process the output of the primary capsules in cooperation with the dynamic routing, and high-level capsules representing different class semantic attributes are generated;

[0012] Step six: performing classification and reconstruction on the digital capsule result, wherein the classification processes the digital capsule result through L2 norm to form a vector with n elements, the index corresponding to the maximum value element in the vector is the class code, and then the target class to which the classification result belongs is determined, and the reconstruction is realized by a fully connected layer;

[0013] Step seven: performing precision evaluation on the fine classification result of the cultivation area.

[0014] Further, in step one, the principal component analysis dimension reduction comprises: performing principal component analysis dimension reduction processing on the hyperspectral images to compress the number of wavebands to 1 / 4 of the original number, and removing the high redundancy and noise between the wavebands of the hyperspectral images; and the image blocking comprises: dividing the data after the principal component analysis dimension reduction into image blocks with the size of 11x11x64, and taking the image blocks as the input of the classification network.

[0015] Further, in step two, the image blocks are respectively subjected to maximum value pooling and average value pooling in the channel dimension, the pooling results are respectively cascaded in a stacked manner to form a cascaded feature with the size of 11x11x2, convolution and activation operations are sequentially performed on the basis of the cascaded feature to form a spatial attention matrix weight value with the value range of 0-1, and the spatial attention matrix weight value is multiplied with the original image block element by element to complete the weighting operation; the spatial attention weighting operation process is represented by the following formula:

[0016]

[0017] where F represents the input feature data, AvgPool(·) and MaxPool(·) represent average value pooling and maximum value pooling respectively, Conv(·) represents a convolution operation with a convolution kernel size of 7x7, σ(·) represents a Sigmoid activation function, O s is the obtained spatial attention matrix weight value; wherein the convolution operation involved is represented as follows:

[0018]

[0019] wherein, is the activation output of the βth channel in the convolution layer l; f (·) represents an activation function, which adopts a Sigmoid; is the feature to be activated of the βth channel in the convolution layer l, which is obtained by performing convolution sum operation on the feature map of the previous layer and the corresponding convolution kernel and then adding the bias ; is an input feature map subset used to obtain ; "*" represents a convolution operator.

[0020] Further, in step three, on the basis of the output result of step two, a convolution operation is first performed to generate 11x11x128 intrinsic features, and then a Depthwise convolution operation is performed, and the calculation process is as follows:

[0021]

[0022] where Y represents intrinsic features after a regular convolution, u is an input feature, and the convolution kernel ∈ m x k x k x c;

[0023]

[0024] wherein, i,j represent performing the jth linear operation on the ith intrinsic feature yi of the intrinsic feature set Y to generate the corresponding jth enhanced feature y ij , wherein the last layer i,s represents an identity mapping on the intrinsic feature, and the final obtained enhanced feature set is {y 11 , y 12 , …, y ms}; the relationship between the number of intrinsic features m, the number of linear operations s, and the number of final enhanced features n is described as n=m·s (m≤n), and without counting the last identity transformation, the relationship is further expressed as m(s-1)=n / s·(s-1).

[0025] Further, in step four, the channel attention operation is performed on the enhanced features output in step three, the feature data is first compressed in the spatial dimension, and the average value is pooled in the spatial dimension scale. The information exchange of the current channel and its k neighborhood channels is considered, and one-dimensional convolution is used to achieve it, and then the channel dimension attention weight value is generated. Multiply it with the enhanced feature matrix by element, and complete the channel dimension weighting operation; the calculation process is as follows:

[0026]

[0027] In the formula, y represents the input channel feature data, C1D(·) represents one-dimensional convolution, and σ(·) represents the Sigmoid activation function, The channel attention weight value obtained is k, which represents the size of the one-dimensional convolution kernel, and is adaptively determined as follows:

[0028]

[0029] In the formula, C represents the number of channels of the input data, k represents the size of the one-dimensional convolution kernel obtained by calculation, odd represents that the obtained value is an odd number, b=1, and γ=2.

[0030] Further, in step five, the primary capsule converts the features into a vector form of the capsule by using the convolution operation. It contains multiple parallel small convolution kernels, and the convolution kernel size is 4x4x8, a total of 32. Each convolution kernel extracts a group of features to form a capsule vector;

[0031] The information transmission process of the vector neuron in the capsule layer includes affine transformation, weighted summation and nonlinear activation; the affine transformation is the abstract spatial position relationship between the encoding bottom capsule and the high-level feature. The input bottom capsule is multiplied by the weight matrix W ij (i=1, 2, 3, …) to obtain the high-level feature; the weighted summation uses the coupling coefficient c ij (i=1, 2, 3, …) to weight and add the high-level feature to obtain the input vector required by the high-level capsule, and the calculation formula is:

[0032]

[0033] In the formula, u i represents the input bottom capsule, W ij is the weight matrix, U j|i represents the high-level feature, c ij is the coupling coefficient with the characteristics of non-negativity and sum of 1, j is the input vector required by the high-level capsule;

[0034] The nonlinear activation utilizes a squashing function squash(·) to squash the vector The length of j is normalized to the interval (0, 1) and the direction of the vector is kept unchanged, so that the length of the final output high-level capsule vj is used to represent the probability of the existence of the object entity:

[0035]

[0036] In the formula, ||·|| represents the L2 norm, which is the length of the vector, and ||·||2 represents the square of the L2 norm; the first half of the calculation formula is to squash the vector, and the second half is the direction vector, which realizes the normalization operation of the vector consistent with the original direction as a whole;

[0037] The dynamic routing is used to update and determine the coupling coefficient c ij The operation process of the dynamic routing includes: the input low-level capsule u i is multiplied by the weight matrix W i to obtain the high-level feature U i , and on this basis, the softmax normalization, vector weighted summation, vector squashing squash(·), and updating of the coupling coefficient operation are performed, wherein the initial value of the process variable b is 0; through multiple rounds of iteration in this way, the high-level capsule is finally obtained.

[0038] Further, in step six, the expression of the reconstruction operation is:

[0039]

[0040] In the formula, x is the input vector data, W is the weight matrix, Reshape(·) represents the reshaping operation, and O r represents the reconstructed data.

[0041] The loss function is used to constrain the training process of the classification network, and a joint loss function including classification and reconstruction loss is constructed, wherein the reconstruction loss plays a regularization role, and the sum of squares of the difference between the reconstruction output in the decoding stage and the original input is calculated.

[0042]

[0043] In the formula, L c represents the classification loss of a single capsule neuron, c is the category value, T c is an indicator function, and T cValue is 1, otherwise 0; m+ and m- represent the upper edge threshold and the lower edge threshold respectively, the former is to punish the error situation that the classifier predicts the existence of a certain class but actually does not exist, and the latter is to punish the error situation that actually exists but the classifier predicts the non-existence of a certain class, m+=0.9, m-=0.1; Lambda is a sparse coefficient for adjusting the proportion, taking 0.5; Ltotal represents the total loss, I is the original input, Or is the decoder reconstruction output, is a weight factor, taking 2x10 -5 .

[0044] Further, in step seven, the indicators for evaluation include overall accuracy, average classification accuracy and Kappa coefficient K; the classification results are compared with the ground reference data to calculate the confusion matrix M, and each evaluation indicator is calculated, and the formula is as follows:

[0045]

[0046] In the formula, n is the number of categories, N is the total number of test samples, M uu represents the number of samples of which the category u is correctly identified as u, M uv represents the number of samples of which the category v is identified as u, u [1, n], v [1, n].

[0047] The application has the following beneficial effects: the multi-class crop cultivation area classification method of the fusion attention mechanism and the capsule layer, by fusing lightweight convolution, spatial and channel attention and capsule module in the coding and decoding structure, the network can enhance the extraction and representation ability of the spatial and spectral features of the hyperspectral remote sensing image. At the same time, the joint classification and reconstruction loss is used to improve the classification generalization performance of the model, so as to realize high-precision and fine classification of complex cultivation areas. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced as follows: obviously, for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0049] Figure 1 is the technical flow chart of the present application;

[0050] Figure 2 is the regional hyperspectral image and land class label of the embodiment;

[0051] Figure 3 is the dynamic routing execution flow;

[0052] Figure 4 is the fine classification result of the cultivation area. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. The technical solutions provided by the embodiments of the present application are described in detail below in combination with the drawings.

[0054] Please refer to Figures 1 to 4 The embodiments of the present application provide a multi-class crop cultivation area classification method fusing attention mechanism and capsule layer, and a technical process thereof is as shown in Figure 1 The technical process comprises the following steps.

[0055] Step one, obtaining a hyperspectral image of a multi-class crop cultivation area, performing principal component analysis dimension reduction and image blocking on the hyperspectral image to obtain an image block.

[0056] The hyperspectral image is obtained from open-source data of a remote sensing community, which is collected by a full-spectrum multi-modal imaging spectrometer sensor, and the collection site is a multi-class crop cultivation area. The image spectral range is 400-1000 nm, the number of bands is 256, the image size is 1580×3750 pixels, and the spatial resolution is 0.5 m. The data has 21 kinds of ground object categories in total, among which there are 14 categories of seedlings, economic and food crops, and 1% of samples of each category are sampled layer by layer during model training. The hyperspectral image coverage area is as shown in Figure 2 The category names and detailed sample information involved are shown in Table 1.

[0057] Table 1: Ground object categories and sample information of the hyperspectral image of a multi-class crop cultivation area

[0058]

[0059] The principal component analysis dimension reduction comprises: performing principal component analysis dimension reduction processing on the hyperspectral image, compressing the number of bands to 1 / 4 of the original, removing the high redundancy and noise between the bands of the hyperspectral image, and improving the signal-to-noise ratio and subsequent classification efficiency. The image blocking comprises: dividing the data after the principal component analysis dimension reduction into an image block with a size of 11×11×64, which is used as the input of the classification network.

[0060] Step two, performing a spatial attention weighting operation on the image block.

[0061] The image blocks are respectively subjected to maximum value pooling and average value pooling in the channel dimension, and the pooling results are respectively concatenated in a stacked manner to form a concatenated feature with a size of 11x11x2, taking one image block as an example. On this basis, convolution and activation operations are sequentially performed to form a spatial attention matrix weight value (11x11x1) with a value range of 0~1, and the spatial attention matrix weight value is multiplied with the original image block element by element to complete the weighting operation; the spatial attention weighting operation process is expressed by the following formula:

[0062]

[0063] In the formula, F represents input feature data, AvgPool(·) and MaxPool(·) represent average value pooling and maximum value pooling respectively, Conv(·) represents convolution operation, the convolution kernel size is 7x7, σ(·) represents Sigmoid activation function, O s is the obtained spatial attention matrix weight value; wherein the convolution operation involved is expressed as follows:

[0064]

[0065] In the formula, is the activation output of the βth channel in the convolution layer l; f(·) represents an activation function, and Sigmoid is adopted; is the feature to be activated in the βth channel of the convolution layer l, which is obtained by performing convolution sum operation on the feature map of the previous layer and the corresponding convolution kernel and obtaining by adding the bias ; is the input feature map subset used to obtain ; "*" represents a convolution operator.

[0066] Step three, on the basis of the output result of step two, a lightweight convolution module is constructed.

[0067] On the basis of the output result of step two, a convolution operation is first performed to produce intrinsic features with a size of 11x11x128, and then a Depthwise convolution operation is performed, which can be regarded as a linear operation, and the calculation process is as follows:

[0068]

[0069] In the formula, Y represents intrinsic features after a regular convolution, u is an input feature, and the convolution kernel ∈ m×k×k×c;

[0070]

[0071] In the formula, i,j represents the i-th eigen feature y i performs the j-th linear operation, and further generates the corresponding j-th reinforced feature y ij , wherein the last layer i,s represents the identity mapping of the eigen feature, and the final reinforced feature set is {y 11 , y 12 , …, y ms}; the relationship between the number of eigen features m, the number of linear operations s and the number of final reinforced feature channels n is described as n=m·s (m≤n), and the relationship is further expressed as m(s-1)=n / s·(s-1) without considering the last identity transformation.

[0072] Step four, performing channel attention weighting operation on the output result of step three.

[0073] Performing channel attention operation on the reinforced feature output in step three, first performing spatial dimension compression on the feature data, and then performing average value pooling on the spatial dimension scale, considering the information exchange between the current channel and its k neighborhood channels, using one-dimensional convolution to achieve, and then generating channel dimension attention weight value, and multiplying it with the reinforced feature to complete the channel dimension weighting operation; the calculation process is as follows:

[0074]

[0075] In the formula, y represents the input channel feature data, C1D(·) represents one-dimensional convolution, σ(·) represents Sigmoid activation function, to obtain the channel attention weight value, k represents the size of the one-dimensional convolution kernel, which is adaptively determined by the following formula:

[0076]

[0077] In the formula, C represents the number of channels of the input data, k represents the size of the one-dimensional convolution kernel obtained by calculation, odd represents that the obtained value is an odd number, b=1, and γ=2.

[0078] Step five, constructing the capsule layer based on the output result of step four, the capsule layer includes primary capsules, dynamic routing and digital capsules, the vector-based neuron represents the capsule, and the modulus and direction of the vector are used to represent the probability of the existence of the instance and the pose attribute; the primary capsule converts the feature into a vector-form capsule by convolution operation, the digital capsule cooperates with the dynamic routing to process the output of the primary capsule, and generates a high-level capsule representing different class semantic attributes.

[0079] The primary capsules convert the features into vector-form capsules using convolutional operations, which contain multiple parallel small convolutional kernels with a size of 4x4x8, a total of 32, each of which extracts a set of features to form a capsule vector, with an example size of [32(x16), 8]. The numeric capsules further process the outputs of the primary capsules in cooperation with the dynamic routing to generate high-level capsules representing semantic attributes of different classes, with an example size of [n, 16], where n represents the number of classes.

[0080] The information transmission process of the vector neurons in the capsule layer includes three parts: affine transformation, weighted summation, and nonlinear activation. The affine transformation encodes the abstract spatial positional relationship between the bottom capsules and the high-level features. The input bottom capsules are multiplied by the weight matrix W ij (i=1, 2, 3,...) to obtain the high-level features. The weighted summation uses the coupling coefficients c ij (i=1, 2, 3,...) to weight and add the high-level features to obtain the input vector required by the high-level capsules, with the calculation formula being:

[0081]

[0082] where u i represents the input bottom capsules, W ij is the weight matrix, U j|i represents the high-level features, c ij is the coupling coefficient with the properties of non-negativity and sum of 1, j is the input vector required by the high-level capsules.

[0083] The nonlinear activation uses the squash(·) function to normalize the length of the vector v j to the interval (0, 1) while keeping the vector direction unchanged, so that the length of the final output high-level capsule v j is used to represent the probability of the existence of the object entity:

[0084]

[0085] where ||·|| represents the L2 norm, which is the length of the vector, and ||·||2 represents the square of the L2 norm; the first half of the calculation formula is to squash the vector, and the second half is the direction vector, which realizes the normalization operation of the vector consistent with the original direction; the main function of the dynamic routing is to update and determine the coupling coefficient c ij , so as to solve how to selectively transmit the target feature information from the bottom capsules to the high-level capsules with appropriate weights. The operation process of the dynamic routing includes: the input bottom capsules u i are multiplied by the weight matrix W i to obtain the high-level features U iOn this basis, the softmax normalization, vector weighted sum, vector squashing (·), updating coupling coefficient and other operations are performed, wherein the initial value of the process variable b is 0; the dynamic routing execution process is as shown in Figure 3 The input: u1, u2; high-level features: U1, U2; input vector required by high-level capsules: Process variables: a, b; coupling coefficient: Output: v. Through multiple iterations, the high-level capsules are finally obtained. It can be seen that when the inner product of the process vector ai and the high-level feature Ui is larger, the coupling coefficient value obtained by updating is also larger, at this time the bottom layer capsule will pass more rich information to the high-level capsule.

[0086] Step six, classifying and reconstructing the digital capsule result, the classification uses L2 norm to process the digital capsule result to form a vector with n elements, the index corresponding to the maximum value element in the vector is the class code, and then the target class to which the classification result belongs is determined, and the reconstruction is realized by relying on the full connection layer.

[0087] The second to fifth steps are the encoding stage, and this step is the decoding stage, mainly including classification and reconstruction. The reconstruction is mainly realized by relying on the full connection layer, and the corresponding size change of the data is

[512] ->

[1024] ->[11x11xC / 4], and the expression of the reconstruction operation is:

[0088]

[0089] In the formula, x is the input vector data, W is the weight matrix, Reshape(·) represents the reshaping operation, and Or represents the reconstructed data.

[0090] The loss function is used to constrain the training process of the classification network, and a joint loss function including classification and reconstruction loss is constructed, wherein the reconstruction loss plays a regularization role, and the sum of squares of the difference between the reconstruction output in the decoding stage and the original input is calculated.

[0091]

[0092] In the formula, L c represents the classification loss of a single capsule neuron, c is the class value, T c is an indicator function, and T c1, otherwise 0; m+ and m- represent the upper edge threshold and the lower edge threshold, respectively, the former is used to punish the error situation that the classifier predicts the existence of a certain class but actually does not exist, and the latter is used to punish the error situation that actually exists but the classifier predicts the non-existence of a certain class, m+=0.9, m-=0.1; λ is a sparse coefficient for adjusting the proportion, and is 0.5; Ltotal represents the total loss, I is the original input, and Or is the decoder reconstruction output, is a weight factor, and is 2x10 -5 .

[0093] Step seven, precision evaluation is performed on the fine classification result of the cultivated area.

[0094] The fine classification result of a certain multi-class crop cultivated area is as shown in Figure 4 The evaluation indexes include overall accuracy (Overall Accuracy, OA), average classification accuracy (Average Accuracy, AA) and Kappa coefficient (κ); the classification result is compared with the ground reference data to obtain a confusion matrix M, and each evaluation index is calculated, and the formula is as follows:

[0095]

[0096] In the formula, n is the number of categories, N is the total number of test samples, M uu represents the number of samples of which the category u is correctly identified as u, M uv represents the number of samples of which the category v is identified as u, u∈[1, n], v∈[1, n]. Based on the method proposed in the application, the precision indexes OA, AA and κ of the fine classification result of the cultivated area are 99.19%, 96.41% and 0.99, respectively.

[0097] In summary, the fine classification method for multi-class crop cultivated areas fusing attention mechanism and capsule layer proposed in the application introduces lightweight convolution in the encoding stage, and combines spatial and channel attention mechanisms, which effectively improves the extraction and filtering ability of the network for the joint spectral feature in the hyperspectral remote sensing image, and relieves the interference caused by complex background and broken land. In the feature expression stage, the vectorized neuron hierarchical spatial relationship modeling layer of the capsule layer is used, and in the decoding stage, the classification and reconstruction loss are combined, so as to enhance the discrimination and classification generalization performance of the model for the multi-class crop cultivated area, and realize high-precision and fine classification.

[0098] The embodiment of the present application further provides a storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to implement some or all steps in each embodiment of the multi-class crop cultivation area classification method combining the attention mechanism and the capsule layer provided by the present application. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0099] Those skilled in the art can clearly understand that the technology in the embodiment of the present application can be realized by means of software and necessary general hardware platforms. Based on such understanding, the technical solutions in the embodiment of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disc, an optical disc, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some part of the embodiment of the present application.

[0100] The above-mentioned embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A method for classifying multi-crop cultivated areas by integrating attention mechanisms and capsule layers, characterized in that, include: Step 1: Acquire hyperspectral images of various crop cultivation areas, perform principal component analysis for dimensionality reduction and image segmentation on the hyperspectral images to obtain image blocks; Step 2: Perform spatial attention weighting on the image block; Step 3: Construct a lightweight convolutional module based on the output of Step 2; Step four: Perform channel attention weighting on the output of step three; Step 5: Construct the capsule layer based on the output of Step 4. The capsule layer includes primary capsules, dynamic routing, and digital capsules. Vector-based neurons represent capsules, and the magnitude and direction of the vectors are used to represent the probability and pose attributes of the existence of instances, respectively. The primary capsules use convolution operations to convert features into vector capsules. The digital capsules work with the dynamic routing to process the output of the primary capsules and generate high-level capsules that represent different classes of semantic attributes. Step 6: Classify and reconstruct the digital capsule results. The classification uses the L2 norm to process the digital capsule results, forming a vector with n elements. The index corresponding to the maximum value element in the vector is the target class. The reconstruction is achieved by a fully connected layer. The expression for the refactoring operation is: In the formula, x is the input vector data, W is the weight matrix, Reshape(·) represents the reshaping operation, and Or represents the reconstructed data; The loss function is used to constrain the training process of the classification network. A joint loss function is constructed that includes classification and reconstruction losses. The reconstruction loss plays a regularization role and calculates the sum of squares of the differences between the reconstruction output and the original input during the decoding stage. In the formula, L c This represents the classification loss for a single capsule neuron, where c is the number of categories, and T is the classification loss. c It is an indicator function, T when the category is c. c The value is 1 otherwise; m+ and m- represent the upper and lower edge thresholds, respectively. The former penalizes the classifier's prediction of a class that does not actually exist, while the latter penalizes the classifier's prediction of a class that does not actually exist. m+ = 0.9, m- = 0.1; λ is the sparsity coefficient, used to adjust the weight, with a value of 0.5; Ltotal represents the total loss, I is the original input, and Or is the decoder's reconstructed output. The weighting factor has a value of 2 × 10. -5 ; Step 7: Accuracy assessment of the fine classification results of cultivated areas.

2. The method for classifying multi-crop cultivated areas by integrating attention mechanisms and capsule layers as described in claim 1, characterized in that, In step one, the principal component analysis dimensionality reduction includes: performing principal component analysis dimensionality reduction on the hyperspectral image to compress its band number to 1 / 4 of the original, and removing high redundancy and noise between hyperspectral image bands; the image segmentation includes: dividing the data after principal component analysis dimensionality reduction into image blocks with a length × width × number of channels of 11 × 11 × 64, which are used as input to the classification network.

3. The method for classifying multi-crop cultivated areas by integrating attention mechanisms and capsule layers as described in claim 1, characterized in that, In step two, the image patches are subjected to max pooling and mean pooling in the channel dimension, respectively. The pooling results are then concatenated in a stacked manner to form a concatenated feature of size 11×11×2. Convolution and activation operations are then performed sequentially on these features to generate spatial attention matrix weights with values ​​ranging from 0 to 1. The spatial attention matrix weights are then element-wise multiplied with the original image patches to complete the weighting operation. The spatial attention weighting process is expressed by the following formula: In the formula, F represents the input feature data, AvgPool(·) and MaxPool(·) represent average pooling and max pooling, respectively, Conv(·) represents the convolution operation with a kernel size of 7×7, σ(·) represents the Sigmoid activation function, and O s The spatial attention matrix weights are obtained; the convolution operations involved are represented as follows: In the formula, f(·) represents the activation output of the β-th channel in convolutional layer l; f(·) represents the activation function, which is Sigmoid. It is the feature to be activated in the β-th channel of convolutional layer l, derived from the feature map of the previous layer. With the corresponding convolution kernel Perform convolution summation operations and, by adjusting the bias... Obtained by addition; It is used to obtain The input feature map subset; "*" represents the convolution operator.

4. The method for classifying multi-crop cultivated areas by integrating attention mechanisms and capsule layers as described in claim 1, characterized in that, In step three, based on the output of step two, a convolution operation is first performed to generate 11×11×128 intrinsic features. Then, a depthwise convolution operation is performed on these features. The calculation process is as follows: In the formula, Y represents the intrinsic features after regular convolution, u is the input feature, and the convolution kernel is... ∈ m×k×k×c; In the formula, i,j Represents the i-th eigenfeature y of the eigenfeature set Y. i Perform the j-th linear operation to generate the corresponding j-th enhanced feature y. ij The last layer i,s This indicates that by performing an identity mapping on the intrinsic features, the final set of enhanced features obtained is {y}. 11 , y 12 , …, y ms The relationship between the number of intrinsic features m, the number of linear operations s, and the number of final enhanced feature channels n is described as n = m·sm ≤ n. Ignoring the last identity transformation, this relationship is further expressed as m(s-1) = n / s·(s-1).

5. The method for classifying multi-crop cultivated areas by integrating attention mechanisms and capsule layers as described in claim 1, characterized in that, In step four, channel attention operations are performed on the enhanced features output in step three. First, spatial compression is performed on the feature data, followed by average pooling on the spatial scale. Considering the information exchange between the current channel and its k neighboring channels, one-dimensional convolution is used to generate channel-dimensional attention weight values. These weights are then multiplied element-wise with the enhanced features to complete the channel-dimensional weighting operation. The calculation process is as follows: In the formula, y represents the input channel feature data, C1D(·) represents one-dimensional convolution, and σ(·) represents the Sigmoid activation function. The channel attention weights are obtained, where k represents the size of the one-dimensional convolution kernel, adaptively determined by the following formula: In the formula, C represents the number of channels of the input data, k represents the size of the calculated one-dimensional convolution kernel, odd indicates that the obtained value is odd, b=1, and γ=2.

6. The method for classifying multi-crop cultivated areas by integrating attention mechanisms and capsule layers as described in claim 1, characterized in that, In step five, the primary capsule uses convolution operations to convert features into capsules in vector form. It contains multiple parallel small convolution kernels with a kernel size of 4×4×8, totaling 32 kernels. Each convolution kernel extracts a set of features to form a capsule vector. The information transmission process of vector neurons in the capsule layer includes three parts: affine transformation, weighted summation, and nonlinear activation. Affine transformation encodes the abstract spatial relationship between low-level capsules and high-level features. The input low-level capsules are transformed through the weight matrix W. ij i=1, 2, 3, … multiply to obtain higher-level features; Weighted summation using coupling coefficient c ij For i=1,2,3,…, the high-level features are weighted and summed to obtain the input vector required for the high-level capsule. The calculation formula is as follows: In the formula, u i W represents the underlying capsule of the input. ij U is the weight matrix. j|i c represents a high-level feature. ij It is a coupling coefficient that has the properties of non-negativity and summing to 1. j This is the input vector required for the high-level capsule; Nonlinear activation utilizes the squash function (·) to transform the vector The magnitude of j is normalized to the interval (0, 1) while maintaining the vector direction, so that the magnitude of the final output high-level capsule vj can be used to characterize the probability of the existence of the object entity: In the formula, ||·|| represents the L2 norm, which is the magnitude of the vector, and ||·||2 represents the square of the L2 norm. The first part of the calculation formula is to compress the vector, and the second part is the direction vector. The whole formula achieves a normalization operation on the vector to be consistent with the original direction. Dynamic routing is used to update and determine the coupling coefficient c. ij The dynamic routing process includes: the input underlying capsule u i With weight matrix W i After multiplication, the high-level feature U is obtained. i Based on this, softmax normalization, vector weighted summation, vector squash (·), and coupling coefficient update operations are performed, where the process variable b is initialized to 0; through multiple iterations, the high-level capsule is finally obtained.

7. The method for classifying multi-crop cultivated areas by integrating attention mechanisms and capsule layers as described in claim 1, characterized in that, In step seven, the evaluation metrics include overall accuracy, average classification accuracy, and the Kappa coefficient κ. The classification results are compared with ground reference data to obtain the confusion matrix M, and the evaluation metrics are calculated using the following formulas: In the formula, n is the number of categories, N is the total number of test samples, and M is the total number of samples. uu M represents the number of samples in which class u is correctly identified as u. uv This represents the number of samples whose category v is identified as u, where u∈[1, n], v∈[1, n].

Images