Crop remote sensing hyperspectral image classification method, device, equipment and medium

Abstract

The present application relates to a method, apparatus, device, and medium for classifying crop remote sensing hyperspectral images. The method comprises: determining a first context information feature map and a second context information feature map in a context information fusion module; inputting a two-dimensional convolution feature map and a three-dimensional convolution feature map into a feature filter to determine a first filter feature map and a second filter feature map; fusing the first context information feature map with the first filter feature map, and the second context information feature map with the second filter feature map, to determine a first fused feature map and a second fused feature map; concatenating the first fused feature map and the second fused feature map to determine a comprehensive fused feature map, and inputting the comprehensive fused feature map into a multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image. The present application can effectively improve the accuracy and efficiency of crop remote sensing hyperspectral image classification.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for classifying crop remote sensing hyperspectral images, a corresponding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Compared to traditional optical, infrared, or synthetic aperture radar imagery, hyperspectral imagery (HSI) boasts high resolution and a large number of bands. Each pixel in a hyperspectral image contains dozens or even hundreds of spectral bands, meaning HSI can provide rich spatial and spectral information, helping to distinguish different land cover categories. This provides powerful support for environmental monitoring, Earth resource surveys, and agricultural and forestry monitoring. This has made HSI classification methods a research hotspot in recent years.

[0003] In recent years, thanks to the powerful spatial information extraction capabilities of convolutional neural networks (CNN), experts and scholars have applied CNN to HSI classification methods. For example, a deep belief network is used as a classifier and end-to-end learning and feature extraction are performed to achieve accurate classification. And the dual spatial information fusion method is used to improve the classification performance of hyperspectral images through multi-level feature learning and comprehensive utilization of multi-source resource information. These methods have achieved satisfactory classification results thanks to the powerful ability of CNN to extract local spatial information. However, HSI classification methods based solely on CNN also have some shortcomings: 1. Limited to the use of fixed-size convolution kernels, CNN methods are prone to fail to capture subtle differences between adjacent spectral bands. 2. The ability to obtain contextual information is relatively weak, and the global features in hyperspectral data are lost.

[0004] In addition, the Visual Transformer (ViT), as a new model, has performed well in the field of image processing. This is due to its key module, the self-attention technology, which can capture global information by encoding position. This provides a new idea for HSI classification, and the Transformer network has been applied by researchers to HSI classification. For example, Qing et al. introduced a spectral attention mechanism combined with the Transformer's multi-head self-attention mechanism to effectively capture continuous spectral relationships. Hong et al. designed a spectral Transformer classification method. By spectrally grouping hyperspectral images and connecting Transformer encoder modules across layers, the spectral representation of adjacent bands between groups is learned, thereby improving the utilization efficiency of spectral features. Although the Transformer performs well in capturing spectral features, it fails to capture local spatial-spectral fusion information.

[0005] To sum up, in view of the fact that the existing convolutional neural network model cannot combine spectral dimension features and local spatial feature information, lacks the ability to obtain contextual information, and the Transformer model has difficulty in capturing local spatial features and cannot integrate local spatial features and spectral dimension information, the applicant has made corresponding explorations to solve these problems. Summary of the Invention

[0006] The purpose of this application is to solve the above problems and provide a method for classifying crop remote sensing hyperspectral images, a corresponding device, an electronic device and a computer-readable storage medium.

[0007] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0008] A crop remote sensing hyperspectral image classification method proposed to meet one of the purposes of this application includes:

[0009] In response to a crop remote sensing hyperspectral image classification instruction, the crop remote sensing hyperspectral image from the drone's onboard camera is obtained, the improved SENet network is called, the global pooling is updated to a one-to-one one-dimensional feature encoding, and two pooling kernels are used to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the region of interest in the crop remote sensing hyperspectral image;

[0010] Inputting the region of interest into a crop remote sensing image classification model that has been trained to a convergent state, and performing parallel processing in a two-dimensional convolutional branch network and a three-dimensional convolutional branch network of a feature extraction module to determine a two-dimensional convolutional feature map and a three-dimensional convolutional feature map, wherein the crop remote sensing image classification model is constructed by a convolutional neural network and a Transformer model;

[0011] In the context information fusion module, the cross attention mechanism module in the Transformer model is used to extract context information features from the two-dimensional convolution feature map and the three-dimensional convolution feature map to determine a first context information feature map and a second context information feature map; the two-dimensional convolution feature map and the three-dimensional convolution feature map are respectively input into a feature filter to determine a first filtered feature map and a second filtered feature map;

[0012] Performing feature map fusion on the first context information feature map and the first filtering feature map, and on the second context information feature map and the second filtering feature map, respectively, to determine a first fused feature map and a second fused feature map;

[0013] The first fusion feature map and the second fusion feature map are spliced ​​to determine a comprehensive fusion feature map, and the comprehensive fusion feature map is input into a multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image, so as to complete the classification of the crop remote sensing hyperspectral image.

[0014] Optionally, calling the improved SENet network, updating the global pooling to a one-to-one one-dimensional feature encoding, and using two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the region of interest in the crop remote sensing hyperspectral image, includes:

[0015] The improved SENet network is called to update the global pooling to a one-to-one one-dimensional feature encoding, and two pooling kernels are used to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the horizontal aggregation features and the vertical aggregation features;

[0016] The horizontally aggregated features and the vertically aggregated features are spliced ​​to determine spliced ​​features, the spliced ​​features are subjected to 1x1 convolution, normalization, and ReLU activation function to extract channel feature weights, and the feature weights are applied to the crop remote sensing hyperspectral image to determine a region of interest in the crop remote sensing hyperspectral image.

[0017] Optionally, performing parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map includes:

[0018] The two-dimensional convolutional branch network includes at least one layer of two-dimensional convolution operation, which is used to extract the correlation and texture information of the input data in the spatial dimension, with a convolution kernel size of (3,3), a stride of (1,1), and a padding of (1,1);

[0019] The three-dimensional convolutional branch network includes at least one layer of three-dimensional convolution operation, which is used to capture the feature information of the input data in the spectral dimension. The convolution kernel size is (3,3,3), the stride is (1,1,1), and the padding method is (1,1,1).

[0020] Optionally, performing parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map includes:

[0021] In the two-dimensional convolution branch network and the three-dimensional convolution branch network, new two-dimensional convolution features and three-dimensional convolution features are respectively extracted through convolution operations;

[0022] The two are fused through a cross-mapping operation. In the two-dimensional convolution branch network, the new three-dimensional convolution feature is reduced in dimension through the Rearrange function and concatenated with the new two-dimensional convolution feature through the Concat function before further two-dimensional convolution operation to determine the first two-dimensional convolution feature map.

[0023] In the three-dimensional convolution branch network, the Squeeze function is used to expand the new two-dimensional convolution feature to five-dimensional data, and the three-dimensional convolution operation is continued after being concatenated with the new three-dimensional convolution feature through the Concat function to determine the first three-dimensional convolution feature;

[0024] The first two-dimensional convolution feature map and the first three-dimensional convolution feature are stacked twice respectively to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map.

[0025] Optionally, the step of using a cross attention mechanism module in the Transformer model to respectively extract context information features from the two-dimensional convolutional feature map and the three-dimensional convolutional feature map to determine a first context information feature map and a second context information feature map includes:

[0026] The two-dimensional convolutional feature map and the three-dimensional convolutional feature map are respectively input into the cross attention mechanism module, and the channels of the input feature maps are compressed through three 1x1 convolutional layers to obtain query feature maps, key feature maps, and value feature maps, where the dimensions of the feature maps are C×H×W, where C is the number of channels, and H and W are the height and width of the feature maps respectively;

[0027] Extract feature vectors at the same position from the query feature map and the key feature map, perform matrix multiplication on the u-th row in the query feature map and the u-th column in the key feature map to determine a first product result, normalize the first product result using a Sigmoid function to obtain an attention weight matrix, and multiply the attention weight matrix with the value feature map to generate a first context information feature map and a second context information feature map, respectively.

[0028] Optionally, the step of inputting the two-dimensional convolution feature map and the three-dimensional convolution feature map into feature filters respectively to determine a first filtered feature map and a second filtered feature map includes:

[0029] The two-dimensional convolution feature map and the three-dimensional convolution feature map are respectively input into the feature filter, processed by two 1x1 two-dimensional convolution layers, and the stride of each convolution layer is set to (1, 1), and two feature maps after convolution processing are obtained;

[0030] The concat operation is used to merge the two convolution-processed feature maps, and a weight map is generated by the Sigmoid function. The weight map is subjected to a matrix operation with the original feature map to generate the first filtered feature map and the second filtered feature map respectively.

[0031] Optionally, the crops include one or more of rice, wheat, corn, soybean, and sorghum.

[0032] A crop remote sensing hyperspectral image classification device provided for another purpose of the present application includes:

[0033] An image preprocessing module is configured to respond to a crop remote sensing hyperspectral image classification instruction, obtain a crop remote sensing hyperspectral image from a camera onboard a UAV, call an improved SENet network, update global pooling to a one-to-one one-dimensional feature encoding, and use two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine a region of interest in the crop remote sensing hyperspectral image;

[0034] a convolutional feature map determination module configured to input the region of interest into a crop remote sensing image classification model that has been trained to a convergence state, and perform parallel processing in a two-dimensional convolutional branch network and a three-dimensional convolutional branch network of a feature extraction module to determine a two-dimensional convolutional feature map and a three-dimensional convolutional feature map, wherein the crop remote sensing image classification model is constructed by a convolutional neural network and a Transformer model;

[0035] The context information fusion module is configured to, in the context information fusion module, use a cross attention mechanism module in the Transformer model to respectively extract context information features from the two-dimensional convolutional feature map and the three-dimensional convolutional feature map to determine a first context information feature map and a second context information feature map; and input the two-dimensional convolutional feature map and the three-dimensional convolutional feature map into a feature filter to determine a first filtered feature map and a second filtered feature map.

[0036] a feature map fusion module configured to perform feature map fusion on the first context information feature map and the first filtered feature map, and on the second context information feature map and the second filtered feature map, respectively, to determine a first fused feature map and a second fused feature map;

[0037] The crop category determination module is configured to perform information splicing on the first fusion feature map and the second fusion feature map to determine a comprehensive fusion feature map, and input the comprehensive fusion feature map into a multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image, so as to complete the classification of the crop remote sensing hyperspectral image.

[0038] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the crop remote sensing hyperspectral image classification method described in the present application.

[0039] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the crop remote sensing hyperspectral image classification method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0040] Compared with the existing technology, this application addresses the problems that the existing convolutional neural network model cannot combine spectral dimension features and local spatial feature information, lacks context information, and the Transformer model has difficulty capturing local spatial features and cannot integrate local spatial features and spectral dimension information. This application includes but is not limited to the following beneficial effects:

[0041] First, efficient feature extraction: Through the improved SENet network and parallel convolution and Transformer networks, spatial and spectral features in images can be better extracted.

[0042] Second, accurate positioning of regions of interest: The extraction of regions of interest can reduce interference from irrelevant areas and ensure more accurate classification.

[0043] Third, deep integration of contextual information: The cross-attention mechanism effectively combines contextual information of different dimensions, making the model more stable in complex environments.

[0044] Fourth, highly robust classification capability: The fused feature maps can better adapt to different types and growth states of crops, thereby improving the robustness of the classification model under different conditions.

[0045] Furthermore, the proposed crop remote sensing hyperspectral image classification method achieves excellent results in HSI classification. This is primarily due to its combination of a two-dimensional convolutional neural network and a three-dimensional convolutional neural network to extract local spatial and spectral information. After convolution, cross-mapping fusion is performed to impart both spatial and spectral information to the feature map. Cross-attention captures dependencies between distant pixels, and a cross-residual connection is proposed to exchange contextual information, enriching the spectral features of the two-dimensional convolutional feature map and complementing the spatial information of the three-dimensional convolutional feature map.

[0046] Furthermore, the crop remote sensing hyperspectral image classification method of this application is based on a classification method that integrates an improved SENet network, a convolutional neural network, a Transformer model and an attention mechanism, which can effectively improve the accuracy and efficiency of crop remote sensing hyperspectral image classification and is suitable for remote sensing image analysis tasks in actual agricultural monitoring and refined agricultural management. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0048] Figure 1 Schematic diagram of the process of crop remote sensing hyperspectral image classification method in the embodiment of the present application;

[0049] Figure 2 This is an exemplary network architecture of the crop remote sensing hyperspectral image classification system in the embodiments of this application;

[0050] Figure 3 This is a principle block diagram of the crop remote sensing hyperspectral image classification device in an embodiment of the present application;

[0051] Figure 4 Schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0052] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0053] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0054] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0055] It will be understood by those skilled in the art that the terms "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, i.e., devices that only have wireless signal receivers without transmission capabilities, and devices that have receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers and tablet computers, which have single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / Intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; and conventional laptop and / or palmtop computers or other devices, which have and / or include a radio frequency receiver. As used herein, the terms "client," "terminal," or "terminal device" may be portable, transportable, or installed in a vehicle (air, sea, and / or land), or may be adapted and / or configured to operate locally and / or in a distributed manner at any other location on Earth and / or in space. As used herein, the terms "client," "terminal," or "terminal device" may also refer to a communication terminal, an Internet terminal, or a music / video playback terminal, such as a PDA, an MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or may include a smart TV, a set-top box, or other device.

[0056] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. Computer programs are stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0057] It should be noted that the concept of "server" referred to in this application can also be extended to server clusters. Based on the network deployment principles understood by those skilled in the art, the servers described should be logically divided. In physical space, these servers can be independent of each other but callable through interfaces, or integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method of this application.

[0058] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for implementation.

[0059] Unless expressly specified otherwise, the neural network models referenced or may be referenced in this application may be deployed on a remote server and remotely called on the client, or may be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence may be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0060] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as they are suitable for being called by the technical solution of this application.

[0061] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus exhibit commonality, unless otherwise specified, these methods can be independently executed. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept. Therefore, concepts with the same expression, as well as concepts that are appropriately transformed for convenience despite different expression, should be understood as equivalent.

[0062] Unless expressly stated to be mutually exclusive, the various embodiments disclosed in this application may be cross-combined with the relevant technical features of the various embodiments to flexibly construct new embodiments, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or resolve certain deficiencies in the prior art. Those skilled in the art should be aware of such flexibility.

[0063] See also Figure 1 In one embodiment, the crop remote sensing hyperspectral image classification method of the present application includes:

[0064] Step S10: responding to the crop remote sensing hyperspectral image classification instruction, obtaining the crop remote sensing hyperspectral image from the drone's onboard camera, calling the improved SENet network, updating the global pooling to a one-to-one-dimensional feature encoding, and using two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the region of interest in the crop remote sensing hyperspectral image;

[0065] The crop remote sensing hyperspectral image classification system can respond to crop remote sensing hyperspectral image classification instructions, obtain crop remote sensing hyperspectral images from a drone-mounted camera, call an improved SENet network, update global pooling to a one-to-one one-dimensional feature encoding, and use two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the area of ​​interest in the crop remote sensing hyperspectral image; wherein the crops include rice, wheat, corn, soybeans, sorghum, etc.

[0066] In some embodiments, see Figure 2 , Figure 2 This is an exemplary network architecture of the crop remote sensing hyperspectral image classification system in the embodiment of this application, where CA represents the preprocessing Coordinate Attention mechanism to enhance feature expression, Mapping represents feature mapping to facilitate the fusion of two-dimensional convolution kernel and three-dimensional convolution features, AttentionGate is a feature filter, CCAtention is cross attention to obtain context information, and Class Prediction is an MLP layer to map the feature data dimensions to the classification category of agricultural remote sensing images.

[0067] In some embodiments, the steps of calling an improved SENet network, updating global pooling to a one-to-one one-dimensional feature encoding, and using two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the region of interest in the crop remote sensing hyperspectral image include:

[0068] Step S101: calling the improved SENet network, updating the global pooling to a one-to-one-dimensional feature encoding, and using two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the horizontal aggregation feature and the vertical aggregation feature;

[0069] Step S102: splicing the horizontally aggregated features and the vertically aggregated features to determine spliced ​​features, performing 1x1 convolution, normalization, and ReLU activation function on the spliced ​​features to extract channel feature weights, and applying the feature weights to the crop remote sensing hyperspectral image to determine a region of interest in the crop remote sensing hyperspectral image.

[0070] Specifically, the original agricultural remote sensing hyperspectral image data is similar to a set of two-dimensional data, and different crop categories have corresponding digital labels. The crop remote sensing image classification model first obtains the original hyperspectral image data, and overlays the surrounding image data with non-zero pixels as the center as a new image, thereby dividing the original image into many images as training samples.

[0071] Calling the improved SENet network to process the above-mentioned agricultural remote sensing hyperspectral image can improve the model's attention to the features of different channels. The improved SENet network is an improvement on Squeeze-and-Excitation Networks (SENet). SENet uses global pooling to only focus on the relationship between channels and ignores position information. The improved SENet network of this application fills this gap, replacing global pooling with a one-dimensional feature encoding. Two pooling kernels AdaptiveAvgPool2d(H, 1) and AdaptiveAvgPool2d(1, W) are used to encode each channel in the horizontal and vertical directions respectively, thereby obtaining aggregated features in two spatial directions. After concatenation, two-dimensional convolution (convolution kernel is 1x1), normalization and Rule activation function are performed to obtain feature weights. These weights are then applied to the input image to achieve the effect of enhancing the features of the input image. The preprocessing module takes into account spatial position information. The calculations in the horizontal and vertical directions can more accurately determine the position of the object of interest. In addition, the channels are modeled and the correlation between channels is learned, which can improve the feature discrimination ability. After a series of operations, it is passed to the feature extraction module of the crop remote sensing image classification model.

[0072] Step S20: inputting the region of interest into a crop remote sensing image classification model that has been trained to a convergent state, and performing parallel processing in a two-dimensional convolutional branch network and a three-dimensional convolutional branch network of a feature extraction module to determine a two-dimensional convolutional feature map and a three-dimensional convolutional feature map, wherein the crop remote sensing image classification model is constructed by a convolutional neural network and a Transformer model;

[0073] After determining a region of interest in the crop remote sensing hyperspectral image, the region of interest is input into a crop remote sensing image classification model that has been trained to a convergence state, and parallel processing is performed in a two-dimensional convolutional branch network and a three-dimensional convolutional branch network of a feature extraction module to determine a two-dimensional convolutional feature map and a three-dimensional convolutional feature map, wherein the crop remote sensing image classification model is constructed by a convolutional neural network (CNN) and a Transformer model;

[0074] In some embodiments, the steps of performing parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map include:

[0075] The two-dimensional convolution branch network includes at least one layer of two-dimensional convolution operation, which is used to extract the correlation and texture information of the input data in the spatial dimension. The convolution kernel size is (3,3), the stride is (1,1), and the padding method is (1,1); the three-dimensional convolution branch network includes at least one layer of three-dimensional convolution operation, which is used to capture the feature information of the input data in the spectral dimension. The convolution kernel size is (3,3,3), the stride is (1,1,1), and the padding method is (1,1,1).

[0076] In a further embodiment, the step of performing parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map includes:

[0077] Step S201: extracting new two-dimensional convolution features and three-dimensional convolution features respectively in the two-dimensional convolution branch network and the three-dimensional convolution branch network through convolution operations;

[0078] Step S202: The two are fused through a cross-mapping operation. In the two-dimensional convolution branch network, the new three-dimensional convolution feature is reduced in dimension using the Rearrange function, and is concatenated with the new two-dimensional convolution feature through the Concat function, followed by a further two-dimensional convolution operation to determine a first two-dimensional convolution feature map.

[0079] Step S203: In the three-dimensional convolution branch network, the new two-dimensional convolution feature is expanded to five-dimensional data using the Squeeze function, and the three-dimensional convolution operation is continued after being concatenated with the new three-dimensional convolution feature using the Concat function to determine the first three-dimensional convolution feature.

[0080] Step S204: stack the first two-dimensional convolution feature map and the first three-dimensional convolution feature map twice to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map.

[0081] Specifically, the feature extraction module of the crop remote sensing image classification model of the present application is composed of a mixture of two-dimensional convolution and three-dimensional convolution, and consists of two parallel branches: a two-dimensional convolution branch and a three-dimensional convolution branch. The two-dimensional convolution extracts the spatial correlation and texture information between bands, and the three-dimensional convolution captures the spectral feature information between bands. The preprocessed data is five-dimensional data, including the training batch, the number of channels, and the length, height, and width of the spectral image.

[0082] The two-dimensional convolution branch network requires that the input data only contain four dimensions, and uses the Rearrange function to merge the number of channels and the height of the image. The preprocessed data is input into the two-dimensional convolution branch and the three-dimensional convolution branch for convolution. The convolution kernel of the two-dimensional convolution is (3, 3), the stride is (1, 1), and the padding is (1, 1); while the convolution kernel of the three-dimensional convolution branch network is (3, 3, 3), the stride is (1, 1, 1), and the padding is (1, 1, 1), obtaining new two-dimensional convolution features and three-dimensional convolution features.

[0083] Then, we do cross mapping. In the three-dimensional convolution branch, we use the squeeze function to expand the new two-dimensional convolution feature by one dimension to become five-dimensional data. Then, we concatenate it with the new three-dimensional convolution feature through the Concat function and continue to perform three-dimensional convolution. At the same time, in the two-dimensional convolution branch, we use the Rearrange function to reduce the dimension of the new three-dimensional convolution feature and concatenate it with the new two-dimensional convolution feature through the Concat function to further perform two-dimensional convolution to extract features.

[0084] The feature extraction module is stacked twice. The output of the first step serves as the input for the second step. The second step uses an additional data normalization layer, BatchNorm, and a maximum pooling layer, MaxPool. The two-dimensional convolution branch uses MaxPool2d(2, 2), while the three-dimensional convolution branch uses Maxpool3d(2, 2, 2). This downsamples the feature maps, reducing the spatial dimensionality and the number of model parameters. The output data of both branches serves as the input for the next module.

[0085] Step S30: In the context information fusion module, the cross attention mechanism module in the Transformer model is used to extract context information features from the two-dimensional convolutional feature map and the three-dimensional convolutional feature map to determine a first context information feature map and a second context information feature map; the two-dimensional convolutional feature map and the three-dimensional convolutional feature map are respectively input into feature filters to determine a first filtered feature map and a second filtered feature map;

[0086] Step S40: performing feature graph fusion on the first context information feature graph and the first filtering feature graph, and on the second context information feature graph and the second filtering feature graph, respectively, to determine a first fused feature graph and a second fused feature graph;

[0087] After performing parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine a two-dimensional convolution feature map and a three-dimensional convolution feature map, in the context information fusion module, the cross attention mechanism module in the Transformer model is used to extract context information features in the two-dimensional convolution feature map and the three-dimensional convolution feature map respectively to determine a first context information feature map and a second context information feature map; the two-dimensional convolution feature map and the three-dimensional convolution feature map are respectively input into the feature filter to determine a first filtered feature map and a second filtered feature map; the first context information feature map is respectively fused with the first filtered feature map, and the second context information feature map is fused with the second filtered feature map to determine a first fused feature map and a second fused feature map;

[0088] In some embodiments, the step of using the cross attention mechanism module in the Transformer model to extract context information features from the two-dimensional convolutional feature map and the three-dimensional convolutional feature map respectively to determine the first context information feature map and the second context information feature map includes:

[0089] Step S301: Input the two-dimensional convolutional feature map and the three-dimensional convolutional feature map into the cross attention mechanism module respectively, and compress the channels of the input feature map through three 1x1 convolutional layers to obtain a query feature map, a key feature map, and a value feature map, wherein the dimension of the feature map is C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively;

[0090] Step S302: extract the feature vectors at the same position from the query feature map and the key feature map, take the u-th row in the query feature map and the u-th column in the key feature map to perform matrix multiplication operation to determine a first product result, use the Sigmoid function to normalize the first product result to obtain an attention weight matrix, multiply the attention weight matrix with the value feature map to generate a first context information feature map and a second context information feature map, respectively.

[0091] In some embodiments, the step of inputting the two-dimensional convolution feature map and the three-dimensional convolution feature map into feature filters respectively to determine a first filtered feature map and a second filtered feature map includes:

[0092] The two-dimensional convolution feature map and the three-dimensional convolution feature map are respectively input into the feature filter, processed by two 1x1 two-dimensional convolution layers, and the stride of each convolution layer is set to (1, 1), and two feature maps after convolution processing are obtained;

[0093] The concat operation is used to merge the two convolution-processed feature maps, and a weight map is generated by the Sigmoid function. The weight map is subjected to a matrix operation with the original feature map to generate the first filtered feature map and the second filtered feature map respectively.

[0094] Specifically, the context information fusion module receives input data from both the 2D and 3D convolution branches, and continues the process across these two branches. To facilitate feature fusion, the 3D convolution branch's feature data undergoes a dimensionality transformation using the Rearrange function, converting five-dimensional data into four-dimensional data. Below, the 2D convolution branch is used as an example. The 3D convolution branch performs the same operation, but the 2D convolution branch is divided into two steps: one for acquiring context information and the other for feeding it into the feature filter.

[0095] To obtain context information: first obtain the output data of the feature extraction module in the previous step, and pass the data into the cross attention to obtain context information. It mainly uses 3 1x1 convolution compression channels to obtain the query (Q) feature map, key (K) feature map and value (V) feature map. The data dimension is CxHxW. Take the same position u in the query (Q) feature map and the key (K) feature map, take the u row of the query (Q) feature map, and take the u column of the key (K) feature map. After matrix multiplication, use the Sigmoid function for normalization to obtain the attention weight, which is multiplied by the value (V) feature map to obtain the context feature.

[0096] For the input feature filter, two 2D convolutions are performed with a kernel of (1, 1) and a stride of (1, 1) to obtain two feature maps. These are then concatenated using the Concat function and weighted using the Sigmoid function. A matrix operation is then performed on the original image to obtain the filtered feature map. Finally, the context feature map and the filtered feature map are concatenated using the Concat function.

[0097] Finally, the information of the two branches of the Concat function is concatenated and input into the MLP layer (multi-layer perceptron layer) for mapping. The final one-dimensional data is mapped from 256 dimensions into the categories in the hyperspectral image, completing the classification of agricultural remote sensing images.

[0098] Step S50: splicing the first fusion feature map and the second fusion feature map to determine a comprehensive fusion feature map, inputting the comprehensive fusion feature map into a multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image, so as to complete the classification of the crop remote sensing hyperspectral image.

[0099] After determining the first fusion feature map and the second fusion feature map, the first fusion feature map and the second fusion feature map are spliced ​​to determine a comprehensive fusion feature map, and the comprehensive fusion feature map is input into the multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image, so as to complete the classification of the crop remote sensing hyperspectral image.

[0100] In the crop remote sensing image classification model, the feature extraction module is composed of a mixture of two-dimensional convolution and three-dimensional convolution, which consists of two branches in parallel: a two-dimensional convolution branch network and a three-dimensional convolution branch network. The dimension of the input image X is CxDxHxW, where C represents the number of channels, D represents the spectral dimension, and H and W represent the spatial dimensions. is the output of the 2D convolution, is the output of the 3D convolution,

[0101]

[0102] The feature map obtained by three-dimensional convolution is integrated into the two-dimensional feature y2, and the feature map obtained by two-dimensional convolution is transformed into the three-dimensional feature y3. We have

[0103] φ i =Concat[T θj→θi ], i∈(2,3)

[0104] y i =O i +φ j , (i=2, j=3||i=3, j=2),

[0105] T is the dimension transformation of data to facilitate feature fusion.

[0106] When i=2, θ i =2 represents the feature map of two-dimensional convolution, and the T operation converts the three-dimensional convolution feature θ j＝3 Perform data dimensionality reduction and integrate it into two-dimensional convolution features, where the dimension transformation is: C×D×H×W→(C×D)×H×W).

[0107] When i=3, θ i =3 represents the feature map of three-dimensional convolution, and the T operation converts the two-dimensional convolution feature θ j＝2 The dimension is upgraded and integrated into the three-dimensional convolution feature, where the dimension is converted to: C×H×W→C×1×H×W.

[0108] The final two-dimensional convolution result y2 can contain both two-dimensional spatial information and spectral dimension information. Similarly, the three-dimensional convolution feature map y3 expands the two-dimensional spatial information on the original basic spectral dimension information. Feature fusion allows the two-dimensional convolution and three-dimensional convolution kernels to influence each other. The main manifestation is: the feature extraction module is stacked twice, and the two-dimensional convolution features and three-dimensional convolution features obtained from the first stacking will be used as the input for the second time. The fused features can promote the learning and optimization of the convolution kernel in the secondary convolution, reducing the complexity of learning the three-dimensional convolution kernel. The formula is:

[0109]

[0110] E i =F i +φ, (i=2, j=3|||i=3, j=2),

[0111] Among them, y i The feature maps are obtained for the first feature extraction module, namely the two-dimensional convolution feature map and the three-dimensional convolution feature map. Indicates normalization processing (BatchNorn). We add normalization and maximum pooling to reduce the network calculation complexity and obtain F i Perform dimension transformation to obtain the new φ i To further integrate the spatial and spectral information, we designed a feature extraction module to capture the local spatial and spectral dimension information of the hyperspectral image; to fully integrate the spatial and spectral information.

[0112] For the context information extraction module, the cross attention mechanism is used instead of the self-attention mechanism to optimize the computational complexity of the model. The query (Q) feature map, key (K) feature map, and value (V) feature map are obtained by 1x1 convolution. The affinity between pixels is calculated as a metric, which can be expressed as follows: taking the row at position u of the query (Q) feature map as the feature, the feature set in the same column can be obtained at position u in the key (K) feature map. The affinity of point u to the same row and column is calculated as follows:

[0113]

[0114] Among them, d u Represents Q u and Ω u The correlation degree between them is calculated by performing the same operation on each pixel in Q to obtain the feature map D with the dimension of (H+W-1)x(H+W). Then, a softmax layer is used on the spectral dimension of D to obtain the attention map A with the dimension of (H+W-1)x(H+W). The feature map V is multiplied by the attention map A to obtain the context information. The formula is as follows:

[0115]

[0116] Among them, the feature filter is used to perform weight filtering on the input features and exchange the context information of the two branches. The filter mainly consists of two 1x1 two-dimensional convolutions. The specific formula is as follows:

[0117] M i =E i Sigmod(Conv(Ei)+Conv(E i )), i∈(2,3)

[0118]

[0119] Among them, M i＝2 and M i＝3 These refer to feature maps obtained by weighting the 2D convolutional feature map and the 3D convolutional feature map, respectively. A 2D convolution is performed on the input features, followed by Sigmod to obtain attention weights, which are then applied to the contextual features. The features in the 2D and 3D convolutional feature maps are not equally useful. Passing AttentionGate reduces information redundancy and strengthens feature expression. Finally, the two feature maps are concatenated using a fully connected layer and SoftMax to obtain the classification result. In short, the contextual information extraction module obtains contextual information from the feature map, enriching the spatial and spectral information of the two branches.

[0120] As can be seen from the above embodiments, compared with the prior art, the present application addresses the problems that the existing convolutional neural network model cannot combine spectral dimension features and local spatial feature information, lacks context information, and the Transformer model has difficulty in capturing local spatial features and cannot integrate local spatial features and spectral dimension information. The present application includes but is not limited to the following beneficial effects:

[0121] First, efficient feature extraction: Through the improved SENet network and parallel convolution and Transformer networks, spatial and spectral features in images can be better extracted.

[0122] Second, accurate positioning of regions of interest: The extraction of regions of interest can reduce interference from irrelevant areas and ensure more accurate classification.

[0123] Third, deep integration of contextual information: The cross-attention mechanism effectively combines contextual information of different dimensions, making the model more stable in complex environments.

[0124] Fourth, highly robust classification capability: The fused feature maps can better adapt to different types and growth states of crops, thereby improving the robustness of the classification model under different conditions.

[0125] Furthermore, the proposed crop remote sensing hyperspectral image classification method achieves excellent results in HSI classification. This is primarily due to its combination of a two-dimensional convolutional neural network and a three-dimensional convolutional neural network to extract local spatial and spectral information. After convolution, cross-mapping fusion is performed to impart both spatial and spectral information to the feature map. Cross-attention captures dependencies between distant pixels, and a cross-residual connection is proposed to exchange contextual information, enriching the spectral features of the two-dimensional convolutional feature map and complementing the spatial information of the three-dimensional convolutional feature map.

[0126] Furthermore, the crop remote sensing hyperspectral image classification method of this application is based on a classification method that integrates an improved SENet network, a convolutional neural network, a Transformer model and an attention mechanism, which can effectively improve the accuracy and efficiency of crop remote sensing hyperspectral image classification and is suitable for remote sensing image analysis tasks in actual agricultural monitoring and refined agricultural management.

[0127] See also Figure 3A crop remote sensing hyperspectral image classification device provided to meet one of the purposes of this application includes an image preprocessing module 1100, a convolution feature map determination module 1200, a context information fusion module 1300, a feature map fusion module 1400 and a crop category determination module 1500. Among them, the image preprocessing module 1100 is configured to respond to the crop remote sensing hyperspectral image classification instruction, obtain the crop remote sensing hyperspectral image in the drone's airborne camera, call the improved SENet network, update the global pooling to a one-to-one-dimensional feature encoding, and use two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine the region of interest in the crop remote sensing hyperspectral image; the convolution feature map determination module 1200 is configured to input the region of interest into the crop remote sensing image classification model that has been trained to a convergence state, and perform parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine a two-dimensional convolution feature map and a three-dimensional convolution feature map, wherein the crop remote sensing image classification model is constructed by a convolutional neural network and a Transformer model; the context information fusion module 1300 is configured to use the Transformer model in the context information fusion module. The cross attention mechanism module in the invention extracts the context information features in the two-dimensional convolution feature map and the three-dimensional convolution feature map respectively to determine the first context information feature map and the second context information feature map; the two-dimensional convolution feature map and the three-dimensional convolution feature map are respectively input into the feature filter to determine the first filter feature map and the second filter feature map; the feature map fusion module 1400 is configured to perform feature map fusion on the first context information feature map and the first filter feature map, and on the second context information feature map and the second filter feature map respectively to determine the first fusion feature map and the second fusion feature map; the crop category determination module 1500 is configured to perform information splicing on the first fusion feature map and the second fusion feature map to determine the comprehensive fusion feature map, and input the comprehensive fusion feature map into the multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image to complete the classification of the crop remote sensing hyperspectral image.

[0128] Based on any embodiment of this application, please refer to Figure 4 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 4As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a method for classifying hyperspectral images of remote sensing crops. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the method for classifying hyperspectral images of remote sensing crops of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0129] In this embodiment, the processor is used to execute Figure 3 The memory stores the program code and various data required to execute the specific functions of each module and its submodule in the device. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the crop remote sensing hyperspectral image classification device of this application. The server can call the server's program code and data to execute the functions of all submodules.

[0130] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the crop remote sensing hyperspectral image classification method described in any embodiment of the present application.

[0131] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the crop remote sensing hyperspectral image classification method described in any embodiment of the present application.

[0132] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0133] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A crop remote sensing hyperspectral image classification method, characterized in that: include: In response to a crop remote sensing hyperspectral image classification instruction, a crop remote sensing hyperspectral image from a camera onboard a UAV is obtained, an improved SENet network is called, global pooling is updated to a one-to-one one-dimensional feature encoding, two pooling kernels are used to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions respectively to determine horizontal aggregation features and vertical aggregation features; the horizontal aggregation features and the vertical aggregation features are spliced ​​to determine spliced ​​features, the spliced ​​features are subjected to 1x1 convolution, normalization, and ReLU activation function to extract channel feature weights, and the feature weights are applied to the crop remote sensing hyperspectral image to determine a region of interest in the crop remote sensing hyperspectral image; Inputting the region of interest into a crop remote sensing image classification model that has been trained to a convergent state, and performing parallel processing in a two-dimensional convolutional branch network and a three-dimensional convolutional branch network of a feature extraction module to determine a two-dimensional convolutional feature map and a three-dimensional convolutional feature map, wherein the crop remote sensing image classification model is constructed by a convolutional neural network and a Transformer model; In the context information fusion module, the cross attention mechanism module in the Transformer model is used to extract context information features from the two-dimensional convolution feature map and the three-dimensional convolution feature map to determine a first context information feature map and a second context information feature map; the two-dimensional convolution feature map and the three-dimensional convolution feature map are respectively input into a feature filter to determine a first filtered feature map and a second filtered feature map; Performing feature map fusion on the first context information feature map and the first filtering feature map, and on the second context information feature map and the second filtering feature map, respectively, to determine a first fused feature map and a second fused feature map; The first fusion feature map and the second fusion feature map are spliced ​​to determine a comprehensive fusion feature map, and the comprehensive fusion feature map is input into a multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image, so as to complete the classification of the crop remote sensing hyperspectral image.

2. The crop remote sensing hyperspectral image classification method according to claim 1, characterized in that: The steps of performing parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map include: The two-dimensional convolutional branch network includes at least one layer of two-dimensional convolution operation, which is used to extract the correlation and texture information of the input data in the spatial dimension. The convolution kernel size is (3, 3), the stride is (1, 1), and the padding is (1, 1). The three-dimensional convolutional branch network includes at least one layer of three-dimensional convolution operation, which is used to capture the feature information of the input data in the spectral dimension. The convolution kernel size is (3, 3, 3), the stride is (1, 1, 1), and the padding is (1, 1, 1).

3. The crop remote sensing hyperspectral image classification method according to claim 2, characterized in that: The steps of performing parallel processing in the two-dimensional convolution branch network and the three-dimensional convolution branch network of the feature extraction module to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map include: In the two-dimensional convolution branch network and the three-dimensional convolution branch network, new two-dimensional convolution features and three-dimensional convolution features are respectively extracted through convolution operations; The two are fused through a cross-mapping operation. In the two-dimensional convolution branch network, the new three-dimensional convolution feature is reduced in dimension through the Rearrange function and concatenated with the new two-dimensional convolution feature through the Concat function before further two-dimensional convolution operation to determine the first two-dimensional convolution feature map. In the three-dimensional convolution branch network, the Squeeze function is used to expand the new two-dimensional convolution feature to five-dimensional data, and the three-dimensional convolution operation is continued after being concatenated with the new three-dimensional convolution feature through the Concat function to determine the first three-dimensional convolution feature; The first two-dimensional convolution feature map and the first three-dimensional convolution feature are stacked twice respectively to determine the two-dimensional convolution feature map and the three-dimensional convolution feature map.

4. The crop remote sensing hyperspectral image classification method according to claim 1, characterized in that: The step of respectively extracting context information features from the two-dimensional convolutional feature map and the three-dimensional convolutional feature map using a cross attention mechanism module in the Transformer model to determine a first context information feature map and a second context information feature map includes: Input the two-dimensional convolutional feature map and the three-dimensional convolutional feature map into the cross attention mechanism module respectively, and compress the channels of the input feature map through three 1x1 convolutional layers to obtain the query feature map, key feature map and value feature map, where the dimension of the feature map is C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively; Extract the feature vectors at the same position from the query feature map and the key feature map, perform matrix multiplication on the u-th row in the query feature map and the u-th column in the key feature map to determine a first product result, normalize the first product result using a Sigmoid function to obtain an attention weight matrix, and multiply the attention weight matrix with the value feature map to generate a first context information feature map and a second context information feature map, respectively.

5. The crop remote sensing hyperspectral image classification method according to claim 1, characterized in that: Inputting the two-dimensional convolution feature map and the three-dimensional convolution feature map into feature filters respectively to determine a first filtered feature map and a second filtered feature map, comprising: Input the two-dimensional convolution feature map and the three-dimensional convolution feature map into the feature filter respectively, Processing is performed through two 1x1 two-dimensional convolutional layers, with the stride of each convolutional layer set to (1, 1), and two convolutional feature maps are obtained; The two convolution-processed feature maps are merged using a Concat operation, and a weight map is generated using a Sigmoid function. A matrix operation is performed on the weight map and the original feature map to generate the first filtered feature map and the second filtered feature map, respectively.

6. The crop remote sensing hyperspectral image classification method according to any one of claims 1 to 5, characterized in that: The crops include one or more of rice, wheat, corn, soybean, and sorghum.

7. A crop remote sensing hyperspectral image classification device, characterized in that: include: An image preprocessing module is configured to respond to a crop remote sensing hyperspectral image classification instruction, obtain a crop remote sensing hyperspectral image from a camera onboard a drone, call an improved SENet network, update global pooling to a one-to-one one-dimensional feature encoding, and use two pooling kernels to encode the crop remote sensing hyperspectral image in the horizontal and vertical directions, respectively, to determine horizontal aggregation features and vertical aggregation features; splicing the horizontal aggregation features and the vertical aggregation features to determine spliced ​​features, performing 1x1 convolution, normalization, and a ReLU activation function on the spliced ​​features to extract channel feature weights, and applying the feature weights to the crop remote sensing hyperspectral image to determine a region of interest in the crop remote sensing hyperspectral image; a convolutional feature map determination module configured to input the region of interest into a crop remote sensing image classification model that has been trained to a convergence state, and perform parallel processing in a two-dimensional convolutional branch network and a three-dimensional convolutional branch network of a feature extraction module to determine a two-dimensional convolutional feature map and a three-dimensional convolutional feature map, wherein the crop remote sensing image classification model is constructed by a convolutional neural network and a Transformer model; The context information fusion module is configured to, in the context information fusion module, use a cross attention mechanism module in the Transformer model to respectively extract context information features from the two-dimensional convolutional feature map and the three-dimensional convolutional feature map to determine a first context information feature map and a second context information feature map; and input the two-dimensional convolutional feature map and the three-dimensional convolutional feature map into a feature filter to determine a first filtered feature map and a second filtered feature map. a feature map fusion module configured to perform feature map fusion on the first context information feature map and the first filtered feature map, and on the second context information feature map and the second filtered feature map, respectively, to determine a first fused feature map and a second fused feature map; The crop category determination module is configured to perform information splicing on the first fusion feature map and the second fusion feature map to determine a comprehensive fusion feature map, and input the comprehensive fusion feature map into a multi-layer perceptron layer for mapping to determine the category of crops in the crop remote sensing hyperspectral image, so as to complete the classification of the crop remote sensing hyperspectral image.

8. An electronic device comprising a central processing unit and a memory, characterized in that: The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that It stores a computer program implemented according to the method described in any one of claims 1 to 6 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images