Remote sensing image classification method and system and storage medium

Abstract

The invention provides a remote sensing image classification method and system and a storage medium, and relates to the technical field of remote sensing image classification. Preprocessing images in the remote sensing image set; extracting multilayer features of the preprocessed remote sensing image through a pre-trained CNN model; performing dimension reduction processing on the extracted multi-layer features by applying PCA, extracting non-overlapping image blocks from the features after dimension reduction, performing spatial feature extraction on the extracted image blocks by applying 3D-CNN flow, and performing spectrum feature extraction on the extracted image blocks by applying CV-CNN flow; carrying out series fusion on the spatial features and the spectrum features; the fused features are enhanced through an SE attention mechanism; and inputting the enhanced features into an SVM (Support Vector Machine) for classification. The method integrates the advantages of multiple technologies, accurately extracts features, optimizes calculation, is high in adaptability, and effectively improves the classification precision and application value of the remote sensing image.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image classification, and in particular to a remote sensing image classification method, system and storage medium. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images have become an important data source for geographic information acquisition and analysis, playing a vital role in many fields such as land use planning, environmental monitoring, and disaster warning. However, remote sensing image classification faces many challenges. Its image data is highly complex, the types of objects are rich and diverse, and there are high requirements for classification accuracy. These factors make remote sensing image classification a challenging task.

[0003] Traditional remote sensing image classification methods mainly rely on manually designed features and classifiers. Due to the complexity of remote sensing image data and the diversity of land object types, manually designed features often fail to fully capture the key information in the image, resulting in limited classification accuracy. This method has significant limitations in feature extraction and classifier design, and is difficult to meet the needs of high-precision classification.

[0004] In recent years, the rise of deep learning technology has brought new solutions to remote sensing image classification. In particular, the successful application of convolutional neural networks (CNN) in the field of image recognition has provided strong technical support for remote sensing image classification.

[0005] However, existing remote sensing image classification methods still have problems under the deep learning framework. A single CNN model often has difficulty in effectively capturing both the spatial structure and spectral characteristics of an image at the same time, which limits its performance in remote sensing image classification. In addition, although some complex deep learning models may improve classification accuracy, they usually require expensive computing resources and time, which is impractical in practical applications, especially in remote sensing image classification scenarios that require fast response and real-time processing. At the same time, existing methods lack effective feature fusion strategies and attention mechanisms when dealing with complex content, which makes it difficult for the model to accurately identify key information in the image, thus affecting classification performance.

[0006] To sum up, the shortcomings of the existing technology in remote sensing image classification provide room for invention and innovation. The present invention is committed to solving these problems to improve the accuracy, efficiency and practicality of remote sensing image classification and meet the urgent needs of related fields for high-quality remote sensing image classification technology. Summary of the invention

[0007] To this end, the embodiments of the present invention provide a remote sensing image classification method, system and storage medium, which are used to solve the problems in the prior art that the traditional remote sensing image classification method relies on manually designed features and classifiers, is difficult to fully capture key information, has limited accuracy and has great limitations; under the deep learning framework, a single CNN model is difficult to capture both spatial structure and spectral characteristics at the same time, complex models require expensive computing resources and time, and are impractical in practical applications; and existing methods lack effective feature fusion strategies and attention mechanisms, which affects classification performance.

[0008] In order to solve the above problem, an embodiment of the present invention provides a remote sensing image classification method, which includes: S1: Establish remote sensing image set; S2: Preprocess the images in the remote sensing image set, including cropping and normalization, data enhancement, and data set division; S3: Extract multi-layer features of pre-processed remote sensing images through pre-trained CNN model; S4: Apply principal component analysis to reduce the dimensionality of the extracted multi-layer features, extract non-overlapping image blocks from the reduced dimensionality features, apply 3D-CNN flow to extract spatial features from the extracted image blocks, and apply CV-CNN flow to extract spectral features from the extracted image blocks; S5: tandem fusion of spatial features and spectral features; S6: Enhance the fused features through the SE attention mechanism; S7: Input the enhanced features into the support vector machine for classification.

[0009] Preferably, in step S3, the pre-trained CNN model is VGG-VD16, and the conv3-3, conv4-3 and conv5-3 feature maps of VGG-VD16 are used for aggregation.

[0010] Preferably, in step S4, the process of applying principal component analysis to perform dimensionality reduction processing on the extracted multi-layer features includes: Arrange the extracted multi-layer features into a matrix and calculate the covariance matrix; Select the direction with the largest variance as the principal component, perform eigenvalue decomposition on the covariance matrix, and obtain the eigenvalue and eigenvector; According to the size of the eigenvalue, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected as the principal components; The original data is projected into k-dimensional space to achieve dimensionality reduction, and non-overlapping image blocks are extracted from the reduced features.

[0011] Preferably, in step S4, the specific formula for applying the 3D-CNN flow to extract spatial features from the extracted image blocks is: ; in is the input original image block, is the weight of 3D-CNN, is the bias term, is the mapping function of 3D-CNN.

[0012] Preferably, in step S4, the process of applying the CV-CNN flow to extract spectrum features from the extracted image blocks includes: Apply FFT to the extracted image block features to obtain the spectrum representation , where FFT is fast Fourier transform, is the input original image block; The spectral representation is channel-weighted through the CV-CNN flow, and the spectral feature after channel weighting is expressed as ,in is the weight of CV-CNN, is the bias term, is the mapping function of CV-CNN.

[0013] Preferably, in step S5, the process of serially fusing the spatial features and the spectral features includes: Perform dimension alignment operations on spatial features and spectral features; The feature vectors extracted by the 3D-CNN stream and the CV-CNN stream are concatenated in the last dimension to form a new feature vector with a dimension of ,in is the sample size, is the spatial feature dimension, is the spectral feature dimension.

[0014] Preferably, in step S6, the process of strengthening the fused features by the SE attention mechanism includes: Compression: The feature map of each channel is compressed through the global average pooling layer and converted into a scalar value. The output dimension becomes ,in is the number of channels; Motivation: Two fully connected layers are used to capture the dependencies between channels. The first fully connected layer reduces the dimension, and the second fully connected layer restores the original number of channels. The ReLU activation function is added in the middle to increase nonlinearity, and finally the sigmoid activation function is used to obtain the weight value of each channel. Feature recalibration: Multiply the obtained channel weights with the original feature map channel by channel to obtain the output feature map of the SE module.

[0015] Preferably, in step S7, when the support vector machine performs classification, the feature vector with a dimension of K is input into the support vector machine for final classification.

[0016] The embodiment of the present invention further provides a remote sensing image classification system, which is used to implement the remote sensing image classification method described above, and specifically includes: An image set building module, used for building a remote sensing image set; Image preprocessing module, used to preprocess images in the remote sensing image set, including cropping and normalization, data enhancement, and data set division; The feature extraction module is used to extract multi-layer features of the pre-processed remote sensing image through the pre-trained CNN model; A dimension reduction and feature extraction module, which is used to perform dimension reduction processing on the extracted multi-layer features by applying principal component analysis, extract non-overlapping image blocks from the reduced-dimensional features, apply 3D-CNN flow to perform spatial feature extraction on the extracted image blocks, and apply CV-CNN flow to perform spectral feature extraction on the extracted image blocks; Feature fusion module, used to fuse spatial features and spectral features in series; The attention mechanism module is used to enhance the fused features through the SE attention mechanism; The classification module is used to input the enhanced features into the support vector machine for classification.

[0017] An embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores a computer software product, and the computer software product includes a number of instructions for enabling a computer device to execute the remote sensing image classification method described above.

[0018] It can be seen from the above technical solutions that the present invention has the following beneficial effects: (1) Significantly improve classification accuracy and robustness: Adopt a dual-branch deep feature fusion network and attention mechanism to process multi-scale features in parallel, accurately capture key features, reduce information loss, enhance the understanding of complex images, demonstrate excellent performance in the comparison of multiple data sets, and effectively cope with changes in different scenarios.

[0019] (2) Deeply enhance feature expression capabilities: The dual branches extract global and local features respectively. The fusion strategy is combined with the attention mechanism to dynamically empower the model, so that the model can learn richer and more discriminative features, optimize the feature fusion process, and improve classification accuracy.

[0020] (3) Effectively optimize computing performance: Carefully design the network architecture, fusion strategy, and application attention mechanism to ensure high-precision classification while reducing computing resource consumption. It has significant advantages when processing large-scale data sets, combining high precision and high efficiency.

[0021] (4) Widely adaptable and versatile: It does not rely on specific image types and resolutions. It is suitable for a variety of remote sensing scenarios such as urban monitoring, agricultural production estimation, and disaster assessment. It can process different data sets and provide accurate classification solutions for multiple fields.

[0022] (5) Vigorously promote the development and application of remote sensing technology: Innovative technologies will improve the level of remote sensing image processing, promote the advancement of classification technology, and promote its widespread application in environmental protection, resource surveys, urban planning and other fields, thereby generating significant social and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them: Figure 1 A flowchart of a remote sensing image classification method provided in an embodiment; Figure 2 It is a schematic diagram of the overall framework of a dual-branch deep feature fusion network for remote sensing image classification based on attention in an embodiment; Figure 3 This is a comparison result diagram of classification accuracy (%) of different remote sensing image classification methods on the UC data set in the embodiment; Figure 4 This is a comparison result diagram of classification accuracy (%) of different remote sensing image classification methods on the AID dataset in the embodiment; Figure 5 This is a comparison result diagram of classification accuracy (%) of different remote sensing image classification methods on the NWPU-RESISC45 data set in the embodiment; Figure 6 It is a block diagram of a remote sensing image classification system provided in an embodiment. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1

[0025] In order to solve the problems in the existing technology that traditional remote sensing image classification methods rely on manually designed features and classifiers, it is difficult to fully capture key information, the accuracy is limited and there are great limitations; under the deep learning framework, a single CNN model is difficult to capture both spatial structure and spectral characteristics, complex models require expensive computing resources and time, and are impractical in practical applications; and existing methods lack effective feature fusion strategies and attention mechanisms, which affect classification performance. Figure 1 As shown, an embodiment of the present invention provides a remote sensing image classification method, the method comprising: S1: Establish remote sensing image set; S2: Preprocess the images in the remote sensing image set, including cropping and normalization, data enhancement, and data set division; S3: Extract multi-layer features of pre-processed remote sensing images through pre-trained CNN model; S4: Apply principal component analysis to reduce the dimensionality of the extracted multi-layer features, extract non-overlapping image blocks from the reduced dimensionality features, apply 3D-CNN flow to extract spatial features from the extracted image blocks, and apply CV-CNN flow to extract spectral features from the extracted image blocks; S5: tandem fusion of spatial features and spectral features; S6: Enhance the fused features through the SE attention mechanism; S7: Input the enhanced features into the support vector machine for classification.

[0026] It can be seen from the above technical solutions that the present invention proposes a remote sensing image classification method. First, by establishing a diversified image set, a rich data foundation is provided for classification. In the preprocessing step, operations such as cropping and normalization ensure data consistency and validity, data enhancement improves data diversity, and data set division helps to accurately evaluate model performance. The pre-trained CNN model extracts multi-layer features and can obtain key information at different abstract levels. Principal component analysis dimensionality reduction and 3D-CNN and CV-CNN streams extract spatial and spectral features respectively, achieving comprehensive and efficient extraction of image features and solving the limitations of a single CNN model. Serial fusion and SE attention mechanism enhance feature expression capabilities, enable the model to focus on key areas, optimize computational efficiency, and improve classification accuracy and robustness. It does not rely on specific image types and has a wide range of adaptability, which has strongly promoted the development and application of remote sensing image classification technology in multiple fields.

[0027] The present invention proposes an attention-based dual-branch deep feature fusion network for remote sensing image classification. By combining the multi-layer feature extraction capability of convolutional neural network (CNN), the dimension reduction technology of principal component analysis (PCA), the spectrum analysis capability of fast Fourier transform (FFT) and SE attention mechanism, efficient classification of remote sensing images is achieved. DBFF-Net (Dual-Branch Deep Feature Fusion Network) is mainly composed of two parallel branches: 3D-CNN stream and CV-CNN stream. The 3D-CNN stream focuses on processing the spatial information of the image, while the CV-CNN stream focuses on the spectral characteristics of the image. The features extracted by the two branches are fused in the feature fusion layer, and the weights are assigned through the attention mechanism, and finally classified by the support vector machine (SVM). Figure 2 shown.

[0028] In this embodiment, the remote sensing image set of the present invention uses three public remote sensing image sets, namely NWPU-RESISC45, UC Merced Land-Use and Aerial Image. NWPU-RESISC45 is a large remote sensing image dataset created by Northwestern Polytechnical University, containing 45 categories of scenes, 700 256×256 pixel images in each category. UC Merced Land-Use is a remote sensing ground feature image dataset collected and manually annotated by the United States Geological Survey, with 21 categories of scenes, 100 256×256 pixel images in each category. AerialImage is a large-scale aerial image dataset, containing 30 categories with a total of 10,000 images, each category consisting of 220 to 420 images, the image size is fixed at 600×600 pixels in RGB space, and has multiple resolutions, with pixel resolutions varying from about 8 meters to about 0.5 meters. It should be noted here that other suitable public remote sensing image datasets can also be used to construct a remote sensing image set according to actual needs.

[0029] In this embodiment, the images in the remote sensing image set are preprocessed, including cropping and normalization, data enhancement, and data set division.

[0030] Cropping and normalization: For the UC dataset and the NWPU dataset, the original 256×256 pixel RGB images are used directly. For the AID dataset, the images are uniformly cropped to 224×224 pixels. Then, all datasets are normalized to normalize the image pixel values ​​to the range [0,1].

[0031] Data augmentation: All images in the dataset are randomly rotated from -10° to 10°, and some images are randomly flipped horizontally and vertically to increase data diversity.

[0032] Dataset division: For the UC dataset, 80% of the data is used as the training set and 20% as the test set. For the AID dataset, 20% and 50% of the data are used as the training set, respectively, to evaluate the impact of different training data amounts on model performance. For the NWPU dataset, 10% and 20% of the data are used as the training set, respectively, to evaluate the impact of different training data amounts on model performance. To ensure the accuracy of the performance of the evaluation method, the experiment was repeated ten times, and each training set was composed of randomly selected samples and updated. The final overall accuracy and standard deviation were determined by the average of the ten results.

[0033] In this embodiment, the pre-trained CNN model VGG-VD16 is used to extract the multi-layer deep features of the pre-processed remote sensing image, and specifically the conv3-3, conv4-3 and conv5-3 feature maps of VGG-VD16 are used for aggregation. In two-dimensional image processing, the convolution operation is a sliding window traversal operation on the entire image after the dot product operation between the input image X and the convolution kernel K, which is mathematically expressed as: ; in It is the feature map after convolution. Line Column pixel value, represents the convolution operation, Represents the convolution kernel All elements are traversed and summed. is the pixel value at the corresponding position of the input image, is the convolution kernel Corresponding parameter values. In the CNN model, convolutional layers are stacked to form a multi-layer convolution structure. Each layer uses multiple different convolution kernels to extract different features. As the network level deepens, the simple features extracted by the previous layers can be combined to obtain more complex features. The layer output feature map can be expressed as ,in is the output feature map of the previous layer, is the convolution kernel of the current layer, is the bias term, It is an activation function. The features extracted by different convolution layers have different levels of abstraction. The convolution kernels in the early layers capture simple features such as edges, colors, and textures, while the deep layers can combine these simple features to extract complex features such as object parts and shapes.

[0034] In this embodiment, principal component analysis (PCA) is applied to reduce the dimension of the extracted multi-layer features. PCA is a linear dimensionality reduction technique that reduces the data dimension by finding the direction with the maximum variance (principal component) in the data. The extracted multi-layer features are organized into a matrix, each row represents a sample (image), each column represents a feature dimension, and the covariance matrix is ​​calculated to reflect the feature correlation. The direction with the maximum variance (i.e., the least relevant feature) is selected as the principal component, and the covariance matrix is ​​subjected to eigenvalue decomposition to obtain eigenvalues ​​and eigenvectors. The eigenvalue represents the amount of information in the direction of the corresponding eigenvector. The eigenvectors corresponding to the first k largest eigenvalues ​​are selected as the principal component according to the size of the eigenvalue. The original data is projected into the k-dimensional space to achieve dimensionality reduction, and the reduced-dimensional data matrix is ​​obtained, and then non-overlapping image blocks are extracted from the reduced-dimensional features.

[0035] In this embodiment, the 3D-CNN flow is applied to extract spatial features from the extracted image blocks. Specifically, the non-overlapping image blocks extracted from the features after dimensionality reduction are processed by a carefully designed 3D convolutional neural network (3D-CNN) flow for spatial weighting. 3D-CNN can capture the structural information of the image block features in the spatial dimension. Assume that the input original image block is , the weight of 3D-CNN is , the bias term is , the feature after spatial weighting is expressed as ,in It is the mapping function of 3D-CNN.

[0036] In this embodiment, the CV-CNN flow is used to extract the spectrum features of the extracted image blocks. Specifically, firstly, the fast Fourier transform (FFT) is applied to the acquired image block features to obtain the spectrum representation , then these spectral features (complex patch features) are channel-weighted through an efficient channel convolutional neural network (CV-CNN) flow. Let the weight of CV-CNN be , the bias term is , the spectral characteristics after channel weighting are expressed as ,in It is the mapping function of CV-CNN, thereby mining the spectral characteristics of the image.

[0037] In this embodiment, the spatial features and the spectral features are fused in series. Specifically, the spatial features obtained by the 3D-CNN stream and the spectral features obtained by the CV-CNN stream are processed, and their dimensions are made the same by zero padding, truncation or interpolation, and feature alignment is performed. Suppose the feature vector extracted by the 3D-CNN stream is , the dimension is ( is the sample size, is the spatial feature dimension), the feature vector extracted by the CV-CNN flow is , the dimension is ( is the spectral feature dimension), concatenate the two feature vectors on the last dimension to obtain the concatenated feature vector , whose dimensions are ,The new feature vector contains both image space and spectrum information, and has stronger expressive power.

[0038] In this embodiment, the fused features are enhanced through the SE attention mechanism. Specifically, the features after serial fusion are enhanced through the SE attention mechanism. The SE module workflow mainly includes two steps: Squeeze and Excitation.

[0039] Squeeze compresses each channel feature map through the global average pooling layer and converts it into a scalar value with a global receptive field, which can represent the global information of the channel. Assume that the input feature map dimension is ( are the feature map height and width respectively, is the number of channels), the output dimension after global average pooling becomes , the compression formula is ,in is the first The feature map of each channel, is the global feature descriptor of the channel (a compressed scalar value).

[0040] Excitation uses one or more fully connected layers to capture the dependencies between channels. It usually includes two fully connected layers. The first fully connected layer reduces the dimension and reduces the amount of calculation. The second fully connected layer restores the original number of channels. The ReLU activation function is added in the middle to increase nonlinearity. Finally, the sigmoid activation function is used to limit the output between 0 and 1 to obtain the weight value of each channel, indicating the importance of the channel. The calculation formula is: ,in , are the weight matrices of the two fully connected layers, is the dimensionality reduction coefficient, is the ReLU activation function, is the sigmoid activation function, Is the length A vector representing the importance (i.e., weight value) of each channel.

[0041] Feature recalibration: The final channel weights With the original feature map Multiply channel by channel to get the SE module output feature map ,in yes Corresponding to The weight value of each channel, It is The original feature map of channels, is the first The feature map of each channel.

[0042] In this embodiment, the enhanced features are input into a support vector machine for classification. Specifically, the feature vector (with a dimension of K) enhanced by the SE attention mechanism is input into a support vector machine (SVM) for final classification.

[0043] In order to evaluate the feasibility and effectiveness of the method of the present invention, it is compared with five commonly used methods, namely GoogleNet, VGG_VD16, DCA, MCNN, and SPP-Net. Figure 3 , Figure 4 , Figure 5 As shown in the figure, the comparison results show that the method of the present invention performs well on different data sets. For example, the overall accuracy on the NWPU-RESISC45 data set is 10% higher than that of the CNN method, and the performance on the AID data set with 50% training samples is 95.83%, which is better than DCA (performance is about 89.71%) that integrates the last two FC layers of the CNN model and SPP-Net that uses SPP to aggregate the last convolutional features (the method of the present invention performs better on all data sets). It is 1% higher than the DCA method on the UC data set, which proves that the model of the present invention can effectively extract discriminative features from remote sensing images and achieve accurate classification.

[0044] Embodiment 2

[0045] like Figure 6 As shown, the present invention provides a remote sensing image classification system, which is used to implement the remote sensing image classification method of the above embodiment 1, and specifically includes: An image set building module 100 is used to build a remote sensing image set; An image preprocessing module 200 is used to preprocess images in a remote sensing image set, including cropping and normalization, data enhancement, and data set division; A feature extraction module 300 is used to extract multi-layer features of the pre-processed remote sensing image through a pre-trained CNN model; A dimension reduction and feature extraction module 400, for performing dimension reduction processing on the extracted multi-layer features by applying principal component analysis, extracting non-overlapping image blocks from the reduced-dimensional features, performing spatial feature extraction on the extracted image blocks by applying a 3D-CNN flow, and performing spectral feature extraction on the extracted image blocks by applying a CV-CNN flow; A feature fusion module 500 is used to fuse spatial features and spectral features in series; An attention mechanism module 600 is used to enhance the fused features through the SE attention mechanism; The classification module 700 is used to input the enhanced features into a support vector machine for classification.

[0046] A remote sensing image classification system of the present embodiment is used to implement the aforementioned remote sensing image classification method. Therefore, the specific implementation method of the remote sensing image classification system can be seen in the embodiment part of the aforementioned remote sensing image classification method. For example, the image set establishment module 100, the image preprocessing module 200, the feature extraction module 300, the dimension reduction and feature extraction module 400, the feature fusion module 500, the attention mechanism module 600, and the classification module 700 are respectively used to implement steps S1, S2, S3, S4, S5, S6, and S7 in the aforementioned remote sensing image classification method. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part. In order to avoid redundancy, it will not be repeated here. Embodiment 3

[0047] An embodiment of the present invention provides a computer storage medium, which stores a computer software product. The computer software product includes several instructions for enabling a computer device to execute the above-mentioned remote sensing image classification method. In order to avoid redundancy, it will not be repeated here.

[0048] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0049] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0050] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0051] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A remote sensing image classification method, characterized in that: include: S1: Establish remote sensing image set; S2: Preprocess the images in the remote sensing image set, including cropping and normalization, data enhancement, and data set division; S3: Extract multi-layer features of pre-processed remote sensing images through pre-trained CNN model; S4: Apply principal component analysis to reduce the dimensionality of the extracted multi-layer features, extract non-overlapping image blocks from the reduced dimensionality features, apply 3D-CNN flow to extract spatial features from the extracted image blocks, and apply CV-CNN flow to extract spectral features from the extracted image blocks; S5: The spatial features and spectral features are fused in series, including: Perform dimension alignment operations on spatial features and spectral features; The feature vectors extracted by the 3D-CNN stream and the CV-CNN stream are concatenated in the last dimension to form a new feature vector with a dimension of ,in is the sample size, is the spatial feature dimension, is the spectral feature dimension; S6: Enhance the fused features through the SE attention mechanism; S7: Input the enhanced features into the support vector machine for classification.

2. The remote sensing image classification method according to claim 1, characterized in that: In step S3, the pre-trained CNN model is VGG-VD16, and the conv3-3, conv4-3 and conv5-3 feature maps of VGG-VD16 are used for aggregation.

3. The remote sensing image classification method according to claim 1, characterized in that: In step S4, the process of applying principal component analysis to perform dimensionality reduction processing on the extracted multi-layer features includes: Arrange the extracted multi-layer features into a matrix and calculate the covariance matrix; Select the direction with the largest variance as the principal component, perform eigenvalue decomposition on the covariance matrix, and obtain the eigenvalue and eigenvector; According to the size of the eigenvalue, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected as the principal components; The original data is projected into k-dimensional space to achieve dimensionality reduction, and non-overlapping image blocks are extracted from the reduced features.

4. The remote sensing image classification method according to claim 1, characterized in that: In step S4, the specific formula for applying the 3D-CNN flow to extract spatial features from the extracted image blocks is: ； in is the input original image block, is the weight of 3D-CNN, is the bias term, is the mapping function of 3D-CNN.

5. The remote sensing image classification method according to claim 1, characterized in that: In step S4, the process of applying the CV-CNN flow to extract spectrum features from the extracted image blocks includes: Apply FFT to the extracted image block features to obtain the spectrum representation , where FFT is fast Fourier transform, is the input original image block; The spectral representation is channel-weighted through the CV-CNN flow, and the spectral feature after channel weighting is expressed as ,in is the weight of CV-CNN, is the bias term, is the mapping function of CV-CNN.

6. The remote sensing image classification method according to claim 1, characterized in that: In step S6, the process of strengthening the fused features through the SE attention mechanism includes: Compression: The feature map of each channel is compressed through the global average pooling layer and converted into a scalar value. The output dimension becomes ,in is the number of channels; Motivation: Two fully connected layers are used to capture the dependencies between channels. The first fully connected layer reduces the dimension, and the second fully connected layer restores the original number of channels. The ReLU activation function is added in the middle to increase nonlinearity, and finally the sigmoid activation function is used to obtain the weight value of each channel. Feature recalibration: Multiply the obtained channel weights with the original feature map channel by channel to obtain the output feature map of the SE module.

7. The remote sensing image classification method according to claim 1, characterized in that: In step S7, when the support vector machine performs classification, the feature vector with a dimension of K is input into the support vector machine for final classification.

8. A remote sensing image classification system, characterized in that: The system is used to implement the remote sensing image classification method according to any one of claims 1 to 7, specifically comprising: An image set building module, used for building a remote sensing image set; Image preprocessing module, used to preprocess images in the remote sensing image set, including cropping and normalization, data enhancement, and data set division; The feature extraction module is used to extract multi-layer features of the pre-processed remote sensing image through the pre-trained CNN model; A dimension reduction and feature extraction module, which is used to perform dimension reduction processing on the extracted multi-layer features by applying principal component analysis, extract non-overlapping image blocks from the reduced-dimensional features, apply 3D-CNN flow to perform spatial feature extraction on the extracted image blocks, and apply CV-CNN flow to perform spectral feature extraction on the extracted image blocks; Feature fusion module, used to fuse spatial features and spectral features in series; The attention mechanism module is used to enhance the fused features through the SE attention mechanism; The classification module is used to input the enhanced features into the support vector machine for classification.

9. A computer storage medium, characterized in that The computer storage medium stores a computer software product, and the computer software product includes a number of instructions for enabling a computer device to execute the remote sensing image classification method according to any one of claims 1 to 7.

Images