Unmanned aerial vehicle remote sensing image classification method, equipment and medium

Abstract

The invention belongs to the technical field of unmanned aerial vehicle remote sensing image processing, and particularly relates to an unmanned aerial vehicle remote sensing image classification method and device and a medium, and the method comprises the following steps: S1, model construction: constructing a training model which comprises a basic module, a convolution attention mechanism module, an adaptive module and a maximum classifier difference structure; s2, model training: adopting transfer learning to train the training model to obtain an unmanned aerial vehicle remote sensing image classification model; and S3, carrying out unmanned aerial vehicle remote sensing image classification by adopting the unmanned aerial vehicle remote sensing image classification model. According to the method, an adaptive module is added to the model, and the distance between the source domain data and the target domain data is measured by using the multi-core maximum mean value difference, so that the problem of cross-domain distribution difference is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle remote sensing image processing, and in particular to a method, device and medium for classifying unmanned aerial vehicle remote sensing images. Background Art

[0002] With the continuous development of remote sensing technology, remote sensing images taken by drones can provide real-time monitoring without being affected by weather, and support comparative analysis of historical data, with the advantages of wide coverage, efficient data acquisition, multi-spectral information and high resolution. They are suitable for land cover, environmental monitoring and other fields. However, due to the multi-source heterogeneity of remote sensing data and the differences in data distribution between different fields, remote sensing image classification faces severe challenges.

[0003] Traditional remote sensing image classification methods rely on low-level visual features, such as texture, color, spatial and spectral information, or a combination of them. These methods can only extract shallow features of the image and have difficulty in obtaining deeper information. With the improvement of computing power and the accumulation of large-scale data sets, deep learning technology has shown powerful capabilities in the field of image classification. Deep learning models such as convolutional neural networks can automatically learn multi-level features in images without relying on manually designed feature extraction processes.

[0004] The shortcomings of existing deep learning methods include: 1) Due to the differences in transferability between different regions of the image, negative transfer problems may occur during training.

[0005] 2) Existing methods based on deep convolutional neural networks generally assume that training samples and test samples follow independent and identical distribution, but in practical applications, training data and test data may come from different data sets, which leads to distribution difference problems.

[0006] In summary, there is an urgent need for a UAV remote sensing image classification method, equipment and medium to solve the problems in the existing technology. Summary of the invention

[0007] The purpose of the present invention is to provide a method, device and medium for classifying remote sensing images of unmanned aerial vehicles. The specific technical scheme is as follows: A method for classifying remote sensing images from unmanned aerial vehicles comprises the following steps: S1: Model construction, constructing a training model, wherein the training model includes a basic module, a convolutional attention mechanism module, an adaptive module, and a maximum classifier difference structure; S2: Model training, using transfer learning to train the training model; S3: Use the UAV remote sensing image classification model to classify the UAV remote sensing images.

[0008] Preferably, in S1, the basic module is obtained as follows: remove the global pooling and classifier from the pre-trained ResNet50 model to obtain the basic module.

[0009] Preferably, in S1, the convolutional attention mechanism module includes sequentially arranged spatial attention and channel attention.

[0010] Preferably, in S1, the adaptive module includes a multi-core maximum mean difference, which is used to measure the difference between the source domain dataset and the target domain dataset in transfer learning.

[0011] Preferably, in S1, the maximum classifier difference structure includes a classifier layer and a domain discrimination layer; The classifier layer is used to classify the remote sensing image data with labels in the source domain data set, and the training loss adopts the difference loss; The domain discriminator is used to align the marginal distribution of the source domain dataset and the target domain dataset, and the training loss adopts the domain discrimination loss.

[0012] Preferably, in S2, initial training parameters are set, the training parameters are adjusted based on the loss, and the UAV remote sensing image classification model is obtained by training using the test set data.

[0013] Preferably, in S2, the total loss of the training process is as follows: ; ; ; ; in, represents the difference loss, and Represent the source domain dataset and the target domain dataset respectively, and Respectively and For categories The probability output is Indicates the total number of categories; represents the domain discrimination loss, represents the domain discriminator, represents the feature extractor, represents the cross entropy loss function, and Represent the source domain category and the target domain category respectively, represents the source domain sample set, represents the target domain sample set, Representation sample The domain label of represents the adaptive loss, represents the learnable hyperparameters, Indicates the maximum mean difference.

[0014] In addition, the present invention also discloses a computer device, including a memory and a processor; The memory is used to store a computer program executable on the processor; The processor is used to implement the steps of the above-mentioned UAV remote sensing image classification method when executing the computer program.

[0015] In addition, the present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned UAV remote sensing image classification method are implemented.

[0016] The application of the technical solution of the present invention has the following beneficial effects: The present invention discloses a method for classifying remote sensing images from unmanned aerial vehicles. Starting from the two perspectives of hybrid attention mechanism and domain adaptation, targeted improvements are made to the current problems based on the existing deep learning and transfer learning technologies. The method of the present invention solves the negative transfer problem caused by areas with low mobility during the training process and the difference in spatial distribution of categories between the source domain and the target domain. The present invention adopts an adversarial method of a hybrid attention mechanism, so that the network can automatically learn which parts need to be paid attention to during the migration process, thereby improving the negative transfer problem. Furthermore, the present invention adds an adaptive module to the model, and uses the multi-core maximum mean difference to measure the distance between the source domain and the target domain data to solve the problem of cross-domain distribution differences.

[0017] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 is a flowchart of the steps of the UAV remote sensing image classification method in the preferred embodiment of the present invention; Figure 2 is a model structure diagram of a training model in a preferred embodiment of the present invention; Figure 3is a structural diagram of a channel attention module in a preferred embodiment of the present invention; Figure 4 is a structural diagram of a spatial attention module in a preferred embodiment of the present invention; Figure 5 is a sample example diagram of the UC Merced data set in a preferred embodiment of the present invention; Figure 6 is a sample example diagram of an AID data set in a preferred embodiment of the present invention; Figure 7 This is a sample example diagram of the NWPU-RESISC45 data set in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] like Figure 1 As shown, this embodiment discloses a method for classifying UAV remote sensing images, comprising the following steps: S1: Model construction, constructing a training model, wherein the training model includes a basic module, a convolutional attention mechanism module, an adaptive module, and a maximum classifier difference structure; S2: Model training, using transfer learning to train the training model to obtain a UAV remote sensing image classification model; S3: Use the UAV remote sensing image classification model to classify the UAV remote sensing images.

[0022] In a specific implementation case, in S1, the basic module is obtained as follows: remove the global pooling and classifier from the pre-trained ResNet50 model to obtain the basic module.

[0023] In a specific implementation case, in S1, the convolutional attention mechanism module (CBAM, Convolutional Block Attention Module) includes sequentially arranged spatial attention and channel attention, that is, the convolutional attention mechanism module includes a channel attention mechanism module and a spatial attention mechanism module.

[0024] Furthermore, the flow chart of the channel attention mechanism module in CBAM is as follows Figure 2As shown in the figure, the input feature map is first subjected to global maximum pooling and global average pooling respectively, and the feature map is compressed based on two dimensions to obtain two feature descriptions of different dimensions. The pooled feature maps share a multi-layer perceptron network, first reduce the number of channels through a fully connected layer, and then restore the number of channels through another fully connected layer. The two feature maps are stacked in the channel dimension, and the weight of each channel of the feature map is normalized to between 0 and 1 through the sigmoid activation function. The normalized weight is multiplied by the input feature map.

[0025] Furthermore, the spatial attention mechanism module in CBAM is as follows Figure 3 As shown in the figure, the output feature map of the channel attention mechanism is processed in the spatial domain. First, the input feature map is subjected to maximum pooling and average pooling in the channel dimension, and the two pooled feature maps are stacked in the channel dimension. Then, , or The convolution kernel of different sizes fuses channel information, and the shape of the feature map changes from become Finally, the convolution result is passed through the sigmoid function to normalize the spatial weight of the feature map, and then the input feature map is multiplied by the weight.

[0026] In a specific implementation case, in S1, the adaptive module includes a multi-core maximum mean difference, which is used to measure the difference between the source domain dataset and the target domain dataset in transfer learning.

[0027] It should be noted that the maximum mean difference (MMD) is the most commonly used measurement criterion in transfer learning, and the expression is as follows: ; in, Representation distribution and distribution The distance between them, i.e. the maximum mean difference; Indicates that the distribution is satisfied The source domain dataset, Indicates that the distribution is satisfied The target domain dataset, Indicates that Mapped into the Reproducing Kernel Hilbert Space (RKHS), Indicates that the distribution The expectations below, Indicates that the distribution The expectations below, Indicates passing The regenerating Hilbert space of a kernel map.

[0028] Furthermore, in this embodiment, the multi-core maximum mean difference (MMD) method is used. Compared with the single-core maximum mean difference (MMD) method, the multi-core maximum mean difference (MK-MMD) sets a coefficient for each kernel function and performs weighted addition. Since the selection of the kernel function has a great influence on the effect of the maximum mean difference distance calculation, the multi-core maximum mean difference method can effectively reduce the possibility of selecting a kernel function with poor effect. The kernel function of MK-MMD is shown in Formula 2: ; in, Represents the kernel function of MK-MMD, which is constructed by multiple kernels. represents the number of kernel functions, Indicates the proportion of each kernel function. The adaptive module in this embodiment uses MK-MMD to measure the difference between the source domain and the target domain.

[0029] In a specific implementation case, in S1, the maximum classifier difference structure includes a classifier layer and a domain discrimination layer; The classifier layer is used to classify the remote sensing image data with labels in the source domain data set, and the training loss adopts the difference loss; The domain discriminator is used to align the marginal distribution of the source domain dataset and the target domain dataset, and the training loss adopts the domain discrimination loss.

[0030] In a specific implementation case, in S2, initial training parameters are set, the training parameters are adjusted based on the loss, and the UAV remote sensing image classification model is trained using the test set data. The training parameters include the initial learning rate, batch size, and number of training rounds, and the learning rate is adjusted according to the verification loss to obtain the optimal model.

[0031] Preferably, in S2, the total loss of the training process is as follows: ; ; ; ; in, represents the difference loss, and Represent the source domain dataset and the target domain dataset respectively, and Respectively and For categories The probability output is Indicates the total number of categories; represents the domain discrimination loss, represents the domain discriminator, represents the feature extractor, represents the cross entropy loss function, and Represent the source domain category and the target domain category respectively, represents the source domain sample set, represents the target domain sample set, Representation sample The domain label of represents the adaptive loss, represents the learnable hyperparameters, Indicates the maximum mean difference.

[0032] The following is a data verification of the method of this embodiment: The model training and testing platform of the present invention adopts: Microsoft Windows 11 operating system, CPU model is 13th Gen Intel(R) Core(TM) i9-13900KF, GPU model is GeForce RTX 4090, and GPU available video memory is 24G. The Pytorch deep learning framework is used to implement the proposed remote sensing image classification method. The optimizer of the classification network model is Adam, the loss function is the cross entropy loss function, the initial learning rate is 1e-3, the learning rate in the fine-tuning stage is 1e-6, the batch size is 16, and the number of training iterations is 100 epochs.

[0033] In the image classification task, the evaluation indicators include the accuracy and confusion matrix during the training process. The accuracy calculation formula is: ; in, (True Positive) indicates the true class, that is, the true class is the positive class and the predicted class is also the positive class. (False Negative) represents the false negative class, that is, the true value of the sample is the positive class, and the predicted value is the negative class. (FalsePositive) represents a false positive class, that is, the true value of the sample is a negative class, and the predicted value is a positive class. (True Negative) represents the true negative class, that is, the true class of the sample is the negative class, and the predicted class is also the negative class. Through the confusion matrix, this embodiment can clearly know the classification effect of the model on each class.

[0034] In order to call the pre-trained model in PyTorch to implement a deep adversarial domain adaptation method based on hybrid attention and classifier constraints for drone remote sensing image classification, the hybrid attention mechanism is first defined, and a model based on the pre-trained ResNet50 is built, the global pooling and classification parts are removed, and the hybrid attention layer and the custom fully connected layer are added. Then the total loss function is defined to combine the classification loss and the adversarial loss to optimize the model. Data augmentation and preprocessing are used in the data preparation stage, and training and validation data are loaded through ImageFolder and DataLoader. During the model training process, the device, loss function and optimizer are set, the model parameters are iteratively updated in multiple rounds, and the loss and accuracy of the validation set are evaluated at the end of each round. Finally, a prediction function is written to load new images and use the trained model for classification prediction, which improves the accuracy of remote sensing image classification.

[0035] The models used for comparison in this data verification are as follows: The ResNet network is a deep convolutional neural network architecture. This network design overcomes the common degradation problems in deep network training by introducing an innovative residual learning framework, enabling the network to achieve more accurate image recognition and classification capabilities through a deeper architecture. The basic module of the ResNet network is the residual block, which contains jump connections. During the training process, a deep network can be constructed by stacking multiple residual blocks. The ResNet network also introduces a global average pooling layer to reduce the spatial dimension of the feature map, and finally uses a fully connected layer to output the classification results.

[0036] The DAN network algorithm is a method that uses a deep adaptation network to learn transferable features, solving the problems of transfer learning and domain adaptation through a deep learning framework. The network quantifies and minimizes the distribution differences of cross-domain tasks in the feature space by using an adaptive metric function as the loss function for domain adaptation. Using this method, DAN is able to learn a domain-invariant feature representation that performs well in both the source and target domains, thereby improving the generalization ability of the model in the target domain. Structurally, DAN directly optimizes the feature extraction layer by embedding the MK-MMD loss in the deep network, making the learned features more consistent in the source and target domains.

[0037] The DANN network proposes an innovative method in the field of transfer learning, which mainly achieves unsupervised domain adaptation through back propagation. The core idea of ​​this algorithm is to use the gradient reversal layer to simultaneously optimize the feature extractor, so that the model can reduce the distribution difference between the source domain and the target domain while maintaining good performance on the source domain task. Structurally, the gradient reversal layer introduces normal computational loss during forward propagation, and reverses the direction of the gradient during back propagation, forcing the domain classifier to be unable to distinguish input data from different domains, thereby encouraging the model to learn guiding features that do not change with the domain.

[0038] The core of the DDC network algorithm is the introduction of domain confusion loss, which aims to make it difficult for the model to distinguish whether the data is from the source domain or the target domain during the learning process, thereby promoting the learned feature representation to be invariant between different domains. The structural feature of this algorithm is that it directly adds a mechanism to consider domain invariance in the feature extraction stage, so that the model can effectively learn feature representations that are useful for the task and common between different domains.

[0039] The MCD model is a model that uses the maximum mean difference method. First, sample data from different categories or distributions are collected, and then a suitable kernel function is selected to map the data to a high-dimensional space, and the mean of each sample is calculated. The maximum mean difference (MMD) formula is then used to measure the difference between the two means. Finally, a decision is made based on the calculated MMD value.

[0040] The CBAM-MCD method combines the convolutional block attention module (CBAM) and the maximum class difference (MCD) metric to enhance the performance of tasks such as image classification and object detection. CBAM dynamically adjusts the weights of feature maps through channel and spatial attention mechanisms to emphasize important feature areas, thereby improving the model's feature expression capabilities. MCD uses the mean difference of samples of different categories to evaluate the distribution difference between categories to optimize the model's classification capabilities in complex scenes.

[0041] The MK-MCD method combines the multi-kernel technology and the maximum class difference metric model. This method captures the diversity of data by introducing multiple kernel functions to better represent sample characteristics in high-dimensional space. By calculating the mean of samples of different categories and using the maximum class difference (MCD) metric to evaluate the difference in distribution between classes, MK-MCD can effectively enhance the model's ability to distinguish between categories.

[0042] In order to verify the effect of the method of the present invention, this embodiment conducts experiments of different degrees on the UC Merced, AID and NWPU-RESISC45 datasets, where U, A and N represent the abbreviations of the three datasets respectively.

[0043] The UC Merced dataset is a widely used public dataset for remote sensing image classification. It contains 21 different categories of high-resolution remote sensing images. Each category contains 100 color images of size 1 pixel. The goal of this dataset is to classify these images to achieve the recognition and analysis of different objects and scenes. The UC Merced dataset is characterized by high image quality and rich and diverse categories, and is suitable for the research and evaluation of remote sensing image classification algorithms. Some samples of the UCMerced dataset are as follows: Figure 5 shown.

[0044] The AID dataset is a large-scale dataset for remote sensing image scene classification. It consists of 30 scene types, including airports, bare land, baseball fields, beaches, bridges, and centers. All images are annotated by remote sensing image experts. The AID dataset has a total of 10,000 images. Some samples of each category are as follows: Figure 6 shown.

[0045] The NWPU-RESISC45 dataset is a widely used public dataset for remote sensing image classification. It contains remote sensing image samples of 45 land object categories from Northwestern Polytechnical University in China. The dataset contains images from different regions, seasons and resolutions, covering a variety of land object types such as cities, farmlands, forests, rivers, etc. Each category contains about 700 images, with a total of about 31,500 images. The goal of this dataset is to promote the research and performance evaluation of remote sensing image classification algorithms. Some samples of the three domains of the NWPU-RESISC45 dataset are as follows: Figure 7 shown.

[0046] It can be seen from Table 1 that the classification performance of the adversarial domain adaptation method on the UC Merced, AID and NWPU-RESISC45 data sets is generally better than that of the traditional domain adaptation method, and the method of the present invention is better than the MCD model that only uses the maximum mean difference. At the same time, for the ResNet model, the deep adversarial domain adaptation model that selects ResNet as the feature extractor is better than ResNet. Overall, the method of the present invention has the best classification effect on remote sensing images, with accuracy rates of 84.35%, 90.49% and 95.87% respectively. Secondly, this embodiment conducts comparative experiments on domain adaptation methods between multiple data sets, among which the comparative methods are ResNet (deep neural network architecture), DAN (deep adaptation network), DANN (deep adversarial domain adaptation network), DDC (domain adversarial classifier adaptation algorithm), MCD (mean difference) and DAAN (combined with deep transfer learning).

[0047] Table 1 Comparison of classification performance of different methods on remote sensing datasets

[0048] The experimental results are shown in Table 2. It can be seen from the table that compared with other methods, the method of the present invention has achieved the best classification effect in the six migration tasks of U→A, N→A, U→N, A→N, A→U and N→U, with different degrees of improvement between 1% and 2.3% in each migration task. Among them, their average accuracy can reach 87.9%, which is 1.1% higher than the suboptimal method. The experimental results show that the classification method proposed by the present invention has a better classification effect on the remote sensing data set, and has different degrees of improvement compared with other methods. This also proves the effectiveness of the method of the present invention.

[0049] Table 2 Comparison of classification accuracy (%) of different methods among remote sensing datasets

[0050] In order to further verify the effectiveness of the method proposed in the present invention, ablation experiments were carried out on the UC Merced, AID and NWPU-RESISC45 datasets, and experiments on six migration tasks of U→A, N→A, U→N, A→N, A→U and N→U were carried out on three remote sensing datasets. The experimental results are shown in Table 3. From the experimental results in the table, it can be seen that the classification effect of the method of the present invention between the three remote sensing datasets is better than that of the feature extraction network ResNet, the backbone network MCD, the CBAM_MCD that only introduces the attention mechanism, and the MK_MCD that only adds the multi-core maximum mean difference metric, which further verifies the effect of the method of the present invention. At the same time, the improved domain adaptation layer also significantly improves the classification effect in the six migration tasks. The method of the present invention achieved the best results in the six tasks, and the average classification accuracy of the method of the present invention was improved by nearly 2.5% compared with the MCD network.

[0051] Table 3 Comparison of ablation experiment accuracy (%) on remote sensing datasets

[0052] In order to verify that the improved algorithm of the present invention can effectively extract discriminative features, this embodiment performs CAM visualization on the feature map. By randomly extracting target domain images from the UC Merced, AID and NWPU-RESISC45 data sets for comparison, the area identified by the method proposed in the present invention is more accurate than the baseline algorithm. For example, for identifying an airplane in an image, the baseline algorithm mainly focuses on the local features of the airplane, and is not very complete. In addition, the network with the added attention mechanism can well identify one wing of the airplane. In contrast, the network that only fuses the domain adaptation method ignores the tail features and only highlights the wings of the airplane, especially those that are important for classification and migration. Of course, by successfully extracting potential fine-grained features using the attention mechanism and the adaptive module, the method of the present invention improves the ability of the basic quasi-algorithm to transfer knowledge.

[0053] In addition, this embodiment also discloses a computer device, including a memory and a processor; The memory is used to store a computer program executable on the processor; The processor is used to implement the steps of the above-mentioned UAV remote sensing image classification method when executing the computer program.

[0054] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the computer device.

[0055] The computer device may be a computing device such as a mobile phone, a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may include, but is not limited to, a processor and a memory. For example, the computer device may also include an input / output device, a network access device, a bus, etc.

[0056] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0057] The memory can be used to store the computer program and / or module, and the processor implements the computer program by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0058] Wherein, if the module / unit integrated in the computer device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0059] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned UAV remote sensing image classification method are implemented.

[0060] The embodiment of the present invention discloses a method for classifying remote sensing images from unmanned aerial vehicles. Starting from the two perspectives of hybrid attention mechanism and domain adaptation, targeted improvements are made to the current problems based on the existing deep learning and transfer learning technologies. The method of the present invention solves the negative transfer problem caused by areas with low mobility during the training process and the difference in spatial distribution of categories between the source domain and the target domain. The present invention adopts an adversarial method of a hybrid attention mechanism, so that the network can automatically learn which parts need to be paid attention to during the migration process, thereby improving the negative transfer problem. Furthermore, the present invention adds an adaptive module to the model, and uses the multi-core maximum mean difference to measure the distance between the source domain and the target domain data to solve the problem of cross-domain distribution differences.

[0061] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for classifying unmanned aerial vehicle remote sensing images, characterized in that: The steps include: S1: Model construction, constructing a training model, wherein the training model includes a basic module, a convolutional attention mechanism module, an adaptive module, and a maximum classifier difference structure; S2: Model training, using transfer learning to train the training model to obtain a UAV remote sensing image classification model; S3: Use the UAV remote sensing image classification model to classify the UAV remote sensing images.

2. The method for classifying unmanned aerial vehicle remote sensing images according to claim 1, characterized in that: In S1, the basic module is obtained as follows: remove the global pooling and classifier from the pre-trained ResNet50 model to obtain the basic module.

3. The method for classifying UAV remote sensing images according to claim 2, characterized in that: In S1, the convolutional attention mechanism module includes sequentially arranged spatial attention and channel attention.

4. The method for classifying UAV remote sensing images according to claim 3, characterized in that: In S1, the adaptation module includes a multi-core maximum mean difference, which is used to measure the difference between the source domain dataset and the target domain dataset in transfer learning.

5. The method for classifying unmanned aerial vehicle remote sensing images according to claim 4, characterized in that: In S1, the maximum classifier difference structure includes a classifier layer and a domain discrimination layer; The classifier layer is used to classify the remote sensing image data with labels in the source domain data set, and the training loss adopts the difference loss; The domain discriminator is used to align the marginal distribution of the source domain dataset and the target domain dataset, and the training loss adopts the domain discrimination loss.

6. The method for classifying unmanned aerial vehicle remote sensing images according to claim 5, characterized in that: In S2, the initial training parameters are set, the training parameters are adjusted based on the loss, and the UAV remote sensing image classification model is trained using the test set data.

7. The method for classifying UAV remote sensing images according to claim 6, characterized in that: In S2, the total loss of the training process as follows: ； ； ； ； in, represents the difference loss, and Represent the source domain dataset and the target domain dataset respectively, and Respectively and For categories The probability output is Indicates the total number of categories; represents the domain discrimination loss, represents the domain discriminator, represents the feature extractor, represents the cross entropy loss function, and Represent the source domain category and the target domain category respectively, represents the source domain sample set, represents the target domain sample set, Representation sample The domain label of represents the adaptive loss, represents the learnable hyperparameters, Indicates the maximum mean difference.

8. A computer device, characterized in that: including memory and processor; The memory is used to store a computer program executable on the processor; The processor is used to implement the steps of the UAV remote sensing image classification method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the UAV remote sensing image classification method according to any one of claims 1 to 7 are implemented.

Images