A remote sensing image scene classification method based on multi-instance learning and a storage medium

Abstract

The application discloses a remote sensing image scene classification method based on multiple example learning and a storage medium, and the method comprises the following steps: extracting multi-scale convolution features of a current collected remote sensing image, fusing the features to obtain a fused multi-scale feature map, inputting the fused multi-scale feature map into an example classifier to obtain class prediction probabilities of each example in the multi-scale feature map, aggregating the class prediction probabilities of each example into bag prediction probabilities, and determining the class of the remote sensing image according to the bag prediction probabilities. Since the application extracts each example in the remote sensing image to obtain example prediction probabilities after multi-scale feature fusion of the remote sensing image, aggregates each example probability into the whole bag prediction probability, and finally determines the scene class according to the bag prediction probability, the inner relationship between each example in the original remote sensing image is considered, important local areas related to the scene label can be highlighted better, and therefore the precision of remote sensing scene classification with complex semantic information is improved.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method, apparatus, device, and storage medium for remote sensing image scene classification based on multi-instance learning. Background Technology

[0002] Currently, with the continuous development of remote sensing technology, remote sensing images, due to their rich spatial and semantic information, are widely used in many fields such as agriculture, environmental monitoring, disaster prevention, mapping, and urban construction and management.

[0003] In existing technologies, scene classification of remote sensing images is usually based on depth features at a single scale and angle as the basis for distinguishing different scenes. However, when dealing with target areas with richness and similarity, and multiple objects coexisting and being related, the classification effect is often poor. Therefore, how to improve the classification accuracy and precision of remote sensing images in complex scenes has become an urgent problem to be solved.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is related technology. Summary of the Invention

[0005] The main objective of this invention is to provide a remote sensing image scene classification method and storage medium based on multi-instance learning, aiming to solve the technical problem of how to improve the accuracy and precision of complex remote sensing scene classification.

[0006] To achieve the above objectives, this invention provides a remote sensing image scene classification method based on multi-instance learning, the method comprising the following steps: Extract multi-scale convolutional features from the currently acquired remote sensing image, and fuse the features to obtain a fused multi-scale feature map; The fused multi-scale feature map is input into the example classifier to obtain the class prediction probability of each example in the multi-scale feature map; The class prediction probabilities of the various examples are aggregated into bag prediction probabilities, and the category of the remote sensing image is determined based on the bag prediction probabilities.

[0007] Optionally, the step of extracting multi-scale convolutional features from the currently acquired remote sensing image and fusing the features to obtain a fused multi-scale feature map includes: Obtain scaled images of the currently acquired remote sensing image at different scales; Convolutional features of images at different scales are extracted by a preset multi-scale convolutional feature extractor to obtain multi-scale feature maps; The multiple scale feature maps are fused to obtain a fused multi-scale feature map.

[0008] Optionally, the step of extracting convolutional features from images of different scales using a preset multi-scale convolutional feature extractor to obtain multiple scale feature maps includes: The preset multi-scale convolutional feature extractor is constructed using the dilated convolution operator; The convolutional features of the images at different scales are extracted according to the preset multi-scale convolutional feature extractor; The convolutional features of the images at different scales are represented by convolutional profile maps to obtain the multi-scale feature maps.

[0009] Optionally, the step of inputting the fused multi-scale feature map into an example classifier to obtain the class prediction probability of each example in the multi-scale feature map includes: The fused multi-scale feature map is input into the example classifier, and the activation value at each position of the multi-scale feature map is calculated to obtain the activation value at each position, wherein the dimension of the activation value matches the dimension of the currently acquired remote sensing image. Each example is represented by a feature vector formed by the activation values ​​at each position, thus obtaining the example feature vector; Obtain the class prediction probability of each example in the multi-scale feature map.

[0010] Optionally, the step of aggregating the class prediction probabilities of the various examples into bag prediction probabilities, and determining the category of the remote sensing image based on the bag prediction probabilities, includes: The class prediction probabilities of the individual examples are aggregated into bag prediction probabilities using a pooling model. The category of the remote sensing image is determined based on the bag prediction probability.

[0011] Optionally, the step of aggregating the class prediction probabilities of the various examples into bag prediction probabilities using a pooling model includes: The attention weights for each example are obtained using a spatial attention model; The class prediction probabilities of the examples are aggregated into bag prediction probabilities based on the attention weights of the examples.

[0012] Optionally, the step of aggregating the class prediction probabilities of the various examples into bag prediction probabilities using a pooling model includes: Multiple example predictions are aggregated into a single bag prediction using a pre-defined pooling function; The preset pooling function is: In the formula, For bag prediction, For example prediction, g( ) is the preset pooling function; in, In the formula, For attention weights.

[0013] Optionally, the attention weights are calculated using an attention network; The formula for calculating the attention weight is: In the formula, For example feature vectors, as well as Let be the trainable weight parameters in the attention network, and b be the bias parameter.

[0014] Optionally, before the step of aggregating the class prediction probabilities of the individual examples into bag prediction probabilities using a pooling model, the method includes: The pooling model is optimized by selecting a network loss function. The network loss function is used to minimize the loss between the bag prediction probability and the true label. The network loss function is: In the formula, L is the network loss function, and N is the number of scene categories. Indicates the true label of the category scenario. This indicates the predicted label for the category / scene.

[0015] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a remote sensing image scene classification program based on multiple instance learning, wherein when the remote sensing image scene classification program based on multiple instance learning is executed by a processor, it implements the steps of the remote sensing image scene classification method based on multiple instance learning as described above.

[0016] This invention extracts multi-scale convolutional features from currently acquired remote sensing images and fuses these features to obtain a fused multi-scale feature map. This fused multi-scale feature map is then input into an example classifier to obtain the class prediction probability of each example in the multi-scale feature map. The class prediction probabilities of each example are aggregated into a bag prediction probability, and the category of the remote sensing image is determined based on the bag prediction probability. Because this invention extracts and fuses multi-scale features from remote sensing images to obtain a multi-scale feature map with the features of the original remote sensing image, then extracts the prediction probability of each example in the multi-scale feature map, and finally aggregates the probabilities of each example into the overall bag prediction probability, it takes into account the semantic information of complex scenes and the inherent relationships between examples in the original remote sensing image. This better highlights important local regions related to scene labels, thereby improving the accuracy of classifying remote sensing scenes with complex semantic information. Attached Figure Description

[0017] Figure 1This is a flowchart illustrating the first embodiment of the remote sensing image scene classification method based on multi-instance learning of the present invention; Figure 2 This invention relates to a multi-instance relationship graph in a remote sensing image scene classification method based on multi-instance learning. Figure 3 This is a flowchart of the multi-instance image classification process in the remote sensing image scene classification method based on multi-instance learning of the present invention; Figure 4 This is a flowchart illustrating the second embodiment of the remote sensing image scene classification method based on multi-instance learning of the present invention; Figure 5 These are example diagrams of different hole roll operators in the remote sensing image scene classification method based on multi-instance learning of this invention; Figure 6 This is an example image of the convolutional profile in the remote sensing image scene classification method based on multi-instance learning of this invention; Figure 7 This is a flowchart illustrating the third embodiment of the remote sensing image scene classification method based on multi-instance learning of the present invention; Figure 8 This is a network architecture diagram of the remote sensing image scene classification method based on multi-instance learning of the present invention.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] This invention provides a remote sensing image scene classification method based on multi-instance learning, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the remote sensing image scene classification method based on multi-instance learning according to the present invention.

[0020] In this embodiment, the remote sensing image scene classification method based on multi-instance learning includes the following steps: Step S10: Extract the multi-scale convolutional features of the currently acquired remote sensing image, and fuse the features to obtain the fused multi-scale feature map.

[0021] It should be noted that the executing entity of the method in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a mobile phone, tablet computer, or personal computer. It can also be other electronic devices capable of performing the same or similar functions, and this embodiment does not impose any limitations on it. Here, a personal computer will be used as an example to describe various embodiments of the remote sensing image scene classification method based on multi-instance learning.

[0022] It is understood that the currently acquired remote sensing images can be obtained from different remote sensing data sources on the network. Based on the differences in remote sensing technology, remote sensing data can be divided into optical remote sensing data, microwave radar data, and lidar data; this embodiment does not impose any limitations on this. The currently acquired images can also be remote sensing images obtained from a preset dataset. The preset dataset includes datasets of multiple types of remote sensing images, such as the UC Merced dataset, the NWPU-RESISC45 dataset, the AID dataset, and the WHU-RS19 dataset. In specific implementations, the dataset can be selected according to actual needs; this embodiment does not impose any limitations on this.

[0023] Furthermore, considering the phenomenon of large intraclass differences (significant differences in texture and color between remote sensing images of the same type of land cover) caused by shooting from multiple angles and directions, as well as the influence of irregular noise, banding, and bright spots, after acquiring the remote sensing images, data enhancement processing can be performed on the remote sensing images. The data enhancement processing includes, but is not limited to, image rotation and flipping, etc., and this embodiment does not impose any restrictions on this.

[0024] It should be noted that after acquiring the current remote sensing image, it can be downsampled successively to obtain a multi-scale image represented in the form of an image pyramid. The image pyramid is a type of multi-scale image representation, an effective yet conceptually simple structure for interpreting images at multiple resolutions. An image pyramid is a set of image resolutions that gradually decrease in a pyramid shape (from bottom to top) and originate from the same original image. It is obtained through stepwise downsampling until a certain termination condition is met.

[0025] It is understandable that multi-scale image technology refers to the expression of image sampling at multiple scales and processing them separately at different scales. Since it is not easy to see or obtain all the characteristics of an image at a single scale in different scenes, while these characteristics are easier to discover and extract at other scales, multi-scale technology can be used when extracting image features.

[0026] It should be understood that multi-scale feature fusion can be performed by inputting all the obtained image features into a preset convolutional neural network model in a multi-scale input manner. The preset convolutional neural network model can be an existing VGG-Net 16 model, or it can be a convolutional neural network model for fusing multi-scale features obtained by improving an existing model based on different scenarios. This embodiment does not limit this.

[0027] In the specific implementation, remote sensing images of different sizes obtained through tiered sampling are subjected to feature extraction and feature aggregation to obtain convolutional features at different scales. The multi-scale convolutional features are then input into a pre-defined convolutional neural network model for fusing multi-scale features to obtain a fused multi-scale feature map.

[0028] Step S20: Input the fused multi-scale feature map into the example classifier to obtain the class prediction probability of each example in the multi-scale feature map.

[0029] Understandably, this example can represent sub-regions within a complex scene in a remote sensing image. These sub-regions are divided according to target objects, such as: medium-sized residential areas, densely populated residential areas, palaces, lakes, grasslands, etc. Furthermore, multiple objects within this division coexist and can be interconnected within the scene; here, we take... Figure 2 The diagram illustrates the relationship between multiple instances, where the selected multiple instance images are remote sensing images of an airport. In this scenario, instances can be categorized as "buildings," "grassland," "runway," and "airplane."

[0030] It should be noted that the classifier is used to classify each sub-region, i.e., each example, in a remote sensing image. Classification is the process of recognizing, understanding, and assigning ideas or objects to a predefined category or "subgroup." Machine learning programs use pre-classified training datasets and can classify future datasets using various algorithms. Common classification algorithms include, for example, Naive Bayes, nearest neighbor, decision trees, and support vector machines. This embodiment does not limit the algorithm used in the selected classifier.

[0031] In practice, the obtained fused multi-scale feature map is input into the classifier to obtain multiple features based on the map, and then the class prediction probability based on each example in the multi-scale feature map is obtained based on an example represented by each feature.

[0032] Step S30: Aggregate the class prediction probabilities of each example into bag prediction probabilities, and determine the category of the remote sensing image based on the bag prediction probabilities.

[0033] Understandably, considering the classification of multi-example images in various scenarios, the class prediction probabilities of each example can be aggregated into bag prediction probabilities by combining Multiple Instance Learning (MIL) with convolutional neural networks.

[0034] It's important to note that MIL (Multi-Instance Binary Classification) is a weakly supervised learning method, evolved from supervised learning algorithms. It defines a "bag" as a collection of multiple instances and has wide applications. Instead of receiving a set of individually labeled instances, the learner receives a set of labeled bags, each containing multiple instances. In the simple case of multi-instance binary classification, a bag is labeled negative if all instances in it are negative. Conversely, a bag is labeled positive if at least one instance is positive. This is combined with... Figure 3 This section explains the classification of multiple example images. Figure 3 The flowchart for multi-instance image classification is as follows: the sample bag is established by treating each image as a bag, and each bag consists of a series of samples represented by the divided sample regions. The key to implementing MIL in computer vision tasks is to use the sub-regions of the downsampled image as samples to build a multi-instance model. Then, by analyzing the relationships between samples and between samples and labels, the scene label is finally determined from the relevant samples.

[0035] In the specific implementation, the weights of each example are obtained by combining MIL with a convolutional neural network. Based on the weights of different examples, the class prediction probabilities of each example are aggregated into bag prediction probabilities, thereby obtaining the category to which the remote sensing image belongs.

[0036] This embodiment acquires multi-scale convolutional features from the currently acquired remote sensing image, fuses these features to obtain a fused multi-scale feature map, inputs the fused multi-scale feature map into an example classifier, obtains the class prediction probability of each example in the multi-scale feature map, aggregates the class prediction probabilities of each example into a bag prediction probability, and determines the category of the remote sensing image based on the bag prediction probability. Since this embodiment extracts and fuses multi-scale features from the remote sensing image to obtain a multi-scale feature map with the features of the original remote sensing image, extracts the prediction probability of each example in the multi-scale feature map, and finally aggregates the probability of each example into the overall bag prediction probability, it takes into account the semantic information of complex scenes and the inherent relationships between examples in the original remote sensing image. This approach can better highlight important local regions related to scene labels, thereby improving the accuracy of classifying remote sensing scenes with complex semantic information.

[0037] refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the remote sensing image scene classification method based on multi-instance learning according to the present invention.

[0038] Based on the first embodiment described above, in this embodiment, step S10 includes: Step S11: Obtain scale images of the currently acquired remote sensing image at different scales.

[0039] It is understood that remote sensing images can be input into a preset image pyramid and subjected to upward and / or downward image sampling processing according to image sampling rules to obtain images of different scales. The preset image pyramid includes, but is not limited to, Gaussian pyramids and Laplacian pyramids. Gaussian pyramids can be used for downward image sampling, while Laplacian pyramids are used to reconstruct unsampled images from lower-level images, i.e., upward image sampling. The image sampling rules can be understood as matching the corresponding image pyramid according to the scale size and performing corresponding pattern sampling processing according to the matched pyramid.

[0040] Step S12: Extract convolutional features from the images at different scales using a preset multi-scale convolutional feature extractor to obtain multi-scale feature maps.

[0041] Furthermore, step S12 may also include: Step S121: Construct the preset multi-scale convolution feature extractor using the dilated convolution operator.

[0042] Understandably, due to their fixed receptive field, traditional convolutional filters struggle to perceive key objects of varying sizes. Objects larger than the receptive field typically fail to achieve robust semantic representation, and directly combining multiple convolutions with different window sizes is more prone to overfitting. Therefore, this embodiment employs a dilated convolution operator to extract multi-scale features. Specifically, a dilation ratio of... The dilated convolution operator adds r-1 zeros between each filter value. Without increasing the parameters, the dilated convolution operator can be viewed as an extension of the traditional convolution filter with more zero padding. This is combined with... Figure 5 The dilated convolution operator will be explained in detail. Figure 5 Examples of different voiding operators are shown in the figure. (a), (b), and (c) represent examples when r is 1, 2, and 3, respectively. Typically, an inflation ratio of r is used. Convolution kernels can be expanded to Convolution kernel. When r is 1, no 0 is added; in this case, the dilated convolution operator works in the same way as a traditional convolution filter.

[0043] Step S122: Extract convolutional features of the images at different scales according to the preset multi-scale convolutional feature extractor.

[0044] Step S123: Represent the convolutional features of the images at different scales using convolutional profile maps to obtain the multi-scale feature maps.

[0045] Understandably, a convolutional profile consists of a set of convolutional feature maps extracted by a series of convolution operators with different window sizes. This is combined with... Figure 6 Explanation of the convolution profile diagram. Figure 6 This is an example diagram of a convolutional profile. Let... This represents the input convolutional feature map. This represents a set of convolution operators with different window sizes, where , The set representing the convolution profile plots is expressed as: In the specific implementation, you can choose The convolution kernel uses the dilated convolution operator as its foundation, and the dilation rate r is used to control the window size. With n scales, the dilated convolution operator for a feature map of scale i is expressed as follows: The convolution profile can be represented as: Step S13: Fuse the multiple scale feature maps to obtain a fused multi-scale feature map.

[0046] In the specific implementation, the expansion ratio r takes the value of In multi-scale convolutional feature extractors In China, based on The convolution kernel size increases with the dilation ratio, and the window size becomes... Finally, the extracted multi-scale features are fused, and the expression is as follows: Where F represents the fused feature map, i.e., the fused multi-scale feature map. This indicates a fusion operation.

[0047] This embodiment performs image sampling processing on the acquired remote sensing image by inputting it into a preset image pyramid to obtain scaled images at different scales corresponding to the scaled remote sensing image. These scaled images are then sequentially input into a feature extractor and a feature aggregator for feature extraction and aggregation processing to obtain image features. Compared to existing technologies that directly reduce or crop the remote sensing image, resulting in reduced pixel information and affecting image classification, this embodiment achieves high-quality restoration and input of the acquired remote sensing image, ensuring that the features of the original image are preserved and used, further improving the accuracy and precision of remote sensing image scene classification.

[0048] refer to Figure 7 , Figure 7 This is a flowchart illustrating the third embodiment of the remote sensing image scene classification method based on multi-instance learning of the present invention.

[0049] Based on the above embodiments, in this embodiment, step S20 includes: Step S21: Input the fused multi-scale feature map into the example classifier and calculate the activation value at each position of the multi-scale feature map to obtain the activation value at each position, wherein the dimension of the activation value matches the dimension of the currently acquired remote sensing image.

[0050] Step S22: Each example is represented by a feature vector formed by the activation values ​​at each position, thus obtaining the example feature vector.

[0051] Step S23: Obtain the class prediction probability of each example in the multi-scale feature map.

[0052] In the specific implementation, the feature map F with multi-scale features calculates the activation value at each position (i,j) through convolution in the example classifier. And each activation value This forms a feature vector to represent an example. Since the example-level classifier outputs the class prediction of the example, The dimension needs to match the number of scene categories.

[0053] Further, step S30 includes: Step S31: Aggregate the class prediction probabilities of each example into bag prediction probabilities using a pooling model.

[0054] Understandably, this pooling model can obtain the attention weights of each example through the spatial attention model, and aggregate the class prediction probabilities of each example into bag prediction probabilities based on the attention weights of each example.

[0055] It should be noted that spatial attention mechanism is a resource allocation scheme among the main means of solving the information overload problem. It allocates computing resources to more important tasks. Based on spatial attention mechanism, the weight of objects in intuitive sense can be determined, which determines which part of the information is more important.

[0056] In the specific implementation, multiple example predictions can be aggregated into a single bag prediction using a preset pooling function. The preset pooling function is as follows: in, Predict the probability for the bag. Predict the probability for the example. This is a pre-defined pooling function; Furthermore, the preset pooling function is: in, This represents the attention weight.

[0057] Furthermore, the formula for calculating attention weights is: in, For example feature vectors, as well as These are the trainable weight parameters in the attention network. For the deviation parameter, use Functional attention weights The sum is 1.

[0058] Step S32: Determine the category of the remote sensing image based on the bag prediction probability.

[0059] Furthermore, prior to step S31, the following steps are also included: Step S31': Select a network loss function to optimize the loss of the pooling model.

[0060] It should be noted that a loss function is a function that maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. In applications, loss functions are often used as learning criteria in relation to optimization problems; that is, the model is solved and evaluated by minimizing the loss function. The loss function is used to evaluate the degree to which the model's predicted values ​​differ from the true values. Common loss functions include: 0-1 loss function, squared loss function, exponential loss function, Hinge loss function, and cross-entropy loss function. This embodiment does not limit itself to these; instead, the cross-entropy loss function is selected for illustration.

[0061] In the specific implementation, the cross-entropy loss function is chosen to minimize the loss between the bag prediction probability and the true label. After calculating the loss value during the forward propagation process, the entire model is optimized based on the loss value during the backward propagation process. The formula for calculating the cross-entropy loss function is as follows: Where L is the network loss function, and N is the number of scene categories. Indicates the true label of the category scenario. This indicates the predicted label for the category / scene.

[0062] It should be noted that the pooling model provided in this embodiment can be built on a network architecture capable of executing the remote sensing image scene classification method based on multi-instance learning of this invention. This is combined with... Figure 8 To explain, Figure 8 This is a network architecture diagram for implementing the remote sensing image scene classification method based on multi-instance learning of the present invention. Figure 8The model uses the first 13 convolutional layers of VGG-16 as the backbone network (CNNBackbone). First, a set of multi-scale feature extractors is constructed through a series of dilated convolution operators (scale 1, scale 2, ..., scale n) to extract multi-scale convolutional feature maps (features scale 1, features scale 2, ..., features scale n). Then, the multi-scale feature maps are fused and input into the class score. Finally, a multiple instance learning pooling model based on the attention mechanism is used to obtain the bag probability, thereby determining the class belonging to one of multiple predicted labels, such as forest, river, baseball diamond, chaparral, etc.

[0063] This embodiment proposes a MIL pooling method based on an attention mechanism. The example class probabilities obtained by the example classifier are aggregated into bag probabilities through a pooling model combined with an attention mechanism. The classic cross-entropy loss function is used in advance to minimize the loss between bag-level predictions and true labels. This solves the problem of classification difficulties caused by the richness and similarity of objects in the target region of remote sensing images in complex scenes, and the coexistence and interrelation of multiple objects, thereby further improving the classification accuracy of remote sensing scenes.

[0064] Furthermore, this embodiment of the invention also proposes a storage medium storing a remote sensing image scene classification program based on multiple instance learning. When the remote sensing image scene classification program based on multiple instance learning is executed by a processor, it implements the steps of the remote sensing image scene classification method based on multiple instance learning described above.

[0065] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0066] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0067] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0068] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A remote sensing image scene classification method based on multi-instance learning, characterized in that, The method includes: Extract multi-scale convolutional features from the currently acquired remote sensing image, and fuse the features to obtain a fused multi-scale feature map; The fused multi-scale feature map is input into the example classifier to obtain the class prediction probability of each example in the multi-scale feature map. Each example is a sub-region in the remote sensing image divided according to the target region object. The target region object is an object that is related to each other in the same scene. The class prediction probabilities of each example are aggregated into a bag prediction probability, and the category of the remote sensing image is determined based on the bag prediction probability. The bag prediction probability is obtained by weighting the class prediction probabilities of each example according to the attention weight of each example. The step of inputting the fused multi-scale feature map into an example classifier to obtain the class prediction probability of each example in the multi-scale feature map includes: The fused multi-scale feature map is input into the example classifier, and the activation value at each position of the multi-scale feature map is calculated to obtain the activation value at each position, wherein the dimension of the activation value matches the dimension of the currently acquired remote sensing image. Each example is represented by a feature vector formed by the activation values ​​at each position, thus obtaining the example feature vector; The class prediction probability of each example in the multi-scale feature map is obtained based on the feature vectors of each example.

2. The remote sensing image scene classification method based on multi-instance learning as described in claim 1, characterized in that, The process of extracting multi-scale convolutional features from the currently acquired remote sensing image and fusing these features to obtain a fused multi-scale feature map includes: Obtain scaled images of the currently acquired remote sensing image at different scales; Convolutional features of images at different scales are extracted by a preset multi-scale convolutional feature extractor to obtain multi-scale feature maps; The multiple scale feature maps are fused to obtain a fused multi-scale feature map.

3. The remote sensing image scene classification method based on multi-instance learning as described in claim 2, characterized in that, The step of extracting convolutional features from images of different scales using a preset multi-scale convolutional feature extractor to obtain multi-scale feature maps includes: The preset multi-scale convolutional feature extractor is constructed using the dilated convolution operator; The convolutional features of the images at different scales are extracted according to the preset multi-scale convolutional feature extractor; The convolutional features of the images at different scales are represented by convolutional profile maps to obtain the multi-scale feature maps.

4. The remote sensing image scene classification method based on multi-instance learning as described in claim 1, characterized in that, The step of aggregating the class prediction probabilities of the various examples into bag prediction probabilities, and determining the category of the remote sensing image based on the bag prediction probabilities, includes: The class prediction probabilities of the individual examples are aggregated into bag prediction probabilities using a pooling model. The category of the remote sensing image is determined based on the bag prediction probability.

5. The remote sensing image scene classification method based on multi-instance learning as described in claim 4, characterized in that, The aggregation of class prediction probabilities of the various examples into bag prediction probabilities using a pooling model includes: The attention weights for each example are obtained using a spatial attention model; The class prediction probabilities of the examples are aggregated into bag prediction probabilities based on the attention weights of the examples.

6. The remote sensing image scene classification method based on multi-instance learning as described in claim 4, characterized in that, The aggregation of class prediction probabilities of the various examples into bag prediction probabilities using a pooling model includes: Multiple example predictions are aggregated into a single bag prediction using a pre-defined pooling function; The preset pooling function is: In the formula, For bag prediction, For example prediction, This represents the attention weight.

7. The remote sensing image scene classification method based on multi-instance learning as described in claim 6, characterized in that, The attention weights are calculated using an attention network. The formula for calculating the attention weight is: In the formula, For example feature vectors, as well as Let be the trainable weight parameters in the attention network, and b be the bias parameter.

8. The remote sensing image scene classification method based on multi-instance learning as described in claim 4, characterized in that, Before the step of aggregating the class prediction probabilities of the individual examples into bag prediction probabilities using a pooling model, the following steps are included: The pooling model is optimized by selecting a network loss function. The network loss function is used to minimize the loss between the bag prediction probability and the true label. The network loss function is: In the formula, L is the network loss function, and N is the number of scene categories. Indicates the true label of the category scenario. This indicates the predicted label for the category / scene.

9. A storage medium, characterized in that, The storage medium stores a remote sensing image scene classification program based on multiple instance learning. When the remote sensing image scene classification program based on multiple instance learning is executed by the processor, it implements the steps of the remote sensing image scene classification method based on multiple instance learning as described in any one of claims 1 to 8.

Images