Remote sensing image scene classification method and system based on joint trusted learning

Abstract

The invention belongs to the field of intelligent monitoring technology and remote sensing detection cross technology, and relates to a remote sensing image scene classification method and system based on joint credible learning. The system comprises remote sensing image acquisition equipment and a data processor, wherein the data processor comprises a generative feature extraction module, a discriminant feature extraction module, an uncertainty quantification module and a credible decision fusion module. The method comprises the steps of obtaining a remote sensing scene image of a target area; generating features and discriminant features are extracted from the obtained remote sensing scene image; using an uncertainty quantification method based on a subjective logic theory to estimate uncertainty contained in the generative features and the discriminant features for scene classification; and classifying scenes in the remote sensing scene image by using a decision-level fusion method based on a D-S evidence theory combination rule according to an uncertainty quantization result. According to the invention, the target scene in the remote sensing image can be quickly and accurately classified, and the intelligent monitoring capability of the system is improved.

Description

Technical Field

[0001] The present invention belongs to the cross-technical field of intelligent monitoring technology and remote sensing detection, and relates to a remote sensing image scene classification method and system for monitoring the surface of the earth, which is oriented to optical remote sensing sensors. Background Art

[0002] Remote sensing image scene classification is a technology that uses remote sensing technology to analyze and classify surface images. It can quickly and accurately obtain information about objects in large areas and realize dynamic monitoring and analysis of surface changes. It is of great significance for scientific research, resource management, environmental protection, and response to natural disasters. However, due to the earth observation perspective of remote sensing images, there are many fuzzy targets with similar visual features in the images. Therefore, in order to achieve accurate remote sensing image scene classification, it is necessary to extract features that can capture the differences between different categories and the similarities within the same category.

[0003] Generative methods and discriminative methods are two types of data-driven deep feature learning methods for improving the accuracy of remote sensing image scene classification. The former aims to capture the global semantics and local details in the image by encoding and decoding the input and reconstructing the input with high fidelity in an unsupervised manner; the latter extracts the most significant feature semantics in the image by minimizing the empirical error between the predicted probability and the true value label.

[0004] Both the generative method and the discriminative method are currently the most advanced remote sensing image scene classification methods. The generative features and discriminative features of remote sensing images extracted by them are crucial for scene classification. However, the potential synergistic effect of these two types of features has not been studied in this field, resulting in deficiencies in the comprehensiveness and effectiveness of the extracted features in the existing methods, which limits the further development of remote sensing image scene classification performance. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the purpose of the present invention is to provide a remote sensing image scene classification method and system based on joint credible learning. The present invention adopts remote sensing image scene classification based on generative and discriminative methods, and a joint credible learning method, by quantifying the uncertainty of generative and discriminative features, and credibly fusing the scene recognition results based on these two features, which can greatly increase the credibility of remote sensing image scene classification.

[0006] The present invention discovers the complementary relationship between the generative feature learning method and the discriminative feature learning method of remote sensing images through research, thereby providing a method for integrating and comprehensively expressing the generative and discriminative features of remote sensing images by using joint trusted learning, and on this basis proposes a scene classification method with outstanding effectiveness and reliability, which can integrate the respective advantages of the generative and discriminative methods from optical remote sensing image data to improve its scene classification accuracy.

[0007] The technical solution of the present invention is:

[0008] A remote sensing image scene classification method based on joint trusted learning, the steps of which include:

[0009] Detect the target area and obtain the remote sensing scene image of the target area;

[0010] Extracting generative features and discriminative features from the acquired remote sensing scene images of the target area;

[0011] Use uncertainty quantification methods based on subjective logic theory to estimate the uncertainty contained in generative and discriminative features for scene classification;

[0012] According to the uncertainty quantification results, the scenes in the remote sensing scene images are classified using a decision-level fusion method based on the combination rules of DS evidence theory.

[0013] Furthermore, the target area is detected using a remote sensing image acquisition device, and the remote sensing scene image of the target area is obtained and sent to a data processor; the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module, and a trusted decision fusion module.

[0014] Furthermore, the generative feature extraction module and the discriminative feature extraction module in the data processor respectively extract features from the input remote sensing image data set, and the remote sensing image data set is recorded as Where (x i ,y i ) represents the i-th sample and its label, N represents the total number of samples, and the process can be expressed as f g (x) and f d (x), where x represents the input sample, f g and f d They respectively represent the feature mapping functions of the pre-trained generative model (such as the convolutional autoencoder model) and the discriminative model (such as the ResNet18 model) on the remote sensing image, thus obtaining the image features of the same scene under two different feature extractors.

[0015] Furthermore, the uncertainty quantification module uses an uncertainty quantification method based on subjective logic theory to quantify the uncertainty of generative features and discriminative features for scene recognition respectively; the specific steps include:

[0016] First, the classification evidence obtained by calculating the generative features and discriminative features: g (x) = log(1 + exp(f g (x))),e d (x) = log(1 + exp(f d (x)));

[0017] Then, using evidence g (x),e d (x) is used to calculate the belief quality b and overall uncertainty u of generative features and discriminative features: u g (x i )=K / S g (x i ),u d (x i )=K / S d (x i ),in The parameter α of the Dirichlet distribution is then expressed as in, Represents input x i The confidence of the generative model in predicting the jth category, K represents the total number of categories, Represents input x i When is the evidence of the generative model's prediction of the jth category, Represents input x i The confidence of the discriminant model for the prediction result of the jth category, Represents input x i When the discriminative model predicts the jth category, u g (x i ) represents the input x i The uncertainty of the generative model prediction, u d (x i ) represents the input x i The uncertainty of the prediction of the time-discriminative model;

[0018] Then, the loss function of the uncertainty quantification model is optimized based on the above definition:

[0019]

[0020] where ψ(·) is the digamma function; y ij For label y iThe jth dimension of and The input samples x are i Then the parameter α is calculated g and α d The jth dimension of . Use the corresponding model optimization algorithm to optimize L, so that the uncertainty quantification module learns and quantifies the uncertainty of the features for the remote sensing image scene recognition results from the generative features and discriminant features. and

[0021] Furthermore, the trusted decision fusion module uses a decision-level fusion method based on the DS evidence theory combination rule to fuse the decision results of the generative model and the discriminative model. The specific calculation process is: Among them, b j (x i ) represents the input x i The confidence of the model in predicting the jth category, u(x i ) represents the input x i The uncertainty of the model prediction, C(x i ) represents the input x i This step not only achieves the classification result of the scene category in the remote sensing image (i.e. the prediction result of the label y of the input image x), but also obtains the quantitative estimation of the confidence and uncertainty of the classification result (i.e. ).

[0022] Preferably, the remote sensing image acquisition device is a network camera.

[0023] Preferably, the data processor is a computer or an embedded motherboard.

[0024] Preferably, the generative feature extraction module is a convolutional autoencoder pre-training model.

[0025] Preferably, the discriminant feature extraction module is a ResNet18 pre-trained model.

[0026] A remote sensing image scene classification system based on joint trusted learning, comprising a remote sensing image acquisition device and a data processor; the data processor comprises a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module and a trusted decision fusion module; the remote sensing image acquisition device is connected to the generative feature extraction module and the discriminative feature extraction module in the data processor; the generative feature extraction module and the discriminative feature extraction module in the data processor are connected to the uncertainty quantification module in the data processor, and the uncertainty quantification module in the data processor is connected to the trusted decision fusion module in the data processor;

[0027] The remote sensing image acquisition device detects the target area, obtains the remote sensing scene image of the target area and sends it to the data processor;

[0028] The generative feature extraction module uses a remote sensing image scene feature extraction method based on generative deep learning to extract features from optical remote sensing images to obtain generative features;

[0029] The discriminant feature extraction module uses a remote sensing image scene feature extraction method based on discriminant deep learning to extract features from optical remote sensing images to obtain discriminant features;

[0030] The uncertainty quantification module uses an uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative and discriminant features for scene classification;

[0031] The trusted decision fusion module classifies scenes in optical remote sensing images according to uncertainty quantification results and adopts a decision-level fusion method based on DS evidence theory combination rules.

[0032] The specific implementation of each module refers to the above description of the method of the present invention.

[0033] The beneficial effects of the above technical solution of the present invention are as follows:

[0034] In the above scheme, the remote sensing image acquisition device can pre-process the data to produce optical remote sensing images, and the discriminative feature extraction module adopts a remote sensing image feature extraction method based on discriminative deep learning, which can quickly and accurately classify the target scene. Compared with the discriminative scene classification method, it introduces effective information in the generative features, and uses uncertainty quantification to express the uncertainty and conflict in the two types of features. Through trusted decision fusion, a higher scene classification accuracy is obtained, which has high robustness and reliability in different scenarios. Through the trusted fusion of generative and discriminative joint trusted learning, the accuracy of remote sensing image scene classification can be greatly increased, and the intelligent monitoring capability of the system is improved. At the same time, the data processor can be an embedded motherboard, which can be built on a variety of intelligent monitoring devices, improving the portability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the remote sensing image scene classification method based on joint trusted learning of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below through examples and drawings.

[0037] See also Figure 1 An embodiment of the present invention provides a remote sensing image scene classification system based on joint trusted learning, the system comprising a remote sensing image acquisition device 1 and a data processor 2; the data processor 2 comprises: a generative feature extraction module 3, a discriminative feature extraction module 4, an uncertainty quantification module 5 and a trusted decision fusion module 6; the uncertainty quantification module 5 is connected to the trusted decision fusion module 6, the generative feature extraction module 3 and the discriminative feature extraction module 4 are connected to the uncertainty quantification module 5; the remote sensing image acquisition device 1 is connected to the generative feature extraction module 3 and the discriminative feature extraction module 4 in the data processor 2.

[0038] In the above embodiment, the remote sensing image acquisition device 1 may be a camera or other image acquisition devices.

[0039] In the above embodiment, the data processor 2 may be a computer or an embedded motherboard.

[0040] In the above embodiment, the generative feature extraction module 3 may use a deep learning method to extract features from the optical remote sensing image.

[0041] In the above embodiment, the discriminant feature extraction module 4 may use a deep learning method to extract features from the optical remote sensing image.

[0042] In the above embodiment, the uncertainty quantification module 5 may use an uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative and discriminant features for scene classification.

[0043] In the above embodiment, the trusted decision fusion module 6 may classify the scenes in the optical remote sensing image by using a decision-level fusion method based on the DS evidence theory combination rule.

[0044] A remote sensing image scene classification method based on joint trusted learning in an embodiment of the present invention has the following specific workflow:

[0045] First, the remote sensing image acquisition device detects the target area to obtain an optical remote sensing image of the target scene and inputs it into the data processor 2.

[0046] Secondly, the generative feature extraction module 3 and the discriminative feature extraction module 4 in the data processor 2 are used to extract the input remote sensing image data set, which is recorded as The feature extraction process can be expressed as f g (x) and f d (x), where f g and f d They respectively represent the feature mapping functions of the pre-trained generative model (such as the convolutional autoencoder model) and the discriminative model (such as the ResNet18 model) on the remote sensing image, which obtains the image features of the same scene under two different feature extractors.

[0047] Then, the uncertainty quantification module 5 uses the uncertainty quantification method based on subjective logic theory to quantify the uncertainty of the generative and discriminative features in the input scene recognition. The classification evidence obtained by calculating the generative features and the discriminative features is: g (x) = log(1 + exp(f g (x))),e d (x) = log(1 + exp(f d Without loss of generality, the output of the generative model and the discriminative model is expressed as the predicted probability distribution σ = [σ 1 ,…,σ K ], where K represents the total number of mutually exclusive categories, and each σ j ∈[0,1], and Σ j σ j = 1. According to subjective logic theory, the probability is assigned belief quality b to reflect the confidence in different events, which is specifically defined as where u represents the overall uncertainty and u,b j ≥0. Using evidence e=[e 1 ,…,e K] to calculate belief quality and overall uncertainty: b j =e j / S,u=K / S,where S=Σ j (e j +1), thus expressing the relationship between the predicted probability distribution and the evidence as:

[0048] Dir is the Dirichlet distribution, α is the parameter of the Dirichlet distribution and α j =e j +1; B(α) is the K-dimensional polynomial beta function, S K is the unit simplex of dimension K:

[0049]

[0050] Classification evidence using generative and discriminative features g (x i ),e d (x i ) Calculate the corresponding belief quality Overall uncertainty u g (x i ),u d (x i ), parameter S g and S d , and parameter α g and α d , thus obtaining the loss function required for the model learning:

[0051]

[0052] Among them, ψ(·) is the digamma function, and The input samples x are i Then the parameter α is calculated g and α d The jth dimension of y ij For label y i The jth dimension of . Use the corresponding model optimization algorithm to optimize L, prompting the uncertainty quantification module 5 to learn and quantify the uncertainty of the features for the remote sensing image scene recognition results from the generative and discriminative features and

[0053] Finally, the trusted decision fusion module 6 uses a decision-level fusion method based on the DS evidence theory combination rule to fuse the decision results of the generative and discriminative models. The specific calculation process is:

[0054]

[0055] After the above steps, while achieving the classification of scene categories in remote sensing images, quantitative estimates of the confidence and uncertainty of the classification results are also obtained.

[0056] A remote sensing image scene classification method and system based on joint trusted learning in an embodiment of the present invention can quickly classify and process scene-level remote sensing images acquired by a remote sensing image acquisition device. A generative feature extraction module 3 and a discriminative feature extraction module 4 use a deep learning method to extract two different types of remote sensing image features. An uncertainty quantification module 5 estimates the uncertainty in generative and discriminative remote sensing image features, and uses a trusted decision fusion module 6 to fuse the decision results of the two types of features for scene classification. By explicitly modeling and optimizing the confidence and uncertainty of the decision to obtain a higher accuracy rate, the system has high robustness and reliability in different scenarios, thereby improving the intelligent monitoring capability of the system.

[0057] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0058] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0059] In order to illustrate the purpose of the present invention, the above embodiments are only for description and not for limiting the scope of the present invention. The scope of the present invention is determined by the claims, and any equivalent substitutions and modifications that do not deviate from the principles and core features of the embodiments of the present invention should be included in the scope of the present invention.

[0060] It is obvious to those skilled in the art that the specific forms of the embodiments of the present invention are not limited to the details in the above exemplary embodiments. Therefore, these embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is limited by the claims rather than the above description. Any figure mark should not be regarded as limiting the claims. In addition, the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules or devices can be implemented by the same unit, module or device through software or hardware.

[0061] Finally, although the above preferred embodiments are described in detail, they are only for describing the technical solutions of the embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art should understand that the technical solutions may be modified or replaced by equivalents, but should not depart from the principles and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image scene classification method based on joint trusted learning, characterized in that: The following steps are involved: Detect the target area and obtain the remote sensing scene image of the target area; Extracting generative features and discriminative features from the acquired remote sensing scene images of the target area; Use uncertainty quantification methods based on subjective logic theory to estimate the uncertainty contained in generative and discriminative features for scene classification; According to the uncertainty quantification results, the scenes in the remote sensing scene images are classified using a decision-level fusion method based on the DS evidence theory combination rule.

2. The method according to claim 1, characterized in that The target area is detected by using a remote sensing image acquisition device, and the acquired remote sensing scene image of the target area is sent to a data processor; the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module, and a trusted decision fusion module.

3. The method according to claim 2, characterized in that The generative feature extraction module and the discriminative feature extraction module respectively extract features from the input remote sensing image dataset, and the remote sensing image dataset is denoted as Where (x i ,y i ) represents the i-th sample and its label, N represents the total number of samples, and the feature extraction process is represented as f g (x) and f d (x), where x represents the input sample, f g and f d They represent the feature mapping functions of the generative model and the discriminative model pre-trained on the remote sensing image, respectively, so as to obtain the image features of the same scene under two different feature extraction modules.

4. The method according to claim 2, characterized in that: The uncertainty quantification module uses an uncertainty quantification method based on subjective logic theory, and its steps include: First, calculate the classification evidence obtained by generative features and discriminative features: e g (x)=log(1+exp(f g (x))),e d (x)=log(1+exp(f d (x))); Then, using evidence g (x),e d (x) is used to calculate the belief quality b and overall uncertainty u of generative features and discriminative features: u g (x i )=K / S g (x i ),u d (x i )=K / S d (x i ),in thereby The parameter α of the Dirichlet distribution is expressed as in, Represents input x i The confidence of the generative model in predicting the jth category, K represents the total number of categories, Represents input x i When is the evidence of the generative model's prediction of the jth category, Represents input x i The confidence of the discriminant model for the prediction result of the jth category, Represents input x i When the discriminative model predicts the jth category, u g (x i ) represents the input x i The uncertainty of the generative model prediction, u d (x i ) represents the input x i The uncertainty of the prediction of the time-discriminative model; Then, the loss function of the uncertainty quantification model is optimized: Where ψ(·) is the double gamma function, y ij For label y i The jth dimension of and The input samples x are i Then the parameter α is calculated g and α d The jth dimension of ; L is optimized using a model optimization algorithm, prompting the uncertainty quantification module to learn and quantify the uncertainty of features for remote sensing image scene recognition results from generative features and discriminative features and 5. The method according to claim 2, characterized in that: The trusted decision fusion module uses a decision-level fusion method based on the DS evidence theory combination rule to fuse the decision results of the generative model and the discriminant model to obtain The calculation process is: Among them, b j (x i ) represents the confidence of the model for the prediction result of the jth category when input xi is given, u(x i ) represents the input x i The uncertainty of the model prediction, C(x i ) represents the input x i It is a measure of the degree of conflict between the output results of the generative model and the discriminative model.

6. The method according to claim 1, characterized in that The remote sensing image acquisition device is a network camera, and the data processor is a computer or an embedded motherboard.

7. The method according to claim 1, characterized in that The generative feature extraction module is a convolutional autoencoder pre-training model, and the discriminative feature extraction module is a ResNet18 pre-training model.

8. A remote sensing image scene classification system based on joint trusted learning, characterized in that: It includes a remote sensing image acquisition device and a data processor; the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module and a trusted decision fusion module; The remote sensing image acquisition device detects the target area, obtains the remote sensing scene image of the target area and sends it to the data processor; The generative feature extraction module uses a remote sensing image scene feature extraction method based on generative deep learning to extract features from optical remote sensing images to obtain generative features; The discriminant feature extraction module uses a remote sensing image scene feature extraction method based on discriminant deep learning to extract features from optical remote sensing images to obtain discriminant features; The uncertainty quantification module uses an uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative features and discriminant features for scene classification; The trusted decision fusion module classifies scenes in optical remote sensing images according to uncertainty quantification results and adopts a decision-level fusion method based on DS evidence theory combination rules.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images