Remote Sensing Image Scene Classification Method and System Based on Federated Trusted Learning

Abstract

The present invention belongs to the cross - technical field of intelligent monitoring technology and remote sensing detection, and relates to a method and system for remote sensing image scene classification based on joint trusted learning. The present invention includes a remote sensing image acquisition device and a data processor, and the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module, and a trusted decision fusion module. The method includes: acquiring a remote sensing scene image of a target area; extracting generative features and discriminative features from the acquired remote sensing scene image; using an uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative features and discriminative features for scene classification; and classifying the scene in the remote sensing scene image using a decision - level fusion method based on the combination rule of D - S evidence theory. The present invention can quickly and accurately classify the target scene in the remote sensing image, and improves the intelligent monitoring ability of the system.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of intelligent monitoring technology and remote sensing detection, and relates to a remote sensing image scene classification method and system for monitoring the ground surface, which is oriented to optical remote sensing sensors. Background Art

[0002] Remote sensing image scene classification is a technology that uses remote sensing technology to analyze and classify surface images. It can quickly and accurately obtain ground object information in large - area regions, realize dynamic monitoring and analysis of surface changes, and is of great significance for scientific research, resource management, environmental protection, and response to natural disasters. However, limited by the ground - observation perspective of remote sensing images, there are many fuzzy targets with similar visual features in the images. Therefore, to achieve accurate remote sensing image scene classification, features that can capture the differences between different categories and the similarities within the same category must be extracted.

[0003] For improving the accuracy of remote sensing image scene classification, generative methods and discriminative methods are two types of data - driven deep feature learning methods. The former aims to reconstruct the input with high fidelity in an unsupervised manner by encoding and decoding the input, so as to capture the global semantics and local details in the image; the latter extracts the most significant feature semantics in the image by minimizing the empirical error between the predicted probability and the true label.

[0004] Both generative methods and discriminative methods are currently the most advanced remote sensing image scene classification methods. The generative features and discriminative features extracted from remote sensing images are crucial for scene classification. However, the potential synergistic effect between these two types of features has not been studied in this field, resulting in deficiencies in the comprehensiveness and effectiveness of the features extracted by existing methods, and restricting the further development of remote sensing image scene classification performance. Summary of the Invention

[0005] Aiming at the technical problems existing in the prior art, the purpose of the present invention is to provide a remote sensing image scene classification method and system based on joint credible learning. The present invention adopts remote sensing image scene classification based on generative and discriminative methods, as well as a joint credible learning method. By quantifying the uncertainty of generative and discriminative features, the scene recognition results based on these two types of features are credibly fused, which can greatly increase the credibility of remote sensing image scene classification.

[0006] Through research, the present invention discovers the complementary association between the generative feature learning method and the discriminative feature learning method of remote sensing images, thereby providing a method for fusing and comprehensively expressing the generative and discriminative features of remote sensing images using joint credible learning, and based on this, proposes a scene classification method with outstanding effectiveness and reliability, which can synthesize the respective advantages of the generative and discriminative methods from optical remote sensing image data to improve the accuracy of its scene classification.

[0007] The technical solution of the present invention is as follows:

[0008] A remote sensing image scene classification method based on joint credible learning, the steps of which include:

[0009] Detect the target area to obtain the remote sensing scene image of the target area;

[0010] Extract the generative features and discriminative features from the obtained remote sensing scene image of the target area;

[0011] Use the uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative features and discriminative features for scene classification;

[0012] According to the uncertainty quantification result, use the decision-level fusion method based on the D-S evidence theory combination rule to classify the scene in the remote sensing scene image.

[0013] Furthermore, use the remote sensing image acquisition device to detect the target area, obtain the remote sensing scene image of the target area and send it to the data processor; the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module, and a credible decision fusion module.

[0014] Furthermore, the generative feature extraction module and the discriminative feature extraction module in the data processor respectively perform feature extraction on the input remote sensing image data set, and this remote sensing image data set is denoted as where (x i , y i ) represents the i-th sample and its label, N represents the total number of samples, and this process can be expressed as f g (x) and f d (x), where x represents the input sample, and f g and f d respectively represent the feature mapping functions of the pre-trained generative model (such as a convolutional autoencoder model) and discriminative model (such as a ResNet18 model) on the remote sensing image, and thus the image features of the same scene under two different feature extractors are obtained.

[0015] Furthermore, the uncertainty quantification module uses an uncertainty quantification method based on subjective logic theory to quantify the uncertainties of generative features and discriminative features for scene recognition respectively. The specific steps are as follows:

[0016] First, calculate the classification evidence obtained from generative features and discriminative features: e g (x) = log(1 + exp(f g (x))), e d (x) = log(1 + exp(f d (x)));

[0017] Then, use the evidence e g (x), e d (x) to calculate the belief mass b and the overall uncertainty u of generative features and discriminative features: u g (x i ) = K / S g (x i ), u d (x i ) = K / S d (x i ), where Thus, represent the parameter α of the Dirichlet distribution as where represents the confidence of the generative model's prediction result for the j-th category when the input is x i , K represents the total number of categories, represents the evidence of the generative model's prediction result for the j-th category when the input is x i , represents the confidence of the discriminative model's prediction result for the j-th category when the input is x i , represents the evidence of the discriminative model's prediction result for the j-th category when the input is x i , u g (x i ) represents the uncertainty of the generative model's prediction when the input is x i , u d (x i ) represents the uncertainty of the discriminative model's prediction when the input is x i ;

[0018] Then, optimize the loss function of the uncertainty quantification model based on the above definitions:

[0019]

[0020] where ψ(·) is the digamma function; y ij is the label y iThe j-th dimension of and are respectively the input sample x i After calculation, the parameter α g and α d The j-th dimension. Use the corresponding model optimization algorithm to optimize L, prompting the uncertainty quantification module to learn from the generative features and discriminative features and quantify the uncertainty of the features for the remote sensing image scene recognition result and

[0021] Furthermore, the reliable decision fusion module uses a decision-level fusion method based on the D-S evidence theory combination rule to fuse the decision results of the generative model and the discriminative model The specific calculation process is as follows: where b j (x i ) represents the confidence of the model in the prediction result of the j-th category when the input is x i u(x i ) represents the uncertainty predicted by the model when the input is x i C(x i ) represents the measure of the conflict degree between the output results of the generative and discriminative models when the input is x i . This step realizes the classification result of the scene category in the remote sensing image (i.e., the prediction result of the label y of the input image x), and at the same time obtains the quantitative estimation of the confidence and uncertainty of the classification result (i.e., ).

[0022] Preferably, the remote sensing image acquisition device is a network camera.

[0023] Preferably, the data processor is a computer or an embedded mainboard.

[0024] Preferably, the generative feature extraction module is a convolutional autoencoder pre-training model.

[0025] Preferably, the discriminative feature extraction module is a ResNet18 pre-training model.

[0026] A remote sensing image scene classification system based on federated trusted learning, comprising a remote sensing image acquisition device and a data processor; the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module, and a trusted decision fusion module; the remote sensing image acquisition device is connected to the generative feature extraction module and the discriminative feature extraction module in the data processor; the generative feature extraction module and the discriminative feature extraction module in the data processor are connected to the uncertainty quantification module in the data processor, and the uncertainty quantification module in the data processor is connected to the trusted decision fusion module in the data processor;

[0027] The remote sensing image acquisition device detects a target area, acquires a remote sensing scene image of the target area and sends it to the data processor;

[0028] The generative feature extraction module uses a remote sensing image scene feature extraction method based on generative deep learning to extract features from an optical remote sensing image to obtain generative features;

[0029] The discriminative feature extraction module uses a remote sensing image scene feature extraction method based on discriminative deep learning to extract features from an optical remote sensing image to obtain discriminative features;

[0030] The uncertainty quantification module uses an uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative and discriminative features for scene classification;

[0031] The trusted decision fusion module classifies the scene in the optical remote sensing image by using a decision-level fusion method based on the D-S evidence theory combination rule according to the uncertainty quantification result.

[0032] For the specific implementation manners of each module, refer to the description of the method of the present invention above.

[0033] The beneficial effects of the above technical solutions of the present invention are as follows:

[0034] In the above solution, the remote sensing image acquisition device can generate optical remote sensing images through data preprocessing. The discriminative feature extraction module adopts a remote sensing image feature extraction method based on discriminative deep learning, which can quickly and accurately classify target scenes. Moreover, compared with the discriminative scene classification method, it introduces the effective information in the generative features and uses uncertainty quantification to express the uncertainty and conflict in the two types of features, and obtains a higher scene classification accuracy through credible decision fusion, and has high robustness and reliability in different scenes. Through the credible fusion of generative and discriminative joint credible learning for the two types of features, the accuracy of remote sensing image scene classification can be greatly increased, and the intelligent monitoring ability of the system is improved. At the same time, the data processor can be an embedded mainboard, which can be built on a variety of intelligent monitoring devices, improving the portability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 FIG. is a schematic diagram of the remote sensing image scene classification method based on joint credible learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The present invention will be further described in detail below through embodiments and drawings.

[0037] See Figure 1 , an embodiment of the present invention provides a remote sensing image scene classification system based on joint credible learning. The system includes a remote sensing image acquisition device 1 and a data processor 2; the data processor 2 includes: a generative feature extraction module 3, a discriminative feature extraction module 4, an uncertainty quantification module 5, and a credible decision fusion module 6; the uncertainty quantification module 5 is connected to the credible decision fusion module 6, and the generative feature extraction module 3 and the discriminative feature extraction module 4 are connected to the uncertainty quantification module 5; the remote sensing image acquisition device 1 is connected to the generative feature extraction module 3 and the discriminative feature extraction module 4 in the data processor 2.

[0038] In the above embodiment, the remote sensing image acquisition device 1 can be a camera or other image acquisition devices.

[0039] In the above embodiment, the data processor 2 can be a computer or an embedded mainboard.

[0040] In the above embodiment, the generative feature extraction module 3 can adopt a deep learning method to extract features from optical remote sensing images.

[0041] In the above embodiment, the discriminative feature extraction module 4 can adopt a deep learning method to extract features from optical remote sensing images.

[0042] In the above embodiments, the uncertainty quantification module 5 may adopt an uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative and discriminative features for scene classification.

[0043] In the above embodiments, the reliable decision fusion module 6 may adopt a decision-level fusion method based on the combination rule of D-S evidence theory to classify the scenes in the optical remote sensing images.

[0044] A method for remote sensing image scene classification based on joint reliable learning according to an embodiment of the present invention has the following specific working process:

[0045] First, the remote sensing image acquisition device detects the target area to obtain the optical remote sensing image of the target scene and inputs it into the data processor 2.

[0046] Secondly, the generative feature extraction module 3 and the discriminative feature extraction module 4 in the data processor 2 perform feature extraction on the input remote sensing image data set, denoted as respectively. This process can be expressed as f g (x) and f d (x), where f g and f d respectively represent the feature mapping functions of the pre-trained generative (such as convolutional autoencoder model) and discriminative (such as ResNet18 model) on the remote sensing image, and thus the image features of the same scene under two different feature extractors are obtained.

[0047] Then, the uncertainty quantification module 5 uses an uncertainty quantification method based on subjective logic theory to quantify the uncertainty of the generative and discriminative features in the recognition of the input scene respectively. Calculate the classification evidence obtained from the generative feature and the discriminative feature: e g (x) = log(1 + exp(f g (x))), e d (x) = log(1 + exp(f d (x))). Without loss of generality, represent the outputs of the generative model and the discriminative model as the predicted probability distribution σ = [σ1,..., σ K , where K represents the total number of mutually exclusive categories, each σ j ∈[0, 1], and Σ j σ j = 1. According to the subjective logic theory, assign belief mass b to the probability to reflect the confidence in different events, and the specific definition is where u represents the overall uncertainty and u, b j ≥0. Use the evidence e = [e1,..., e K to calculate the belief mass and the overall uncertainty: bj = e j / S, u = K / S, where S = Σ j (e j + 1), thus representing the relationship between the predicted probability distribution and the evidence as:

[0048] Dir is the Dirichlet distribution, α is the parameter of the Dirichlet distribution and α j = e j + 1; B(α) is the K - dimensional multinomial beta function, S K is the K - dimensional unit simplex:

[0049]

[0050] Using the classification evidence e g (x i ), e d (x i ) to calculate the corresponding belief mass The overall uncertainty u g (x i ), u d (x i ), the parameter S g and S d , as well as the parameters α g and α d , thus obtaining the loss function required for the model to learn:

[0051]

[0052] where, ψ(·) is the digamma function, and are respectively the j - th dimension of the parameters α i and α g calculated after inputting the sample x d , y ij is the j - th dimension of the label y i . Using the corresponding model optimization algorithm to optimize L, prompting the uncertainty quantification module 5 to learn from the generative and discriminative features and quantify the uncertainty of the features for the remote sensing image scene recognition result and

[0053] Finally, the credible decision - making fusion module 6 uses the decision - level fusion method based on the D - S evidence theory combination rule to fuse the decision results of the generative and discriminative models The specific calculation process is as follows:

[0054]

[0055] After the above steps, while realizing the classification of scene categories in remote sensing images, a quantitative estimation of the confidence and uncertainty of the classification results is also obtained.

[0056] A method and system for remote sensing image scene classification based on joint credible learning according to an embodiment of the present invention can quickly classify scene-level remote sensing images acquired by a remote sensing image acquisition device. The generative feature extraction module 3 and the discriminative feature extraction module 4 use deep learning methods to extract two different types of remote sensing image features. The uncertainty quantification module 5 estimates the uncertainty in the generative and discriminative remote sensing image features, and uses the credible decision fusion module 6 to fuse the decision results of the two types of features for scene classification. By explicitly modeling and optimizing the confidence and uncertainty of the decision, a higher accuracy is obtained, and it has high robustness and reliability in different scenarios, improving the intelligent monitoring ability of the system.

[0057] Another embodiment of the present invention provides a computer device (such as a computer, a server, a smart phone, etc.), which includes a memory and a processor. The memory stores a computer program, and the computer program is configured to be executed by the processor. The computer program includes instructions for executing the steps in the method of the present invention.

[0058] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, a disk, an optical disc). The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the various steps of the method of the present invention are implemented.

[0059] For the purpose of clarifying the object of the present invention, the above embodiments are only for description and do not limit the scope of the present invention. The scope of the present invention is determined by the claims, and any equivalent substitutions and modifications that do not deviate from the principles and core features of the embodiments of the present invention should be included in the scope of the present invention.

[0060] For those skilled in the art, it is obvious that the specific forms of the embodiments of the present invention are not limited to the details in the above exemplary embodiments. Therefore, these embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the claims rather than the above description. Any reference signs should not be regarded as limiting the claims. In addition, the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules or devices can be implemented by the same unit, module or device through software or hardware.

[0061] Finally, although the above best embodiments have been described in detail, it is only for describing the technical solutions of the embodiments of the present invention rather than limiting. Those skilled in the art should understand that the technical solutions can be modified or equivalently substituted, but should not deviate from the principles and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image scene classification method based on joint trusted learning, characterized in that: The following steps are involved: Detect the target area and obtain the remote sensing scene image of the target area; Extracting generative features and discriminative features from the acquired remote sensing scene images of the target area; Use uncertainty quantification methods based on subjective logic theory to estimate the uncertainty contained in generative and discriminative features for scene classification; According to the uncertainty quantification results, the scenes in the remote sensing scene images are classified using the decision-level fusion method based on the DS evidence theory combination rule; The steps of the uncertainty quantification method based on subjective logic theory include: First, calculate the classification evidence obtained by generative features and discriminative features: e g (x)=log(1+exp(f g (x))),e d (x)=log(1+exp(f d (x))); Then, using evidence g (x),e d (x) is used to calculate the belief quality b and overall uncertainty u of generative features and discriminative features: u g (x i )=K / S g (x i ),u d (x i )=K / S d (x i ),in thereby The parameter α of the Dirichlet distribution is expressed as in, Represents input x i The confidence of the generative model in predicting the jth category, K represents the total number of categories, Represents input x i When is the evidence of the generative model's prediction of the jth category, Represents input x i The confidence of the discriminant model for the prediction result of the jth category, Represents input x i When the discriminative model predicts the jth category, u g (x i ) represents the input x i The uncertainty of the generative model prediction, u d (x i ) represents the input x i The uncertainty of the prediction of the time-discriminative model; Then, the loss function of the uncertainty quantification model is optimized: Among them, ψ(·) is the double gamma function, y ij For label y i The jth dimension of and The input samples x are i Then the parameter α is calculated g and α d The jth dimension of ; L is optimized using a model optimization algorithm, prompting the uncertainty quantification module to learn and quantify the uncertainty of features for remote sensing image scene recognition results from generative features and discriminative features and 2. The method according to claim 1, characterized in that: The target area is detected by using a remote sensing image acquisition device, and the acquired remote sensing scene image of the target area is sent to a data processor; the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module, and a trusted decision fusion module.

3. The method according to claim 2, characterized in that The generative feature extraction module and the discriminative feature extraction module respectively extract features from the input remote sensing image dataset, and the remote sensing image dataset is denoted as Where (x i ,y i ) represents the i-th sample and its label, N represents the total number of samples, and the feature extraction process is represented as f g (x) and f d (x), where x represents the input sample, f g and f d They represent the feature mapping functions of the generative model and the discriminative model pre-trained on the remote sensing image, respectively, so as to obtain the image features of the same scene under two different feature extraction modules.

4. The method according to claim 2, characterized in that: The trusted decision fusion module uses a decision-level fusion method based on the DS evidence theory combination rule to fuse the decision results of the generative model and the discriminant model to obtain The calculation process is: Among them, b j (x i ) represents the input x i The confidence of the model in predicting the jth category, u(x i ) represents the input x i The uncertainty of the model prediction, C(x i ) represents the input x i It is a measure of the degree of conflict between the output results of the generative model and the discriminative model.

5. The method according to claim 2, characterized in that: The remote sensing image acquisition device is a network camera, and the data processor is a computer or an embedded mainboard.

6. The method according to claim 1, characterized in that The generative feature extraction module is a convolutional autoencoder pre-training model, and the discriminative feature extraction module is a ResNet18 pre-training model.

7. A remote sensing image scene classification system based on joint trusted learning, characterized in that: It includes a remote sensing image acquisition device and a data processor; the data processor includes a generative feature extraction module, a discriminative feature extraction module, an uncertainty quantification module and a trusted decision fusion module; The remote sensing image acquisition device detects the target area, obtains the remote sensing scene image of the target area and sends it to the data processor; The generative feature extraction module uses a remote sensing image scene feature extraction method based on generative deep learning to extract features from optical remote sensing images to obtain generative features; The discriminant feature extraction module uses a remote sensing image scene feature extraction method based on discriminant deep learning to extract features from optical remote sensing images to obtain discriminant features; The uncertainty quantification module uses an uncertainty quantification method based on subjective logic theory to estimate the uncertainty contained in the generative features and discriminant features for scene classification; The trusted decision fusion module classifies scenes in optical remote sensing images according to uncertainty quantification results and adopts a decision-level fusion method based on DS evidence theory combination rules; The steps of the uncertainty quantification method based on subjective logic theory include: First, the classification evidence obtained by calculating the generative features and discriminative features: g (x) = log(1 + exp(f g (x))),e d (x) = log(1 + exp(f d (x))); Then, using evidence g (x),e d (x) is used to calculate the belief quality b and overall uncertainty u of generative features and discriminative features: u g (x i )=K / S g (x i ),u d (x i )=K / S d (x i ),in The parameter α of the Dirichlet distribution is then expressed as in, Represents input x i The confidence of the generative model in predicting the jth category, K represents the total number of categories, Represents input x i When is the evidence of the generative model's prediction of the jth category, Represents input x i The confidence of the discriminant model for the prediction result of the jth category, Represents input x i When the discriminative model predicts the jth category, u g (x i ) represents the input x i The uncertainty of the generative model prediction, u d (x i ) represents the input x i The uncertainty of the prediction of the time-discriminative model; Then, the loss function of the uncertainty quantification model is optimized: Among them, ψ(·) is the double gamma function, y ij For label y i The jth dimension of and The input samples x are i Then the parameter α is calculated g and α d The jth dimension of ; L is optimized using a model optimization algorithm, prompting the uncertainty quantification module to learn and quantify the uncertainty of features for remote sensing image scene recognition results from generative features and discriminative features and 8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 6 is implemented.

Images