Remote sensing image security protection method and device based on adversarial sample, and storage medium
By generating adversarial samples in remote sensing images, using dynamic sampling non-local gradient descent method of neural network classification model, misleading deep learning models to protect the security of remote sensing images, solving the limited protection range of the existing technology in the context of deep learning. The problem is to achieve security guarantees in diversified application scenarios of remote sensing images.
Patent Information
- Application Number
- CN202411188781.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2025-01-14
AI Technical Summary
Existing remote sensing image security protection technology has failed to effectively prevent the malicious extraction and identification of sensitive geographical targets by deep learning models, and the protection scope is limited in the context of deep learning, which cannot meet the needs of current diversified usage scenarios.
Adopting security protection methods based on adversarial samples, dynamic sampling non-local gradients are calculated through neural network classification models, and adversarial samples with adversariality and visual authenticity are generated, misleading deep learning models to protect the security of remote sensing images.
It effectively resists the identification of sensitive geographical targets by attacker network models, improves the security of remote sensing images, and is suitable for different application scenarios, making up for the gap in the field of remote sensing image sensitive information protection in the context of deep learning.
Smart Images
Figure CN119313935A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of remote sensing image processing, and in particular to a remote sensing image security protection method, device and storage medium based on adversarial samples. Background Art
[0002] Remote sensing images have been widely used in economic development and social construction, and their sharing, circulation and application needs are also increasing. Existing remote sensing image security protection methods mainly focus on digital watermarking methods, data encryption methods and access control methods. In recent years, artificial intelligence technology represented by deep learning has developed rapidly. Its powerful learning and analysis capabilities meet the processing needs of massive remote sensing images. For example, high-resolution satellite images contain important surface targets such as ports, airports and ships. Many studies have used artificial intelligence technology to identify these targets. Since remote sensing images contain sensitive information such as spatial location information and important surface facilities (such as airports, ports, military bases, etc.), in the context of deep learning, how to prevent important geographical targets contained in remote sensing images from being maliciously extracted and identified by attackers using deep learning models has become a difficult problem.
[0003] The existing remote sensing image security protection technology mainly has the following three problems:
[0004] First of all, the existing remote sensing image digital watermarking technology combines watermark information containing data copyright, users and other content with the carrier data. It exists in the entire process of data generation, transmission and use, realizing the copyright protection of data, but the confidentiality of data is not protected enough, and the efficiency and applicability of remote sensing image digital watermarking algorithms also have some shortcomings.
[0005] Secondly, existing remote sensing image data encryption protection technologies usually convert raw data into ciphertext data to ensure data confidentiality, but this hinders the sharing and circulation of remote sensing images.
[0006] Finally, the existing remote sensing image access control protection technology generally declares that users have data access rights. Users can access the data only after identity authentication and authorization, thereby accurately controlling the spatial scope and time interval of access to geographic spatial data. However, the current processing of massive remote sensing images and diversified application scenarios such as social networks and self-media have greatly increased the difficulty of identity authentication and authorization in access control schemes.
[0007] Although the existing remote sensing image protection technology can meet the requirements of ensuring the security of remote sensing images to a certain extent, with the rapid development of artificial intelligence technologies such as deep learning, the use scenarios of remote sensing images have become diversified and complicated. Therefore, it is necessary to closely combine the essential characteristics of remote sensing images with the open sharing requirements in the Internet environment, analyze the current application status of deep learning models in remote sensing images, and study how to protect the security of sensitive content in remote sensing images under the current trend of great development of artificial intelligence. Summary of the invention
[0008] The purpose of the present invention is to propose a remote sensing image security protection method, device and storage medium based on adversarial samples, so as to solve the technical problem that traditional remote sensing image protection does not take deep learning into consideration, resulting in a limited protection scope and cannot meet current needs.
[0009] The present invention provides a remote sensing image security protection method based on adversarial samples, comprising the following steps:
[0010] S1: Acquire remote sensing target images;
[0011] S2: Input the remote sensing target image as the initial sample into the neural network classification model, and find the sampling point of the current input sample in each iteration;
[0012] S3: According to the sampling point gradient of the neural network classification model and the gradient of the current input sample, the dynamic sampling non-local gradient descent method is used to calculate the dynamic sampling non-local gradient of the model loss function with respect to the current input sample;
[0013] S4: Use non-local gradients to make perturbations, and add the perturbations to the input samples of the current iteration to obtain the adversarial samples in this iteration;
[0014] S5: Use the adversarial sample generation algorithm DSN-I-FGSM to update the perturbation through multi-step iterative cycles to maximize the model loss function value;
[0015] S6: Obtain adversarial examples of remote sensing images that are both adversarial and visually realistic.
[0016] A storage medium stores instructions and data for implementing a remote sensing image security protection method based on adversarial samples.
[0017] A remote sensing image security protection device based on adversarial samples comprises: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a remote sensing image security protection method based on adversarial samples.
[0018] The beneficial effects provided by the present invention are as follows: the present invention provides a remote sensing image security protection system and method based on adversarial samples, in view of the fact that there is no security protection technology for remote sensing images in deep learning scenarios in the current domestic market. The present invention calculates the gradient of the target model by the gradient descent method, so that the gradient direction is always the direction of the rising loss value. Dynamic sampling is used to find sampling points, and multi-point weighted gradients are used to make adversarial samples, so as to avoid the local optimal problem of random gradients, improve the efficiency of making adversarial samples, and take into account the visual authenticity and adversarial nature of adversarial samples. In the above manner, the present invention can mislead and disrupt the remote sensing image intelligent recognition system, causing the attacker's network to output incorrect classification results, so as to effectively resist the attacker's network model from identifying sensitive geographical targets, and improve the security of remote sensing images. The present invention can be used to provide protection for the protection of sensitive information of remote sensing images in different application scenarios, filling the gap in the field of sensitive information protection of remote sensing images in the context of deep learning in China today. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a schematic flow chart of the method of the present invention;
[0020] Figure 2 Update the process graph for the disturbance;
[0021] Figure 3 The remote sensing image recognition effect diagram before and after the application of adversarial samples;
[0022] Figure 4 Create diagrams for adversarial examples;
[0023] Figure 5 Generate effect graphs for adversarial samples at different iterations;
[0024] Figure 6 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION
[0025] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0026] Before formally describing the present invention, the scheme of the present invention is first generally described for easy understanding.
[0027] Please refer to Figure 1 , Figure 1 It is a schematic flow diagram of the method of the present invention;
[0028] The present invention provides a remote sensing image security protection method based on adversarial samples, comprising the following steps:
[0029] S1: Acquire remote sensing target images;
[0030] S2: Input the remote sensing target image as the initial sample into the neural network classification model, and find the sampling point of the current input sample in each iteration;
[0031] It should be noted that the neural network classification model in step S2 includes: Inception v3 network structure, Inception v4 network structure and Inception Resnet v2 network structure. The present invention is not limited to a certain network structure.
[0032] It should be noted that in step S2, by finding sampling points, the perturbation in the adversarial sample is always in the direction of increasing loss function value, wherein the calculation process of finding sampling points is:
[0033]
[0034] in represents the sampling point of the i-th iteration, x i represents the input sample of the i-th iteration, is a random variable used to find the sampling point for the i-th iteration, clip [,] is a clipping operator with a value range of [,b], which is used to ensure that the data remains within the specified range and is usually used for bounded data such as images; random variables From the standard deviation Parameterized independent and identically distributed P σ , P σ is a sampling distribution, σ is called the sampling distribution parameter, which determines x i The extent of the nearby sampling neighborhood.
[0035] S3: According to the sampling point gradient of the neural network classification model and the gradient of the current input sample, the dynamic sampling non-local gradient descent method is used to calculate the dynamic sampling non-local gradient of the model loss function with respect to the current input sample;
[0036] As an embodiment, step S3 is specifically as follows:
[0037] S31: Before calculating the gradient, first update the weights of the current iteration sampling point gradient and the historical sampling point gradient. The calculation formula is:
[0038]
[0039] Among them, x t is the input sample of the current iteration, is the sampling point in the i-th attack iteration; if the current sample x t and a sampling point in the sampling history If the distance is greater than σ, then reduce The importance in the gradient calculation, otherwise, the sampling points that are still within the sample neighborhood defined by σ are given a larger weight;
[0040] S32: Create a perturbation by calculating the dynamic sampled non-local gradient of the loss function with respect to the input sample. The calculation method of the dynamic sampled non-local gradient is:
[0041]
[0042] Among them, F θ It is a neural network classification model, with the model parameters as θ; is the neural network classification model F θ The loss function, x t is the sample given in the current iteration, y true is the correct category of the original sample, represents the sampling point of the i-th iteration, w i is related to the loss function The weight of the gradient.
[0043] S4: Use non-local gradients to make perturbations, and add the perturbations to the input samples of the current iteration to obtain the adversarial samples in this iteration;
[0044] As an embodiment, step S4 is specifically as follows:
[0045] S41: Use dynamic sampling non-local gradient to make perturbations, the calculation method is:
[0046]
[0047] Among them, δ represents the perturbation, x represents the original sample, and y true represents the correct category of x, Represents the loss function of the target model with respect to the current input sample Dynamic sampling non-local gradients;
[0048] S42: After each disturbance is made, the sampling distribution parameters of the next iteration are updated, and the calculation method is:
[0049]
[0050] Among them, λ σ is the artificially set learning rate for the distribution hyperparameter σ, x t is the input sample in the tth iteration, is the random variable in the tth iteration, taken from the sampling distribution parameter σ t The sampling distribution defined by t+1 is the updated sampling distribution parameter in the t+1th iteration;
[0051] S43: Add the perturbation to the input sample to obtain the adversarial sample of the current iteration, which is calculated as:
[0052]
[0053] Where δ represents the disturbance, It means that the input sample in the current iteration cycle The adversarial sample generated after adding perturbations is also the input sample of the next iteration cycle.
[0054] S5: Use the adversarial sample generation algorithm DSN-I-FGSM to update the perturbation through multi-step iterative cycles to maximize the model loss function value;
[0055] The calculation formula for maximizing the model loss function in step S5 is:
[0056]
[0057] Among them, ∈ is added to the adversarial sample x * , y is the true category label of the original sample x, and subscript 2 is the norm type; calculate the loss function value of the model for the adversarial sample, and check whether the loss function value reaches the maximum convergence condition. If so, the training ends and goes to step S6; otherwise, return to step S2 and continue iterating.
[0058] S6: Obtain adversarial examples of remote sensing images that are both adversarial and visually realistic.
[0059] It should be noted that the loss function of the model for adversarial samples reaches the maximum, indicating that the process of the model making adversarial samples has reached convergence and the iteration has terminated. The obtained adversarial samples can deceive the classification model and make it misclassify, thereby achieving the protection of the original target remote sensing image.
[0060] The present invention provides an embodiment of constructing an adversarial sample; please refer to Figure 2 , Figure 2 Update the process graph for the disturbance;
[0061] Each white arrow represents an iteration. First, find a sampling point in this iteration, and then work with the gradients of all sampling points before the current iteration (indicated by red arrows). By setting different weights to determine the contribution of each sampling point gradient to the final gradient, we can get the dynamic sampling non-local gradient of the input sample in this iteration. Secondly, use the non-local gradient to make a perturbation. The perturbation is calculated as follows:
[0062]
[0063] Among them, δ represents the disturbance, F θis a neural network classification model with parameters θ, is the neural network model F θ The loss function, Y True is the correct category of the original sample, represents the sampling point of the i-th iteration, w i is related to the loss function The weight of the gradient.
[0064] Finally, the perturbation is added to the input sample to construct an adversarial sample. The adversarial sample obtained in this iteration is also the input sample in the next iteration. The perturbation is continuously updated according to the customized number of iterations, and finally an adversarial sample of the remote sensing target image is generated. Figure 3 Generate renderings for adversarial samples of remote sensing images.
[0065] like Figure 3 As shown, the remote sensing image adversarial samples produced by the present invention are very similar to the original images in human vision, but can cause the neural network classification model to recognize errors with high confidence.
[0066] The perturbations in adversarial samples have two characteristics: concealment and deceptiveness. The concealment of perturbations means that the perturbations are very small and have an effect that is imperceptible to the human eye after being added to the original sample; the deceptiveness of perturbations means that for deep learning models, after the perturbations are added to specific pixels of the original image, the new image generated can induce the target model to misclassify. Therefore, when this specific adversarial perturbation is added to the image to obtain an adversarial image, it is difficult for the naked eye to detect the difference between the two, but the deep learning model gives different classification results.
[0067] The above method is used to realize the application of adversarial samples on remote sensing images. Figure 4 Create diagrams for adversarial examples; Figure 5 Generate effect graphs for adversarial samples at different iterations.
[0068] See also Figure 6 , Figure 6 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, and the hardware device specifically includes: a remote sensing image security protection device 401 based on adversarial samples, a processor 402 and a storage medium 403.
[0069] A remote sensing image security protection device 401 based on adversarial samples: The remote sensing image security protection device 401 based on adversarial samples implements the remote sensing image security protection method based on adversarial samples.
[0070] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the remote sensing image security protection method based on adversarial samples.
[0071] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the remote sensing image security protection method based on adversarial samples.
[0072] The beneficial effects of the present invention are as follows: the present invention provides a remote sensing image security protection system and method based on adversarial samples, in view of the fact that there is no security protection technology for remote sensing images in deep learning scenarios in the current domestic market. The present invention calculates the gradient of the target model by the gradient descent method, so that the gradient direction is always the direction of the rising loss value. Dynamic sampling is used to find sampling points, and multi-point weighted gradients are used to make adversarial samples, so as to avoid the local optimal problem of random gradients, improve the efficiency of making adversarial samples, and take into account the visual authenticity and adversarial nature of adversarial samples. In the above manner, the present invention can mislead and disrupt the remote sensing image intelligent recognition system, causing the attacker's network to output incorrect classification results, so as to effectively resist the attacker's network model from identifying sensitive geographical targets, and improve the security of remote sensing images. The present invention can be used to provide protection for the protection of sensitive information of remote sensing images in different application scenarios, filling the gap in the field of sensitive information protection of remote sensing images in the context of deep learning in China today.
[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A remote sensing image security protection method based on adversarial samples, characterized by: The method comprises the following steps: S1: Acquire remote sensing target images; S2: Input the remote sensing target image as the initial sample into the neural network classification model, and find the sampling point of the current input sample in each iteration; S3: According to the sampling point gradient of the neural network classification model and the gradient of the current input sample, the dynamic sampling non-local gradient descent method is used to calculate the dynamic sampling non-local gradient of the model loss function with respect to the current input sample; S4: Use non-local gradients to make perturbations, and add the perturbations to the input samples of the current iteration to obtain the adversarial samples in this iteration; S5: Use the adversarial sample generation algorithm DSN-I-FGSM to update the perturbation through multi-step iterative cycles to maximize the model loss function value; S6: Obtain adversarial examples of remote sensing images that are both adversarial and visually realistic.
2. The remote sensing image security protection method based on adversarial samples according to claim 1, characterized in that: The neural network classification model described in step S2 includes: Inception v3 network structure, Inceptionv4 network structure and Inception Resnetv2 network structure.
3. The remote sensing image security protection method based on adversarial samples according to claim 1, characterized in that: In step S2, the calculation process for finding the sampling point is: in represents the sampling point of the i-th iteration, x i represents the input sample of the i-th iteration, is a random variable used to find the sampling point for the i-th iteration, clip [a,b] is a clipping operator with a range of values [a,b]; the random variable From the standard deviation Parameterized independent and identically distributed P σ , P σ is a sampling distribution, σ is called the sampling distribution parameter, which determines x i The extent of the nearby sampling neighborhood.
4. The remote sensing image security protection method based on adversarial samples as claimed in claim 3, characterized in that: Step S3 is specifically as follows: S31: Before calculating the gradient, first update the weights of the current iteration sampling point gradient and the historical sampling point gradient. The calculation formula is: Among them, x t is the input sample of the current iteration, is the sampling point in the i-th attack iteration; if the current sample x t and a sampling point in the sampling history If the distance is greater than σ, then the The importance in the gradient calculation, otherwise, the sampling points that are still within the sample neighborhood defined by σ are given a larger weight; S32: Create a perturbation by calculating the dynamic sampled non-local gradient of the loss function with respect to the input sample. The calculation method of the dynamic sampled non-local gradient is: Among them, F θ It is a neural network classification model, with the model parameters as θ; is the neural network classification model F θ The loss function, x t is the sample given in the current iteration, y true is the correct category of the original sample, represents the sampling point of the i-th iteration, w i is related to the loss function The weight of the gradient.
5. The remote sensing image security protection method based on adversarial samples as claimed in claim 4, characterized in that: Step S4 is specifically as follows: S41: Use dynamic sampling non-local gradient to make perturbations, the calculation method is: Among them, δ represents the perturbation, x represents the original sample, and y true represents the correct category of x, Represents the loss function of the target model with respect to the current input sample Dynamic sampling non-local gradients; S42: After each disturbance is made, the sampling distribution parameters of the next iteration are updated, and the calculation method is: Among them, λ σ is the artificially set learning rate for the distribution hyperparameter σ, x t is the input sample in the tth iteration, is the random variable in the tth iteration, taken from the sampling distribution parameter σ t The sampling distribution defined by t+1 is the updated sampling distribution parameter in the t+1th iteration; S43: Add the perturbation to the input sample to obtain the adversarial sample of the current iteration, which is calculated as: Where δ represents the disturbance, It means that the input sample in the current iteration cycle The adversarial sample generated after adding perturbations is also the input sample of the next iteration cycle.
6. The remote sensing image security protection method based on adversarial samples as claimed in claim 5, characterized in that: The calculation formula for maximizing the model loss function in step S5 is: Among them, ∈ is added to the adversarial sample x * , y is the true category label of the original sample x, and subscript 2 is the norm type; calculate the loss function value of the model for the adversarial sample, and check whether the loss function value reaches the maximum convergence condition. If so, the training ends and goes to step S6; otherwise, return to step S2 and continue iterating.
7. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a remote sensing image security protection method based on adversarial samples as described in any one of claims 1 to 6.
8. A remote sensing image security protection device based on adversarial samples, characterized in that: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement a remote sensing image security protection method based on adversarial samples as described in any one of claims 1 to 6.