Image recognition system defense method based on multi-source remote sensing interference target intelligent forgery

Abstract

The application discloses a kind of multi-source remote sensing interference target intelligent image recognition system of false image identification method for defense, comprising: obtaining target identification network and training the target identification network using target image dataset, obtain trained target identification network model;The original image to be identified is input into the trained target identification network, and the position coordinate information and category information of real target are obtained;The real target identified is subjected to counter-interference, and the counter-sample image that can make real target detection invalid is generated;Scene information in original image is identified to determine the position range of false target in original image using identified scene information;Acquire false target data and build the false target in the corresponding scene of the counter-sample image, form camouflage image.The application makes real detection frame invalid at the same time, according to the scene identified, adds false target, to confuse target identification network, so that the camouflage process is not easy to detect.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target recognition, and particularly relates to an image recognition system defense method based on multi-source remote sensing interference target intelligent forgery. BACKGROUND

[0002] At present, information security problems are increasingly serious, and many existing technical means can easily obtain important confidential information. Artificial intelligence has been widely applied to the interpretation link of remote sensing pictures, and a target recognition network can accurately identify the position and category of a target and has been deployed in various application scenarios. Therefore, protecting one's own target information from being leaked has become a popular research field. Existing research shows that the recognition result of a network can be disturbed by attacking samples to reduce the recognition accuracy of the network model, thereby preventing others from obtaining one's own target information. Among these attacks, the most famous is the adversarial sample attack. An adversarial sample refers to a new data object generated by adding perturbations to an original data object (such as an image). The commonly used method for generating adversarial samples is to interfere by generating patches or modifying pixels.

[0003] However, there has been no targeted research on the recognition interference of multi-source images. Multi-source images refer to new images generated after pixel-level, feature-level or decision-level fusion of simultaneous phase remote sensing images (i.e. remote sensing images of the same time, same phase and same scene) from different sensors. The new images have more feature information. Existing interference methods can only make the detection frame of the real target invalid, but such interference methods are easy to be recognized by some network models.

[0004] Deep learning has been successful because of its strong expression ability, but it has also caused it to learn some unexplainable results and some counterintuitive properties. The input-output mapping learned by a deep neural network is largely discontinuous, which causes images to be misclassified by applying some imperceptible perturbations that are found by maximizing the prediction error of the network.

[0005] Existing adversarial interference mostly focuses on making existing target detection frames invalid or interfering with the recognition network to recognize as other wrong types. However, when making the real target detection invalid, no new false target is generated or other categories of false targets are generated, which is easy to be recognized as an attack behavior in a certain environment. For example, after conducting adversarial attacks at an airport location, no "airplane" or other targets are detected, but some category targets that do not match the environment are detected. Or in the case where the target is known to exist, the target is not recognized and the naked eye is used for recognition, which will make the other party aware that the target detection is being attacked and switch to other target recognition methods, thereby being not conducive to protecting the target information of the other party. SUMMARY

[0006] In order to solve the above problems existing in the prior art, the present application provides an image recognition system defense method based on multi-source remote sensing interference target intelligent forgery.

[0007] The present application provides a multi-source remote sensing interference target intelligent forgery image recognition system defense method, comprising:

[0008] Obtain a target recognition network and train the target recognition network using a target image dataset to obtain a trained target recognition network model;

[0009] Input the original image to be identified into the trained target recognition network to obtain position coordinate information and category information of the real target;

[0010] Perform adversarial interference on the identified real target to generate an adversarial sample image that can make the real target detection invalid;

[0011] Identify the scene information in the original image to determine the position range of the false target that can be generated in the original image using the identified scene information;

[0012] Obtain false target data and construct the false target in the corresponding scene of the adversarial sample image to form a camouflage image.

[0013] In an embodiment of the present application, obtaining a target recognition network and training the target recognition network using a target image dataset to obtain a trained target recognition network model comprises:

[0014] Construct a target recognition network, wherein the target recognition network is a YOLOv4 network;

[0015] Train the target recognition network using a training dataset containing target labels to obtain a trained target recognition network model, wherein the trained target recognition network model can be used to identify target information in an image.

[0016] In an embodiment of the present application, performing adversarial interference on the identified real target to generate an adversarial sample image that can make the real target detection invalid comprises:

[0017] Perform adversarial interference on the identified real target using an adversarial patch to generate an adversarial sample image containing a patch to make the detection of the real target invalid.

[0018] In an embodiment of the present application, performing adversarial interference on the identified real target to generate an adversarial sample image that can make the real target detection invalid comprises:

[0019] In the process of generating the patch, an input target recognition network model, a target category, and a scale parameter of the patch are identified; after a given image with a patch is given, a gradient descent optimizer is used to minimize the classification loss of the target recognition network model, and the loss will decrease during the training process, and finally an adversarial sample image is generated.

[0020] In an embodiment of the present application, the scene information in the original image is identified to determine the position range of the false target in the original image using the identified scene information, comprising:

[0021] The scene category information and position information in the original image are obtained using a fuzzy C-means clustering method, SVM, decision tree, or normalized vegetation index, or a trained scene recognition network model.

[0022] In an embodiment of the present application, the scene recognition network model is a YOLOv4 network.

[0023] In an embodiment of the present application, the false target data is obtained and the false target is constructed in the corresponding scene of the adversarial sample image to form a camouflage image, comprising:

[0024] A material library for generating false targets is constructed by collecting various target images of the same type as the original image;

[0025] A target image is obtained from the material library as a false target to be used;

[0026] The false target is constructed in the corresponding scene range of the original image.

[0027] In an embodiment of the present application, the false target data is obtained and the false target is constructed in the corresponding scene of the adversarial sample image to form a camouflage image, comprising:

[0028] The category information of the real target is obtained as a false target to be used;

[0029] The false target is constructed in a position different from the real target in the corresponding scene range of the original image.

[0030] Another aspect of the present application provides a storage medium having a computer program stored therein, the computer program being used to execute the steps of the multi-source remote sensing interference target intelligent camouflage image recognition system defense method of any one of the above embodiments.

[0031] Still another aspect of the present application provides an electronic device comprising a memory and a processor, the memory having a computer program stored therein, and the processor calling the computer program in the memory to implement the steps of the multi-source remote sensing interference target intelligent camouflage image recognition system defense method of any one of the above embodiments.

[0032] Compared with the prior art, the present application has the beneficial effects that:

[0033] 1、The image recognition system defense method based on multi-source remote sensing interference target intelligent forgery of the present application can make the real detection frame invalid, increase the target or copy the real target corresponding to the scene material library, and move the real target to appear in different positions, so as to achieve the purpose that the target recognition network can recognize the same category and quantity information as the real target, but the recognized position has some deviation, so as to make the recognition result invalid.

[0034] 2、The scene recognition idea adopted in the present application is to first recognize the scene information of the original picture, which can generate high false target credibility. In order to prevent the target recognition network with high recognition accuracy from recognizing the false target as the real target, scene recognition is first performed, which can avoid the false target from being generated at the wrong position and improve the defense strength.

[0035] 3、The present application establishes a false target material library when generating a false target, collects images related to target information, is conducive to obtaining image information such as texture information and spectral information of the target, helps to analyze the characteristics of the target, improves the authenticity of the false target, and achieves better defense effect.

[0036] The present application will be further described in detail below in combination with the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flow chart of the image recognition system defense method based on multi-source remote sensing interference target intelligent forgery provided by the embodiment of the present application;

[0038] Figure 2 is a detailed flow chart of the image recognition system defense method based on multi-source remote sensing interference target intelligent forgery provided by the embodiment of the present application;

[0039] Figure 3 is a generation process schematic diagram of the adversarial sample image provided by the embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined invention purpose, the image recognition system defense method based on multi-source remote sensing interference target intelligent forgery according to the present application is described in detail below in combination with the drawings and specific embodiments.

[0041] The foregoing and other technical contents, features and effects of the present application can be clearly presented in the following detailed description of the embodiments with reference to the accompanying drawings. Through the description of the embodiments, the technical means and effects adopted by the present application to achieve the predetermined purposes can be understood more deeply and specifically. However, the accompanying drawings are provided for reference and illustration only, and are not intended to limit the technical solutions of the present application.

[0042] It should be noted that, in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant are intended to cover non-exclusive inclusion, so that the article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed. Without more limitation, the element defined by the sentence "including a…" does not exclude the presence of other identical elements in the article or device including the element.

[0043] The embodiment provides an image recognition system defense method based on multi-source remote sensing interference target intelligent forgery, please see Figure 1 and Figure 2 , the method specifically comprises the following steps:

[0044] S1: obtaining a target recognition network and training the target recognition network with a target image dataset to obtain a trained target recognition network model.

[0045] The target recognition network is constructed, and the target recognition network is trained by using a training dataset containing target labels to obtain a trained target recognition network model. Assuming that the target to be recognized is a vehicle, the training dataset for training the target recognition network is a vehicle dataset, that is, each picture in the training dataset contains the type and position labels of the vehicle; if the target to be recognized has two or more, the training dataset for training the target recognition network is also a picture dataset containing multiple target categories and positions. For example, the target recognition network is used to recognize a scene picture containing an airplane, a vehicle and a ship, and the target recognition network is trained using an image dataset containing target labels including an airplane, a vehicle and a ship.

[0046] Illustratively, the embodiment uses an airplane image dataset to train the target recognition network, and the target recognition network of the embodiment selects a YOLOv4 network.

[0047] The YOLOv4 network mainly consists of three main components: (1) Backbone: a convolutional neural network that aggregates and forms image features at different image granularities; (2) Neck: a series of network layers that mix and combine image features, and pass the image features to the prediction layer; (3) Head: predicting image features to generate bounding boxes and predict classes; and YOLOv4 is a one-stage network.

[0048] S2: inputting the original image to be identified into the trained target recognition network to obtain position coordinate information and class information of the real target.

[0049] In this embodiment, the original image to be detected is input into the trained target recognition network YOLOv4 model, and the aircraft target is identified to obtain the class and position of the aircraft target in the original image.

[0050] S3: performing adversarial interference on the identified real target to generate an adversarial sample that can cause the real target detection to fail.

[0051] For the real target identified in step S2, certain processing can be performed to make the detection frame invalid, and an adversarial sample is generated.

[0052] Specifically, the method of generating an adversarial sample includes a method of generating an adversarial patch and a pixel adversarial method. The adversarial patch is a means of generating an adversarial sample that causes the network to recognize invalid by replacing the local pixels of the input image with an artificially synthesized "patch" picture. The pixel adversarial method refers to intentionally adding some subtle interference that is not easy to detect in the input image, causing the target recognition network model to give an incorrect output with high confidence, i.e. changing part of the pixels on the image (compared with the patch, the pixels are scattered), which can make the target detection frame invalid.

[0053] In this embodiment, the method of Adversarial Patch Attack (adversarial patch attack) is used to generate a patch that is irrelevant to the original image, and then the patch can be placed in a specific position in the original image, so that the identification result of the opponent's target recognition network can be destroyed.

[0054] Given a target recognition network Pr[y|x], given input image data x∈R n , select target class y and maximum disturbance ε, find an input to maximize But subject to constraints.

[0055] The process of generating the patch is to calculate the gradient of the target recognition network model (i.e. the target recognition network model in step S1) for the input image, and then update the generated adversarial sample image input accordingly. When updating, instead of calculating the gradient for each pixel, the corresponding area of the original image is first replaced with a patch, and then the gradient is only calculated for the patch area. Secondly, training needs to be performed on multiple pictures, because the attack target is any picture plus our patch that can deceive the model.

[0056] In the process of generating the patch, the input is the target recognition network model to be attacked (the target recognition network model in step S1), the target category, and the size parameter of the patch. After the image with the patch is given, the SGD (stochastic gradient descent) optimizer is used to minimize the classification loss of the target recognition network model, and the loss will decrease during the training process, and finally the adversarial sample image is generated.

[0057] Specifically, please refer to Figure 3 , Figure 3 is a schematic diagram of the generation process of an adversarial sample image provided by an embodiment of the present application. By completely replacing a part of the image with a patch, the target information of others can be prevented from being stolen. A mask patch is used to make it have any shape, and then various images are trained, random translation, scaling and rotation are applied to the patch in each image, and gradient descent is used for optimization. In particular, for a given image x∈R w×h×c , patch p, patch position l, and patch transformation parameter set t (including rotation parameters, scaling parameters), a patch application operator A (p, x, l, t) can be defined. In the actual process of generating an adversarial sample, the patch transformation parameter t is first applied to the patch p, and then the transformed patch p is placed on the position l of the image x.

[0058] In practice, the patch needs to be trained to optimize the expected probability of the target category. The patch can be obtained by the following formula:

[0059]

[0060] Where X is the image training set, T is the distribution of patch transformation, L is the distribution of position in the image, Pr() represents the expected meaning, and Pr() is a function of the target recognition network.

[0061] It should be noted that the image training set here refers to a data set that has no category intersection with the data set in step S1, and the data set is used as a training patch, so that when the trained patch image is superimposed on the recognition target, the recognition frame of the target recognition network will be invalid.

[0062] S4: scene information in the original image is recognized to determine the location range in which the false target can be generated in the original image by using the recognized scene information.

[0063] In this embodiment, the scene type information and location information in the original image are obtained by using a fuzzy C-means clustering method, an SVM, a decision tree, or a normalized vegetation index or a trained scene recognition network model.

[0064] Since a remote sensing image belongs to a big data image, it contains a lot of information and a lot of scenes. One remote sensing image data can cover many scenes. For example, an image can contain different scenes such as a school, a residential area, and a river. If a vehicle or the like that can only appear on land is generated on the water surface, or a ship is generated on the lawn, the camouflage behavior will be easily exposed. Therefore, before generating a false target in the image to be recognized, it is necessary to first determine the region range in which the target can be generated. When the target scene to be recognized is a large scene that is easy to recognize (such as a lawn and a water area), a non-intelligent algorithm can be used. For example, when a lawn is recognized, a fuzzy C-means clustering (FCMA) method can be used. At this time, the algorithm does not need to be processed in advance. However, for a scene with higher location accuracy requirement, a smaller positioning range is required. At this time, a target recognition algorithm can be used for scene recognition. Specifically, a scene data set can be added while training the target recognition network. For example, a data set with both airplane and airfield labels is used to train the target recognition network, so that the trained target recognition network model can simultaneously perform target recognition and scene recognition.

[0065] In this embodiment, since the recognition accuracy of the airfield is required to be high, a target recognition network is used for recognition. A trained YOLOv4 network is used to recognize the location of the airfield. In other cases, such as generating a ship in a water area, only the water area position needs to be recognized. The water area position can be recognized by using a more basic classifier such as an SVM (support vector machines) or a decision tree. For example, in the case of generating a target in a lawn, a normalized vegetation index (ndvi) can be used to realize vegetation classification mapping.

[0066] For example, the recognized target of this embodiment is an airplane. Therefore, the subsequent airplane false target needs to be generated in a suitable scene such as an airfield, so as not to easily expose the camouflage behavior.

[0067] The scene recognition idea used in this embodiment first recognizes the scene information of the original image, which can improve the credibility of generating a high false target. To prevent the target recognition network with high recognition accuracy from recognizing the false target as a real target, scene recognition is performed first, so as to avoid generating the false target in a wrong position and improve the defense strength.

[0068] S5: obtaining the false target data and constructing the false target in the corresponding scene of the adversarial sample image to form a camouflage image.

[0069] In one specific embodiment of the present application, the S5 comprises:

[0070] S51: constructing a material library for generating the false target by collecting various target images of the same type as the original image;

[0071] S52: obtaining a target image from the material library as the false target to be used;

[0072] S53: constructing the false target in the corresponding scene range of the original image.

[0073] Specifically, the material library for generating the false target is constructed by collecting various target images of the same type as the original image. In order to generate the false target that can be recognized in the scene, the material library of the target of this type can be prepared in advance. The material library has small-size target images of the same type as the original image, and the images only include the image of the target part, and the size is small compared with the original data. For example, the original image is hyperspectral data (spectral image with spectral resolution in the order of 10-2λ), and the hyperspectral image has multiple wavebands. At this time, the target data in the prepared database is also hyperspectral data. Assuming that the material library does not have target pictures matching the original image, the real target data recognized in step S2 can also be used as the false target.

[0074] In this embodiment, the material library including the aircraft target is constructed, which can be the same model as the aircraft in the original image, or different models of aircraft, and the category can be determined according to specific requirements.

[0075] This embodiment establishes the false target material library when generating the false target, and collects the images of the target information, which is beneficial to obtain the image information such as the texture information and spectral information of the target, and is helpful to analyze the characteristics of the target, improve the authenticity of the false target, and achieve better defense effect.

[0076] In another embodiment of the present application, if there is no suitable data, the aircraft image recognized in step S2 can be used. Specifically, the S5 can further comprise:

[0077] S51': obtaining the category information of the real target as the false target to be used;

[0078] S52': constructing the false target in the corresponding scene range of the original image and at a position different from the real target.

[0079] Specifically, as described above, the position coordinate information and the category information of the real target in the original image to be identified have been acquired in step S2. Using the scene coordinate information recorded in step S4, a target of the same type as the target identified in step S2 is generated within the coordinate range. The original image data is replaced by the target data prepared in step S5 at the determined scene coordinate.

[0080] In summary, if an aircraft sample in the material library is selected to be added, an aircraft false target in the material library is directly generated within the identified scene range. If an aircraft false target in the current scene is selected, the aircraft target identified in step S2 is processed to remove the surrounding environment influence and only keep the pixel points related to the target, and then another position different from the position coordinate of the real target in the identified scene position is generated. In this way, when someone wants to identify the type information and position information of the target of the remote sensing image, the position and type data of the false target will be obtained through the method of the embodiment of the application, so that the correct target information is protected from being leaked.

[0081] The embodiment of the application first uses a patch or other technical means to invalidate the real sample detection frame, and then generates a new target at another position in the same scene, so as to achieve the purpose of generating an adversarial sample to confuse the target recognition network. The embodiment of the application enriches the current adversarial sample generation scheme, and does not completely cover up the target information in the real scene, but modifies the existing target to a certain extent. When the target recognition network is in the process of target recognition, the target type closely related to the scene can still be identified, but the position of the target has been modified to a false position at this time, so that the identified information points to a false target, thereby confusing the opponent to a certain extent, and protecting the correct target information from being leaked.

[0082] Still another embodiment of the present application provides a storage medium having stored therein a computer program for performing the steps of the image recognition system defense method based on intelligent forgery of multi-source remote sensing interference targets as described in the above embodiments. In still another aspect, the present application provides an electronic device comprising a memory having stored therein a computer program and a processor, wherein the processor invokes the computer program stored in the memory to implement the steps of the image recognition system defense method based on intelligent forgery of multi-source remote sensing interference targets as described in the above embodiments. Specifically, the integrated modules implemented in the form of software functional modules described above can be stored in a computer readable storage medium. The software functional modules described above are stored in a storage medium, including a number of instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0083] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application should not be limited to these descriptions. For ordinary skilled persons in the art to which the present application belongs, a number of simple deductions or substitutions can be made without departing from the concept of the present application, and all of these should be considered to fall within the protection scope of the present application.

Claims

1. A multi-source remote sensing jamming target intelligent fake image recognition system defense method, characterized in that, Comprise: S1: obtain a target recognition network and train the target recognition network using a target image data set to obtain a trained target recognition network model; S2: input the original image to be identified into the trained target recognition network to obtain position coordinate information and category information of the real target; S3: perform adversarial interference on the identified real target to generate an adversarial sample image that can make the real target detection invalid; S3: identify the scene information in the original image to determine the position range of the false target in the original image using the identified scene information; S5: obtain false target data and construct the false target in the corresponding scene of the adversarial sample image to form a camouflage image; Identifying the scene information in the original image to determine the position range of the false target in the original image using the identified scene information, comprising: Using fuzzy C-means clustering method, SVM, decision tree or normalized vegetation index or trained scene recognition network model, obtain scene category information and position information in the original image; Obtaining false target data and constructing the false target in the corresponding scene of the adversarial sample image to form a camouflage image, comprising: Construct a material library for generating false targets by collecting various target images of the same type as the original image; Obtain a target image from the material library as a false target to be used; Construct the false target in the corresponding scene range of the original image; if there is no target picture matching the original image in the material library, use the real target data identified in step S2 as the false target, comprising: Obtain the category information of the real target as the false target to be used; Construct the false target in the corresponding scene range of the original image at a position different from the real target; if the false target under the current scene is selected, process the target identified in step S2, remove the surrounding environment influence, and only keep the pixel points related to the target, and then generate at another position different from the position coordinate of the real target in the identified scene position.

2. The multi-source remote sensing jamming target intelligent counterfeiting image recognition system defense method according to claim 1, characterized in that, Obtaining a target recognition network and training the target recognition network using a target image data set to obtain a trained target recognition network model, comprising: Constructing a target recognition network, the target recognition network being a YOLOv4 network; Training the target recognition network using a training data set containing target labels to obtain a trained target recognition network model, the trained target recognition network model being used to identify target information in an image.

3. The multi-source remote sensing jamming target intelligent counterfeiting image recognition system defense method of claim 1, wherein, Performing adversarial interference on the identified real target to generate an adversarial sample image that can make the real target detection invalid, comprising: Performing adversarial interference on the identified real target using an adversarial patch to generate an adversarial sample image containing a patch to make the detection of the real target invalid.

4. The multi-source remote sensing jamming target intelligent counterfeiting image recognition system defense method according to claim 3, characterized in that, Performing adversarial interference on the identified real target to generate an adversarial sample image that can make the real target detection invalid, comprising: In the process of generating the patch, an input target recognition network model, a target category and a scale parameter of the patch are identified; after a given image with the patch is given, a gradient descent optimizer is used to minimize the classification loss of the target recognition network model, and the loss will decrease with the training process, and finally an adversarial sample image is generated.

5. The multi-source remote sensing jamming target intelligent counterfeiting image recognition system defense method according to claim 4, characterized in that, The scene recognition network model is a YOLOv4 network.

6. A storage medium, characterized by The storage medium has a computer program stored therein, and the computer program is used for executing the steps of the image recognition system defense method for intelligent forgery of multi-source remote sensing interference targets according to any one of claims 1 to 5.

7. An electronic device, comprising: The computer program is stored in the memory, and the processor calls the computer program in the memory to implement the steps of the image recognition system defense method for intelligent forgery of multi-source remote sensing interference targets according to any one of claims 1 to 5.

Images