Remote sensing marine target positioning anti-interference matching algorithm based on deep hash

Abstract

The invention discloses a remote sensing marine target positioning anti-interference matching algorithm based on deep Hash. The algorithm comprises the following steps: step 1, training an original deep Hash network; and step 2, carrying out anti-interference antagonism training on the original deep hash network. The algorithm disclosed by the invention makes up for the deficiency of the anti-interference capability of the ocean remote sensing image on the target positioning image preprocessing method, and is beneficial to the development of ocean remote sensing image feature rapid target positioning. Based on a marine remote sensing image deep Hash feature rapid matching algorithm, a trained original deep Hash network is used as an original model, a class center Hash code of each class image is constructed in a discriminant learning mode, and the class center Hash code is used for guiding generation of an interference image example and iteration of adversarial training, so that a deep Hash feature is obtained. And finally, the anti-interference capability of the deep hash network is effectively improved.

Description

Technical Field

[0001] The invention belongs to the field of marine target remote sensing image processing, and in particular relates to a marine target positioning remote sensing image anti-interference feature fast matching algorithm based on deep hashing in the field. Background Art

[0002] With the development of science and technology, the number of aerospace platforms has increased year by year. At the same time, the increase in the number of remote sensing imaging sensors on these platforms has brought people a large amount of large-scale, high-precision marine target observation data. Marine remote sensing big data provides more opportunities for marine target monitoring, marine electromagnetic equipment layout planning, management and application. In recent years, in order to solve the problem of big data processing of marine remote sensing targets, people have carried out a lot of research. Among them, achieving feature matching of specific targets in large-scale marine remote sensing images is one of the most widely studied topics. At present, there are many algorithms that can detect targets in marine remote sensing images, but most of them can only manage small images, and the size of large-scale remote sensing data is very large. Therefore, when these detection algorithms are used to process large-scale remote sensing images, the storage cost and computing time increase dramatically.

[0003] As a method that maps high-dimensional features to low-dimensional binary features and reflects the similarity between images by calculating the Hamming distance of binary feature vectors, feature hashing can significantly reduce computing and storage costs, and is therefore widely used in target feature matching tasks in marine remote sensing images. However, the existing hashing network has weak anti-interference capabilities. By changing a very small number of pixels in an image, it is possible to significantly change the feature discrimination results of the hashing network for the image. Therefore, improving the anti-interference ability of the algorithm is crucial for deep hashing algorithms.

[0004] At present, according to different implementation methods, the anti-interference of marine remote sensing image target hashing algorithm can be divided into two categories: traditional anti-interference method and anti-interference based on deep learning method. Traditional image anti-interference technology usually processes images through statistical filtering, frequency domain conversion, image contrast enhancement, etc. Although it has certain anti-interference ability, it will significantly reduce the visual quality of the input image. At present, the rise of deep learning has provided a new idea for realizing anti-interference technology for target recognition in marine remote sensing images. Anti-interference based on deep learning method mainly trains the model through deep neural network, learns the mapping relationship between noise model and denoising model, so as to achieve accurate anti-interference effect. Although the existing methods have certain effectiveness in image anti-interference, they are not very applicable in the field of remote sensing image hashing. Summary of the invention

[0005] Aiming at the technical problem that the existing methods in the field of ocean remote sensing image target positioning hashing are insufficient in anti-interference ability, which restricts the rapid matching of ocean remote sensing image features, the present invention provides an ocean remote sensing image anti-interference target positioning and matching algorithm based on deep hashing, which realizes the rapid positioning of remote sensing scene features from ocean remote sensing images with interference, thereby providing technical support for ocean remote sensing data processing.

[0006] The present invention adopts the following technical solution:

[0007] A remote sensing marine target positioning anti-interference matching algorithm based on deep hashing, the improvement of which is that it includes the following steps:

[0008] Step 1, original deep hashing network training:

[0009] The deep hash learning steps include feature extraction and hash code generation. In the feature extraction stage, a well-trained convolutional neural network will extract continuous feature values ​​from the image input to the model; in the hash code generation stage, the activation function will convert the continuous feature values ​​into binary values, such as the sigmoid function, and finally constrain the binary values ​​through the loss function, which can prompt the network to generate efficient hash codes;

[0010] Step 2, anti-interference adversarial training of the original deep hashing network:

[0011] The purpose of adversarial training is to generate a batch of adversarial samples with interference capabilities, and to improve the identification ability of the deep hash network through supervision. The hash code of the adversarial sample in the Hamming space is similar to the interference image, which is far away from the hash code of its original category and close to the hash code of other categories. Through continuous training iterations, the hash network gradually makes correct judgments on this type of image, generates hash codes close to the original category, and restores its correct matching ability for this type of image. Specifically, the adversarial training of hash anti-interference is mainly divided into two steps, namely, class-center hash code generation and adversarial training based on class-center hash code;

[0012] The purpose of class-centered hash code generation is to obtain the most representative hash code for each category, so as to help the deep hash network understand the difference between each category more easily in subsequent training, thereby improving the network's anti-interference ability through corresponding adversarial training. In the process of class-centered hash code generation, the deep hash network first projects all training samples into the Hamming space to form corresponding hash codes; then, a master code is constructed for each class through discriminative learning, and the hash codes of images in the same category as the master code are made closer, and the hash codes of images in different categories from the master code are further away; finally, the constructed master code is used as the class-centered hash code for each category to guide subsequent adversarial training;

[0013] The main purpose of adversarial training based on class-centered hash codes is to improve the deep hash network's ability to discriminate interference by constructing adversarial examples similar to the original images, thereby avoiding being affected by them and generating hash codes far from the original categories. In adversarial training based on class-centered hash codes, the training data is a set of image data, their class labels, and the class-centered hash codes of each class; first, the image samples are subjected to feature extraction, full connection, and tanh activation by the deep hash network to generate the corresponding hash code, and the corresponding adversarial example image is generated by maximizing the Hamming distance between the hash code and the class-centered hash code and back-propagating; then, the generated adversarial example image is input into the deep hash network again to generate the corresponding hash code, and the network parameters are updated by minimizing the Hamming distance between the hash code and the class-centered hash code and back-propagating, thereby gradually improving the deep hash network's ability to resist interference.

[0014] Furthermore, the training process of the algorithm is as follows: first, the structural parameters of the original deep hash network are imported, and then the number of iterations is judged. If the number of iterations is reached, the model is saved and exited; if the number of iterations is not reached, the deep hash network parameters are updated according to the loss function; after the parameters are updated, the original image sample is input into the deep hash network to generate the corresponding class center hash code and adversarial sample, and then the overall loss function of the adversarial training is calculated and back propagation is completed to achieve the anti-interference performance training of the deep hash network.

[0015] Furthermore, for the original image x, the objective function F1 of its class center hash code h is defined as:

[0016]

[0017] In the above formula, f(.) is the approximate hash function obtained by the deep hash model, D is the distance metric function, and w i and w j represents the corresponding distance weight, S p is the number of positive samples, S n is the number of negative samples, is a positive sample related to the original sample, is a negative sample that has nothing to do with the original sample;

[0018] Based on this formula, the objective function formula for a single class center hash code is obtained as follows:

[0019]

[0020] In the above formula, K is the length of the hash code, D Ham is the Hamming distance metric function, is a positive sample related to the original sample The hash code of is a negative sample that has nothing to do with the original sample. The hash code of

[0021] The optimal solution of formula (2) is the class center hash code, which is calculated by the deviation between the weighted sum of positive and negative hash code samples:

[0022]

[0023] Furthermore, the target generation formula of adversarial samples is defined as follows:

[0024]

[0025] In the above formula, x a is an adversarial example of the input image x, h * is the class center hash code of the input image, ε is a positive constant close to zero, and for any hash codes h1 and h2, the Hamming distance metric function is:

[0026]

[0027] The above target generation formula is specifically described as:

[0028]

[0029] Using the function f with respect to the variable θ θ Substituting the function f in formula (11), the final target generation formula is as follows:

[0030]

[0031] Furthermore, the adversarial loss function is defined as follows:

[0032]

[0033] In the above formula, N is the maximum number of categories, is the original sample of the input image x i The class-centric hash code, x a is the adversarial sample corresponding to the input image sample. For any hash codes h1 and h2, the Hamming distance metric function is:

[0034] D Ham (h1,h2)=(K-h1 T h2) / 2

[0035] The above adversarial loss function is described as follows:

[0036]

[0037] Using the function f with respect to the variable θθ Replacing the function f in equation (14), the final adversarial loss function is obtained as follows:

[0038]

[0039] The overall objective function of adversarial training of class-centered hash codes is as follows:

[0040] L t =L o +λL a (16)

[0041] In the above formula, λ is the weighting factor, L o is the loss function of the original deep hashing network.

[0042] The beneficial effects of the present invention are:

[0043] The algorithm disclosed in the present invention makes up for the lack of anti-interference ability of marine remote sensing images to target positioning image preprocessing methods, which is beneficial to the development of rapid target positioning of marine remote sensing image features. Based on the deep hash feature fast matching algorithm of marine remote sensing images, the trained original deep hash network is used as the original model, and the class center hash code of each category image is constructed by discriminant learning, and it is used to guide the generation of interfering image examples and the iteration of adversarial training, which ultimately effectively improves the anti-interference ability of the deep hash network. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the algorithm of the present invention;

[0045] Figure 2 This is the schematic diagram of the generation of class-centric hash codes;

[0046] Figure 3 This is the principle diagram of adversarial training based on class center hash code;

[0047] Figure 4 It is a detailed diagram of the training process of the algorithm model;

[0048] Figure 5 This is a comparison chart of the matching results of the hash anti-interference network for the original image and the interference image;

[0049] Figure 6 This is a comparison chart of target accuracy curves of different hash anti-interference networks;

[0050] Figure 7 This is a comparison chart of PR curves of different hash anti-interference networks. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] Example 1, in order to solve the problem of weak anti-interference ability of existing image matching methods in the field of remote sensing hashing, this example discloses a marine remote sensing image hash anti-interference matching algorithm based on class center hash code guided adversarial training, which improves the anti-interference ability of the algorithm to achieve the task of rapid target matching of features of marine remote sensing images with interference. On the basis of the deep hashing algorithm, the trained original deep hashing network is used as the initial model, and the class center hash code of each category image is constructed by discriminative learning, and it is used to guide the generation of interfering image examples and the iteration of adversarial training, which effectively improves the anti-interference ability of the deep hashing network.

[0053] Marine remote sensing image noise reduction is a method to reduce or eliminate the influence of noise in images through algorithms and techniques. It can improve the stability of the model's discrimination of image features, so it can be used to train the anti-interference ability of image models. Combined with marine remote sensing image noise reduction algorithms, such as Figure 1 As shown, this embodiment designs an adversarial training for the original hash network. The purpose of adversarial training is to use the idea of ​​interference algorithm to generate a batch of adversarial samples, and to supervise the deep hash network to improve its ability to identify such images. The hash code of the adversarial sample in the Hamming space is similar to the interference image, which is far away from its original category hash code and close to the hash code of other categories. Through continuous training iterations, the hash network gradually makes correct judgments on such images, generates hash codes close to the original category, and restores its correct matching ability for such images, so that hash anti-interference can be achieved. In summary, image anti-noise, as a method that can significantly suppress the influence of noise on images, is a good choice for completing hash image anti-interference.

[0054] In order to improve the anti-interference ability of the deep hash network and achieve anti-interference, this embodiment uses the idea of ​​image anti-noise algorithm to perform image adversarial training on the trained original hash network. Specifically, the adversarial training of hash anti-interference is mainly divided into two steps, namely the generation of class center hash code and adversarial training based on class center hash code. The generation of class center hash code is mainly to obtain the most representative hash code for each category, so as to help the deep hash network to more easily understand the semantic differences of each category in subsequent training, so as to obtain anti-interference ability through corresponding adversarial training.

[0055] The principle of class-centric hash code generation is as follows Figure 2As shown in Figure 1. The training data is a set of image data and their class labels. In the process of generating class-centered hash codes, the deep hash network first projects all training samples into the Hamming space to form corresponding hash codes. After that, a master code is constructed for each class through discriminative learning, and the hash codes of images in the same category as the master code are made closer, and the hash codes of images in different categories from the master code are made further away. Finally, the constructed master code is used as the class-centered hash code of each category to guide the subsequent adversarial training.

[0056] The main purpose of adversarial training based on class-centered hash codes is to improve the ability of the deep hash network to distinguish interference by constructing adversarial examples similar to the original image, thereby avoiding being affected by the interference and generating hash codes far from the original category. Figure 3 As shown in the figure, the adversarial training principle based on class center hash code, the gray, red and blue arrows represent forward, backward and forward propagation respectively. The training data for adversarial training based on class center hash code is a set of image data, their class labels and the class center hash code of each class. First, the image sample is extracted by the deep hash network and fully connected and tanh activated to generate the corresponding hash code. By maximizing the Hamming distance between the hash code and the class center hash code and back propagating, the corresponding adversarial example image is generated. After that, the generated adversarial example image is input into the deep hash network again to generate the corresponding hash code. By minimizing the Hamming distance between the hash code and the class center hash code and back propagating, the network parameters are updated, so that the deep hash network can gradually improve its anti-interference ability.

[0057] The training process of the hash anti-interference algorithm is as follows Figure 4 As shown in the figure, firstly, the structural parameters of the original deep hash network are imported, and then the number of iterations is judged. If the number of iterations is reached, the model is saved and exited; if the number of iterations is not reached, the deep hash network parameters are updated according to the loss function; after the parameters are updated, the original image sample is input into the deep hash network to generate the corresponding class center hash code and adversarial sample, and then the overall loss function of the adversarial training is calculated and back propagation is completed to realize the anti-interference performance training of the deep hash network.

[0058] In order to construct a discriminative semantic representation of the image content (i.e., class center hash code) to guide the adversarial training process, the semantic similarity from the training data to the center code must be preserved. To ensure credibility, this goal can be achieved by minimizing the Hamming distance between the center code of the original image and its semantically related samples, while maximizing the distance to irrelevant samples. Therefore, for the original image x, the objective function F1 of its class center hash code h is defined as:

[0059]

[0060] In the above formula, f(.) is the approximate hash function obtained by the deep hash model, D is the distance metric function, and w i and w j represents the corresponding distance weight, S p is the number of positive samples, S n is the number of negative samples, is a positive sample related to the original sample, is a negative sample that has nothing to do with the original sample. Based on the optimization of this objective function, the class center hash code gradually approaches the hash code of the semantically related sample and gradually moves away from the hash code of the semantically irrelevant sample. In theory, the optimal semantic preservation of the class center hash code can be achieved.

[0061] Based on this formula, the objective function formula for a single class center hash code is obtained as follows:

[0062]

[0063] In the above formula, K is the length of the hash code, D Ham is the Hamming distance metric function, is a positive sample related to the original sample The hash code of is a negative sample that has nothing to do with the original sample. The hash code of

[0064] At the same time, due to the binary characteristics of hash codes, the optimal solution of formula (2) is the class-centered hash code, which is calculated by the deviation between the weighted sum of positive and negative hash code samples:

[0065]

[0066] The proof that this formula satisfies the optimal solution condition is as follows:

[0067] First, this embodiment defines the following function:

[0068]

[0069] Since the class center hash code requires the optimal solution of the objective function, proving that formula (3) is the optimal solution is equivalent to proving the following formula:

[0070]

[0071] Let h = {h1,h2,...,h K},but:

[0072]

[0073] Among them, ξ is a constant. Similarly, we can also get ψ(h* ) is as follows:

[0074]

[0075] make: Then we have:

[0076]

[0077] Due to the properties of absolute value, we can get:

[0078]

[0079] Therefore, it can be concluded that formula (3) is the optimal solution of formula (2). By using formula (3), the calculation of the center hash code of each category of the input image sample can be completed quickly, and it can provide guidance for subsequent adversarial training.

[0080] After completing the image sample input and class center hash code calculation, the algorithm will next compare the hash code of each sample with the class center hash code. In the hash-based retrieval algorithm, adversarial examples are generated by inputting hash codes that are far away from the original category, thereby confusing the deep hash network and retrieving irrelevant results. Since the class center hash code can effectively represent the semantic information of each category of the image, this embodiment updates the parameters of the deep hash network through back propagation, and generates adversarial sample examples under the condition that the generated image hash code is far away from the class center hash code. Based on this principle, the target generation formula of adversarial samples is defined as follows:

[0081]

[0082] In the above formula, x a is an adversarial example of the input image x, h * is the class center hash code of the input image, ε is a positive constant close to zero, and for any hash codes h1 and h2, the Hamming distance metric function is:

[0083]

[0084] The above target generation formula is specifically described as:

[0085]

[0086] In order to alleviate the gradient vanishing problem of function f during training, we use the function f about the variable θ θ Replacing the function f in formula (11), the final target generation formula of the adversarial sample is obtained as follows:

[0087]

[0088] After generating adversarial samples, this embodiment uses them as enhanced data to optimize the deep hash network to make it anti-interference, that is, adversarial training. The most direct and effective idea for adversarial training of deep hash networks is to reconstruct the semantic similarity between adversarial samples and original samples so that adversarial samples can still produce correct matching results when input into the deep hash network. Since the class center hash code has retained the significant semantic information of the category as the optimal code, the deep hash network parameters can be updated by minimizing the Hamming distance between the adversarial sample hash code and the class center hash code, thereby increasing the semantic similarity between the adversarial sample and the original sample. Based on this principle, this embodiment defines the adversarial loss function as follows:

[0089]

[0090] In the above formula, N is the maximum number of categories, is the original sample of the input image x i The class-centric hash code, x a is the adversarial sample corresponding to the input image sample. For any hash codes h1 and h2, the Hamming distance metric function is:

[0091] D Ham (h1,h2)=(K-h1 T h2) / 2

[0092] The above adversarial loss function is described as follows:

[0093]

[0094] Similar to formula (12), in order to alleviate the gradient vanishing problem of function f during training, the function f with respect to variable θ is used θ Replacing the function f in equation (14), the final adversarial loss function is obtained as follows:

[0095]

[0096] In summary, the overall objective function of the adversarial training of the class center hash code proposed in this embodiment is as follows:

[0097] L t =L o +λL a (16)

[0098] In the above formula, λ is the weighting factor, L o is the loss function of the original deep hashing network, which ensures that the good retrieval performance of the hashing network is preserved.

[0099] In order to verify the effectiveness of the algorithm, this embodiment conducted experiments on the SpaceNet dataset. The dataset includes 10 classes and a total of 6,800 remote sensing images. 6,300 images were used to train the deep hash network, and 500 images were used to generate interference samples to test the effectiveness of the algorithm proposed in this embodiment. This embodiment was experimented on an RTX2080ti graphics processing unit (GPU) using the PyTorch framework. The learning rate was set to 0.05, the batch size was set to 32, and the number of epochs was set to 100. ResNet50 pre-trained on ImageNet was used as the basic backbone of the deep learning feature extraction method. Since the work done in this embodiment is based on the application of hashing methods in remote sensing images, an indicator widely used to evaluate the image matching performance of hashing methods, namely the target average precision (MAP), is used to evaluate the image matching effect after the deep hashing network anti-interference training. The experimental results show that after the remote sensing image hashing anti-interference training, each deep hashing network still has good image matching performance for the interference image, which verifies the effectiveness of the algorithm in image anti-interference.

[0100] In order to more intuitively compare the image matching performance changes of the deep hash network before and after remote sensing image hash anti-interference training, such as Figure 5-7 As shown, this embodiment plots the target accuracy curve and PR curve of the DPH and DPSH hash networks before and after anti-interference training under 64-bit hash codes. It can be seen from the figure that the target accuracy value of the interference image after the remote sensing image hash anti-interference training is only slightly lower than the target accuracy value of the original image. In addition, with the improvement of the return results and the recall rate, the target accuracy value of the interference image after the image hash anti-interference training decreases at a rate similar to the target accuracy value of the original image, further proving that the adaptability of the network to interference images after the remote sensing image hash anti-interference training is greatly improved.

Claims

1. A remote sensing marine target positioning anti-interference matching algorithm based on deep hashing, characterized in that: The steps include: Step 1, original deep hashing network training: It includes feature extraction and hash code generation. In the feature extraction stage, the convolutional neural network will extract continuous feature values ​​from the image input to the model; in the hash code generation stage, the activation function will convert the continuous feature values ​​into binary values, and finally constrain the binary values ​​through the loss function to generate hash codes; Step 2, anti-interference adversarial training of the original deep hashing network: Including class-centered hash code generation and adversarial training based on class-centered hash codes; In the process of class-centered hash code generation, the deep hash network first projects all training samples into the Hamming space to form the corresponding hash codes; Then, a master code is constructed for each class through discriminative learning, and the hash codes of images of the same category as the master code are made closer, while the hash codes of images of different categories from the master code are made further away; finally, the constructed master code is used as the class center hash code of each category; In adversarial training based on class-center hash codes, the training data is a set of image data, their class labels, and the class-center hash codes of each class; first, the image samples are subjected to feature extraction, full connection, and tanh activation by the deep hash network to generate the corresponding hash code, and the corresponding adversarial example image is generated by maximizing the Hamming distance between the hash code and the class-center hash code and back-propagating; then, the generated adversarial example image is input into the deep hash network again to generate the corresponding hash code, and the network parameters are updated by minimizing the Hamming distance between the hash code and the class-center hash code and back-propagating.

2. According to claim 1, the remote sensing marine target positioning anti-interference matching algorithm based on deep hashing is characterized by: The training process of the algorithm is as follows: first, the structural parameters of the original deep hash network are imported, and then the number of iterations is judged. If the number of iterations is reached, the model is saved and exited; if the number of iterations is not reached, the deep hash network parameters are updated according to the loss function; after the parameters are updated, the original image sample is input into the deep hash network to generate the corresponding class center hash code and adversarial sample, and then the overall loss function of the adversarial training is calculated and back propagation is completed to realize the anti-interference performance training of the deep hash network.

3. According to claim 2, the remote sensing marine target positioning anti-interference matching algorithm based on deep hashing is characterized in that: For the original image x, the objective function F1 of its class center hash code h is defined as: In the above formula, f(.) is the approximate hash function obtained by the deep hash model, D is the distance metric function, and w i and w j represents the corresponding distance weight, S p is the number of positive samples, S n is the number of negative samples, is a positive sample related to the original sample, is a negative sample that has nothing to do with the original sample; Based on this formula, the objective function formula for a single class center hash code is obtained as follows: In the above formula, K is the length of the hash code, D Ham is the Hamming distance metric function, is a positive sample related to the original sample The hash code of is a negative sample that has nothing to do with the original sample. The hash code of The optimal solution of formula (2) is the class center hash code, which is calculated by the deviation between the weighted sum of positive and negative hash code samples:

4. According to claim 2, the remote sensing marine target positioning anti-interference matching algorithm based on deep hashing is characterized in that: The target generation formula of adversarial samples is defined as follows: In the above formula, xa is the adversarial sample of the input image x, h* is the class center hash code of the input image, ε is a positive constant close to zero, and for any hash codes h1 and h2, the Hamming distance metric function is: The above target generation formula is specifically described as: Using the function f with respect to the variable θ θ Substituting the function f in formula (11), the final target generation formula is as follows:

5. According to claim 2, the remote sensing marine target positioning anti-interference matching algorithm based on deep hashing is characterized in that: The adversarial loss function is defined as follows: In the above formula, N is the maximum number of categories, is the original sample of the input image x i The class center hash code, x a is the adversarial sample corresponding to the input image sample. For any hash codes h1 and h2, the Hamming distance metric function is: The above adversarial loss function is described as follows: Using the function f with respect to the variable θ θ Replacing the function f in equation (14), the final adversarial loss function is obtained as follows: The overall objective function of adversarial training of class-centered hash codes is as follows: THE t =L o +λL a (16) In the above formula, λ is the weighting factor, L o is the loss function of the original deep hashing network.

Images